Data processing method and apparatus, and storage medium

Abstract

The application discloses a data processing method and device and a storage medium, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and Internet of Vehicles, and the method comprises the steps of receiving a data processing request sent by a target terminal; performing a hash operation on first identification information and second target identification information carried by the data processing request to obtain a first target hash value of the target terminal; mapping the target terminal to a hash ring according to the first target hash value; at least two servers are mapped on the hash ring, and the at least two servers are servers corresponding to the target object; taking the position of the target terminal on the hash ring as a starting point, searching for a server along the clockwise direction of the hash ring, and taking the first server found as a target server; and sending the data processing request to the target server to enable the target server to process the data processing request. The application improves the processing efficiency of the data processing request and reduces the failure rate of the server.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet communication, and particularly relates to a data processing method and device and a storage medium. BACKGROUND

[0002] Enterprise Instant Messaging (Enterprise IM) is a communication service tool for enterprise terminal users. As an office collaboration product, message transmission is the most basic function. In the prior art, one server is usually set for one enterprise. When a large enterprise with a large number of users holds a large-scale activity, the enterprise message is extremely active, and the load pressure of the one server is large, the response speed is slow, and even the server overload is triggered, causing business failure.

[0003] Therefore, it is necessary to provide a data processing method and device and a storage medium, which can improve the processing efficiency of data processing requests and reduce the failure rate of the server. SUMMARY

[0004] The present application provides a data processing method and device and a storage medium, which can improve the processing efficiency of data processing requests and reduce the failure rate of the server.

[0005] In one aspect, the present application provides a data processing method, which comprises:

[0006] receiving a data processing request sent by a target terminal; the data processing request carries first identification information of a target object corresponding to the target terminal and second target identification information of the target terminal; the target object corresponds to a target number of terminals, and the target number is greater than a preset number threshold;

[0007] performing a hash operation on the first identification information and the second target identification information to obtain a first target hash value of the target terminal; the first target hash value is a value in a preset range;

[0008] mapping the target terminal to a hash ring according to the first target hash value; at least two servers are mapped to the hash ring, and the at least two servers are servers corresponding to the target object; the at least two servers are mapped to the hash ring according to a second hash value corresponding to each server, and the second hash value is a value in the preset range;

[0009] taking the position of the target terminal on the hash ring as a starting point, searching for a server in the clockwise direction of the hash ring, and taking the first server found as a target server;

[0010] sending the data processing request to the target server, so that the target server processes the data processing request.

[0011] Another aspect provides a data processing device, the device comprising:

[0012] a request receiving module configured to receive a data processing request sent by a target terminal, wherein the data processing request carries first identification information of a target object corresponding to the target terminal and second target identification information of the target terminal, the target object corresponds to a target number of terminals, and the target number is greater than a preset number threshold;

[0013] a first target hash value determining module configured to perform hash operation on the first identification information and the second target identification information to obtain a first target hash value of the target terminal, wherein the first target hash value is a value in a preset range;

[0014] a hash value mapping module configured to map the target terminal to a hash ring according to the first target hash value, wherein the hash ring has at least two servers mapped thereon, the at least two servers are servers corresponding to the target object, and the at least two servers are mapped to the hash ring according to second hash values corresponding to the servers, wherein the second hash values are values in the preset range;

[0015] a target server determining module configured to take a position of the target terminal on the hash ring as a starting point, search for a server along a clockwise direction of the hash ring, and take a first server found as a target server;

[0016] a request sending module configured to send the data processing request to the target server, so that the target server processes the data processing request.

[0017] Another aspect provides a data processing device, the device comprising a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the data processing method as described above.

[0018] Another aspect provides a computer storage medium, the computer storage medium storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by a processor to implement the data processing method as described above.

[0019] The other aspect provides a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs to realize the data processing method as described above.

[0020] The data processing method, device and storage medium provided by the application have the following technical effects:

[0021] The application receives a data processing request sent by a target terminal; the data processing request carries first identification information of a target object corresponding to the target terminal and second target identification information of the target terminal; the first identification information and the second target identification information are subjected to a hash operation to obtain a first target hash value of the target terminal; the target terminal is mapped to a hash ring according to the first target hash value; at least two servers are mapped to the hash ring according to second hash values corresponding to the at least two servers, and the at least two servers are servers corresponding to the target object; a first server found along a clockwise direction of the hash ring is taken as a target server, starting from a position of the target terminal on the hash ring; the data processing request is sent to the target server, so that the target server processes the data processing request; thus, when the number of terminals corresponding to a target object is large, each terminal can be sequentially matched with a corresponding server to process a corresponding data processing request; the present application allocates multiple servers to an enterprise, determines a hash value of each user according to an enterprise identification and a user identification, matches a corresponding server for each user in the enterprise according to a consistent hash algorithm, uniformly disperses user requests in the same enterprise to specified part servers of a cluster, improves the processing efficiency of the requests, and reduces the failure rate of the servers. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0023] Figure 1 is a schematic diagram of a data processing system provided by an embodiment of the present application;

[0024] Figure 2 is a flowchart of a data processing method provided by an embodiment of the present application;

[0025] Figure 3 is a flowchart of a method for obtaining a first target hash value of a target terminal provided by an embodiment of the present application;

[0026] Figure 4 is a flowchart of a method for determining a storage server of data in each terminal provided by an embodiment of the present application;

[0027] Figure 5 is a flowchart of a method for determining an updated hash ring provided by an embodiment of the present application;

[0028] Figure 6 is a flowchart of a data migration method in a server provided by an embodiment of the present application;

[0029] Figure 7 is a flowchart of a consistent hash algorithm provided by an embodiment of the present application;

[0030] Figure 8 is a structure diagram of a multi-node hash ring provided by an embodiment of the present application;

[0031] Figure 9 is a structure diagram of a hash ring with added nodes provided by an embodiment of the present application;

[0032] Figure 10 is a structure diagram of a hash ring with deleted nodes provided by an embodiment of the present application;

[0033] Figure 11 is a structure diagram of a data processing apparatus provided by an embodiment of the present application;

[0034] Figure 12 is a structure diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0036] It should be noted that the terms "first", "second" and the like in the description and in the claims of the present application and in the above description of the drawings merely mean different instances of similar objects and do not necessarily imply a specific order or chronology. It is understood that the data thus designated can be interchanged, where appropriate, so that the embodiments of the present application described herein can be carried out in other than the order shown or described herein. Furthermore, the terms "comprise" and "have" and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or server that includes a list of steps or units not necessarily limited to those clearly identified as such, but can include other not clearly recited steps or units inherent therein.

[0037] Referring to Figure 1 , Figure 1 is a schematic diagram of a data processing system provided by an embodiment of the present application, as shown in Figure 1 , the data processing system can at least include a first server 01, a second server 02 and a client 03.

[0038] Specifically, in the embodiment of the present application, the first server 01 can include a standalone server, or a distributed server, or a server cluster composed of multiple servers, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The first server 01 can include a network communication unit, a processor, a memory and the like. Specifically, the first server 01 can be configured to receive a data processing request sent by the client 03, determine a second server 02 corresponding to the client 03, and send the data processing request to the second server 02.

[0039] Specifically, in the embodiment of the present application, the second server 02 can include a standalone server, or a distributed server, or a server cluster composed of multiple servers, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The second server 02 can include a network communication unit, a processor, a memory and the like. Specifically, the second server 02 can be configured to process the data processing request of the client 03.

[0040] Specifically, in the embodiments of the present application, the client 03 can include a smart phone, a desktop computer, a tablet computer, a notebook computer, a digital assistant, a smart wearable device, a smart speaker, a vehicle terminal, a smart television and the like, and can also include software running in the entity device, for example, a web page provided by a service provider to a user, and can also be an application provided by the service provider to the user. Specifically, the client 03 can be used to send a data processing request to the first server 01.

[0041] The following describes a data processing method of the present application, Figure 2 is a flowchart of a data processing method provided by the embodiments of the present application. The present specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps can be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one of the many execution orders, and does not represent the only execution order. In actual system or server product execution, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0042] Specifically as Figure 2 indicated, the method can include:

[0043] S201: receiving a data processing request sent by a target terminal; the data processing request carries first identification information of a target object corresponding to the target terminal and second target identification information of the target terminal; the target object corresponds to a target number of terminals, and the target number is greater than a preset number threshold.

[0044] In the embodiments of the present application, the target object can be a large enterprise, company, unit or the like with a target number of users, wherein the target number can be greater than a preset number threshold; the preset number threshold can be set according to actual conditions. The target terminal can be a user terminal corresponding to the target object, and the terminal mentioned in the embodiments is a user terminal corresponding to the target object; the first identification information of the target object can include but is not limited to the name (including full name and abbreviation) of the target object, ID and the like; the second target identification information of the target terminal can include but is not limited to the ID of the target terminal, user identification information (including name, ID number, mobile phone number, email, nickname and the like) corresponding to the target terminal.

[0045] In the embodiments of the present application, the data processing request can include but is not limited to a data acquisition request, a data analysis request, a data modification request and the like.

[0046] S203: performing a hash operation on the first identification information and the second target identification information to obtain a first target hash value of the target terminal; the first target hash value is a value in a preset range;

[0047] In the embodiment of the present application, the preset range is 0 to 2 32 -1; a hash algorithm maps binary values of arbitrary length to a shorter fixed-length binary value, and this small binary value is called a hash value. The hash value is a unique and extremely compact numerical representation of a piece of data. The ordinary hash algorithm either cannot meet the dynamic addition and removal of nodes or will cause a large range of jumps after meeting the above functions; the jump refers to the migration of data from one server node to another server node. The consistent hash algorithm of the embodiment can realize the dynamic addition and removal of nodes, and can reduce the jump of the request.

[0048] Consistent hashing proposes four adaptive conditions that the hash algorithm should meet in a dynamically changing cache environment:

[0049] 1. Balance

[0050] Balance refers to the fact that the hash result can be distributed to all buffers as much as possible, so that all buffer spaces can be utilized.

[0051] 2. Monotonicity

[0052] Monotonicity refers to the fact that if some content has been assigned to the corresponding buffer through hashing, and a new buffer is added to the system, the hash result should ensure that the original allocated content can be mapped to the new buffer, and will not be mapped to other buffers in the old buffer set. (When the buffer size changes, consistent hashing tries to protect the allocated content from being remapped to the new buffer.)

[0053] 3. Spread

[0054] In a distributed environment, a terminal may not be able to see all the buffers, but only a part of them. When the terminal wants to map the content to the buffer through the hash process, the hash result may be inconsistent due to the different buffer ranges seen by different terminals, and the final result is that the same content is mapped to different buffers by different terminals. This situation should obviously be avoided, because it leads to the same content being stored in different buffers, reducing the efficiency of system storage. The definition of spread is the severity of the above situation. A good hash algorithm should be able to avoid inconsistent situations as much as possible, that is, to reduce the spread as much as possible.

[0055] 4. Load

[0056] Load problem is actually to view the dispersion problem from another angle. Since different terminals can map the same content to different buffers, a specific buffer can also be mapped as different content by different users. As with dispersion, this situation should also be avoided, and therefore a good hash algorithm should be able to minimize the load of the buffer.

[0057] In the embodiments of the present application, a consistent hash algorithm can be used for hash operation, and the steps are as follows:

[0058] First, the hash value of the server (node) is calculated and configured on a circle of 0-2 32 -1.

[0059] Then, the hash value of the key of the stored data is calculated by the same method and mapped to the same circle.

[0060] Finally, the data is saved to the first server found by searching clockwise from the position where the data is mapped.

[0061] In the embodiments of the present application, as Figure 3 shown, the hash operation on the first identification information and the second target identification information is performed to obtain the first target hash value of the target terminal, which includes:

[0062] S2031: Hash operation is performed based on the first identification information to obtain a first identification hash value;

[0063] S2033: Hash operation is performed based on the second target identification information to obtain a second identification hash value;

[0064] S2035: The sum of the first identification hash value and the second identification hash value is determined as the first target hash value of the target terminal.

[0065] In the embodiments of the present application, the hash operation on the identification information can be performed according to the consistent hash algorithm, so as to ensure that the obtained first target hash value is within the range of 0 to 2 32 -1.

[0066] In a specific embodiment, as Figure 7 shown, for a data processing request sent by a terminal, the key corresponding to the request is obtained, and the consistent hash algorithm can be used to determine the target node number corresponding to the key, so as to determine the target node corresponding to the target node number to process the data processing request of the terminal.

[0067] Generally, the enterprise ID is set as the key (key=enterprise ID), that is, only one server is configured for an enterprise; for large enterprises, it is easy to cause server overload, thereby reducing user experience.

[0068] In the embodiments of the present application, the first identification information of the target object can be an enterprise ID, and the second target identification information of the target terminal can be a user ID; the key is set as key1+key2;

[0069] key1=enterprise ID*2 N ,

[0070] key2=Hash(user ID)&((1*2 N )-1).

[0071] Wherein, N=1 or 2 or 3, since the enterprise ID is the same, all users of the enterprise will be scattered to 2 N servers, i.e. 2 or 4 or 8, and are relatively aggregated in the case of being evenly distributed to multiple machines to reduce the load. Hash represents a hash function, which can map the key value (the first target hash value) to the range of 0 to 2 32 -1 by using key1 (the first identification hash value) and key2 (the second identification hash value), so as to map the key to the hash ring. First, a fixed number of servers are allocated to the enterprise according to the consistent hash algorithm, and the servers are uniformly configured in the form of nodes on the hash ring; then the server matched by each user terminal is determined according to the key value corresponding to each user; so as to configure different servers for different users in the enterprise.

[0072] Wherein & is a bitwise operator, representing "bitwise and", which is a binary operator. The bitwise operator operates on bits and performs operations bit by bit, and the operation rules are shown in Table 1:

[0073] Table 1

[0074] p q p & q 0 0 0 0 1 0 1 1 1 1 0 0

[0075] Suppose A=60, B=13, now expressed in binary format, which are as follows:

[0076] A=0011 1100

[0077] B=0000 1101

[0078] A&B=0000 1100.

[0079] In the embodiments of the present application, the consistent hash first hashes the key to a range of 0 to 2 32 -1, to obtain the first hash value, and the data range corresponds to the Figure 8The closed circle (hash ring) is shown. Meanwhile, the three deployed nodes are evenly dispersed on the circle as much as possible (nodes 1, 2, and 3 in the figure), where a node refers to a server for storing data of different terminals and processing different terminal requests. At this time, a request Key is processed by a Hash function to map to the first hash value 04 in Figure 9 , and the request corresponding to the node 2 is calculated by the clockwise rule; therefore, the request corresponding to the first hash value 04 needs to be processed by the node 2.

[0080] In the embodiment of the present application, as shown in Figure 9 , the hash ring in the figure includes server nodes 1, 2, and 3. If it is monitored that the node 2 is a load server, a node 4 can be added between the nodes 1 and 2. According to the consistent hash algorithm, the first node in the clockwise direction of the first hash value between the nodes 1 to 4 changes from the node 4 to the node 2; therefore, the terminal data corresponding to the first hash value between the nodes 1 to 4 needs to be migrated from the node 2 to the node 4, and the data stored in other nodes remains unchanged, so as to avoid a large amount of data migration when a new server node is added, that is, to avoid a large range of jump, and to minimize the number of jumps.

[0081] In the embodiment of the present application, during the running of the server, the running state of each server can be monitored regularly to determine whether a fault server occurs. As shown in Figure 8 , if it is monitored that the node 2 has a fault, the node 2 is removed from the hash ring to obtain the hash ring as shown in Figure 10 , and then the data stored in the node 2 is migrated to the server corresponding to the node 3 according to the clockwise rule, and the data in other nodes remains unchanged, so as to minimize the number of jumps.

[0082] S205: mapping the above target terminal to the hash ring according to the above first target hash value; the hash ring has at least two servers mapped thereon, and the at least two servers are servers corresponding to the target object; the at least two servers are mapped to the hash ring according to a second hash value corresponding to each server, and the second hash value is a value in a preset range.

[0083] In the embodiment of the present application, the second hash value and the first target hash value are both in the preset range, that is, both in the range of 0 to 2 32 -1. A preset number of servers can be set for the target object, and the preset number can be at least two; the servers are mapped to the hash ring, so as to facilitate the search for the corresponding server for the target terminal. The second hash value of each server can be set in advance, so as to ensure that all servers can be evenly distributed on the hash ring.

[0084] S207: Taking the position of the target terminal on the hash ring as a starting point, a server is searched along the clockwise direction of the hash ring, and the first server found is taken as the target server.

[0085] In the embodiments of the present application, when there are multiple target terminals, a consistent hash algorithm can be used to configure a corresponding server for each target terminal, so that user requests in the same enterprise are evenly distributed to specified parts of the cluster machines, improving the processing efficiency of the requests and reducing the failure rate of the servers.

[0086] In the embodiments of the present application, the first server found is taken as the target server, including:

[0087] determining whether the first server found has failed;

[0088] If the first server found has failed, the second server found is taken as the target server.

[0089] In the embodiments of the present application, after the failed server is determined, the failed server can be deleted from the hash ring to obtain an updated hash ring; taking the position of the target terminal on the updated hash ring as a starting point, the first server found is taken as the target server.

[0090] S209: The data processing request is sent to the target server, so that the target server processes the data processing request.

[0091] In the embodiments of the present application, the data processing request includes a target data acquisition request, and the data processing request is sent to the target server so that the target server processes the data processing request, including:

[0092] The target data acquisition request is sent to the target server, so that the target server acquires target data.

[0093] In the embodiments of the present application, the method further includes determining the storage server of data in each terminal; as shown in Figure 4 The determination of the storage server of data in each terminal includes:

[0094] S401: Obtain the second identification information corresponding to each of the target number of terminals;

[0095] S403: Hash processing is performed on the first identification information and the second identification information of each terminal to obtain a first hash value of each terminal, and the first hash value is a value in a preset range;

[0096] In this embodiment, both the first hash value and the first target hash value are within a preset range, that is, both are between 0 and 2. 32 Within the range of -1.

[0097] S405: Map the target number of terminals to the hash ring according to the first hash value corresponding to each terminal;

[0098] S407: Starting from the position of the preset terminal on the hash ring, search for the server in a clockwise direction along the hash ring; the preset terminal is any terminal among the target number of terminals.

[0099] S409: The first server found will be used as the storage server for the aforementioned preset terminal.

[0100] In this embodiment of the application, multiple terminals corresponding to the target object can be uniformly mapped onto the hash ring, thereby configuring a different server for each terminal.

[0101] In this embodiment of the application, during server operation, the above method further includes:

[0102] Based on the operating status of each server, determine whether there are any faulty servers;

[0103] If it exists, determine the next server in a clockwise direction from the faulty server;

[0104] Migrate the data stored on the faulty server to the next server mentioned above;

[0105] Remove the faulty server from the hash ring to obtain an updated hash ring;

[0106] Based on the updated hash ring, the storage server corresponding to each terminal is re-determined according to the consistent hashing algorithm.

[0107] In this embodiment of the application, if there is a faulty server, only the data stored in the faulty server and the first server in the clockwise direction will be affected, while the data stored in other servers will remain unchanged. This avoids a large amount of data migration when deleting server nodes, thus avoiding large-scale server crashes and minimizing the number of crashes.

[0108] In the embodiments of this application, such as Figure 5 As shown, the above method also includes:

[0109] S501: Determine the load value of each server on the hash ring described above;

[0110] S503: Servers with load values ​​greater than a preset threshold are identified as load servers;

[0111] S505: Determine the first server in the counterclockwise direction of the load server on the hash ring as a candidate server;

[0112] S507: Determine the load hash value of the load server and the candidate hash value of the candidate server;

[0113] S509: Set a new server for the target object, and determine the hash value of the new server; the hash value of the new server is between the load hash value and the candidate hash value;

[0114] S5011: Map the new server to the hash ring according to the hash value of the new server to obtain an updated hash ring.

[0115] In the embodiments of the present application, the load value of each server can be monitored regularly during the running of the server, and the load value can be represented by the number of data processing requests received by the server at the same time; one data processing request corresponds to one terminal; when the number of data processing requests received at the same time is greater than a preset threshold, the server is determined as a load server, that is, it is considered that the load of the server is too large, and in order to avoid failure, a server needs to be added for data migration.

[0116] In the embodiments of the present application, if there is a load server, only the data stored in the load server will be affected, and the data stored in other servers remains unchanged, thereby realizing that when a server node is added, a large amount of data migration is avoided, that is, a large-scale jump is avoided, and the minimization of the number of jumps is realized.

[0117] In the embodiments of the present application, as shown in Figure 6 After the new server is mapped to the hash ring according to the hash value of the new server to obtain an updated hash ring, the method further includes:

[0118] S5013: Determine the terminal with a hash value between the candidate hash value and the hash value of the new server in the updated hash ring as a target load terminal;

[0119] S5015: Obtain the data of the target load terminal from the load server;

[0120] S5017: Migrate the data of the target load terminal to the new server.

[0121] In actual use scenarios, for example, in enterprise instant messaging applications, enterprises often have a large number of sudden messages. If the server is configured in the granularity of an enterprise, a large number of requests will be directed to a single server for processing, resulting in message accumulation. Even the server may be overloaded, causing business failure, causing a large number of user complaints, and reducing user experience. On the other hand, if the granularity is dispersed in the user of the enterprise, a single user in the enterprise is too dispersed on all machines in the cluster, which is not conducive to subsequent permission verification and cache data processing.

[0122] The technical solution of the present application can avoid different user requests of the same enterprise falling on a single server, while not making the data corresponding to the user request too dispersed. Through the solution, the user requests in the same enterprise can be evenly dispersed to the specified part of the servers in the cluster, improving the processing efficiency of the request, reducing the failure rate of the server, and improving the stability of the enterprise instant messaging application.

[0123] From the above technical solution provided by the embodiments of the present application, it can be seen that the embodiments of the present application receive a data processing request sent by a target terminal; the data processing request carries first identification information of a target object corresponding to the target terminal and second target identification information of the target terminal; the first identification information and the second target identification information are subjected to a hash operation to obtain a first target hash value of the target terminal; the target terminal is mapped to a hash ring according to the first target hash value; at least two servers are mapped on the hash ring, and the at least two servers are servers corresponding to the target object; the at least two servers are mapped to the hash ring according to a second hash value corresponding to each server; a first server found along the clockwise direction of the hash ring is taken as a target server, starting from the position of the target terminal on the hash ring; the data processing request is sent to the target server, so that the target server processes the data processing request; thereby when the number of terminals corresponding to the target object is large, each terminal can be sequentially matched with a corresponding server to process a corresponding data processing request; the present application allocates multiple servers to an enterprise, and determines the hash value of each user according to the enterprise identification and the user identification; each user in the enterprise is matched with a corresponding server according to the consistent hash algorithm; the user requests in the same enterprise are evenly dispersed to the specified part of the servers in the cluster, improving the processing efficiency of the request, and reducing the failure rate of the server.

[0124] The embodiments of the present application also provide a data processing device, as shown in Figure 11 The device comprises:

[0125] The request receiving module 1110 is configured to receive a data processing request sent by a target terminal; the data processing request carries first identification information of a target object corresponding to the target terminal and second target identification information of the target terminal; the target object corresponds to a target number of terminals, and the target number is greater than a preset number threshold;

[0126] The first target hash value determining module 1120 is configured to perform hash operation on the first identification information and the second target identification information to obtain a first target hash value of the target terminal; the first target hash value is a value in a preset range.

[0127] In some embodiments, the preset range is 0 to 2 32 -1.

[0128] The hash value mapping module 1130 is configured to map the target terminal to a hash ring according to the first target hash value; the hash ring is mapped with at least two servers, and the at least two servers are servers corresponding to the target object; the at least two servers are mapped to the hash ring according to second hash values corresponding to the servers, and the second hash values are values in the preset range.

[0129] In some embodiments, the second hash value and the first target hash value are both in the preset range, i.e., both in the range of 0 to 2 32 -1.

[0130] The target server determining module 1140 is configured to take the position of the target terminal on the hash ring as a starting point, search for a server along the clockwise direction of the hash ring, and take the first found server as a target server.

[0131] The request sending module 1150 is configured to send the data processing request to the target server, so that the target server processes the data processing request.

[0132] In some embodiments, the target server determining module can include:

[0133] The fault judging unit is configured to judge whether the first found server is faulty;

[0134] The target server determining unit is configured to take the second found server as the target server if the first found server is faulty.

[0135] In some embodiments, the data processing request includes a target data acquisition request, and the request sending module can include:

[0136] The data acquisition request sending unit is configured to send the target data acquisition request to the target server, so that the target server acquires the target data.

[0137] In some embodiments, the apparatus can further include:

[0138] The storage server determination module is configured to determine the storage server of the data in each terminal.

[0139] In some embodiments, the storage server determination module can include:

[0140] The second identification information acquisition unit is configured to acquire the second identification information corresponding to each of the target number of terminals;

[0141] The hash processing unit is configured to perform hash processing on the first identification information and the second identification information of each terminal to obtain a first hash value of each terminal, the first hash value being a value within a preset range.

[0142] In some embodiments, the first hash value and the first target hash value are both within a preset range, i.e., both within the range of 0 to 2 32 -1.

[0143] The first hash value mapping unit is configured to map the target number of terminals to the hash ring according to the first hash value corresponding to each terminal.

[0144] The server searching unit is configured to search for a server along the clockwise direction of the hash ring, starting from the position of a preset terminal on the hash ring, the preset terminal being any terminal among the target number of terminals.

[0145] The storage server determination unit is configured to determine the first server found as the storage server of the preset terminal.

[0146] In some embodiments, the apparatus can further include:

[0147] The load value determination module is configured to determine the load value of each server on the hash ring.

[0148] The load server determination module is configured to determine a server with a load value greater than a preset threshold as a load server.

[0149] The candidate server determination module is configured to determine the first server located in the counterclockwise direction of the load server on the hash ring as a candidate server.

[0150] The candidate hash value determination module is configured to determine the load hash value of the load server and the candidate hash value of the candidate server.

[0151] The application further provides a data processing device, which comprises a processor and a memory.

[0152] The application further provides a data processing device, which comprises a processor and a memory.

[0153] In some embodiments, the device can further comprise:

[0154] The target load terminal determination module is configured to determine a terminal with a hash value between the candidate hash value and the hash value of the new server in the updated hash ring as a target load terminal.

[0155] The data acquisition module is configured to acquire data of the target load terminal from the load server.

[0156] The data migration module is configured to migrate the data of the target load terminal to the new server.

[0157] In some embodiments, the first target hash value determination module can comprise:

[0158] The first identification hash value determination unit is configured to perform hash operation based on the first identification information to obtain a first identification hash value.

[0159] The second identification hash value determination unit is configured to perform hash operation based on the second target identification information to obtain a second identification hash value.

[0160] The first target hash value determination unit is configured to determine a sum of the first identification hash value and the second identification hash value as the first target hash value of the target terminal.

[0161] The device in the device embodiment and the method embodiment are based on the same inventive concept.

[0162] The application provides a data processing device, which comprises a processor and a memory.

[0163] The application further provides a computer storage medium, which can be arranged in a terminal to save at least one instruction or at least one program related to a data processing method in the method embodiment.

[0164] The embodiment of the present application further provides a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs a data processing method provided by the above method embodiment.

[0165] Optionally, in the embodiment of the present application, the storage medium can be located in at least one of a plurality of network servers of a computer network. Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0166] The memory provided in the embodiment of the present application can be used to store software programs and modules, and the processor performs various function applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by functions, etc.; and the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide access of the processor to the memory.

[0167] The data processing method provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking the case of running on a server as an example, Figure 12 is a hardware structure block diagram of a server of a data processing method provided in the embodiment of the present application. As shown in Figure 12As shown, the server 1200 can vary greatly in configuration and performance, and can include one or more Central Processing Units (CPU) 1210 (which can include, but is not limited to, a microprocessor, a microcontroller, a programmable logic device, etc.), a memory 1230 for storing data, one or more storage media 1220 (such as one or more mass storage devices) for storing applications 1223 or data 1222. The memory 1230 and the storage media 1220 can be of any type of volatile or non-volatile storage, including, but not limited to, magnetic, optical, or semiconductor-based storage. The applications 1223 stored in the storage media 1220 can include one or more modules, each of which can include a series of instructions for operating a server. Further, the CPU 1210 can be configured to communicate with the storage media 1220 to execute the series of instructions in the storage media 1220 to operate the server 1200. The server 1200 can also include one or more power supplies 1260, one or more wired or wireless network interfaces 1250, one or more input / output interfaces 1240, and / or one or more operating systems 1221, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0168] The input / output interface 1240 can be configured to receive or transmit data via a network. Examples of the network can include a wireless network provided by a communication provider of the server 1200. In one example, the input / output interface 1240 includes a network interface controller (NIC) that can connect to other network devices through a base station to communicate with the Internet. In one example, the input / output interface 1240 can be a radio frequency (RF) module that can communicate with the Internet through a wireless manner.

[0169] Those of ordinary skill in the art can understand that, Figure 12 The structure shown is merely illustrative and does not limit the structure of the electronic device described above. For example, the server 1200 can include more or less components than those shown, or have a different configuration of components than those shown. Figure 12 For example, the server 1200 can include more or less components than those shown, or have a different configuration of components than those shown. Figure 12 For example, the server 1200 can include more or less components than those shown, or have a different configuration of components than those shown.

[0170] According to the data processing method, device, equipment or storage medium provided in the embodiments of the present application, the data processing request sent by the target terminal is received; the data processing request carries the first identification information of the target object corresponding to the target terminal and the second target identification information of the target terminal; the first identification information and the second target identification information are subjected to hash operation to obtain the first target hash value of the target terminal; the target terminal is mapped onto a hash ring according to the first target hash value; at least two servers are mapped onto the hash ring, and the at least two servers are servers corresponding to the target object; the at least two servers are mapped onto the hash ring according to the second hash value corresponding to each server; the first server found along the clockwise direction of the hash ring is taken as the target server, starting from the position of the target terminal on the hash ring; the data processing request is sent to the target server, so that the target server processes the data processing request; and thus when the number of terminals corresponding to the target object is large, each terminal can be sequentially matched with the corresponding server to process the corresponding data processing request; the present application allocates multiple servers to one enterprise, and determines the hash value of each user according to the enterprise identification and the user identification; each user in the enterprise is matched with the corresponding server according to the consistent hash algorithm; the user requests in the same enterprise are evenly dispersed to the specified part of the servers in the cluster, the processing efficiency of the request is improved, and the failure rate of the server is reduced.

[0171] It should be noted that the above-mentioned embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be executed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0172] Each embodiment in the present specification is described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, the device, equipment and storage medium embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0173] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed by a program to complete the related hardware, and the program can be stored in a computer storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0174] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method, characterized by, The method includes: The system receives a data processing request sent by a target terminal; the data processing request carries a first identification information of a target object corresponding to the target terminal and a second target identification information of the target terminal; the target object corresponds to a target number of terminals, the target number being greater than a preset number threshold; the target object is an enterprise with a target number of users, and the target terminals are terminals of users within the enterprise; A hash operation is performed based on the first identification information of the enterprise to obtain the first identification hash value; A hash operation is performed on the second target identifier information of the target terminal corresponding to the user within the enterprise to obtain the second identifier hash value; The sum of the first identifier hash value and the second identifier hash value is determined as the first target hash value of the target terminal; the first target hash value is a value within a preset range; The target terminal is mapped onto a hash ring according to the first target hash value; at least two servers are mapped onto the hash ring, and the at least two servers are the servers corresponding to the target object; the at least two servers are mapped onto the hash ring according to the second hash value corresponding to each server, and the second hash value is a value within the preset range; Starting from the position of the target terminal on the hash ring, search for servers in a clockwise direction along the hash ring, and take the first server found as the target server; The data processing request is sent to the target server so that the target server processes the data processing request.

2. The method of claim 1, wherein, The step of using the first found server as the target server includes: Determine if the first server found has failed; If the first server found fails, the second server found will be used as the target server.

3. The method of claim 1, wherein, The data processing request includes a target data acquisition request, and sending the data processing request to the target server so that the target server processes the data processing request includes: Send the target data acquisition request to the target server so that the target server can acquire the target data.

4. The method according to claim 3, characterized in that, The method further includes: determining the storage server for the data in each terminal; the method for determining the storage server for the data in each terminal includes: Obtain the second identification information corresponding to each of the target number of terminals; The first identification information and the second identification information of each terminal are hashed to obtain the first hash value of each terminal, wherein the first hash value is a value within the preset range; The target number of terminals are mapped onto the hash ring according to the first hash value corresponding to each terminal. Starting from the position of a preset terminal on the hash ring, the server is searched in a clockwise direction along the hash ring; the preset terminal is any terminal among the target number of terminals. The first server found will be used as the storage server for the preset terminal.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Determine the load value of each server on the hash ring; Servers with load values ​​exceeding a preset threshold are identified as load servers; The first server on the hash ring located in the counterclockwise direction from the load server is identified as a candidate server; Determine the load hash value of the load server and the candidate hash value of the candidate server; A new server is set up for the target object, and the hash value of the new server is determined; the hash value of the new server is located between the load hash value and the candidate hash value. The newly added server is mapped onto the hash ring according to its hash value to obtain an updated hash ring.

6. The method according to claim 5, characterized in that, After mapping the newly added server to the hash ring according to the hash value of the newly added server to obtain the updated hash ring, the method further includes: In the updated hash ring, the terminal whose hash value lies between the candidate hash value and the hash value of the newly added server is determined as the target load terminal; Obtain the data of the target load terminal from the load server; The data from the target load terminal is migrated to the newly added server.

7. A data processing apparatus, characterized in that, The device includes: A request receiving module is used to receive a data processing request sent by a target terminal; the data processing request carries a first identification information of a target object corresponding to the target terminal and a second target identification information of the target terminal; the target object corresponds to a target number of terminals, the target number being greater than a preset number threshold; the target object is an enterprise with a target number of users, and the target terminals are terminals of users within the enterprise; The first target hash value determination module is used to perform a hash operation on the first identification information and the second target identification information to obtain the first target hash value of the target terminal; the first target hash value is a value within a preset range. A hash value mapping module is used to map the target terminal onto a hash ring according to the first target hash value; the hash ring is mapped to at least two servers, the at least two servers being the servers corresponding to the target object; the at least two servers are mapped onto the hash ring according to the second hash value corresponding to each server, the second hash value being a value within the preset range; The target server determination module is used to search for servers in a clockwise direction along the hash ring, starting from the position of the target terminal on the hash ring, and to take the first server found as the target server. A request sending module is used to send the data processing request to the target server so that the target server processes the data processing request. The first target hash value determination module includes: The first identifier hash value determination unit is used to perform a hash operation based on the first identifier information of the enterprise to obtain the first identifier hash value; The second identifier hash value determination unit is used to perform a hash operation based on the second target identifier information of the target terminal corresponding to the user within the enterprise to obtain the second identifier hash value. The first target hash value determination unit is used to determine the sum of the first identifier hash value and the second identifier hash value as the first target hash value of the target terminal.

8. The apparatus according to claim 7, characterized in that, The target server determination module includes: The fault diagnosis unit is used to determine whether the first server found has failed. The target server determination unit is used to determine the second server found as the target server if the first server found fails.

9. The apparatus according to claim 7, characterized in that, The data processing request includes a target data acquisition request, and the request sending module includes: A data acquisition request sending unit is used to send the target data acquisition request to the target server so that the target server can acquire the target data.

10. The apparatus according to claim 9, characterized in that, The device further includes: The storage server determination module is used to determine the storage server for the data in each terminal; The storage server determination module includes: The second identification information acquisition unit is used to acquire the second identification information corresponding to each of the target number of terminals; A hash processing unit is used to perform hash processing on the first identification information and the second identification information of each terminal to obtain a first hash value for each terminal, wherein the first hash value is a value within a preset range; The first hash value mapping unit is used to map the target number of terminals onto the hash ring according to the first hash value corresponding to each terminal. The server lookup unit is used to search for a server in a clockwise direction along the hash ring, starting from the position of a preset terminal on the hash ring; the preset terminal is any terminal among the target number of terminals. A storage server determination unit is used to select the first found server as the storage server for the preset terminal.

11. The apparatus according to any one of claims 7-10, characterized in that, The device further includes: The load value determination module is used to determine the load value of each server on the hash ring; The load server determination module is used to identify servers whose load value is greater than a preset threshold as load servers; The candidate server determination module is used to determine the first server on the hash ring located in the counterclockwise direction of the load server as a candidate server. A candidate hash value determination module is used to determine the load hash value of the load server and the candidate hash value of the candidate server; A new hash value determination module is added, used to set up a new server for the target object and determine the hash value of the new server; the hash value of the new server is located between the load hash value and the candidate hash value; The hash ring update module is used to map the newly added server onto the hash ring according to the hash value of the newly added server, so as to obtain an updated hash ring.

12. The apparatus according to claim 11, characterized in that, The device further includes: The target load terminal determination module is used to determine the terminals in the updated hash ring whose hash values ​​are located between the candidate hash values ​​and the hash values ​​of the newly added servers as target load terminals. The data acquisition module is used to acquire data from the target load terminal from the load server; The data migration module is used to migrate the data of the target load terminal to the newly added server.

13. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the data processing method as described in any one of claims 1-6.

14. A computer program product, the computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the data processing method according to any one of claims 1-6.

Images