Counting processing method and device, and electronic equipment

Abstract

The present disclosure provides a counting processing method and device, and an electronic device, which relate to the field of data processing technology. The method applied to any server in a distributed counting cluster includes: receiving operation requests for multiple hotspot data sent by a client within a preset time period; filtering out operation requests for target hotspot data from the operation requests for multiple hotspot data to obtain a target operation request set; superimposing the operation requests in the target operation request set to obtain equivalent operation requests for the target hotspot data within a preset time period; based on the equivalent operation requests, performing consistent counting processing on the target hotspot data in each server in the distributed counting cluster. The technical solution provided by the present disclosure can improve the counting efficiency of the distributed counting cluster, and is conducive to reducing the delay in processing hotspot data.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular, to a counting processing method, a counting processing device, and an electronic device for implementing the counting processing method. Background Art

[0002] Counters are often used in various statistical scenarios, such as statistics on the number of interface requests. In the relevant technical solutions of distributed technical systems, the count value increase and decrease are updated serially, and a consistency protocol (for example, the Paxos protocol) is used to achieve strong consistency of the distributed counting cluster. Therefore, for each counting operation request from the client, a consistency protocol process must be run once. Especially in the case of high-concurrency requests for hot data, the solutions provided by the relevant technologies have the technical problem of low counting efficiency.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the present disclosure is to provide a counting processing method, a counting processing device, an electronic device and a computer-readable storage medium, thereby improving the counting efficiency of a distributed counting cluster to a certain extent and facilitating reducing the delay when processing hotspot data.

[0005] According to a first aspect of the present disclosure, a counting processing method is provided, which is applied to any server in a distributed counting cluster, and the method includes:

[0006] Receiving operation requests for multiple hotspot data sent by a client within a preset time period;

[0007] Filtering out operation requests for target hotspot data from the operation requests for the plurality of hotspot data to obtain a target operation request set;

[0008] Superimposing the operation requests in the target operation request set to obtain equivalent operation requests for the target hotspot data within the preset time period;

[0009] Based on the above equivalent operation request, consistent counting processing is performed on the target hotspot data in each server in the above distributed counting cluster.

[0010] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, an operation request for target hotspot data is screened out from the above-mentioned operation requests for multiple hotspot data to obtain a target operation request set, including:

[0011] Bucket processing of the above operation requests for multiple hot data to obtain N hash buckets, where N is a positive integer;

[0012] All operation requests for the i-th target hotspot data are obtained in the i-th hash bucket to obtain an operation request set corresponding to the i-th target hotspot data, where i is a positive integer less than or equal to N.

[0013] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, bucket processing of the above-mentioned operation requests for multiple hot data includes:

[0014] Based on the coroutine atomic operation, receive the operation request for the i-th hotspot data;

[0015] The above operation request for the i-th hot data is added to the task chain of the i-th hash bucket.

[0016] In an exemplary embodiment of the present disclosure, based on the foregoing embodiment, all operation requests for the i-th target hotspot data are obtained in the i-th hash bucket, including:

[0017] The mutex lock of the i-th hash bucket above is snatched;

[0018] In response to the thread that successfully grabs the lock, all operation requests for the i-th target hotspot data are screened out in the task chain of the i-th hash bucket.

[0019] In an exemplary embodiment of the present disclosure, based on the above embodiment, before superimposing the operation requests in the above target operation request set, the above method further includes:

[0020] It is verified that the operation requests in the target operation request set in the current server are the latest data, so as to superimpose the latest data.

[0021] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the operation requests in the above-mentioned target operation request set are superimposed to obtain an equivalent operation request for the above-mentioned target hotspot data within the above-mentioned preset time period, including:

[0022] The count increase and decrease values ​​corresponding to the operation requests in the target operation request set are accumulated, and the accumulated value obtained is used as the equivalent operation request for the target hotspot data within the preset time period.

[0023] In an exemplary embodiment of the present disclosure, based on the above embodiment, before superimposing the operation requests in the above target operation request set, the above method further includes:

[0024] Verify that the operation requests in the above target operation request set in the current server contain non-latest data, return the processing result of the above target operation request set as failure, and redefine the above target operation request set in another server in the distributed counting cluster, so that the above another server verifies whether the operation requests in the above redetermined target operation request set are the latest data.

[0025] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned equivalent operation request is based on which the target hotspot data in each server in the above-mentioned distributed counting cluster is subjected to consistent counting processing, including:

[0026] A consistency protocol is run in the distributed counting cluster to write the target count value corresponding to the equivalent operation request into each server in the distributed counting cluster, so as to implement consistent counting processing of the target hotspot data in each server in the distributed counting cluster.

[0027] In an exemplary embodiment of the present disclosure, based on the above embodiment, after performing consistency counting processing on the target hotspot data in each server in the above distributed counting cluster, the above method further includes:

[0028] The unique identification code uuid of the operation request corresponding to the above target operation request set is recorded, so as to be deleted when an operation request with the above uuid is received again.

[0029] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the operation request for multiple hotspot data sent by the client is a virtual operation request determined by the client by executing the following steps, including:

[0030] Receiving multiple operation requests from users for target hotspot data;

[0031] Hash mapping is performed on the uuid of each of the above operation requests to obtain multiple virtual operation requests for the above target hotspot data;

[0032] Sending multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster.

[0033] According to a second aspect of the present disclosure, a counting processing method is provided, which is applied to a client interacting with any server in a distributed counting cluster, and the method includes:

[0034] Receiving multiple operation requests from users for target hotspot data;

[0035] Perform hash mapping on the unique identification code uuid of each of the above operation requests to obtain multiple virtual operation requests for the above target hotspot data;

[0036] Sending multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster, so that any server in the distributed counting cluster performs the following steps based on the received virtual operation request:

[0037] Receiving a virtual operation request for multiple hotspot data sent by a client within a preset time period;

[0038] Filtering virtual operation requests for target hotspot data from the virtual operation requests for the plurality of hotspot data to obtain a target operation request set;

[0039] Superimposing the virtual operation requests in the target operation request set to obtain an equivalent operation request for the target hotspot data within the preset time period;

[0040] Based on the above equivalent operation request, consistent counting processing is performed on the target hotspot data in each server in the above distributed counting cluster.

[0041] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, multiple virtual operation requests for the above-mentioned target hotspot data are sent to multiple servers in the above-mentioned distributed counting cluster, including:

[0042] In response to the operation request being a write request, randomly acquiring a virtual operation request from the multiple virtual operation requests of the target hotspot data as a target virtual operation request;

[0043] The target virtual operation request is sent to any server in the distributed counting cluster.

[0044] According to a third aspect of the present disclosure, a counting processing device is provided. The counting processing device is configured in any server in a distributed counting cluster, and includes: a first receiving module, a screening module, a superposition module and a counting processing module.

[0045] The first receiving module is configured to: receive an operation request for multiple hotspot data sent by a client within a preset time period;

[0046] The above-mentioned screening module is configured to: screen out the operation requests for the target hotspot data from the above-mentioned operation requests for the multiple hotspot data, and obtain a target operation request set;

[0047] The superposition module is configured to: superimpose the operation requests in the target operation request set to obtain equivalent operation requests for the target hotspot data within the preset time period; and

[0048] The counting processing module is configured to: perform consistent counting processing on the target hotspot data in each server in the distributed counting cluster based on the equivalent operation request.

[0049] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned screening module includes: a bucket processing submodule and an acquisition submodule.

[0050] The bucket processing submodule is configured to: perform bucket processing on the operation requests for the multiple hotspot data to obtain N hash buckets, where N is a positive integer; and

[0051] The acquisition submodule is configured to: acquire all operation requests for the i-th target hotspot data in the i-th hash bucket, and obtain the operation request set corresponding to the i-th target hotspot data, where i is a positive integer less than or equal to N.

[0052] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned bucket processing submodule is specifically configured as follows:

[0053] Based on the coroutine atomic operation, receiving an operation request for the i-th hotspot data; and,

[0054] The above operation request for the i-th hot data is added to the task chain of the i-th hash bucket.

[0055] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the acquisition submodule is specifically configured as follows:

[0056] The mutex lock of the i-th hash bucket above performs a lock grabbing operation; and,

[0057] In response to the thread that successfully grabs the lock, all operation requests for the i-th target hotspot data are screened out in the task chain of the i-th hash bucket.

[0058] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the counting processing device further includes: a verification module.

[0059] The verification module is configured to verify that the operation requests in the target operation request set in the current server are the latest data before the superposition module superimposes the operation requests in the target operation request set, so as to superimpose the latest data.

[0060] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned superposition module is specifically configured as follows:

[0061] The count increase and decrease values ​​corresponding to the operation requests in the target operation request set are accumulated, and the accumulated value obtained is used as the equivalent operation request for the target hotspot data within the preset time period.

[0062] In an exemplary embodiment of the present disclosure, based on the above embodiment, the verification module is further configured as follows:

[0063] Before the superposition module superimposes the operation requests in the target operation request set, it verifies that the operation requests in the target operation request set in the current server contain non-latest data, returns the processing result of the target operation request set as failure, and redefines the target operation request set in another server in the distributed counting cluster, so that the other server verifies whether the operation requests in the redetermined target operation request set are the latest data.

[0064] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the counting processing module is configured as follows:

[0065] A consistency protocol is run in the distributed counting cluster to write the target count value corresponding to the equivalent operation request into each server in the distributed counting cluster, so as to implement consistent counting processing of the target hotspot data in each server in the distributed counting cluster.

[0066] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the counting processing device further includes: a recording module.

[0067] Among them, the above-mentioned recording module is configured as: after the above-mentioned counting processing module performs consistency counting processing on the target hotspot data in each server in the above-mentioned distributed counting cluster, record the unique identification code uuid of the operation request corresponding to the above-mentioned target operation request set, so as to delete it when the operation request of the above-mentioned uuid is received again.

[0068] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the operation request for multiple hotspot data sent by the client is a virtual operation request determined by the client by executing the following steps, including:

[0069] Receiving multiple operation requests from users for target hotspot data;

[0070] Hash mapping is performed on the uuid of each of the above operation requests to obtain multiple virtual operation requests for the above target hotspot data;

[0071] Sending multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster.

[0072] According to a fourth aspect of the present disclosure, a counting processing device is provided. The counting processing device is configured on a client that interacts with any server in a distributed counting cluster, and includes: a second receiving module, a hash mapping module, and a sending module.

[0073] The second receiving module is configured to: receive multiple operation requests from users for target hotspot data;

[0074] The hash mapping module is configured to: perform hash mapping on the uuid of each of the operation requests to obtain multiple virtual operation requests for the target hotspot data; and

[0075] The sending module is configured to: map multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster, so that any server in the distributed counting cluster performs the following steps based on the received virtual operation request:

[0076] Receiving a virtual operation request for multiple hotspot data sent by a client within a preset time period;

[0077] Filtering virtual operation requests for target hotspot data from the virtual operation requests for the plurality of hotspot data to obtain a target operation request set;

[0078] Superimposing the virtual operation requests in the target operation request set to obtain equivalent operation requests for the target hotspot data within the preset time period; and

[0079] Based on the above equivalent operation request, consistent counting processing is performed on the target hotspot data in each server in the above distributed counting cluster.

[0080] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the sending module is specifically configured as follows:

[0081] In response to the operation request being a write request, randomly acquiring a virtual operation request from the multiple virtual operation requests of the target hotspot data as a target virtual operation request; and

[0082] The target virtual operation request is sent to any server in the distributed counting cluster.

[0083] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the counting processing method described in any embodiment of the above-mentioned first aspect is implemented, and the technical processing method described in any embodiment of the above-mentioned second aspect is implemented.

[0084] According to a sixth aspect of the present disclosure, there is provided an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the counting processing method described in any embodiment of the first aspect above and execute the counting processing method described in any embodiment of the second aspect above by executing the executable instructions.

[0085] The exemplary embodiments of the present disclosure may have the following partial or all beneficial effects:

[0086] In the counting processing technical solution provided by an exemplary embodiment of the present disclosure, any server in the distributed counting cluster receives operation requests for multiple hot data sent by a client within a preset time period, and filters out operation requests for target hot data from these operation requests for multiple hot data, so as to achieve aggregating the operation requests for different hot data within the preset time period, that is, obtaining an operation request set for each hot data. Further, the operation requests in the operation request sets corresponding to each hot data are respectively superimposed to obtain equivalent operation requests for each hot data within the preset time period. Based on the equivalent operation requests, consistent counting processing is performed on the target hot data in each server in the distributed counting cluster.

[0087] It can be seen that in this technical solution, the operation requests for the same hot data within a preset time period are aggregated and superimposed to determine the equivalent operation requests within the preset time period, and then based on the equivalent operation requests, consistent counting processing is implemented in each server in the distributed counting cluster, improving the counting processing efficiency in the distributed counting cluster by means of batch processing of operation requests.

[0088] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] The drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0090] Figure 1 Schematically shows a usage scenario diagram of a counting processing method to which the embodiments of the present disclosure can be applied;

[0091] Figure 2 Schematically shows a flowchart of a counting processing method according to an embodiment of the present disclosure;

[0092] Figure 3 An information interaction diagram schematically illustrates a counting processing method according to an embodiment of the present disclosure;

[0093] Figure 4 A schematic diagram of a solution for processing an operation request by a client according to an embodiment of the present disclosure is shown;

[0094] Figure 5 A flowchart schematically illustrates a method for determining an operation request set according to an embodiment of the present disclosure;

[0095] Figure 6 A flowchart schematically illustrates a method for determining an operation request set according to another embodiment of the present disclosure;

[0096] Figure 7 A schematic diagram of a hash bucket structure for an operation request according to an embodiment of the present disclosure is shown;

[0097] Figure 8 The structure diagram of a counting processing device according to an embodiment of the present disclosure is schematically shown;

[0098] Fig. 9 A structural diagram of a counting processing device according to another embodiment of the present disclosure is schematically shown;

[0099] Fig.10 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0100] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure.

[0101] However, those skilled in the art will appreciate that the technical solutions of the present disclosure can be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0102] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0103] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0104] In the counting processing solution for distributed systems provided by related technologies, if the paxos protocol is used as a strong consistency for each server in the system, and due to the characteristic that the paxos entity can only determine one value at a time, and the time for a resolution through the paxos protocol is 1 to 2 ms, then in the case of high concurrency requests for hot data, the low counting efficiency will cause a large number of requests to be backlogged, and the long request delay will reduce the user experience.

[0105] Based on one or more problems in the related art, the present disclosure provides a counting processing method, device, computer-readable storage mechanism and electronic device.

[0106] The technical solution of the embodiment of the present disclosure is described in detail below:

[0107] first, Figure 1 A schematic diagram of a usage scenario of a counting processing method to which an embodiment of the present disclosure can be applied is shown.

[0108] like Figure 1 As shown, the distributed counting cluster 100 includes multiple hosts, such as servers 105, 105', 105", and the client for the user to send an operation request is set in the terminal device, such as one or more of the terminal devices 101, 102, 103 in the figure. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and any host of the distributed counting cluster 100. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103 can be various electronic devices with display screens, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc.

[0109] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0110] based on Figure 1 The usage scenario of the counting processing solution shown in the figure is as follows to introduce the counting processing method.

[0111] in, Figure 2 The flowchart of the counting processing method according to an embodiment of the present disclosure is schematically shown. Specifically, Figure 2 The execution subject of the counting processing method shown is any server in the distributed counting cluster (such as Figure 1 The server 105 in the figure), the embodiment shown in the figure includes:

[0112] Step S210, receiving an operation request for multiple hotspot data sent by a client within a preset time period;

[0113] Step S220, filtering out operation requests for target hotspot data from the operation requests for the plurality of hotspot data to obtain a target operation request set;

[0114] Step S230, superimposing the operation requests in the target operation request set to obtain an equivalent operation request for the target hotspot data within the preset time period; and

[0115] Step S240: Based on the equivalent operation request, consistent counting processing is performed on the target hotspot data in each server in the distributed counting cluster.

[0116] exist Figure 2 In the technical solution provided by the illustrated embodiment, the operation requests for the same hot data within a preset time are aggregated and superimposed to determine equivalent operation requests within the preset time, and then the consistent counting processing in each server in the distributed counting cluster is implemented based on the equivalent operation requests, and the counting processing efficiency in the distributed counting cluster is improved by batch processing of operation requests. Even in the case of high concurrent requests for hot data, a large number of requests will not be backlogged, which effectively reduces the request delay and improves the user experience.

[0117] The following Figure 2 The specific implementation methods of each step in the embodiment shown are described in detail:

[0118] In an exemplary embodiment, Figure 3 The information interaction flow chart of the counting processing method according to an embodiment of the present disclosure is schematically shown. Figure 3 The counting process information interaction diagram shown is Figure 2 The specific implementation methods of each step are explained in detail:

[0119] In this technical solution, the strong consistency of data storage in each host in the distributed counting cluster is achieved through the paxos protocol. Among them, the paxos entity can only determine the characteristic of one value at a time. In order to solve the bottleneck of single key reading and writing, this technical solution is on the client (with Figure 3 Taking the client 310 in the example as an example) after receiving an operation request for the same hot data key, the following technical solution can be executed first, and then the execution result is sent to the distributed counting cluster:

[0120] Step S410, receiving multiple operation requests from users for target hotspot data; and step S420, performing hash mapping on the uuid of each of the operation requests to obtain multiple virtual operation requests for the target hotspot data.

[0121] In an exemplary embodiment, the operation request includes: the hotspot key to be operated and the increase or decrease value of the hotspot key, and also includes a universally unique identifier (uuid). Figure 4 , the client receives S (S is a positive integer) operation requests regarding the hotspot key_a (i.e., the above-mentioned target hotspot data). Since different operation requests have different uuids, in this embodiment, the uuid of each operation request regarding the hotspot key_a is hash mapped Hash(UUID)%5 to obtain S virtual operation requests regarding the hotspot key_a (key_a_1, key_a_2, key_a_3, key_a_4, key_a_5,...).

[0122] This method of determining virtual operation requests for hot key can map the same hot key on the user side to different Paxos entity processing on different machines in the distributed cluster, which can significantly improve the counting processing efficiency of the hot key.

[0123] Further, step S430 is executed to send multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster. Exemplarily, the operation request includes a write request and a read request. When the operation request is a write request, a random one of the multiple virtual operation requests corresponding to each write request is selected to initiate a write request to the server (refer to Figure 4 Key_a_3 is randomly selected from the source and a write request is initiated to the server. For a read request, the value of the original key can be restored only after the values ​​of all virtual operation requests corresponding to the read request are accumulated.

[0124] It should be noted that although the above solution sacrifices some read performance to enhance write performance, in the actual use scenario of the counter, the number of write requests is often far greater than the number of read requests. Therefore, for the counting processing solution, hash mapping the operation request before sending it to the server has far more advantages than disadvantages.

[0125] It can be seen that the technical solution performs hash mapping on the operation request issued by the user on the client to obtain multiple virtual operation requests for the same hot key, thereby evenly distributing the user's request to multiple machines in the distributed counting cluster, so as to further improve the counting processing efficiency and reduce latency, which is particularly beneficial for improving write performance, that is, improving the counting processing efficiency of write operations.

[0126] In an exemplary embodiment, step S430 distributes the user's operation request relatively evenly to multiple machines in the distributed computing cluster, such as Figure 3 Part of the virtual operation requests are sent to server A in the distributed counting cluster, and part of the virtual operation requests are sent to server B in the distributed counting cluster. In the following embodiments, the technical solution after server A receives the virtual operation request is described as an example.

[0127] In an exemplary embodiment, continue to refer to Figure 3 After server A in the distributed counting cluster 300 receives a virtual operation request within a preset time period, the server processes the virtual operation request in the following steps:

[0128] Exemplarily, the preset duration may be the duration required to run a paxos protocol.

[0129] In step S220', the virtual operation request for the target hotspot data is screened out from the virtual operation requests for the multiple hotspot data to obtain a target operation request set. It should be noted that step S220' is a specific implementation of step S220. Specifically, the processing object "operation request" of step S220 can be either a "virtual operation request" obtained after client hash mapping or an "operation request" that has not been hash mapped.

[0130] Here, the technical solution is described by taking the processing object of "virtual operation request" as an example.

[0131] Specifically, Figure 5 The flowchart of the method for determining the operation request set according to an embodiment of the present disclosure is schematically shown. Figure 5 In step S510, the virtual operation request for the multiple hot data is processed by bucketing to obtain N hash buckets, where N is a positive integer.

[0132] In an exemplary embodiment, in order to achieve concurrent processing to improve data processing efficiency, the technical solution receives a virtual operation request for the i-th (1≤i≤N) hot data based on the coroutine atomic operation (step S610), and adds the virtual operation request for the i-th hot data to the task chain of the i-th hash bucket (step S620). Figure 7 , taking the 6th hash bucket as an example, the hash bucket contains operation requests for hot data key_6.

[0133] Continue to refer Figure 5 After obtaining the hash bucket, in step S520, all virtual operation requests for the i-th target hotspot data are obtained in the i-th hash bucket to obtain a virtual operation request set corresponding to the i-th target hotspot data.

[0134] In an exemplary embodiment, considering that virtual operation requests of different hot keys may be mapped to the same bucket through hash, a screening operation may be performed when determining the virtual operation request set. Figure 6 Taking the first hash bucket (MutexList1) as an example, the virtual operation request of the hotspot key_31 may be mapped to the list of the first hash bucket through hash. When determining the target operation request set for the target hotspot data key_1, the virtual operation request of the hotspot key_31 needs to be screened out (skip).

[0135] Specifically, each hash bucket contains a mutex in its header, and the mutex of the hash bucket performs a lock grabbing operation (step S630), and in response to the thread that successfully grabs the lock, all virtual operation requests of the target hot data key_1 are screened out in the task chain of the hash bucket (step S640), thereby obtaining the target operation request set of the target hot data key_1. For example, if the above-mentioned preset time length can be the time length taken to run a paxos protocol, then the virtual operation request set corresponding to the hotspot key_1 within the time taken for a paxos interaction is: [(key_1,+1), (key_1,+3), (key_1,+2), (key_1,+1), (key_1,-1)].

[0136] pass Figure 5 and Figure 6 The provided technical solution realizes the aggregation of virtual operation requests for different hotspot data within the above preset time, that is, obtaining a virtual operation request set for each hotspot data. Furthermore, the virtual operation requests in the virtual operation request set corresponding to each hotspot data can be superimposed to obtain an equivalent operation request for each hotspot data within the preset time.

[0137] Continue to refer Figure 3In an exemplary embodiment, before superimposing the virtual operation requests in the virtual operation request set corresponding to the hotspot data, in order to ensure the counting accuracy, server A further performs step S250 for the virtual operation set corresponding to each hotspot data: checking whether the operation request in the target operation request set in the current server is the latest data, so as to superimpose the latest data.

[0138] For example, the above verification can be implemented by running the paxos protocol. If the verification result of server A on the virtual operation request set corresponding to the hotspot key_b is that the local data is not the latest, the processing result of the virtual operation request set corresponding to the hotspot key_b is returned as failure, and another server (such as Figure 3 Server B executes the "retry operation" in which the virtual operation request set corresponding to the hotspot key_b is re-determined, so that server B verifies whether the virtual operation request set corresponding to the hotspot key_b is the latest data, so as to determine the latest operation request of the hotspot key.

[0139] If the verification result of the virtual operation request set corresponding to the hotspot key_c by server A is that the local data is the latest, then step S230' is executed: the virtual operation requests in the target operation request set are superimposed to obtain equivalent operation requests for the target hotspot data within the preset time length. That is, server A superimposes the operation requests of the virtual operation request set corresponding to the hotspot key_c to obtain equivalent operation requests for the hotspot key_c within the preset time length, thereby realizing batch processing of the operation requests for the hotspot key.

[0140] Specifically, the count increase and decrease values ​​corresponding to each operation request in the target operation request set are accumulated, and the accumulated value obtained is used as the equivalent operation request for the target hotspot data within the preset time length. Exemplarily, the above preset time length can be the time taken to run a paxos protocol. The virtual operation request set corresponding to the hotspot key_c within the time taken for a paxos interaction is: [(key_c,+5), (key_c,+2), (key_c,-2), (key_c,+6), (key_c,-1)], then within the preset time length, the equivalent operation request for the hotspot key_c is: (key_c,+5+2-2+6-1=+10).

[0141] Exemplarily, the accumulated value obtained by accumulating the increase and decrease values ​​corresponding to all virtual operation requests in the virtual operation request set is added to the value corresponding to the corresponding hotspot key. For example, the initial value corresponding to hotspot key_c is 1010, then the accumulated value + 10 corresponding to the equivalent operation request for hotspot key_c is added to the initial value 1010 corresponding to the corresponding hotspot key_c, and the target count value corresponding to the hotspot key_c is 1020.

[0142] It should be noted that step S230' is a specific implementation of step S230. Specifically, the processing object "operation request" of step S230 can be a "virtual operation request" obtained after the client hash mapping, or an "operation request" that has not been hash mapped. The technical solution is described above by taking the processing object "virtual operation request" as an example.

[0143] Continue to refer Figure 2 / Figure 3 Based on the equivalent operation request, in step S240, consistent counting processing is performed on the target hotspot data in each server in the distributed counting cluster.

[0144] Specifically, the count increase and decrease values ​​corresponding to the operation requests in the target operation request set are accumulated, and the accumulated value is added to the value corresponding to the target hotspot data to obtain the target count value of the target hotspot data (such as the above-mentioned hotspot key_c and its target count value 1020). Then, the consistency protocol (paxos protocol) is run in the distributed counting cluster to write the target count value to each server in the distributed counting cluster to perform consistency counting processing on the target hotspot data in each server in the distributed counting cluster. As a result, the count values ​​for the same hotspot key in each server in the distributed counting cluster are consistent. For example, the count value of the hotspot key_c in each server in the distributed counting cluster is 1020.

[0145] In an exemplary embodiment, the uuid of each virtual operation request corresponding to the equivalent operation request is recorded to prevent the client from timeout and repeated writing, thereby ensuring the counting accuracy.

[0146] This solution is designed to solve the counting problem of high-concurrency hot spots. The operation requests for the same hot spot data within a preset time are aggregated and superimposed to determine the equivalent operation requests within the preset time. Then, based on the equivalent operation requests, consistent counting processing is implemented in each server in the distributed counting cluster. The counting processing efficiency in the distributed counting cluster is improved by batch processing of operation requests.

[0147] This technical solution proposes a high-performance counter solution based on strong storage consistency. For the case of using the paxos protocol to achieve strong data consistency in a distributed counting cluster, all requests within the time taken for a paxos interaction (i.e., within the above-mentioned preset time length) are aggregated to obtain the operation request set corresponding to each hotspot data, and the count values ​​in the aggregated operation request set are further superimposed to obtain equivalent operation requests, and finally paxos is run based on equivalent operation requests. Batch processing of operation requests is implemented, thereby saving the time-consuming overhead of paxos synchronization and eliminating the backlog of hotspot key requests under high concurrency. The caller only needs to call the corresponding client interface to pass in the key to be operated and the value to be increased or decreased, and then it can obtain the high-performance and highly available counting service provided by the server, providing users with convenient and fast counting services.

[0148] Furthermore, it has been found through testing that for a distributed counting cluster consisting of three servers, the counting processing method provided by this technical solution (i.e. Figure 2 The method of batch processing operation requests on the server side as shown in Figure 4 The method of performing hash mapping on the client side for the operation request of the hotspot key to obtain the virtual operation request is shown), and the number of requests that can be processed per second for a single hotspot key is about 1.5 million. If the serial counting processing method provided by the related technology is adopted, only about 1k requests can be processed per second. By comparison, it can be seen that compared with the technical solution provided by the related technology, this technical solution can effectively improve the counting efficiency of the hotspot key.

[0149] Those skilled in the art will appreciate that all or part of the steps to implement the above embodiments are implemented as a computer program executed by a processor (including a CPU and a GPU). When the computer program is executed by the processor, the above functions defined by the above method provided by the present disclosure are performed. The program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk or an optical disk, etc.

[0150] In addition, it should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiment of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0151] Furthermore, in this exemplary embodiment, a counting processing device is also provided. Figure 8 As shown, the counting processing device 800 is configured in any server in the distributed counting cluster, and includes: a first receiving module 801 , a screening module 802 , a superposition module 803 and a counting processing module 804 .

[0152] The first receiving module 801 is configured to: receive an operation request for multiple hotspot data sent by a client within a preset time period;

[0153] The screening module 802 is configured to: screen out the operation requests for the target hotspot data from the operation requests for the plurality of hotspot data to obtain a target operation request set;

[0154] The superposition module 803 is configured to: superimpose the operation requests in the target operation request set to obtain equivalent operation requests for the target hotspot data within the preset time period; and

[0155] The counting processing module 804 is configured to: perform consistent counting processing on the target hotspot data in each server in the distributed counting cluster based on the equivalent operation request.

[0156] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the screening module 802 includes: a bucket processing submodule and an acquisition submodule.

[0157] The bucket processing submodule is configured to: perform bucket processing on the operation requests for the multiple hotspot data to obtain N hash buckets, where N is a positive integer; and

[0158] The acquisition submodule is configured to: acquire all operation requests for the i-th target hotspot data in the i-th hash bucket, and obtain the operation request set corresponding to the i-th target hotspot data, where i is a positive integer less than or equal to N.

[0159] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned bucket processing submodule is specifically configured as follows:

[0160] Based on the coroutine atomic operation, receiving an operation request for the i-th hotspot data; and,

[0161] The above operation request for the i-th hot data is added to the task chain of the i-th hash bucket.

[0162] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the acquisition submodule is specifically configured as follows:

[0163] The mutex lock of the i-th hash bucket above performs a lock grabbing operation; and,

[0164] In response to the thread that successfully grabs the lock, all operation requests for the i-th target hotspot data are screened out in the task chain of the i-th hash bucket.

[0165] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the counting processing device further includes: a verification module.

[0166] The verification module is configured to verify that the operation requests in the target operation request set in the current server are the latest data before the superposition module 803 superimposes the operation requests in the target operation request set, so as to superimpose the latest data.

[0167] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned superposition module 803 is specifically configured as follows:

[0168] The count increase and decrease values ​​corresponding to the operation requests in the target operation request set are accumulated, and the accumulated value obtained is used as the equivalent operation request for the target hotspot data within the preset time period.

[0169] In an exemplary embodiment of the present disclosure, based on the above embodiment, the verification module is further configured as follows:

[0170] Before the superposition module 803 superimposes the operation requests in the target operation request set, it verifies that the operation requests in the target operation request set in the current server contain non-latest data, returns the processing result of the target operation request set as failure, and redefines the target operation request set in another server in the distributed counting cluster, so that the other server verifies whether the operation requests in the redetermined target operation request set are the latest data.

[0171] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the counting processing module 804 is configured as follows:

[0172] A consistency protocol is run in the distributed counting cluster to write the target count value corresponding to the equivalent operation request into each server in the distributed counting cluster, so as to implement consistent counting processing of the target hotspot data in each server in the distributed counting cluster.

[0173] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the counting processing device further includes: a recording module.

[0174] Among them, the above-mentioned recording module is configured as: after the above-mentioned counting processing module 804 performs consistency counting processing on the target hotspot data in each server in the above-mentioned distributed counting cluster, record the unique identification code uuid of the operation request corresponding to the above-mentioned target operation request set, so as to delete it when the operation request of the above-mentioned uuid is received again.

[0175] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the operation request for multiple hotspot data sent by the client is a virtual operation request determined by the client by executing the following steps, including:

[0176] Receiving multiple operation requests from users for target hotspot data;

[0177] Hash mapping is performed on the uuid of each of the above operation requests to obtain multiple virtual operation requests for the above target hotspot data;

[0178] Sending multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster.

[0179] The specific details of each module or unit in the above counting processing device have been described in detail in the counting processing method executed by the execution subject being any server in the distributed counting cluster, so they will not be repeated here.

[0180] Furthermore, in this exemplary embodiment, a counting processing device is also provided. Fig. 9 As shown, the counting processing device 900 is configured on a client that interacts with any server in a distributed counting cluster, and includes: a second receiving module 901 , a hash mapping module 902 and a sending module 903 .

[0181] The second receiving module 901 is configured to: receive multiple operation requests from users for target hotspot data;

[0182] The hash mapping module 902 is configured to: perform hash mapping on the uuid of each of the operation requests to obtain multiple virtual operation requests for the target hotspot data; and

[0183] The sending module 903 is configured to map multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster, so that any server in the distributed counting cluster performs the following steps based on the received virtual operation request:

[0184] Receiving a virtual operation request for multiple hotspot data sent by a client within a preset time period;

[0185] Filtering virtual operation requests for target hotspot data from the virtual operation requests for the plurality of hotspot data to obtain a target operation request set;

[0186] Superimposing the virtual operation requests in the target operation request set to obtain equivalent operation requests for the target hotspot data within the preset time period; and

[0187] Based on the above equivalent operation request, consistent counting processing is performed on the target hotspot data in each server in the above distributed counting cluster.

[0188] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the sending module 903 is specifically configured as follows:

[0189] In response to the operation request being a write request, randomly acquiring a virtual operation request from the multiple virtual operation requests of the target hotspot data as a target virtual operation request; and

[0190] The target virtual operation request is sent to any server in the distributed counting cluster.

[0191] The specific details of each module or unit in the above counting processing device have been described in detail in the counting processing method in which the execution subject is a client interacting with any server in the distributed counting cluster, so they will not be repeated here.

[0192] In the exemplary embodiments of the present disclosure, a computer-readable storage medium capable of implementing the above method is also provided. A program product capable of implementing the above method of the present specification is stored thereon. In some possible embodiments, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the above program product is run on a terminal device, the above program code is used to enable the above terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of the present specification.

[0193] According to the program product for implementing the above method in the embodiment of the present disclosure, it can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0194] The above program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0195] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0196] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0197] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0198] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. Fig.10 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown.

[0199] It should be noted that Fig.10 The computer system 1000 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0200] like Fig.10As shown, the computer system 1000 includes a processor 1001, wherein the processor 1001 may include: a graphics processing unit (GPU), a central processing unit (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 to the random access memory (RAM) 1003. In RAM 1003, various programs and data required for system operation are also stored. The processor (GPU / CPU) 1001, ROM 1002 and RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0201] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0202] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor (GPU / CPU) 1001, various functions defined in the system of the present application are executed. In some embodiments, the computer system 1000 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0203] It should be noted that the computer-readable medium shown in the embodiment of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0204] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0205] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0206] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0207] For example, the electronic device can implement Figure 2 The method shown in and executed by any server in the distributed counting cluster: step S210, receiving operation requests for multiple hotspot data sent by a client within a preset time length; step S220, filtering out operation requests for target hotspot data from the operation requests for multiple hotspot data to obtain a target operation request set; step S230, superimposing the operation requests in the target operation request set to obtain equivalent operation requests for the target hotspot data within the preset time length; and step S240, based on the equivalent operation requests, performing consistency counting processing on the target hotspot data in each server in the distributed counting cluster.

[0208] For another example, the electronic device can implement the following Figure 3The method shown in the figure is performed by a client interacting with any server in a distributed counting cluster: step S410, receiving multiple operation requests of a user for target hotspot data; step S420, performing hash mapping on the unique identification code uuid of each operation request to obtain multiple virtual operation requests for the target hotspot data; and step S430, sending the multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster, so that any server in the distributed counting cluster performs the following steps based on the received virtual operation request:

[0209] Receive virtual operation requests for multiple hotspot data sent by a client within a preset time period; filter out virtual operation requests for target hotspot data from the virtual operation requests for the multiple hotspot data to obtain a target operation request set; superimpose the virtual operation requests in the target operation request set to obtain equivalent operation requests for the target hotspot data within the preset time period; and perform consistency counting processing on the target hotspot data in each server in the distributed counting cluster based on the equivalent operation requests.

[0210] For another example, the electronic device can implement the following Figure 3 Other steps: Figure 5 The steps shown or Figure 6 The steps shown in .

[0211] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0212] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0213] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0214] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A counting processing method, It is characterized in that Applied to any server in a distributed counting cluster, the method comprises: Receiving operation requests for multiple hotspot data sent by a client within a preset time period; Filtering out operation requests for target hotspot data from the operation requests for the plurality of hotspot data to obtain a target operation request set; Superimposing the operation requests in the target operation request set to obtain equivalent operation requests for the target hotspot data within the preset time period; Based on the equivalent operation request, consistent counting processing is performed on the target hotspot data in each server in the distributed counting cluster.

2. The counting processing method according to claim 1, It is characterized in that Filtering out operation requests for target hotspot data from the operation requests for the plurality of hotspot data to obtain a target operation request set includes: Bucket processing of the operation requests for the multiple hotspot data to obtain N hash buckets, where N is a positive integer; All operation requests for the i-th target hotspot data are obtained in the i-th hash bucket to obtain an operation request set corresponding to the i-th target hotspot data, where i is a positive integer less than or equal to N.

3. The counting processing method according to claim 2, It is characterized in that Bucket processing of the operation request for the plurality of hotspot data includes: Based on the coroutine atomic operation, receive the operation request for the i-th hotspot data; The operation request for the i-th hot data is added to the task chain of the i-th hash bucket.

4. The counting processing method according to claim 2, It is characterized in that Get all operation requests for the i-th target hotspot data in the i-th hash bucket, including: The mutex lock of the i-th hash bucket performs a lock grabbing operation; In response to the thread that successfully grabs the lock, all operation requests for the i-th target hotspot data are screened out in the task chain of the i-th hash bucket.

5. The counting processing method according to any one of claims 1 to 4, It is characterized in that Before superimposing the operation requests in the target operation request set, the method further includes: It is verified that the operation requests in the target operation request set in the current server are the latest data, so as to superimpose the latest data.

6. The counting processing method according to claim 5, It is characterized in that The superimposing of the operation requests in the target operation request set to obtain an equivalent operation request for the target hotspot data within the preset time period includes: The count increase and decrease values ​​corresponding to the operation requests in the target operation request set are accumulated, and the accumulated value obtained is used as the equivalent operation request for the target hotspot data within the preset time period.

7. The counting processing method according to any one of claims 1 to 4, It is characterized in that Before superimposing the operation requests in the target operation request set, the method further includes: Verify that the operation requests in the target operation request set in the current server contain non-up-to-date data, return the processing result of the target operation request set as a failure, and re-determine the target operation request set in another server in the distributed counting cluster, so that the other server verifies whether the operation requests in the re-determined target operation request set are up-to-date data.

8. The counting processing method according to any one of claims 1 to 4, characterized in that the consistent counting processing of the target hot data in each server in the distributed counting cluster based on the equivalent operation request includes: Running a consistency protocol in the distributed counting cluster to write the target count value corresponding to the equivalent operation request into each server in the distributed counting cluster, so as to realize the consistent counting processing of the target hot data in each server in the distributed counting cluster.

9. The counting processing method according to any one of claims 1 to 4, characterized in that after the consistent counting processing of the target hot data in each server in the distributed counting cluster, the method further includes: Recording the unique identification code uuid of the operation request corresponding to the target operation request set for deletion when the operation request with the uuid is received again.

10. The counting processing method according to any one of claims 1 to 4, characterized in that the operation request sent by the client for multiple hot data is a virtual operation request determined by the client by performing the following steps, including: Receiving multiple operation requests from the user for the target hot data; Performing hash mapping on the unique identification code uuid of each operation request to obtain multiple virtual operation requests for the target hot data; Sending the multiple virtual operation requests for the target hot data to multiple servers in the distributed counting cluster.

11. A counting processing method, characterized in that applied to a client interacting with any server in a distributed counting cluster, the method includes: Receiving multiple operation requests from the user for the target hot data; Performing hash mapping on the unique identification code uuid of each operation request to obtain multiple virtual operation requests for the target hot data; Sending the multiple virtual operation requests for the target hot data to multiple servers in the distributed counting cluster, so that any server in the distributed counting cluster performs the following steps based on the received virtual operation requests: Receiving virtual operation requests sent by the client for multiple hot data within a preset time period; Filtering out the virtual operation requests for the target hot data from the virtual operation requests for multiple hot data to obtain a target operation request set; Overlaying the virtual operation requests in the target operation request set to obtain an equivalent operation request for the target hot data within the preset time period; Based on the equivalent operation request, performing consistent counting processing on the target hot data in each server in the distributed counting cluster.

12. The counting processing method according to claim 11, It is characterized in that Sending multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster includes: In response to the operation request being a write request, randomly acquiring a virtual operation request from multiple virtual operation requests of the target hotspot data as a target virtual operation request; The target virtual operation request is sent to any server in the distributed computing cluster.

13. A counting processing device, It is characterized in that The device is configured in any server in a distributed counting cluster, and comprises: The first receiving module is configured to: receive an operation request for multiple hotspot data sent by a client within a preset time period; The screening module is configured to: screen out the operation requests for the target hotspot data from the operation requests for the plurality of hotspot data to obtain a target operation request set; The superposition module is configured to: superimpose the operation requests in the target operation request set to obtain an equivalent operation request for the target hotspot data within the preset time period; The counting processing module is configured to: perform consistent counting processing on the target hotspot data in each server in the distributed counting cluster based on the equivalent operation request.

14. A counting processing device, It is characterized in that The device is configured on a client that interacts with any server in a distributed counting cluster, and includes: The second receiving module is configured to: receive multiple operation requests of the user for the target hotspot data; A hash mapping module is configured to: perform hash mapping on the uuid of each operation request to obtain multiple virtual operation requests for the target hotspot data; The sending module is configured to: map multiple virtual operation requests for the target hotspot data to multiple servers in the distributed counting cluster, so that any server in the distributed counting cluster performs the following steps based on the received virtual operation request: Receiving a virtual operation request for multiple hotspot data sent by a client within a preset time period; Filtering virtual operation requests for target hotspot data from the virtual operation requests for the plurality of hotspot data to obtain a target operation request set; Superimposing the virtual operation requests in the target operation request set to obtain an equivalent operation request for the target hotspot data within the preset time period; Based on the equivalent operation request, consistent counting processing is performed on the target hotspot data in each server in the distributed counting cluster.

15. An electronic device, It is characterized in that include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the counting processing method according to any one of claims 1 to 10 by executing the executable instructions, and Execute the counting processing method described in claim 11 or claim 12.

Images