Data distribution method, device, electronic device and storage medium

Abstract

The embodiment of the present invention discloses a data distribution method, device, electronic device and storage medium. The method includes: receiving request data to be distributed, obtaining data source information of the request data; determining the request hash key corresponding to the request data based on the data source information and the consistent hash algorithm; determining the target cluster node corresponding to the request hash key in each cluster node based on the request hash key and the pre-established hash mapping relationship between each cluster node and each hash key, and distributing the request data to the target cluster node. The technical solution of the embodiment of the present invention can distribute request data through data source information and the consistent hash algorithm, solve the problem of uncontrollable data traffic, make better use of cluster resources, and help improve data processing efficiency.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a data distribution method, device, electronic device, and storage medium. Background Art

[0002] With the rapid expansion of data volume, in order to improve the efficiency and capability of data processing, the method of using multiple cluster nodes to process data has been widely used.

[0003] In existing technologies, when allocating cluster nodes to requested data, data is typically categorized to facilitate traceability, with data of the same type assigned to the same cluster node. This allows users to access data of a certain type by accessing only one cluster node. For example, data requests can be categorized based on the carrier used by the requesting user terminal. However, using existing technologies, the data traffic allocated to each cluster node is often uncontrollable, leading to imbalanced allocation, wasted resources, and reduced data processing efficiency. Summary of the Invention

[0004] The embodiments of the present invention provide a data distribution method, device, electronic device and storage medium to achieve better utilization of cluster resources and improve data processing efficiency.

[0005] In a first aspect, an embodiment of the present invention provides a data distribution method, including:

[0006] Receiving request data to be allocated, and obtaining data source information of the request data;

[0007] Determine a request hash key corresponding to the requested data based on the data source information and a consistent hashing algorithm;

[0008] Based on the request hash key and the pre-established hash mapping relationship between each cluster node and each hash key, a target cluster node corresponding to the request hash key is determined in each cluster node, and the request data is distributed to the target cluster node.

[0009] In a second aspect, an embodiment of the present invention further provides a data distribution device, the device comprising:

[0010] A data source information acquisition module is used to receive request data to be allocated and acquire data source information of the request data;

[0011] A request hash key determination module is used to determine a request hash key corresponding to the request data based on the data source information and a consistent hash algorithm;

[0012] The request data allocation module is used to determine the target cluster node corresponding to the request hash key in each cluster node based on the request hash key and the pre-established hash mapping relationship between each cluster node and each hash key, and allocate the request data to the target cluster node.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the data distribution method provided by any embodiment of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data distribution method provided by any embodiment of the present invention.

[0018] A data distribution method provided by an embodiment of the present invention receives request data to be distributed and obtains data source information of the request data; based on the data source information and a consistent hashing algorithm, determines a request hash key corresponding to the request data; based on the request hash key and a pre-established hash mapping relationship between each cluster node and each hash key, determines a target cluster node corresponding to the request hash key in each cluster node, and distributes the request data to the target cluster node. By distributing request data through data source information and a consistent hashing algorithm, the embodiment of the present invention solves the problem of uncontrollable data traffic, enables better utilization of cluster resources, and is conducive to improving data processing efficiency.

[0019] In addition, a data distribution device, electronic device, and storage medium provided by the present invention correspond to the above method and have the same beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A flow chart of a data distribution method provided by an embodiment of the present invention;

[0022] Figure 2 A flowchart of another data distribution method provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a data distribution method in a multi-gateway scenario provided by an embodiment of the present invention;

[0024] Figure 4 A structural diagram of a data distribution device provided by an embodiment of the present invention;

[0025] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0027] It should also be noted that, for ease of description, only the part relevant to the present invention, rather than all of the content, is shown in the accompanying drawings. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processing or methods depicted as flow charts. Although flow charts describe various operations (or steps) as sequential processing, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. When its operation is completed, the processing can be terminated, but can also have additional steps not included in the accompanying drawings. The processing can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0028] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] Example 1

[0030] Figure 1 This is a flowchart of a data distribution method provided by an embodiment of the present invention. The method can be executed by a data distribution device, which can be implemented by software and / or hardware and can be configured in a terminal and / or server to implement the data distribution method in the embodiment of the present invention.

[0031] like Figure 1 As shown, the method of this embodiment may specifically include:

[0032] S101: Receive request data to be allocated and obtain data source information of the request data.

[0033] In a specific implementation, when request data to be allocated is received, data source information of the request data can be determined. Optionally, obtaining the data source information of the request data includes obtaining at least one of an Internet Protocol address of a requesting terminal corresponding to the request data, a requesting user identifier, a requesting terminal identifier, and a requesting organization identifier. Those skilled in the art may modify the content of the data source information based on actual application requirements, and this embodiment of the present invention is not limited thereto.

[0034] S102: Determine the request hash key corresponding to the requested data based on the data source information and the consistent hashing algorithm.

[0035] Specifically, a consistent hash calculation can be performed on the data source information of the request data, and the calculated value is determined as the request hash key corresponding to the request data. The calculated request hash key can be used to determine the corresponding position of the request data in the hash ring space.

[0036] For example, the consistent hashing algorithm organizes the entire hash value space into a virtual ring, with different locations on the ring corresponding to different hash values. A consistent hash calculation is performed on the data source information to determine the request hash key corresponding to the requested data. Based on the value of the request hash key, the location of the requested data on the ring of the hash value space is determined.

[0037] S103 : Based on the request hash key and the pre-established hash mapping relationship between each cluster node and each hash key, determine the target cluster node corresponding to the request hash key in each cluster node, and distribute the request data to the target cluster node.

[0038] In a specific implementation, the request location of the request data on the ring formed by the hash value space can be determined based on the request hash key. Furthermore, the node location of each cluster node on the ring formed by the hash value space can be pre-established. Based on the node location of each cluster node, a hash mapping relationship between each cluster node and each hash key can be set.

[0039] Optionally, determining a target cluster node corresponding to the request hash key in each cluster node based on the request hash key and a pre-established hash mapping relationship between each cluster node and each hash key includes: determining the target cluster node corresponding to the request hash key in each cluster node based on a pre-established determination rule, the request location, and the node location. Exemplarily, the determination rule may be set to determine the cluster node corresponding to the node location closest to the request location as the target cluster node.

[0040] A data distribution method provided by an embodiment of the present invention receives request data to be distributed and obtains data source information of the request data; based on the data source information and a consistent hashing algorithm, determines a request hash key corresponding to the request data; based on the request hash key and a pre-established hash mapping relationship between each cluster node and each hash key, determines a target cluster node corresponding to the request hash key in each cluster node, and distributes the request data to the target cluster node. By distributing request data through data source information and a consistent hashing algorithm, the embodiment of the present invention solves the problem of uncontrollable data traffic, enables better utilization of cluster resources, and is conducive to improving data processing efficiency.

[0041] Example 2

[0042] Figure 2 A flowchart of another data distribution method provided by an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. Optionally, before determining the target cluster node corresponding to the requested hash key in each cluster node, it also includes: obtaining the first node information of each current cluster node; and establishing a hash mapping relationship between each cluster node and each hash key based on each first node information and a consistent hashing algorithm. Among them, the explanations of the terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.

[0043] like Figure 2 As shown, the method of this embodiment may specifically include:

[0044] S201: Receive request data to be allocated and obtain data source information of the request data.

[0045] S202: Determine the request hash key corresponding to the requested data based on the data source information and the consistent hashing algorithm.

[0046] S203: Obtain the first node information of each current cluster node.

[0047] Specifically, each cluster node may be detected, and based on the node status of each cluster node, currently available cluster nodes may be determined. First node information of each cluster node may be obtained, and the first node information may include the name and node identifier of the cluster node.

[0048] S204: Based on the information of each first node and the consistent hashing algorithm, a hash mapping relationship is established between each cluster node and each hash key.

[0049] Specifically, a consistent hash calculation can be performed on the first node information of each cluster node to determine a node hash key. Based on the node hash key of each cluster node, the node position corresponding to each cluster node on the ring formed by the hash value space is determined. Based on the node position of each cluster node, a hash mapping relationship between each cluster node and each hash key can be established.

[0050] It should be noted that the data flow allocated to each cluster node can be set by setting a hash mapping relationship. For example, request data corresponding to different areas on the ring formed by the hash value space can be allocated to different cluster nodes, and the data flow allocated to each cluster node can be controlled by setting the area range of the allocation area.

[0051] Optionally, based on the first node information and the consistent hashing algorithm, a hash mapping relationship is established between each cluster node and each hash key, including: determining whether traffic distribution information for specifying the amount of data to be distributed to each cluster node is received; if so, based on the traffic distribution information, constructing a virtual node corresponding to each cluster node, and determining the second node information of each virtual node; based on the second node information and the consistent hashing algorithm, establishing a hash mapping relationship between each virtual node and each hash key.

[0052] Specifically, when the traffic distribution information for specifying the amount of data to be allocated to each cluster node is not received, the requested data can be allocated according to a pre-set default distribution method. When the traffic distribution information for specifying the amount of data to be allocated to each cluster node is received, a virtual node corresponding to each cluster node can be constructed based on the traffic distribution information. Exemplarily, the traffic distribution information includes the amount of data allocated to each cluster node, the traffic distribution ratio between each cluster node, etc. Furthermore, a consistent hash calculation can be performed on the second node information of the virtual node to determine the virtual hash key of each virtual node, and a hash mapping relationship between each virtual node and each hash key can be established based on the virtual hash key.

[0053] Optionally, the traffic distribution information includes the traffic distribution ratio of each cluster node; based on the traffic distribution information, constructing the virtual nodes corresponding to each cluster node, including: determining the number of virtual nodes corresponding to each cluster node based on the traffic distribution ratio of each cluster node; and constructing the virtual nodes corresponding to each cluster node according to the number of nodes.

[0054] S205 : Based on the request hash key and the pre-established hash mapping relationship between each cluster node and each hash key, determine the target cluster node corresponding to the request hash key in each cluster node, and distribute the request data to the target cluster node.

[0055] Optionally, the method provided by an embodiment of the present invention further includes: detecting status information of each cluster node in the current cluster; determining whether the status information of each cluster node in the current cluster has changed; if so, generating change prompt information for prompting the change of the status information.

[0056] Specifically, cluster nodes may experience node failures and / or new nodes may be added during data processing. To promptly detect the status of each cluster node and avoid impacting the processing of requested data, the status information of each cluster node in the current cluster may be periodically monitored. Exemplarily, this status information includes operating status and fault status. When a change in cluster node status information is detected, it indicates that the current cluster node may be abnormal. To ensure accurate request processing, change notification information may be generated to indicate changes in status information, providing timely notification of changes.

[0057] Optionally, the change information includes additions and / or deletions. After generating the change notification information indicating the change in the status information, the process further includes updating the hash mapping relationship based on the change notification information. To ensure that similar request data can be assigned to the same cluster node even when cluster nodes change, the hash mapping relationship can be updated promptly based on the change notification information.

[0058] The data distribution method provided by the present invention can determine a hash mapping relationship based on traffic distribution information, thereby flexibly controlling the amount of data allocated to each cluster node. Furthermore, the status of each cluster node is monitored to ensure the stability and effectiveness of the request data processing process, enabling better utilization of cluster resources and improving data processing efficiency.

[0059] Example 3

[0060] The above describes the embodiments corresponding to the data distribution method in detail. To make the technical solution of the method more clear to those skilled in the art, a specific application scenario is given below. The data distribution method improved in the embodiments of the present invention can be applied to a gateway.

[0061] Figure 3 A schematic diagram of a data distribution method in a dual gateway scenario provided by an embodiment of the present invention; Figure 3 As shown, gateway 1 receives three request data items: request 1, request 2, and request 3, and determines the user IDs (identity documents) of requests 1, 2, and 3, respectively. Based on the user IDs of each request and the consistent hashing algorithm, the request hash key of each request is determined. Based on the request hash key and the pre-established hash mapping relationship, gateway 1 determines to assign the request to cluster 1 and assign requests 2 and 3 to cluster 2, thereby achieving link request consistency.

[0062] For example, the request hash key for each request data may be determined based on the protocol address of the network interconnection between the requesting terminals of requests 1, 2, and 3, the requesting terminal identifier, and the identifier of the organization corresponding to the request data. Those skilled in the art may determine this based on actual application circumstances, and this is not limited in this embodiment of the present invention.

[0063] Furthermore, after requests 1, 2, and 3 are processed by clusters 1 and 2, respectively, they must be redistributed through gateway 2. Gateway 2 can obtain the request hash key determined when each request was distributed through gateway 1 and redistribute it based on the request hash key and the consistent hash map. If the cluster service is Java or a third-party service, identifying information such as the user's request hash key may be lost during transmission. Gateway 2 can redetermine the request hash key based on information such as the user ID, the protocol address for interconnection between networks, the request terminal identifier, and the identifier of the organization corresponding to the request data.

[0064] Furthermore, when gateway 1 is uncontrollable or consistent hash mapping is not used, in order to achieve consistent distribution, gateway 2 extracts the user ID corresponding to each request and performs consistent hash mapping, thereby forwarding the same user request to a certain cluster, avoiding data inconsistency problems.

[0065] The embodiment of the present invention distributes request data through data source information and a consistent hash algorithm, thereby solving the problem of uncontrollable data traffic, making better use of cluster resources, and improving data processing efficiency.

[0066] Example 4

[0067] Figure 4 This is a structural diagram of a data distribution device provided in an embodiment of the present invention, which is used to execute the data distribution method provided in any of the above embodiments. This device and the data distribution method provided in the above embodiments are based on the same inventive concept. For details not fully described in the embodiments of the data distribution device, please refer to the embodiments of the above data distribution method. This device may specifically include:

[0068] A data source information acquisition module 10 is configured to receive request data to be allocated and acquire data source information of the request data;

[0069] A request hash key determination module 11 is configured to determine a request hash key corresponding to the request data based on the data source information and a consistent hashing algorithm;

[0070] The request data distribution module 12 is configured to determine a target cluster node corresponding to the request hash key in each cluster node based on the request hash key and a pre-established hash mapping relationship between each cluster node and each hash key, and distribute the request data to the target cluster node.

[0071] Based on any optional technical solution in the embodiments of the present invention, optionally, the device further includes:

[0072] A hash mapping relationship establishment module is used to obtain the first node information of each current cluster node before determining the target cluster node corresponding to the requested hash key in each cluster node; based on each first node information and the consistent hashing algorithm, a hash mapping relationship is established between each cluster node and each hash key.

[0073] Based on any optional technical solution in the embodiments of the present invention, optionally, the hash mapping relationship establishment module includes:

[0074] Determine the traffic distribution information unit, which is used to determine whether the traffic distribution information for specifying the amount of data to be distributed to each cluster node is received; if so, based on the traffic distribution information, construct the virtual nodes corresponding to each cluster node and determine the second node information of each virtual node; based on the second node information and the consistent hashing algorithm, establish a hash mapping relationship between each virtual node and each hash key.

[0075] Based on any optional technical solution in the embodiments of the present invention, optionally, the traffic distribution information includes a traffic distribution ratio of each cluster node; and determining the traffic distribution information unit includes:

[0076] The node quantity determination unit is used to determine the number of virtual nodes corresponding to each cluster node based on the traffic distribution ratio of each cluster node; and to construct the virtual nodes corresponding to each cluster node according to the number of nodes.

[0077] Based on any optional technical solution in the embodiment of the present invention, optionally, the data source information acquisition module 10 includes:

[0078] The address acquisition unit is used to acquire at least one of the Internet Protocol address of the requesting terminal, the requesting user identifier, the requesting terminal identifier and the requesting organization identifier corresponding to the request data.

[0079] Based on any optional technical solution in the embodiments of the present invention, optionally, the device further includes:

[0080] The status information detection module is used to detect the status information of each cluster node in the current cluster; determine whether the status information of each cluster node in the current cluster has changed; if so, generate change prompt information for prompting the change of the status information.

[0081] Based on any optional technical solution in the embodiment of the present invention, optionally, the change information includes addition information and / or deletion information; and the detection status information module includes:

[0082] The updating relationship unit is used to update the hash mapping relationship based on the change prompt information after generating the change prompt information for prompting the change of the state information.

[0083] A data distribution device provided by an embodiment of the present invention can implement the following method: receiving request data to be distributed and obtaining data source information of the request data; determining a request hash key corresponding to the request data based on the data source information and a consistent hashing algorithm; determining a target cluster node corresponding to the request hash key in each cluster node based on the request hash key and a pre-established hash mapping relationship between each cluster node and each hash key, and distributing the request data to the target cluster node. By distributing request data through data source information and a consistent hashing algorithm, the embodiment of the present invention solves the problem of uncontrollable data traffic, enables better utilization of cluster resources, and is conducive to improving data processing efficiency.

[0084] It is worth noting that in the embodiment of the above-mentioned data distribution device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0085] Example 5

[0086] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present invention. Figure 5 FIG2 is a block diagram of an exemplary electronic device 20 suitable for implementing an embodiment of the present invention. The electronic device 20 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0087] like Figure 5 As shown, electronic device 20 is a general-purpose computing device. Components of electronic device 20 may include, but are not limited to, one or more processors or processing units 201, system memory 202, and a bus 203 connecting various system components (including system memory 202 and processing unit 201).

[0088] Bus 203 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0089] The electronic device 20 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 20, including volatile and non-volatile media, removable and non-removable media.

[0090] The system memory 202 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 204 and / or cache memory 205. The electronic device 20 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 206 may be used to read and write to non-removable, non-volatile magnetic media. A magnetic disk drive may be provided for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), as well as an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media). In these cases, each drive may be connected to the bus 203 via one or more data media interfaces. The memory 202 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0091] A program / utility 208 having a set (at least one) of program modules 207 may be stored, for example, in memory 202. Such program modules 207 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 207 generally perform the functions and / or methods of the embodiments described herein.

[0092] The electronic device 20 may also communicate with one or more external devices 209 (e.g., a keyboard, a pointing device, a display 210, etc.), one or more devices that enable a user to interact with the electronic device 20, and / or any device that enables the electronic device 20 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may occur via an input / output (I / O) interface 211. Furthermore, the electronic device 20 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 212. As shown, the network adapter 212 communicates with other modules of the electronic device 20 via a bus 203. It should be understood that other hardware and / or software modules may be used in conjunction with the electronic device 20, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0093] The processing unit 201 executes various functional applications and data processing by running programs stored in the system memory 202 .

[0094] The electronic device provided by the present invention can implement the following method: receiving request data to be distributed and obtaining data source information of the request data; determining a request hash key corresponding to the request data based on the data source information and a consistent hashing algorithm; determining a target cluster node corresponding to the request hash key in each cluster node based on the request hash key and a pre-established hash mapping relationship between each cluster node and each hash key, and distributing the request data to the target cluster node. By distributing request data through data source information and a consistent hashing algorithm, the embodiment of the present invention solves the problem of uncontrollable data flow, enables better utilization of cluster resources, and is conducive to improving data processing efficiency.

[0095] Example 6

[0096] An embodiment of the present invention provides a storage medium containing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, the computer-executable instructions are used to perform a data distribution method. The method includes:

[0097] Receive request data to be distributed and obtain data source information of the request data; determine the request hash key corresponding to the request data based on the data source information and a consistent hashing algorithm; determine the target cluster node corresponding to the request hash key in each cluster node based on the request hash key and a pre-established hash mapping relationship between each cluster node and each hash key, and distribute the request data to the target cluster node. The embodiments of the present invention distribute request data through data source information and a consistent hashing algorithm, solving the problem of uncontrollable data traffic, enabling better utilization of cluster resources, and improving data processing efficiency.

[0098] Of course, the computer executable instructions of the storage medium provided by the embodiment of the present invention are not limited to the operations of the method described above, but can also execute related operations in the data distribution method provided by any embodiment of the present invention.

[0099] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by an instruction execution system, device or device or used in combination with it.

[0100] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0101] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0102] The computer program code for performing the operations of the embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data distribution method, characterized in that: include: Receiving request data to be allocated, and obtaining data source information of the request data; Determine a request hash key corresponding to the requested data based on the data source information and a consistent hashing algorithm; Based on the request hash key and the pre-established hash mapping relationship between each cluster node and each hash key, determining a target cluster node corresponding to the request hash key in each cluster node, and distributing the request data to the target cluster node; Determining the allocated data traffic of each cluster node based on the hash mapping relationship; Before determining the target cluster node corresponding to the request hash key in each cluster node, the method further includes: Obtaining first node information of each current cluster node; Based on the first node information and the consistent hashing algorithm, establishing the hash mapping relationship between each cluster node and each hash key; The establishing the hash mapping relationship between each cluster node and each hash key based on each first node information and the consistent hashing algorithm includes: determining whether traffic allocation information for specifying the amount of data to be allocated to each of the cluster nodes is received; the traffic allocation information includes a traffic allocation ratio of each of the cluster nodes; If yes, constructing a virtual node corresponding to each of the cluster nodes based on the traffic distribution information, and determining second node information of each of the virtual nodes; Based on the information of each second node and the consistent hashing algorithm, establishing the hash mapping relationship between each virtual node and each hash key; The constructing of a virtual node corresponding to each of the cluster nodes based on the traffic distribution information includes: Determining the number of virtual nodes corresponding to each cluster node based on the traffic distribution ratio of each cluster node; The virtual nodes corresponding to the cluster nodes are constructed according to the number of nodes.

2. The method according to claim 1, characterized in that The obtaining of data source information of the requested data includes: At least one of an Internet Protocol address of a requesting terminal, a requesting user identifier, a requesting terminal identifier, and a requesting organization identifier corresponding to the request data is obtained.

3. The method according to claim 1, characterized in that Also includes: Detecting status information of each cluster node in the current cluster; Determining whether the status information of each cluster node in the current cluster changes; If yes, change prompt information is generated to prompt the change of the status information.

4. The method according to claim 3, characterized in that The change information includes adding information and / or deleting information; After the change prompt information for prompting the change of the status information is generated, the method further includes: Based on the change prompt information, the hash mapping relationship is updated.

5. A data distribution device, characterized in that: include: A data source information acquisition module is used to receive request data to be allocated and acquire data source information of the request data; A request hash key determination module is used to determine a request hash key corresponding to the request data based on the data source information and a consistent hash algorithm; a request data distribution module configured to determine, from among the cluster nodes, a target cluster node corresponding to the request hash key based on the request hash key and a pre-established hash mapping relationship between each cluster node and each hash key, and distribute the request data to the target cluster node; The allocation request data module is further configured to determine the allocated data flow of each cluster node based on the hash mapping relationship; The apparatus further includes a hash mapping relationship establishing module, configured to obtain first node information of each of the current cluster nodes before determining the target cluster node corresponding to the request hash key in each cluster node; Based on the first node information and the consistent hashing algorithm, establishing the hash mapping relationship between each cluster node and each hash key; The hash mapping relationship establishment module is further used to: determine whether traffic allocation information for specifying the amount of data to be allocated to each cluster node is received; The traffic distribution information includes the traffic distribution ratio of each cluster node; If yes, constructing a virtual node corresponding to each cluster node based on the traffic distribution information, and determining the second node information of each virtual node; establishing the hash mapping relationship between each virtual node and each hash key based on each second node information and the consistent hashing algorithm; The hash mapping relationship establishment module is further used to: determine the number of virtual nodes corresponding to each cluster node based on the traffic distribution ratio of each cluster node; and construct the virtual nodes corresponding to each cluster node according to the number of nodes.

6. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data distribution method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data distribution method according to any one of claims 1 to 4 is implemented.

Images