Data traversal method and device for computing node and computing node
By traversing multiple data shards of the storage cluster in parallel in the computing cluster and reading data directly from the storage cluster, the problem of slow traversal speed of traditional computing clusters is solved, and efficient data traversal and high availability are achieved.
Patent Information
- Application Number
- CN201810340710.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2018-04-16
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2038-04-16
AI Technical Summary
During the data traversal process, traditional computing clusters only support single-thread sequential reading, and cannot take advantage of the concurrency of multiple nodes, and data needs to be transmitted twice, resulting in slow traversal speed and long time.
By obtaining multiple data sharding information and performing read-traversal operations in parallel with other computing nodes, we can directly traverse the reading data from the storage cluster to avoid ETL server redirection, and use a distributed coordination service architecture for information sharing and sharding uniformity. Arrange to ensure that different nodes traverse different shards.
It improves the speed of data traversal, reduces time-consuming, avoids repeated data traversal and single-point overload, and improves the concurrency and high availability of the computing cluster.
Smart Images

Figure CN110399394B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and more specifically, to a data traversal method and device for a computing node, and a computing node. Background Art
[0002] With the rapid development of computer technology, cluster storage has emerged to meet the needs of data-intensive and high-concurrency access industries such as the Internet, streaming media, high-performance computing, and energy.
[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the prior art: the traditional computing cluster traverses the data stored in the storage cluster by the following method: first, the data storage system traversal method is called to read the data stored in the storage cluster to the memory or disk of the ETL server, and then the data is distributed to each computing node. However, in this method, for storage clusters such as Redis and MySQL, since only single-threaded sequential reading is supported, the concurrency advantage of multiple nodes in the computing cluster cannot be brought into play, so data traversal takes a long time; furthermore, the data needs to be transmitted twice on the network, and the traversal speed is limited by the network bandwidth and IO processing capacity of the ETL server, so the data traversal speed is slow. Summary of the invention
[0004] In view of this, the present disclosure provides a data traversal method and device for a computing node, and a computing node.
[0005] A first aspect of the present disclosure provides a data traversal method for a computing node, wherein the data is stored in a plurality of data shards, the method comprising: obtaining a plurality of shard information, each shard information corresponding to a data shard; and performing a read-traversal operation based on the plurality of shard information in parallel with at least one other computing node to traverse and read the data stored in the data shards corresponding to the plurality of shard information, wherein different computing nodes traverse and read the data stored in different data shards.
[0006] According to an embodiment of the present disclosure, the above method also includes writing multiple shard information into a preset storage space; the above read-traversal operation includes: reading one shard information from the multiple shard information from the preset storage space; and based on the read shard information, traversing and reading the data stored in the data shard corresponding to the shard information.
[0007] According to an embodiment of the present disclosure, the above-mentioned multiple data shards are stored on multiple storage nodes of a storage cluster, and each storage node stores one or more data shards. The above-mentioned method also includes: obtaining storage node information of multiple data shards; then writing multiple shard information into a preset storage space includes: based on a preset allocation rule and the storage node information, evenly arranging the multiple shard information so that the shard information of different data shards with the same storage node information are arranged at intervals; and sequentially writing the evenly arranged multiple shard information into the preset storage space; the above-mentioned reading one shard information from the preset storage space includes: sequentially reading one shard information from the multiple evenly arranged shard information.
[0008] According to an embodiment of the present disclosure, the above-mentioned read-traverse operation also includes: when one shard information among multiple shard information is read from a preset storage space, the read shard information is set from the first state to the second state; and when the data stored in the data shard corresponding to the shard information is traversed and read, the shard information is set from the second state to the third state; wherein the first state of the shard information indicates that the shard information can be read by the computing node or at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information cannot be read again.
[0009] According to an embodiment of the present disclosure, the above method also includes: determining whether other computing nodes that have read one of the multiple shard information from a preset storage space are faulty nodes; and when other computing nodes are faulty nodes, setting the shard information read by the faulty node from the second state to the first state for reading by the computing node or at least one other computing node except the faulty node.
[0010] According to an embodiment of the present disclosure, the above method also includes: determining whether multiple shard information are all in the third state; in the case that there is shard information that is not in the third state, returning to re-determine after a preset time interval; and in the case that multiple shard information are all in the third state, completing the traversal of the data.
[0011] According to an embodiment of the present disclosure, the above-mentioned computing node is elected by the computing node and at least one other computing node using a preset election rule; and / or, when the number of multiple shard information is greater than the total number of nodes of the computing node and at least one other computing node, the computing node and / or at least one other computing node cyclically performs the above-mentioned read-traversal operation so that the data stored in the data shards corresponding to the multiple shard information are all traversed and read.
[0012] A second aspect of the present disclosure provides a data traversal method for a computing node, wherein the data is stored in multiple data shards, the method comprising: performing a read-traversal operation based on multiple shard information in parallel with at least one other computing node to traverse and read the data stored in the data shards corresponding to the multiple shard information, wherein the multiple shard information is pre-acquired by a computing node in at least one other computing node, each shard information corresponds to a data shard, and different computing nodes traverse and read the data stored in different data shards.
[0013] According to an embodiment of the present disclosure, the above-mentioned multiple shard information is pre-acquired by a computing node among at least one other computing node and then written into a preset storage space; the above-mentioned read-traversal operation includes: reading one shard information from the preset storage space; and based on the read shard information, traversing the data stored in the data shard corresponding to the read shard information.
[0014] According to an embodiment of the present disclosure, the above-mentioned read-traverse operation also includes: when one shard information among multiple shard information is read from a preset storage space, the read shard information is set from the first state to the second state; and when the data stored in the data shard corresponding to the shard information is traversed and read, the shard information is set from the second state to the third state; wherein the first state of the shard information indicates that the shard information can be read by the computing node or at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information can no longer be read.
[0015] According to an embodiment of the present disclosure, the computing node that pre-acquires multiple shard information is elected by the computing node and at least one other computing node using a preset election rule; and / or, when the number of multiple shard information is greater than the total number of nodes of the computing node and at least one other computing node, the computing node and / or at least one other computing node cyclically executes the above-mentioned read-traversal operation so that the data stored in the data shards corresponding to the multiple shard information are all traversed and read.
[0016] A third aspect of the present disclosure provides a data traversal device for a computing node, wherein the data is stored in multiple data shards, and the device includes: an acquisition module, which acquires multiple shard information, each shard information corresponds to a data shard; and a first processing module, which performs a read-traversal operation based on the multiple shard information, wherein the data traversal device of the computing node and the data traversal device of at least one other computing node perform the read-traversal operation based on the multiple shard information in parallel to traverse and read the data stored in the data shards corresponding to the multiple shard information, and the data traversal devices of different computing nodes traverse and read the data stored in different data shards.
[0017] According to an embodiment of the present disclosure, the above-mentioned device also includes a write operation module, which is used to write multiple shard information into a preset storage space; the above-mentioned first processing module includes a first shard information reading unit and a first data traversal reading unit, the first shard information reading unit is used to read one shard information from multiple shard information from a preset storage space; the first data traversal reading unit is used to traverse and read the data stored in the data shard corresponding to the shard information based on the read shard information.
[0018] According to an embodiment of the present disclosure, the above-mentioned multiple data shards are stored on multiple storage nodes of a storage cluster, and each storage node stores one or more of the data shards; the above-mentioned acquisition module is also used to obtain the storage node information of the multiple data shards; the above-mentioned write operation module includes a shard information arrangement unit and a write operation unit, and the shard information arrangement unit is used to evenly arrange the multiple shard information based on a preset allocation rule and the storage node information, so that the shard information of different data shards with the same storage node information are arranged at intervals; the write operation unit is used to sequentially write the evenly arranged multiple shard information into a preset storage space; the above-mentioned first shard information reading unit sequentially reads a shard information from the evenly arranged multiple shard information.
[0019] According to an embodiment of the present disclosure, the above-mentioned first processing module also includes a first change unit, which is used to set the read shard information from the first state to the second state when the first shard information reading unit reads one shard information among multiple shard information from a preset storage space; and to set the shard information from the second state to the third state when the first data traversal reading unit traverses and reads the data stored in the data shard corresponding to the shard information; wherein the first state of the shard information indicates that the shard information can be read by the data traversal device of the computing node or the data traversal device of at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information cannot be read again.
[0020] According to an embodiment of the present disclosure, the above-mentioned first processing module also includes a fault judgment unit, which is used to judge whether the data traversal device of other computing nodes that has read one shard information among multiple shard information from a preset storage space is a faulty device; the above-mentioned first change unit is also used to, when the data traversal device of other computing nodes is a faulty device, set the shard information read by the data traversal device of the faulty node from the second state to the first state, so as to be read by the data traversal device of the computing node or the data traversal device of at least one other computing node except the faulty node.
[0021] According to an embodiment of the present disclosure, the above-mentioned first processing module also includes a state judgment unit, which is used to judge whether multiple shard information are all in the third state; and in the case that there is shard information that is not in the third state, return to re-judge after a preset time interval; and in the case that multiple shard information are all in the third state, complete the traversal of the data.
[0022] According to an embodiment of the present disclosure, the data traversal device of the above-mentioned computing node is elected by the data traversal device of the computing node and the data traversal device of at least one other computing node using a preset election rule; and / or, when the number of multiple shard information is greater than the total number of node devices of the data traversal device of the computing node and the data traversal device of at least one other computing node, the data traversal device of the computing node and / or the data traversal device of at least one other computing node cyclically performs the above-mentioned read-traversal operation so that the data stored in the data shards corresponding to the multiple shard information are all traversed and read.
[0023] The fourth aspect of the present disclosure provides a data traversal device for a computing node, wherein the data is stored in a plurality of data shards, and the device includes a second processing module for performing a read-traverse operation based on a plurality of shard information, wherein the plurality of shard information is pre-acquired by a data traversal device of a computing node in at least one other computing node, and each shard information corresponds to a data shard. The data traversal device of the computing node and the data traversal device of at least one other computing node perform the above-mentioned read-traverse operation based on the plurality of shard information in parallel, so as to traverse and read the data stored in the data shards corresponding to the plurality of shard information, and the data traversal devices of different computing nodes traverse and read the data stored in different data shards.
[0024] According to an embodiment of the present disclosure, the above-mentioned multiple shard information is pre-acquired by a data traversal device of a computing node in at least one other computing node and then written into a preset storage space; the above-mentioned second processing module includes a second shard information reading unit and a second data traversal reading unit, the second shard information reading unit is used to read one shard information among the multiple shard information from the preset storage space; the second data traversal reading unit is used to traverse and read the data stored in the data shard corresponding to the shard information based on the read shard information.
[0025] According to an embodiment of the present disclosure, the above-mentioned second processing module also includes a second changing unit, which is used to set the read shard information from the first state to the second state when the second shard information reading unit reads one shard information among multiple shard information from a preset storage space; and to set the shard information from the second state to the third state when the second data traversal reading unit traverses and reads the data stored in the data shard corresponding to the shard information; wherein the first state of the shard information indicates that the shard information can be read by the data traversal device of the computing node or the data traversal device of at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information cannot be read again.
[0026] According to an embodiment of the present disclosure, the data traversal device of the computing node that pre-acquires multiple shard information is elected by the data traversal device of the computing node and the data traversal device of at least one other computing node using a preset election rule; and / or, when the number of multiple shard information is greater than the total number of the data traversal device of the computing node and the data traversal device of at least one other computing node, the data traversal device of the computing node and / or the data traversal device of at least one other computing node cyclically performs the above-mentioned read-traversal operation so that the data stored in the data shards corresponding to the multiple shard information are all traversed and read.
[0027] The fifth aspect of the present disclosure provides a computing node, comprising: one or more processors; a storage device for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors respectively, the one or more processors execute the data traversal method of the computing node provided by the first aspect of the present disclosure and the data traversal method of the computing node provided by the second aspect of the present disclosure.
[0028] The sixth aspect of the present disclosure provides a computer-readable medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the data traversal method for computing nodes provided in the first aspect of the present disclosure and the data traversal method for computing nodes provided in the second aspect of the present disclosure.
[0029] According to the embodiments of the present disclosure, the defects that data needs to be transmitted twice and can only be read sequentially by a single thread during data traversal can at least be avoided, thereby improving the data traversal speed and reducing the time consumption of data traversal. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0031] Figure 1 The data traversal method and device of a computing node and the system architecture of the computing node according to an embodiment of the present disclosure are schematically shown;
[0032] Figure 2A A method flow chart of a data traversal method for a computing node according to an embodiment of the present disclosure is schematically shown;
[0033] Figure 2B A method flow chart of a data traversal method for a computing node according to another embodiment of the present disclosure is schematically shown;
[0034] Figure 2C A flowchart of a read-traverse operation according to an embodiment of the present disclosure is schematically shown;
[0035] Figure 2D Schematically shows the reference Figure 2B An operational flow chart of operation S230 is described;
[0036] Figure 3 A flowchart of a read-traverse operation according to another embodiment of the present disclosure is schematically shown;
[0037] Figure 4 A method flow chart of a data traversal method for a computing node according to another embodiment of the present disclosure is schematically shown;
[0038] Figure 5 A method flow chart of a data traversal method for a computing node according to another embodiment of the present disclosure is schematically shown;
[0039] Figure 6A to Figure 6B A flowchart of a read-traverse operation according to another embodiment of the present disclosure is schematically shown;
[0040] Figure 7A to Figure 7B A structural block diagram of a data traversal device for a computing node according to an embodiment of the present disclosure is schematically shown;
[0041] Figures 8A to 8D The structure block diagram of the first processing module according to the embodiment of the present disclosure is schematically shown;
[0042] Fig. 9 The structure block diagram of the write operation module according to the embodiment of the present disclosure is schematically shown;
[0043] Figure 10A to Figure 10B The structure block diagram of the second processing module according to the embodiment of the present disclosure is schematically shown;
[0044] Fig.11 A block diagram suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0045] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0046] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0047] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.
[0048] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). Those skilled in the art should also understand that any transitional conjunctions and / or phrases that substantially indicate two or more optional items, whether in the specification, claims, or drawings, should be understood to give the possibility of including one of these items, either of these items, or both of these items. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B", or "A and B".
[0049] An embodiment of the present disclosure provides a data traversal method for a computing node, wherein the data is stored in multiple data shards, the method comprising: obtaining multiple shard information, each shard information corresponding to a data shard; and performing a read-traversal operation based on the multiple shard information in parallel with at least one other computing node to traverse and read the data stored in the data shards corresponding to the multiple shard information, wherein different computing nodes traverse and read the data stored in different data shards.
[0050] An embodiment of the present disclosure also provides another data traversal method of a computing node, wherein the data is stored in multiple data shards, and the method includes: performing a read-traversal operation based on multiple shard information in parallel with at least one other computing node to traverse and read the data stored in the data shards corresponding to the multiple shard information, wherein the multiple shard information is pre-acquired by a computing node in at least one other computing node, each shard information corresponds to a data shard, and different computing nodes traverse and read the data stored in different data shards.
[0051] Figure 1 The data traversal method and device of a computing node and the system architecture 100 of the computing node according to an embodiment of the present disclosure are schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.
[0052] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a storage cluster and a computing cluster, the storage cluster includes storage nodes 101, 102, 103, 104, and the computing cluster includes computing nodes 105, 106, 107. The storage cluster and the computing cluster are connected via a network, and the network may include various connection types, such as wired, wireless communication links or optical fiber cables, etc., and the computing nodes are interconnected to form a distributed or parallel computing cluster.
[0053] The computing nodes 105, 106, and 107 can interact with the storage nodes 101, 102, 103, and 104 through the network to perform operations such as writing, reading, or deleting data on the storage nodes.
[0054] The data stored in the storage cluster can be stored using a data sharding storage strategy, which means dividing the data into multiple data shards according to a specific sharding algorithm, and storing these data shards on different storage nodes. This makes the storage space not limited by the storage capacity of a single node, and the storage space can be dynamically adjusted by increasing or decreasing the number of storage nodes, and supports concurrent data reading and writing.
[0055] Traditionally, data is read by setting up an ETL server between the storage cluster and the computing cluster. Specifically, the data storage system traversal method is first called to read the data in the storage cluster to the ETL server memory or disk, and then the data is distributed to each computing node of the computing cluster to complete the data reading. In this process, since the default traversal method provided by the storage cluster only supports single-threaded sequential reading, the multi-node concurrency advantage of the cluster cannot be brought into play; furthermore, the data needs to be read to the ETL server first, and then distributed to each node of the computing cluster, so the data needs to be transmitted twice on the network, which reduces the data reading rate; and the reading rate of the entire data reading process is limited by the network bandwidth and IO processing capacity of the ETL server, and the processing speed of ETL is slow, and if the ETL server fails during the data traversal reading process, it will cause data traversal failure.
[0056] In view of the above problems, if Figure 1As shown, in the embodiment of the present disclosure, in the computing cluster, the computing node 105 that plays a coordination and management role is elected through the preset election rules to obtain the shard information of each data shard in the storage cluster, and realize information sharing with other computing nodes through the distributed coordination service architecture, then each computing node 105, 106, 107 in the computing cluster can directly traverse and read the data stored in the data shard corresponding to the read shard information in the storage cluster by reading the shared shard information. And multiple computing nodes can read different shard information, and traverse and read the data stored in the corresponding data shard in parallel, so the data traversal speed can be effectively improved, and the traversal speed is theoretically "MIN (number of data shards, number of computing nodes)" times of the traditional traversal method. For example, in a storage cluster with 10 data shards and a computing cluster environment with 10 nodes, the time consumed for data traversal is only one tenth of the traditional method.
[0057] It should be noted that the data traversal method of computing nodes provided in the embodiment of the present disclosure can generally be executed by computing nodes 105, 106, 107. Accordingly, the data traversal device of computing nodes provided in the embodiment of the present disclosure can generally be set in computing nodes 105, 106, 107.
[0058] It should be understood that Figure 1 The number of storage nodes and computing nodes and the types of various connection relationships in the embodiment are merely illustrative. According to implementation requirements, there may be any number of storage nodes and computing nodes and any type of connection relationship.
[0059] Figure 2A A method flow chart of a data traversal method for a computing node according to an embodiment of the present disclosure is schematically shown.
[0060] like Figure 2A As shown, one aspect of the present disclosure provides a data traversal method for a computing node, the method comprising operations S210 to S220.
[0061] In operation S210, a plurality of shard information is obtained, each shard information corresponds to a data shard.
[0062] The above data is stored in multiple data shards, and each data shard corresponds to a shard information, which uniquely corresponds to the data shard.
[0063] According to an embodiment of the present disclosure, according to an embodiment of the present disclosure, the computing node belongs to a computing node in a computing cluster, for example, Figure 1 The computing cluster also includes at least one other computing node besides the computing node, such as the computing node 105 described in the reference Figure 1The computing nodes 106 and 107 described in the description. The computing nodes that obtain the information of multiple shards can be, for example, elected by all computing nodes in the computing cluster through a preset election rule. For the convenience of description, the elected computing nodes are referred to as management nodes below. Among them, the embodiments of the present disclosure do not limit the preset election rules. The preset election rules can, for example, adopt the FastLeaderElection algorithm.
[0064] According to an embodiment of the present disclosure, when a computing cluster is initialized for the first time or a previously elected management node fails, if any computing node in the computing cluster finds that the management node does not exist or is unavailable when trying to communicate with the management node, an election is triggered to elect a new management node.
[0065] According to the embodiments of the present disclosure, any computing node in the computing cluster can provide an interface for the client to submit a traversal task, and the traversal task carries a traversal condition. When receiving the traversal task submitted by the client, at least one other computing node initiates a traversal request to the management node according to the traversal task through communication with the management node. When obtaining shard information, the management node obtains shard information from multiple data shards stored in the storage cluster based on the traversal request (including the traversal condition).
[0066] According to an embodiment of the present disclosure, when the interfaces provided by multiple computing nodes all receive traversal tasks submitted by the client, the management node can create multiple task nodes in the distributed coordination service architecture after receiving the multiple traversal tasks, and each traversal task corresponds to a task node. Then, the management node reads shard information from multiple data shards according to the traversal conditions carried by the multiple traversal tasks, and writes the returned shard information into the child nodes of each task node in sequence according to the matching of the traversal tasks. Then, the multiple task nodes can be arranged into a task node directory, and each task node includes multiple child nodes, and each child node has a shard information written. The embodiment of the present disclosure does not limit the distributed coordination service architecture. For example, the distributed coordination service architecture can be a Zookeeper (abbreviated as zk) distributed coordination service, etc., to realize information sharing between multiple computing nodes in a computing cluster.
[0067] In operation S220, a read-traverse operation is performed based on the plurality of shard information in parallel with at least one other computing node.
[0068] According to an embodiment of the present disclosure, the read-traverse operation may specifically be: reading one shard information among multiple shard information, and then traversing the data stored in the data shard corresponding to the read shard information.
[0069] The at least one other computing node is: other computing nodes in the computing cluster except for the computing node that obtains the shard information. Since the computing node and at least one other computing node can perform read-traverse operations in parallel based on the acquired shard information to traverse and read the data stored in the data shards corresponding to the multiple shard information, the scheme of the embodiment of the present disclosure can realize concurrent traversal of data and greatly improve the speed of data traversal. Furthermore, since the computing node directly traverses and reads data from the storage cluster based on the shard information, the secondary transfer of data through the ETL server is avoided, so the speed of data traversal can be further improved.
[0070] Figure 2B A method flow chart of a data traversal method for a computing node according to another embodiment of the present disclosure is schematically shown.
[0071] like Figure 2B As shown, according to the embodiment of the present disclosure, Figure 2A The described method for traversing data of a computing node may further include S230 between operations S210 and S220.
[0072] In operation S230, the plurality of slice information is written into a preset storage space.
[0073] According to an embodiment of the present disclosure, a computing node that obtains multiple shard information may write the multiple shard information into a preset storage space, which may be, for example, a shared space for access by each computing node in the computing cluster.
[0074] Specifically, the preset storage space may be, for example, a storage space provided by a distributed coordination service architecture, so as to realize information sharing between various computing nodes in a computing cluster, and each computing node in the computing cluster may read the shard information from the storage space to perform traversal reading of data in the corresponding data shard based on the read shard information. For example, if the management node writes the shard information into the child node of the task node in Zookeeper, then when traversing a single traversal task, each computing node in the computing cluster will read the shard information from the child node of the corresponding task node.
[0075] Figure 2C The flowchart of the read-traverse operation according to the embodiment of the present disclosure is schematically shown.
[0076] like Figure 2C As shown, in the case where the computing node that obtains the shard information writes the shard information into a preset storage space, the read-traverse operation correspondingly includes operations S221 to S222.
[0077] In operation S221, one of the plurality of shard information is read from a preset storage space; in operation S222, based on the read shard information, data stored in the data shard corresponding to the read shard information is traversed.
[0078] According to an embodiment of the present disclosure, when the number of multiple shard information is less than the number of available computing nodes in the computing cluster (the total number of management nodes and at least one other computing node), multiple computing nodes in the computing cluster (equal to the number of multiple shard information) can complete the traversal reading of the data stored in the data shards corresponding to the multiple shard information through one of the above-mentioned read-traverse operations. Of course, the embodiment of the present disclosure does not limit the specific traversal reading process. For example, some computing nodes (less than the number of shard information) among the multiple computing nodes can also be cycled through multiple read-traverse operations so that the multiple shard information are all read, and the data stored in the corresponding data shards are all traversed and read.
[0079] According to an embodiment of the present disclosure, when the number of multiple shard information is greater than the number of available computing nodes in the computing cluster (the total number of nodes of the management node and at least one other computing node), the management node and at least one other computing node cyclically perform read-traverse operations. Specifically, after one of the computing nodes reads the shard information written by the child node of the task node in Zookeeper, the data stored in the data shard corresponding to the shard information is traversed and read; after completing the traversal reading, the computing node returns to read the shard information written by the other child nodes of the task node in Zookeeper (other child nodes that have not been read), and traverses and reads the data stored in the corresponding data shards, and other computing nodes also cyclically perform read-traverse operations in this manner until the shard information written by all child nodes of the task node in Zookeeper has been read, so as to ensure that the data stored in the data shards corresponding to the multiple shard information are traversed and read.
[0080] According to an embodiment of the present disclosure, if there are multiple task nodes in Zookeeper, the available computing nodes in the computing cluster traverse the traversal tasks of the multiple task nodes in turn. After completing the traversal and reading of the data stored in the data shard corresponding to the shard information written by the child node of a task node, the shard information written by the child node of the next task node is read to traverse the traversal task of the next task node.
[0081] Figure 2D Schematically shows the reference Figure 2B An operational flow chart of operation S230 is described.
[0082] According to an embodiment of the present disclosure, multiple data shards may be stored on multiple storage nodes of a storage cluster, and each storage node stores one or more data shards.
[0083] Then, refer to Figure 2A Operation S210 in the example may also be operation S210', obtaining storage node information of multiple shard information and multiple data shards, wherein the storage node information can indicate in which storage node a data shard corresponding to a certain shard information is stored.
[0084] like Figure 2D As shown, operation S230 may include operation S231 and operation S232.
[0085] In operation S231, based on the preset allocation rules and storage node information, multiple shard information is evenly arranged so that the shard information of different data shards with the same storage node information is arranged at intervals; in operation S232, the evenly arranged multiple shard information is sequentially written into the preset storage space.
[0086] Specifically, the principle of the preset allocation rule is: the shard information of the data shards in the same storage node is set apart in the arrangement queue, preferably the distance between them is as far as possible. The preset allocation rule can be, for example, the following simple uniform allocation strategy: the shard information is organized into a two-dimensional array according to the storage node where the corresponding data shard is located, and the shard information of the data shards with the same storage node information is located in the same row, and the shard information of the data shards with different storage node information is located in different rows. For example, multiple shard information can be arranged into a shard information list. When arranging multiple shard information evenly, starting from the first row and first column of the two-dimensional array, one shard information can be taken from each row of the first column of the two-dimensional array and placed at the end of the list; after all the shard information in the first column of the two-dimensional array is arranged, starting from the shard information in the first row and second column, one shard information can be taken from each row of the second column and placed at the end of the list, and so on, until all the shard information in the two-dimensional data is arranged in the shard information list.
[0087] In the case where there are multiple traversal tasks, multiple shard information of each traversal task is evenly arranged, so that the shard information of different data shards with the same storage node information in each traversal task is arranged at intervals.
[0088] Then, in operation S232, the multiple shard information arranged in the shard information list is sequentially written into the preset storage space. For the Zookeeper distributed coordination service architecture, the multiple shard information is sequentially written into the subnodes under the task node according to the order in the shard information list. In the case where there are multiple task nodes, the shard information in the shard information list is written into the subnodes of the corresponding task node according to the corresponding relationship, and the shard information in different shard information lists is written into the subnodes of different task nodes.
[0089] In the read-traverse operation, one of the multiple shard information is read from the preset storage space, which can be sequentially reading one shard information from the multiple evenly arranged shard information, specifically, sequentially reading the shard information from the task node in the order of arrangement of the child nodes.
[0090] Through the above-mentioned settings, the embodiment of the present disclosure can make the shard information corresponding to multiple data shards stored in the same storage node be located as farthest as possible in the task node. Therefore, when reading the shard information in the preset storage space, it can be avoided that the different shard information read by multiple computing nodes is the shard information of different data shards stored in the same storage node. Therefore, it can be avoided that when traversing and reading the data stored in the data shards corresponding to the shard information, multiple computing nodes access the same storage node, thereby avoiding the defect of low traversal and reading speed caused by excessive load of a single storage node, thereby improving the efficiency of data traversal as much as possible.
[0091] In order to ensure the stability of data storage and avoid partial data loss caused by a failure of a computing node in the storage cluster, data can be sharded and saved in multiple copies on different storage nodes. When obtaining shard information, the same shard information will inevitably exist.
[0092] Therefore, when evenly arranging the obtained multiple shard information, duplicate checking can be performed to ensure that only one of the multiple identical shard information with different storage node information is saved after even arrangement. Multiple shard information can also be arranged into a shard information set. Due to the non-repeatable nature of the set, only the first arranged shard information among the multiple identical shard information can be included in the shard information set.
[0093] Through the above method, the stability of data stored in the storage cluster can be guaranteed, and repeated traversal and reading of the same data stored in different storage nodes can be avoided in the subsequent traversal and reading process, thereby avoiding unnecessary additional overhead and improving the efficiency of data traversal of the computing cluster.
[0094] Figure 3 The flowchart of a read-traverse operation according to another embodiment of the present disclosure is schematically shown.
[0095] like Figure 3 As shown, the read-traverse operation includes operations S223 and S224 in addition to operations S221 and S222, wherein operation S223 is performed between operations S221 and S222, and operation S224 is performed after operation S223 is performed.
[0096] In operation S223, the read fragment information is set from the first state to the second state; in operation S224, when the traversal and reading are completed, the fragment information is set from the second state to the third state.
[0097] According to an embodiment of the present disclosure, after the computing node obtains multiple shard information, when the multiple shard information is written into a preset storage space, the multiple shard information is set to an initial state, i.e., a first state, to indicate that the multiple shard information can be read by any computing node (computing node or at least one other computing node) in the computing cluster; the second state of the shard information indicates that the shard information has been read by any computing node in the computing cluster and can no longer be read by a computing node in the computing cluster; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information can no longer be read by a computing node in the computing cluster.
[0098] By setting the shard information status as described above, it is possible to avoid the data stored in the data shard corresponding to any shard information among multiple shard information being repeatedly traversed and read by the computing cluster, and it is possible to avoid the situation where some shard information is not read, thereby minimizing the time spent on data traversal while ensuring the integrity of the read data.
[0099] Figure 4 A method flow chart of a data traversal method for a computing node according to another embodiment of the present disclosure is schematically shown.
[0100] like Figure 4 As shown, the method includes operations S410 to S440.
[0101] In operation S410, the status of multiple shard information stored in a preset storage space is obtained; in operation S420, it is determined whether the multiple shard information are all in the third state; in the case that there is shard information that is not in the third state, operation S430 is performed, and after a preset time interval, operation S410 is returned to be performed to re-judge the status of the multiple shard information; in the case that the multiple shard information are all in the third state, operation S440 is performed to complete the traversal of the data.
[0102] According to an embodiment of the present disclosure, the above-mentioned operation S410 and operation S420 can also be combined into one operation step to directly determine whether multiple shard information are all in the third state, without having to pre-acquire the states of multiple shard information. Instead, the multiple shard information stored in the preset storage space are judged one by one in sequence. If it is determined that the shard information is not in the third state, the judgment is interrupted. After a preset time interval, starting from the shard information that is found not to be in the third state, it is re-determined whether there is shard information that is not in the third state, until no shard information that is not in the third state is found, indicating that the multiple shard information are all in the third state, and the judgment is terminated to complete the data traversal.
[0103] Through the above operation, it can be ensured that the data stored in the data shards corresponding to the multiple shard information are all traversed and read, avoiding the omission of part of the data and ensuring the integrity of the data reading.
[0104] Figure 5 A method flow chart of a data traversal method for a computing node according to another embodiment of the present disclosure is schematically shown.
[0105] like Figure 5 As shown, the method includes operations S510 and S520 in addition to operations S210 to S220.
[0106] In operation S510, determine whether other computing nodes that have read one of the multiple shard information from the preset storage space are faulty nodes; if the other faulty nodes are faulty nodes, execute operation S520 to set the shard information read by the faulty node from the second state to the first state.
[0107] According to an embodiment of the present disclosure, when a computing node reads shard information from a preset storage space, for example, its identification information can be written to the storage location of the read shard information, and when the computing node initially interacts with the preset storage space, a temporary node is created in the preset storage space, and the temporary node matches the identification information of the computing node; when the interaction between the computing node and the preset storage space ends or the computing node fails and cannot interact, the temporary node disappears. Therefore, when determining whether the computing node that reads the shard information is a faulty node, it can be checked whether the shard information is in the second state first. If so, it is determined whether there is a temporary node in the preset storage space that matches the identification information of the computing node written at the storage location of the shard information. If so, it means that the computing node is working normally. If not, it means that the computing node has not traversed and read the data of the data shard corresponding to the shard information, and the computing node is a faulty node.
[0108] In the case that other computing nodes are faulty nodes, the management node is responsible for changing the shard information read by the other computing nodes from the second state to the first state, so that the management node or other normal computing nodes can read it.
[0109] According to the embodiments of the present disclosure, through the judgment and monitoring of the above-mentioned faulty nodes, it is possible to effectively avoid the defect that the faulty nodes fail to traverse the data, and other nodes cannot obtain the corresponding sharding information and thus cannot traverse this part of the data, thereby greatly improving the high availability of the computing cluster and further ensuring the integrity of data traversal and reading.
[0110] Another aspect of the present disclosure further provides a data traversal method for a computing node.
[0111] According to an embodiment of the present disclosure, the data is stored in multiple data shards, and the data traversal method includes the following operations: performing a read-traverse operation based on multiple shard information in parallel with at least one other computing node to traverse and read the data stored in the data shards corresponding to the multiple shard information.
[0112] According to an embodiment of the present disclosure, the computing node is a computing node in a computing cluster, which can be a reference Figure 1 The computing nodes 106 and 107 in FIG. 106 and 107 can also be referenced. Figure 2A to Figure 2B A compute node in at least one other compute node described.
[0113] According to an embodiment of the present disclosure, the plurality of shard information is pre-acquired by a computing node in at least one other computing node in the computing cluster. The computing node that pre-acquires the shard information may be, for example, a reference node. Figure 1 The computing node 105 in the example can also be used as a reference Figure 2A ~Reference Figure 5 Describes the data traversal methods for compute nodes in a compute node.
[0114] According to an embodiment of the present disclosure, the computing node used to obtain shard information in at least one other computing node is elected by the computing cluster (the computing node and at least one other computing node) using a preset election rule. The present disclosure embodiment does not limit the preset election rule, and the preset election rule may adopt, for example, the FastLeaderElection algorithm.
[0115] According to an embodiment of the present disclosure, when a computing cluster is initialized for the first time or a previously elected management node fails, if any computing node in the computing cluster finds that the management node does not exist or is unavailable when trying to communicate with the management node, an election is triggered to elect a new management node.
[0116] According to an embodiment of the present disclosure, each shard information corresponds to a data shard. When the computing node and at least one other computing node traverse and read data, different computing nodes traverse and read data stored in different data shards.
[0117] The data traversal method of the computing node of the embodiment of the present disclosure can greatly improve the speed of data traversal because it performs the read-traversal operation in parallel with at least one other computing node; furthermore, because the computing node can directly traverse and read the data stored in the corresponding data shard according to the shard information, it avoids the secondary transfer of the data through the ETL server, which can further improve the speed of data traversal.
[0118] According to an embodiment of the present disclosure, multiple shard information is pre-acquired by one computing node among at least one other computing node and then written into a preset storage space, so it can be directly read by the computing node and at least one other computing node to perform a read-traversal operation based on the read shard information.
[0119] Figure 6A to Figure 6B The flowchart of a read-traverse operation according to another embodiment of the present disclosure is schematically shown.
[0120] According to the embodiments of the present disclosure, Fig. 6A As shown, the above read-traverse operation includes operations S611 to S612. In operation S611, one of the multiple shard information is read from the preset storage space; in operation S612, based on the read shard information, the data stored in the data shard corresponding to the read shard information is traversed. According to an embodiment of the present disclosure, the read-traverse operation is similar to the reference shard information. Figure 2C The read-traverse operation described above will not be repeated here.
[0121] According to the embodiments of the present disclosure, Figure 6BAs shown, the above-mentioned read-traverse operation may also include, for example, operation S613 and operation S614. When one of the multiple shard information is read from the preset storage space, operation S613 is executed to set the read shard information from the first state to the second state; and when the data stored in the data shard corresponding to the shard information is traversed and read, operation S614 is executed to set the shard information from the second state to the third state. Wherein, when the shard information is written into the preset storage space by one of the at least one other computing nodes, it is set to the first state to indicate that the shard information can be read by the computing node or at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information can no longer be read. According to an embodiment of the present disclosure, the read-traverse operation is similar to reference. Figure 3 The read-traverse operation described above will not be repeated here.
[0122] According to an embodiment of the present disclosure, when the number of multiple shard information is less than the number of available computing nodes in the computing cluster (the total number of computing nodes and at least one other computing node), multiple computing nodes in the computing cluster (equal to the number of multiple shard information) can complete the traversal reading of the data stored in the data shards corresponding to the multiple shard information through one of the above-mentioned read-traverse operations. Of course, the embodiment of the present disclosure does not limit the specific traversal reading process. For example, some computing nodes (less than the number of shard information) among the multiple computing nodes can also be cycled through multiple read-traverse operations so that the multiple shard information are all read, and the data stored in the corresponding data shards are all traversed and read.
[0123] According to an embodiment of the present disclosure, when the number of multiple shard information is greater than the number of available computing nodes in the computing cluster (the total number of computing nodes and at least one other computing node), the computing node and at least one other computing node cyclically perform read-traverse operations. Specifically, after one computing node reads the shard information written by the child node of the task node in Zookeeper, the data stored in the data shard corresponding to the shard information is traversed and read; after completing the traversal reading, the computing node returns to read the shard information written by the other child nodes of the task node in Zookeeper (other child nodes that have not been read), and traverses and reads the data stored in the corresponding data shards, and other computing nodes also cyclically perform read-traverse operations in this manner until the shard information written by all child nodes of the task node in Zookeeper has been read, so that the data stored in the data shards corresponding to the multiple shard information are all traversed and read.
[0124] Figure 7A to Figure 7B The structural block diagram of the data traversal device of the computing node according to the embodiment of the present disclosure is schematically shown.
[0125] like Fig. 7A As shown, another aspect of the present disclosure provides a data traversal device for a computing node. The device 700 includes an acquisition module 710 and a first processing module 720 .
[0126] The data is stored in multiple data slices.
[0127] The acquisition module 710 is used to acquire multiple shard information, each shard information corresponds to a data shard. The first processing module 720 is used to perform a read-traverse operation based on the multiple shard information.
[0128] The data traversal device of the computing node can perform a read-traverse operation based on multiple shard information in parallel with the data traversal device of at least one other computing node to traverse and read the data stored in the data shards corresponding to the multiple shard information. And the data traversal devices of different computing nodes traverse and read the data stored in different data shards. It is understandable that the data traversal device of the computing node can be set in the computing node.
[0129] According to an embodiment of the present disclosure, the acquisition module 710 and the first processing module 720 are respectively used to perform reference Figure 2A The operations S210 and S220 described above are not described in detail here.
[0130] According to the embodiments of the present disclosure, Figure 7B As shown, the data traversal device 700 of the computing node may also include a write operation module 730 .
[0131] The write operation module 730 is used to write the multiple shard information obtained by the acquisition module 710 into a preset storage space. According to an embodiment of the present disclosure, the write operation module 730 can be used to perform a reference Figure 2B The operation S230 described above will not be described in detail here.
[0132] Figures 8A to 8D The structural block diagram of the first processing module according to an embodiment of the present disclosure is schematically shown.
[0133] like Fig. 8A As shown, the first processing module 720 may include a first shard information reading unit 721 and a first data traversal reading unit 722 .
[0134] The first shard information reading unit 721 is used to perform a read operation in a read-traverse operation, that is, to read one shard information from a preset storage space; the first data traversal reading unit 722 is used to perform a traversal operation in a read-traverse operation, that is, based on the read shard information, traverse the data stored in the data shard corresponding to the shard information. According to an embodiment of the present disclosure, the first shard information reading unit 721 and the first data traversal reading unit 722 can be used to respectively perform a reference Figure 2C The operations S221 and S222 described above are not repeated here.
[0135] According to the embodiments of the present disclosure, Figure 8B As shown, the first processing module 720 may further include a first modifying unit 723 .
[0136] The first change unit 723 is used to set the read shard information from the first state to the second state when the first shard information reading unit 721 reads one shard information from the preset storage space; and to set the shard information from the second state to the third state when the first data traversal reading unit 722 traverses and reads the data stored in the data shard corresponding to the shard information. When the shard information is written into the preset storage space by the write operation module 730, it is set to the first state to indicate that the shard information can be read by the data traversal device of the computing node or the data traversal device of at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information can no longer be read. According to an embodiment of the present disclosure, the first change unit 723 can be used to perform reference Figure 3 Operation S223 and operation S224 described above are not repeated here.
[0137] According to the embodiments of the present disclosure, Figure 8C As shown, the first processing module 720 may further include a fault determination unit 724, for example.
[0138] The fault judgment unit 724 is used to judge whether the data traversal device of the other computing node that has read one of the multiple shard information from the preset storage space is a faulty device. In the case where the data traversal device of the other computing node is a faulty device, the first changing unit 723 is also used to change the shard information read by the faulty device from the second state to the first state, so that the data traversal device of the computing node or the data traversal device of at least one other computing node other than the data traversal device of the faulty node can read it. According to an embodiment of the present disclosure, the fault judgment unit 724 can be used, for example, to perform a reference Figure 5The first modification unit 723 may also be used to perform the operation S510 described above. Figure 5 The operation S520 described above will not be described in detail here.
[0139] According to the embodiments of the present disclosure, Fig.8D As shown, the first processing module 720 may further include a state determination unit 725 , for example.
[0140] The state judgment unit 725 is used to judge whether the multiple pieces of information are all in the third state, and if there is a piece of information that is not in the third state, return to re-judge after a preset time interval, and if the multiple pieces of information are all in the third state, complete the data traversal. According to an embodiment of the present disclosure, the state judgment unit 725 can be used, for example, to perform a reference Figure 4 Operations S410 to S440 described above are not repeated here.
[0141] According to an embodiment of the present disclosure, the above-mentioned multiple data shards are stored on multiple storage nodes of the storage cluster, and each storage node stores one or more data shards.
[0142] According to an embodiment of the present disclosure, the acquisition module 710 is also used to obtain storage node information of multiple data shards. For example, the acquisition module 710 can also be used to perform a reference Figure 2B The operation S210' described above will not be described in detail here.
[0143] Fig. 9 The structural block diagram of the write operation module according to the embodiment of the present disclosure is schematically shown.
[0144] like Fig. 9 As shown, the write operation module 730 may include a fragment information arrangement unit 731 and a write operation unit 732 .
[0145] The shard information arrangement unit 731 is used to evenly arrange multiple shard information based on the preset allocation rule and storage node information, so that the shard information of different data shards with the same storage node information are arranged at intervals; the write operation unit 732 is used to sequentially write the evenly arranged multiple shard information into the preset storage space. Then the first shard information reading unit 721 sequentially reads a shard information from the evenly arranged multiple shard information. According to the embodiment of the present disclosure, the shard information arrangement unit 731 and the write operation unit 732 are respectively used to perform reference Figure 2D The operations S231 and S232 described above are not repeated here.
[0146] According to an embodiment of the present disclosure, a data traversal device of a computing node may be provided in the computing node, and the data traversal device of the computing node (hereinafter referred to as a management node) is elected by the data traversal devices of each computing node in the computing cluster (the data traversal device of the computing node and the data traversal device of at least one other computing node) using a preset election rule. The present disclosure embodiment does not limit the preset election rule, and the preset election rule may adopt, for example, the FastLeaderElection algorithm.
[0147] According to an embodiment of the present disclosure, when a computing cluster is initialized for the first time or a data traversal device of a previously elected management node fails, an election is triggered when the data traversal device of any computing node in the computing cluster finds that the management node does not exist or is unavailable when attempting to communicate with the management node, thereby electing a new management node.
[0148] According to an embodiment of the present disclosure, when the number of multiple shard information is less than the number of data traversal devices of available computing nodes in a computing cluster (the total number of data traversal devices of the computing node and the data traversal devices of at least one other computing node), the data traversal devices of multiple computing nodes in the computing cluster (equal to the number of multiple shard information) can complete the traversal reading of the data stored in the data shards corresponding to the multiple shard information through one of the above-mentioned read-traversal operations. Of course, the embodiment of the present disclosure does not limit the specific traversal reading process. For example, the data traversal devices of some computing nodes in the multiple computing nodes (less than the number of shard information) can also be cycled through multiple read-traversal operations so that the multiple shard information are all read and the data stored in the corresponding data shards are all traversed and read.
[0149] According to an embodiment of the present disclosure, when the number of multiple shard information is greater than the number of data traversal devices of available computing nodes in the computing cluster (the total number of data traversal devices of the computing node and the data traversal devices of at least one other computing node), the data traversal device of the computing node and the data traversal device of at least one other computing node cyclically perform read-traverse operations. Specifically, after the data traversal device of one computing node reads the shard information written by the child node of the task node in Zookeeper, the data stored in the data shard corresponding to the shard information is traversed and read; after completing the traversal reading, the data traversal device of the computing node returns to read the shard information written by other child nodes of the task node in Zookeeper (other child nodes that have not been read), and traverses and reads the data stored in the corresponding data shards, and the data traversal devices of other computing nodes also cyclically perform read-traverse operations in this manner until the shard information written by all child nodes of the task node in Zookeeper has been read, so that the data stored in the data shards corresponding to the multiple shard information are traversed and read.
[0150] According to the embodiment of the present disclosure, any one or more of the acquisition module 710, the first processing module 720, the write operation module 730, the first shard information reading unit 721, the first data traversal reading unit 722, the first change unit 723, the fault judgment unit 724, the state judgment unit 725, the shard information arrangement unit 731 and the write operation unit 732, or at least part of the functions of any one or more of them can be implemented in one module. According to the embodiment of the present disclosure, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiment of the present disclosure, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as a computer program module, which may perform corresponding functions when the computer program module is executed.
[0151] For example, any multiple of the acquisition module 710, the first processing module 720, the write operation module 730, the first shard information reading unit 721, the first data traversal reading unit 722, the first change unit 723, the fault judgment unit 724, the state judgment unit 725, the shard information arrangement unit 731 and the write operation unit 732 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 710, the first processing module 720, the write operation module 730, the first shard information reading unit 721, the first data traversal reading unit 722, the first change unit 723, the fault judgment unit 724, the state judgment unit 725, the shard information arrangement unit 731 and the write operation unit 732 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 710, the first processing module 720, the write operation module 730, the first shard information reading unit 721, the first data traversal reading unit 722, the first change unit 723, the fault judgment unit 724, the status judgment unit 725, the shard information arrangement unit 731 and the write operation unit 732 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is executed.
[0152] Figure 10A to Figure 10B The structural block diagram of the second processing module according to the embodiment of the present disclosure is schematically shown.
[0153] Another aspect of the present disclosure provides a data traversal device for a computing node. Fig. 10A As shown, the device includes a second processing module 1010 .
[0154] The second processing module 1010 is used to perform a read-traverse operation based on multiple shard information, the multiple shard information is pre-acquired by a data traversal device of at least one other computing node, and each shard information corresponds to a data shard. The data traversal device of the computing node can perform a read-traverse operation based on multiple shard information in parallel with the data traversal device of at least one other computing node to traverse and read the data stored in the data shards corresponding to the multiple shard information, and the data traversal devices of different computing nodes traverse and read the data stored in different data shards.
[0155] According to an embodiment of the present disclosure, multiple shard information is pre-acquired by a data traversal device of one computing node in at least one other computing node and then written into a preset storage space. The data traversal device of one computing node in the at least one other computing node (hereinafter referred to as a management node) is obtained by election using a preset election rule by the data traversal device of each computing node in the computing cluster (the data traversal device of the computing node and the data traversal device of at least one other computing node). The present disclosure embodiment does not limit the preset election rule, and the preset election rule may adopt, for example, the FastLeaderElection algorithm.
[0156] According to an embodiment of the present disclosure, when a computing cluster is initialized for the first time or a previously elected management node fails, an election is triggered when the data traversal device of any computing node in the computing cluster discovers that the management node does not exist or is unavailable when attempting to communicate with the data traversal device of the management node, thereby electing a new management node.
[0157] According to the embodiments of the present disclosure, Fig. 10A As shown, the second processing module 1010 includes a second shard information reading unit 1011 and a second data traversal reading unit 1012 .
[0158] The second shard information reading unit 1011 is used to read one shard information from a preset storage space, and the second data traversal reading unit 1012 is used to traverse and read the data stored in the data shard corresponding to the shard information based on the read shard information. According to an embodiment of the present disclosure, the second shard information reading unit 1011 and the second data traversal reading unit 1012 can be used to respectively perform reference Fig. 6A The operations S611 and S612 described above are not repeated here.
[0159] According to the embodiments of the present disclosure, Fig. 10B As shown, the second processing module 1010 may further include a second changing unit 1013 .
[0160] The second change unit 1013 is used to set the read shard information from the first state to the second state when the second shard information reading unit 1011 reads one of the multiple shard information from the preset storage space; and to set the shard information from the second state to the third state when the second data traversal reading unit 1012 traverses and reads the data stored in the data shard corresponding to the shard information. Wherein, the shard information is set to the first state when it is written into the preset storage space by the data traversal device of the management node to indicate that the shard information can be read by the data traversal device of the computing node or the data traversal device of at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information can no longer be read. According to an embodiment of the present disclosure, the second change unit can, for example, be used to execute reference Figure 6B Operation S613 and operation S614 described above are not repeated here.
[0161] According to an embodiment of the present disclosure, when the number of multiple shard information is less than the number of data traversal devices of available computing nodes in a computing cluster (the total number of data traversal devices of the computing node and the data traversal devices of at least one other computing node), the data traversal devices of multiple computing nodes in the computing cluster (equal to the number of multiple shard information) can complete the traversal reading of the data stored in the data shards corresponding to the multiple shard information through one of the above-mentioned read-traversal operations. Of course, the embodiment of the present disclosure does not limit the specific traversal reading process. For example, the data traversal devices of some computing nodes in the multiple computing nodes (less than the number of shard information) can also be cycled through multiple read-traversal operations so that the multiple shard information are all read and the data stored in the corresponding data shards are all traversed and read.
[0162] According to an embodiment of the present disclosure, when the number of multiple shard information is greater than the number of data traversal devices of available computing nodes in a computing cluster (the total number of data traversal devices of the computing node and the data traversal devices of at least one other computing node), the data traversal device of the computing node and the data traversal device of at least one other computing node cyclically perform read-traverse operations. Specifically, after one computing node reads the shard information written by the child node of the task node in Zookeeper, the data stored in the data shard corresponding to the shard information is traversed and read; after completing the traversal reading, the data traversal device of the computing node returns to read the shard information written by other child nodes of the task node in Zookeeper (other child nodes that have not been read), and traverses and reads the data stored in the corresponding data shards, and the data traversal devices of other computing nodes also cyclically perform read-traverse operations in this manner until the shard information written by all child nodes of the task node in Zookeeper has been read, so that the data stored in the data shards corresponding to the multiple shard information are traversed and read.
[0163] According to the second processing module 1010, the write operation module 730, the second sharding information reading unit 1011, the second data traversal reading unit 1012 and the second change unit 1013 of the embodiment of the present disclosure, any multiple, or at least part of the functions of any multiple of them can be implemented in one module. According to any one or more of the modules, submodules, units, and subunits of the embodiment of the present disclosure, they can be split into multiple modules for implementation. According to any one or more of the modules, submodules, units, and subunits of the embodiment of the present disclosure, they can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to one or more of the modules, submodules, units, and subunits of the embodiment of the present disclosure, they can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.
[0164] For example, any multiple of the second processing module 1010, the second shard information reading unit 1011, the second data traversal reading unit 1012, and the second change unit 1013 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the second processing module 1010, the second shard information reading unit 1011, the second data traversal reading unit 1012, and the second change unit 1013 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the second processing module 1010, the second shard information reading unit 1011, the second data traversal reading unit 1012 and the second change unit 1013 can be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function can be executed.
[0165] Fig.11 A block diagram suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Fig.11 The computing node shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0166] like Fig.11 As shown, the computing node 1100 according to an embodiment of the present disclosure includes a processor 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage part 1108 to a random access memory (RAM) 1103. The processor 1101 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1101 may also include an onboard processor for caching purposes. The processor 1101 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0167] In RAM 1103, various programs and data required for the operation of computing node 1100 are stored. Processor 1101, ROM 1102 and RAM 1103 are connected to each other through bus 1104. Processor 1101 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1102 and / or RAM 1103. It should be noted that the program can also be stored in one or more memories other than ROM 1102 and RAM 1103. Processor 1101 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.
[0168] According to an embodiment of the present disclosure, the computing node 1100 may further include an input / output (I / O) interface 1105, which is also connected to the bus 1104. The computing node 1100 may further include one or more of the following components connected to the I / O interface 1105: an input portion 1106 including a keyboard, a mouse, etc.; an output portion 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1108 including a hard disk, etc.; and a communication portion 1109 including a network interface card such as a LAN card, a modem, etc. The communication portion 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage portion 1108 as needed.
[0169] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the processor 1101, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.
[0170] The present disclosure also provides a computer-readable medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed, the method flow according to the embodiment of the present disclosure is implemented.
[0171] According to an embodiment of the present disclosure, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical cable, radio frequency signal, etc., or any suitable combination of the above.
[0172] For example, according to an embodiment of the present disclosure, the computer-readable medium may include the ROM 1102 and / or the RAM 1103 described above and / or one or more memories other than the ROM 1102 and the RAM 1103 .
[0173] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0174] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0175] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. A method for traversing data of a computing node, wherein the data is stored in a plurality of data shards, the method comprising: Get multiple shard information, each shard information corresponds to a data shard; as well as In parallel with at least one other computing node, a read-traverse operation is performed based on the plurality of shard information to traverse and read the data stored in the data shards corresponding to the plurality of shard information, Among them, different computing nodes traverse and read data stored in different data shards; The data traversal method of the computing node also includes: Writing the plurality of shard information into a preset storage space; The read-traverse operation includes: Reading one of the plurality of shard information from the preset storage space; and Based on the read shard information, traverse and read the data stored in the data shard corresponding to the shard information; The read-traverse operation also includes: When one of the plurality of shard information is read from the preset storage space, the read shard information is set from the first state to the second state; and When the data stored in the data slice corresponding to the slice information is traversed and read, the slice information is changed from the second state to the third state; Among them, the first state of the shard information indicates that the shard information can be read by the computing node or at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information can no longer be read.
2. The method according to claim 1, wherein: The plurality of data shards are stored on a plurality of storage nodes of a storage cluster, each storage node stores one or more of the data shards, and the method further comprises: Obtaining storage node information of multiple data shards; Writing the plurality of shard information into a preset storage space includes: Based on a preset allocation rule and the storage node information, the plurality of shard information are evenly arranged so that shard information of different data shards having the same storage node information are arranged at intervals; and Sequentially write the evenly arranged multiple shard information into a preset storage space; Reading one of the plurality of shard information from the preset storage space includes: A piece of slice information is sequentially read from the multiple pieces of slice information that are evenly arranged.
3. The method according to claim 1, further comprising: Determine whether other computing nodes that have read one of the plurality of shard information from the preset storage space are faulty nodes; as well as In the case that the other computing node is a faulty node, the shard information read by the faulty node is changed from the second state to the first state, so as to be read by the computing node or at least one other computing node except the faulty node.
4. The method according to claim 1, further comprising: Determining whether the plurality of fragment information are all in the third state; If there is shard information that is not in the third state, return to re-judge after a preset time interval; When the plurality of shard information are all in the third state, the traversal of the data is completed.
5. The method according to claim 1, wherein: The computing node is elected by the computing node and at least one other computing node using a preset election rule; and / or When the number of the multiple shard information is greater than the total number of nodes of the computing node and at least one other computing node, the computing node and / or at least one other computing node cyclically executes the read-traversal operation so that the data stored in the data shards corresponding to the multiple shard information are all traversed and read.
6. A method for traversing data of a computing node, wherein the data is stored in a plurality of data shards, the method comprising: Performing a read-traverse operation based on the plurality of shard information in parallel with at least one other computing node to traverse and read the data stored in the data shards corresponding to the plurality of shard information, The plurality of shard information is pre-acquired by a computing node in at least one other computing node, each shard information corresponds to a data shard, and different computing nodes traverse and read data stored in different data shards; The data traversal method of the computing node also includes: The plurality of shard information is pre-acquired by a computing node in at least one other computing node and then written into a preset storage space; The read-traverse operation includes: Reading one of the plurality of shard information from the preset storage space; and Based on the read shard information, traverse and read the data stored in the data shard corresponding to the shard information; The read-traverse operation also includes: When one of the plurality of shard information is read from the preset storage space, the read shard information is set from the first state to the second state; and When the data stored in the data slice corresponding to the slice information is traversed and read, the slice information is changed from the second state to the third state; Among them, the first state of the shard information indicates that the shard information can be read by the computing node or at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information can no longer be read.
7. The method according to claim 6, wherein: The computing node that pre-acquires the plurality of shard information is elected by the computing node and at least one other computing node using a preset election rule; and / or When the number of the multiple shard information is greater than the total number of nodes of the computing node and at least one other computing node, the computing node and / or at least one other computing node cyclically executes the read-traversal operation so that the data stored in the data shards corresponding to the multiple shard information are all traversed and read.
8. A data traversal device for a computing node, wherein the data is stored in a plurality of data slices, the device comprising: The acquisition module obtains multiple shard information, each of which corresponds to a data shard; as well as A first processing module performs a read-traverse operation based on the plurality of shard information, The data traversal device of the computing node and the data traversal device of at least one other computing node perform the read-traversal operation based on the multiple shard information in parallel to traverse and read the data stored in the data shards corresponding to the multiple shard information, and the data traversal devices of different computing nodes traverse and read the data stored in different data shards; The data traversal device of the computing node also includes: A write operation module, writing the plurality of shard information into a preset storage space; Wherein, the first processing module includes: A first fragment information reading unit, reading one fragment information from the preset storage space; and A first data traversal reading unit, based on the read shard information, traverses and reads the data stored in the data shard corresponding to the shard information; The first processing module further includes a first changing unit: When the first fragment information reading unit reads one fragment information from the preset storage space, the read fragment information is changed from the first state to the second state; and When the first data traversal and reading unit has finished traversing and reading the data stored in the data shard corresponding to the shard information, the shard information is changed from the second state to the third state; Among them, the first state of the shard information indicates that the shard information can be read by the data traversal device of the computing node or the data traversal device of at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information can no longer be read.
9. The device according to claim 8, wherein: The plurality of data shards are stored on a plurality of storage nodes of the storage cluster, and each storage node stores one or more of the data shards; The acquisition module acquires storage node information of the plurality of data shards; The write operation module comprises: A shard information arrangement unit, based on a preset allocation rule and the storage node information, evenly arranges the plurality of shard information so that shard information of different data shards having the same storage node information are arranged at intervals; and A write operation unit, which writes the evenly arranged plurality of shard information sequentially into a preset storage space; The first fragment information reading unit sequentially reads one fragment information from the multiple fragment information evenly arranged.
10. The device according to claim 8, wherein: The first processing module also includes: A fault judgment unit, which judges whether a data traversal device of another computing node that has read one of the plurality of shard information from the preset storage space is a faulty device; The first change unit, when the data traversal device of the other computing node is a faulty device, changes the shard information read by the faulty device from the second state to the first state, so that it can be read by the data traversal device of the computing node or the data traversal device of at least one other computing node except the faulty device.
11. The device according to claim 8, wherein: The first processing module also includes a state judgment unit: Determining whether the plurality of fragment information are all in the third state; If there is shard information that is not in the third state, return to re-judge after a preset time interval; When the plurality of shard information are all in the third state, the traversal of the data is completed.
12. The device according to claim 8, wherein: The data traversal device of the computing node is elected by the data traversal device of the computing node and the data traversal device of at least one other computing node using a preset election rule; and / or When the number of the multiple shard information is greater than the total number of the data traversal devices of the computing node and the data traversal devices of at least one other computing node, the data traversal device of the computing node and / or the data traversal device of at least one other computing node cyclically executes the read-traversal operation so that the data stored in the data shards corresponding to the multiple shard information are all traversed and read.
13. A data traversal device for a computing node, wherein the data is stored in a plurality of data slices, the device comprising: The second processing module performs a read-traverse operation based on a plurality of shard information, wherein the plurality of shard information is pre-acquired by a data traversal device of one of at least one other computing node, and each shard information corresponds to a data shard. The data traversal device of the computing node and the data traversal device of at least one other computing node perform the read-traversal operation based on the multiple shard information in parallel to traverse and read the data stored in the data shards corresponding to the multiple shard information, and the data traversal devices of different computing nodes traverse and read the data stored in different data shards; The data traversal device of the computing node also includes: The plurality of shard information is pre-acquired by a data traversal device of one computing node in at least one other computing node and then written into a preset storage space; Wherein, the second processing module includes: A second fragment information reading unit, reading one fragment information from the preset storage space; and A second data traversal reading unit, based on the read shard information, traverses and reads the data stored in the data shard corresponding to the shard information; The second processing module further includes a second changing unit: When the second fragment information reading unit reads one fragment information from the preset storage space, the read fragment information is changed from the first state to the second state; and When the second data traversal reading unit traverses and reads the data stored in the data slice corresponding to the slice information, the slice information is changed from the second state to the third state, Among them, the first state of the shard information indicates that the shard information can be read by the data traversal device of the computing node or the data traversal device of at least one other computing node; the second state of the shard information indicates that the shard information has been read and cannot be read again; the third state of the shard information indicates that the data stored in the data shard corresponding to the shard information has been traversed and read, and the shard information can no longer be read.
14. The device according to claim 13, wherein: The data traversal device of the computing node that pre-acquires the plurality of shard information is elected by the data traversal device of the computing node and the data traversal device of at least one other computing node using a preset election rule; and / or When the number of the multiple shard information is greater than the total number of the data traversal devices of the computing node and the data traversal devices of at least one other computing node, the data traversal device of the computing node and / or the data traversal device of at least one other computing node cyclically executes the read-traversal operation so that the data stored in the data shards corresponding to the multiple shard information are all traversed and read.
15. A computing node, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are respectively executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 7.
16. A computer-readable medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Distributed type data storing method and equipment
CN102801784A
Method and device for social network service analysis
CN103138981A
Data search method and device
CN103714142A
Distributed file system based file reading and writing method and node server
CN105630808A
Big data platform oriented and memory data locality based scheduling method and system
CN105740068A
