Data storage method

Abstract

The invention discloses a data storage method, and the method comprises the steps: obtaining a storage partition corresponding to target business data, the storage partition corresponding to a Markov chain, and the Markov chain is used for representing the distribution state of initial transition probabilities among storage nodes included in the storage partition; each initial transition probability is determined according to resource data of the corresponding two storage nodes; determining a main storage node from the at least one storage node according to the target service data and the Markov chain; and storing the target business data to the main storage node.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a data storage method. Background Art

[0002] With the rapid development of big data technology, a single database storage method can no longer adapt to the ever-increasing number of users and information storage. A common storage method is to use a distributed multi-cluster multi-node storage method to store user data. However, this method cannot be adjusted and deployed according to the current node's network, memory space and other related conditions, resulting in uneven storage. Summary of the invention

[0003] The embodiment of the present application provides a data storage method, which can solve the problem of uneven data storage.

[0004] In a first aspect, an embodiment of the present application provides a data storage method, the method comprising: obtaining a storage partition corresponding to target business data, wherein the storage partition corresponds to a Markov chain, and the Markov chain is used to represent the distribution state of the initial transition probability between the storage nodes included in the storage partition, each of the initial transition probability is determined based on the resource data of the corresponding two storage nodes; determining a main storage node from at least one of the storage nodes based on the target business data and the Markov chain; and storing the target business data in the main storage node.

[0005] In a second aspect, an embodiment of the present application provides a data storage device, which includes: an acquisition module, used to acquire a storage partition corresponding to target business data, wherein the storage partition corresponds to a Markov chain, and the Markov chain is used to represent the distribution state of the initial transition probability between the storage nodes included in the storage partition, and each of the initial transition probabilities is determined based on the resource data of the corresponding two storage nodes; a determination module, used to determine a main storage node from at least one of the storage nodes based on the target business data and the Markov chain; and a storage module, used to store the target business data to the main storage node.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0008] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0009] In a sixth aspect, an embodiment of the present application provides a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer implements the steps of the method described in the first aspect.

[0010] In an embodiment of the present application, a storage partition corresponding to the target business data is obtained, wherein the storage partition corresponds to a Markov chain, and the Markov chain is used to represent the distribution state of the initial transfer probability between the storage nodes included in the storage partition, and each initial transfer probability is determined according to the resource data of the corresponding two storage nodes, and then according to the target business data and the Markov chain, a main storage node is determined from at least one storage node, and finally the target business data is stored in the main storage node, thereby achieving balanced distribution of storage data and being able to effectively ensure the load balance of each storage node, so that storage resources are reasonably allocated and fully utilized, thereby improving the flexibility and efficiency of data storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram showing a flow chart of a data storage method provided by an embodiment of the present application;

[0012] Figure 2 A schematic diagram of a Markov chain provided in an embodiment of the present application is shown;

[0013] Figure 3 A schematic diagram showing a storage partition provided by an embodiment of the present application is shown;

[0014] Figure 4 A schematic diagram of a hash ring provided in an embodiment of the present application is shown;

[0015] Figure 5 A schematic diagram showing an activation function provided in an embodiment of the present application is shown;

[0016] Figure 6 A schematic diagram of a Markov chain provided in an embodiment of the present application is shown;

[0017] Figure 7 A schematic diagram of a Markov chain provided in an embodiment of the present application is shown;

[0018] Figure 8 A schematic diagram of a distributed storage processing flow provided by an embodiment of the present application is shown;

[0019] Fig. 9 A schematic diagram showing the structure of another data storage device provided in an embodiment of the present application is shown;

[0020] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] A data storage method provided by an embodiment of the present application is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0023] Figure 1 A data storage method provided by an embodiment of the present application is shown, and the method can be executed by an electronic device, and the electronic device may include: a server and / or a terminal device. In other words, the method can be executed by software or hardware installed in the electronic device, and the method includes the following steps:

[0024] S110: Obtain the storage partition corresponding to the target business data.

[0025] The storage partition corresponds to a Markov chain, and the Markov chain is used to represent the distribution state of the initial transition probability between the storage nodes included in the storage partition, and each of the initial transition probability is determined according to the resource data of the corresponding two storage nodes.

[0026] As for Markov chains, it can be understood that Markov chains refer to discrete-time random processes with Markov properties. In this process, given the current state or information, the past (i.e., the historical state before the current period) is irrelevant to predicting the future (i.e., the future state after the current period). In other words, the current state is only determined by the previous state. That is, the state probability result distribution at time t+1 is only related to the state at time t, and has nothing to do with the state before time t. The Markov state formula is as follows:

[0027] P(X n+1 =x|X1,X2,X3....,X n )=P(X n+1 =x|X n )

[0028] Among them, X1, X2, X3..., X n It refers to a set of possible values ​​of a random variable, which is the state space of the nodes of the Markov chain, where X n represents the state result at time n, and X n+1 Represents the state result at time n+1. Therefore, in the embodiment of the present application, the Markov chain may have the following characteristics: the state probability selection of the next storage node depends only on the currently stored node, and has nothing to do with the state of all previous storage nodes; the sum of the probabilities of the current node and all other nodes plus the probability of continuing to hit itself is 1, and the current node can be any storage node in the storage partition.

[0029] In this step, the storage partition includes at least one storage node, and the multiple initial transition probabilities include the probability of each storage node to the next storage node as the initial transition probability, and the probability of each storage node hitting itself. Figure 2 As shown in FIG. 1 , it is a schematic diagram of a Markov chain, where P1 represents the initial transition probability of storage node A hitting itself, P2 represents the initial transition probability of storage node C hitting itself, P3 represents the initial transition probability of storage node B hitting itself, P4 represents the initial transition probability from storage node B to storage node A, and P5 represents the initial transition probability from storage node A to storage node B, etc. In other words, the initial transition probability can represent the selection weight of the next storage node. Each initial transition probability is determined based on the resource data of the two storage nodes corresponding to the initial transition probability, with Figure 2 Taking P6 in the example, P6 is determined based on the resource data of storage node B and the resource data of storage node C. The resource data may include but is not limited to: remaining storage capacity, current load, network bandwidth, and distance from the central node. The central node may be a server or other control node for controlling the distributed storage of data and coordinating the work of each storage node.

[0030] Optionally, the storage node may be a different physical or logical storage device.

[0031] In this step, probability calculation is performed based on the network quality, memory size and other related resource data of the corresponding storage node, which can avoid the problem of some storage nodes crashing due to overloaded storage or some storage nodes wasting resources due to insufficient storage.

[0032] S120: Determine a primary storage node from at least one of the storage nodes according to the target service data and the Markov chain.

[0033] In this step, by considering the weight of the storage node, the load balancing of data storage can be achieved. Even if the performance and capacity of the storage nodes are different, the data can be distributed according to their weights so that the load of each node is relatively balanced. In addition, the Markov chain also has the feature of dynamic update, so that the selection process of the storage node can be more flexible and accurate.

[0034] S130: Storing the target service data in the primary storage node.

[0035] In an embodiment of the present application, a storage partition corresponding to the target business data is obtained, wherein the storage partition corresponds to a Markov chain, and the Markov chain is used to represent the distribution state of the initial transfer probability between the storage nodes included in the storage partition, and each initial transfer probability is determined according to the resource data of the corresponding two storage nodes, and then according to the target business data and the Markov chain, a main storage node is determined from at least one storage node, and finally the target business data is stored in the main storage node, thereby achieving balanced distribution of storage data and being able to effectively ensure the load balance of each storage node, so that storage resources are reasonably allocated and fully utilized, thereby improving the flexibility and efficiency of data storage.

[0036] In one implementation, before obtaining the storage partition corresponding to the target business data, the method also includes: determining a target number by analyzing historical business data, wherein the target number is used to represent the number of partitions; dividing a plurality of the storage nodes into the target number of storage partitions; and constructing a Markov chain corresponding to each of the storage partitions.

[0037] It is understandable that in a distributed storage system, there are usually multiple storage nodes to choose from to store data. Selecting a suitable storage node may require considering multiple factors, such as the current load of the node, network latency, etc. However, due to the large number of storage nodes, it takes longer and is more complex. Therefore, in an embodiment of the present application, multiple storage nodes are partitioned, and each storage partition includes at least one storage node. In the node selection process, there is no need to make decisions for multiple nodes, which can effectively reduce the complexity of the decision-making process and the time required.

[0038] Determining the target number by analyzing historical business data means evaluating the amount of historical business data to determine the number of partitions, which can better match storage needs and avoid performance bottlenecks caused by unbalanced partitions. If too many storage partitions are set, the complexity of the decision-making process and the time required will not be greatly improved. If too few storage partitions are set, the complexity of the decision-making process and the time required will not be greatly improved. Therefore, the appropriate number of partitions can be set by analyzing historical business data. In another implementation method, the number of partitions can be determined using Poisson distribution, and the formula for the Poisson distribution is as follows:

[0039]

[0040] Among them, k represents the target number, that is, the number of partitions; λ represents the average number of data storage times in a unit time period, and the number of data storage times can be obtained based on the analysis of historical business data; in addition, the number of partitions is set in the range of [λ-10, λ+10] (λ>10), and then the memory usage rate is obtained based on the sample data, and the probability value of the P(X=k) value is further obtained. After relevant comparisons, the value of k can be determined.

[0041] In this implementation, the storage nodes are sharded in the form of Poisson distribution, which can more reasonably perform data deployment partitioning.

[0042] Furthermore, in another implementation, the constructing of the Markov chain corresponding to each of the storage partitions includes: for each of the storage partitions, determining the transition probability between each of the storage nodes according to the resource data corresponding to each of the storage nodes; and constructing the Markov chain corresponding to each of the storage partitions based on the transition probabilities between each of the storage nodes and each of the storage nodes.

[0043] That is, after dividing the storage partitions, a Markov chain is constructed for each storage partition, where the initial transfer probability of each storage node to the next storage node can be determined by the following probability calculation formula:

[0044]

[0045] Among them, P point represents the probability of the next storage node being selected for storage, a represents the network bandwidth parameter of the current storage node, and C ′ Indicates the storage memory parameters of the current storage section. represents the total memory of all storage nodes, and L represents the distance factor between the central node and the next storage node. The probability that the current node selects itself again is:

[0046]

[0047] Among them, P self It indicates the probability that the current storage node will select itself again, and Represents the sum of probabilities from the current node to the next node other than itself.

[0048] Since the probability in the above probability calculation formula is proportional to the memory of the current node and the subsequent node, proportional to the network bandwidth, and inversely proportional to the distance value, each storage node can be connected in series according to the probability calculation formula to form the following Figure 2 In the embodiment of the present application, assuming that there are m storage partitions, each storage partition corresponds to a Markov chain, such as Figure 3 shown.

[0049] In one implementation, obtaining the storage partition corresponding to the target business data includes: determining the storage partition corresponding to the target business data through a consistent hashing algorithm.

[0050] As for the consistent hashing algorithm, it can be understood that the consistent hashing algorithm is to map the value corresponding to each storage partition to a ring of fixed size through a hash function, such as a ring-shaped hash space. The target business data calculates the hash value of its storage primary key through the hash function, and then finds the first partition position that is greater than or equal to the hash value along the clockwise direction of the ring. This partition position is the storage partition corresponding to the target business data. In other words, in [0, 2 32 -1] is evenly divided into m partitions, the storage primary key of the target business data is hashed, the obtained hash value is mapped to the hash ring, and the closest storage partition is selected clockwise according to the mapped value. For example, Figure 4 As shown, the hashed value of the storage primary key is mapped to the m-2 partition and the m-1 partition, and according to the clockwise selection, the m-1 partition is the storage partition selected by k. The storage primary key can be determined according to the data volume of the target business data.

[0051] In this implementation, the storage partition corresponding to the target business data can be quickly determined, and the number of the storage partitions can be dynamically expanded.

[0052] In one implementation, determining the main storage node from at least one of the storage nodes based on the target business data and the Markov chain includes: when the storage partition corresponding to the target business data has stored historical business data, obtaining a historical main storage node corresponding to the storage partition, wherein the historical main storage node is the main storage node corresponding to the previous storage operation performed on the storage partition; determining the main storage node from at least one of the storage nodes based on the target business data, the Markov chain and the historical main storage node.

[0053] It can be understood that the storage partition corresponding to the target business data has stored historical business data, which means that the storage partition is not used for the first time, so a storage node in the storage partition has stored business data. Based on the characteristics of the Markov chain: the state probability selection of the next storage node depends only on the current storage node, and has nothing to do with the state of all previous storage nodes. In other words, the selection of the current storage node is related to the main storage node corresponding to the previous storage operation, and has nothing to do with the main storage nodes corresponding to other previous storage operations. It can effectively reduce the complexity of the calculation. Therefore, the historical main storage node can be obtained, and the main storage node can be determined from at least one storage node based on the target business data, the Markov chain and the historical main storage node.

[0054] Furthermore, in another implementation, the determining the main storage node from at least one of the storage nodes based on the target business data, the Markov chain and the historical main storage node includes: determining the target transition probability between the historical main storage node and each of the storage nodes based on the real-time resource data corresponding to each of the storage nodes and the storage size of the target business data; updating the Markov chain based on the target transition probability; and determining the main storage node from at least one of the storage nodes based on the updated Markov chain.

[0055] It is understandable that during the data storage process, the resource data of each storage node may change. In order to accurately determine the main storage node, the Markov chain can be updated first, and the main storage node can be determined according to the updated Markov chain. The update of the Markov chain can be achieved by determining the target transfer probability between the historical main storage node and each of the storage nodes, and the update formula can be:

[0056]

[0057] Wherein, num represents the number of storage times of the storage partition. For example, when the storage partition is selected for the first storage, the main storage node is determined according to the initial Markov chain. When the partition is selected for the second storage operation, the determined target transition probability is matrix-multiplied with the initial transition probability in the Markov chain, and the result of the matrix multiplication is used as the updated initial transition probability. It can be understood that the target transition probability and the initial transition probability can be represented by a state space matrix. For example, it is assumed that the state space matrix corresponding to the initial transition probability is:

[0058]

[0059] The state space matrix corresponding to the target transition probability is:

[0060]

[0061] Then the state space matrix corresponding to the second storage operation is:

[0062]

[0063] After obtaining the state space matrix corresponding to the second storage operation, the updated Markov chain can be obtained. In this way, by adjusting the probability, the solidified probability distribution is destroyed, and the storage weight coefficients of each node are changed accordingly, so that the updated Markov chain is more suitable for the current business scenario and the resource utilization is maximized.

[0064] In one implementation, the determining of the primary storage node from at least one of the storage nodes according to the target business data and the Markov chain includes: when the storage partition corresponding to the target business data does not store historical business data, determining the storage node with the largest initial transfer probability from the Markov chain as the primary storage node. In other words, if the storage partition is used for the first time, the storage node with the largest transfer probability can be determined from the Markov chain as the primary storage node. Among them, in another implementation, when the storage space of the primary storage node does not meet the data size of the target business data, the primary storage node is determined from other storage nodes in sequence according to the order of the initial transfer probability.

[0065] In one implementation, the method further includes: updating the Markov chain.

[0066] It is understandable that the Markov chain may become solid after being used for a period of time, so link perturbation is required to achieve probability balance.

[0067] Furthermore, in one implementation, the updating of the Markov chain includes: when there is a faulty node between at least one of the storage nodes, reducing the initial transition probability from other storage nodes to the faulty node and the initial transition probability of the faulty node selecting itself, and increasing the initial transition probability from the faulty node to other storage nodes, wherein the sum of the initial transition probability of the faulty node selecting itself and the initial transition probability between the storage nodes is 1.

[0068] It is understandable that when there is a faulty node, the probability of other nodes selecting the faulty node can be reduced by reducing the probability of other nodes being connected to the faulty node, and the probability of the faulty node continuing to hit itself can be reduced by increasing the probability of the faulty node being connected to other nodes, thereby achieving the update of the Markov chain. Optionally, the adjustment formula is as follows:

[0069] P reward =P point *2sigmoid(μ)

[0070] μ=C next *a next -C self *a selt

[0071]

[0072] Among them, P reward Represents the adjusted probability. If the adjusted P reward When it is greater than 1, it should be rounded to 1; P point is the probability of node connection in the above formula, the range of 2sigmoid(μ) is [0, 2], C next , a next Indicates the memory and bandwidth of the next node, C self , a selt Indicates the memory and bandwidth of the current node. In addition, the nonlinear activation function sigmoid can be introduced. According to the characteristics of the sigmoid function, when μ>0, the value range of 2sigmoid(μ) is [1, 2]; when μ<0, the value range of 2sigmoid(μ) is [0, 1]; its function graph is as follows Figure 5 Assume that the faulty node is node C, then the updated Markov chain can be as follows Figure 6 As shown, + indicates that the probability value increases, and - indicates that the probability value decreases. In another implementation, the faulty node can be deleted from the Markov chain, such as Figure 7 shown.

[0073] In one implementation, the updating of the Markov chain includes: updating the Markov chain based on resource data of each of the storage nodes.

[0074] It is understandable that after some storage nodes have been precipitated with business data, due to long-term operation, it is easy for the storage nodes to have related problems such as probability deviation, resulting in related phenomena such as "bulky" node storage. The "bulky" means that due to the sharp increase in the probability of storage and access traffic, the storage information will select a storage node with a high probability for storage, causing the memory of the storage node to be in urgent need, and also affecting the throughput, thereby affecting the storage efficiency. Therefore, it is necessary to adjust the Markov chain based on the resource data of each of the storage nodes, and break the existing node solidity by re-adjusting the chain nodes. In another implementation method, storage nodes that are no longer suitable for storage can be deleted from the Markov chain.

[0075] Figure 8 A distributed storage processing system provided by an embodiment of the present application is shown below. Figure 8 The above embodiments are further described. Figure 8 As shown, the following processes may be included:

[0076] (1) Data generation: Data generation mainly generates business data, which refers to the relevant data generated by the business process that needs to be stored.

[0077] (2) Data preprocessing: Data preprocessing involves extracting the data storage primary key (i.e., the partition key of the database, the key value used to select the data partition) from the data to be stored, and performing partition selection after hashing the data primary key.

[0078] (3) Generation of Markov chain node probability weights. The generation of Markov chain node probability weights refers to calculating the storage probability of storage nodes based on the relevant conditions of each node (such as network bandwidth, the distance between the service and the storage node, and the memory size of the storage node itself) and forming a single Markov chain.

[0079] (4) Markov chain construction. Markov chain construction refers to connecting each node according to its probability information to form a Markov chain, and using node partitioning to form multiple Markov chain regions.

[0080] (5) Self-regulation of Markov chain. Self-regulation of Markov chain means that after a long period of business data sedimentation, the Markov chain adjusts the weights of storage nodes according to its own conditions to form a new Markov chain.

[0081] Fig. 9 A schematic diagram showing the structure of a data storage device provided by an embodiment of the present application is shown as follows: Fig. 9 As shown, the data storage device 900 may include: an acquisition module 910 , a determination module 920 and a storage module 930 .

[0082] In this embodiment, an acquisition module 910 is used to acquire a storage partition corresponding to target business data, wherein the storage partition corresponds to a Markov chain, and the Markov chain is used to represent the distribution state of the initial transfer probability between the storage nodes included in the storage partition, and each of the initial transfer probability is determined based on the resource data of the corresponding two storage nodes; a determination module 920 is used to determine a main storage node from at least one of the storage nodes based on the target business data and the Markov chain; and a storage module 930 is used to store the target business data in the main storage node.

[0083] In one implementation, the data storage device 900 may also include: a partitioning module, used to determine a target number by analyzing historical business data, wherein the target number is used to represent the number of partitions; divide the plurality of storage nodes into the target number of storage partitions; and construct a Markov chain corresponding to each of the storage partitions.

[0084] In one implementation, constructing a Markov chain corresponding to each storage partition includes: for each storage partition, determining a transition probability between each storage node based on resource data corresponding to each storage node; and constructing a Markov chain corresponding to each storage partition based on each storage node and the transition probability between each storage node.

[0085] In one implementation, obtaining the storage partition corresponding to the target business data includes: determining the storage partition corresponding to the target business data through a consistent hashing algorithm.

[0086] In one implementation, determining the main storage node from at least one of the storage nodes based on the target business data and the Markov chain includes: when the storage partition corresponding to the target business data has stored historical business data, obtaining a historical main storage node corresponding to the storage partition, wherein the historical main storage node is the main storage node corresponding to the previous storage operation performed on the storage partition; determining the main storage node from at least one of the storage nodes based on the target business data, the Markov chain and the historical main storage node.

[0087] In one implementation, the determining the main storage node from at least one of the storage nodes based on the target business data, the Markov chain and the historical main storage node includes: determining the target transition probability between the historical main storage node and each of the storage nodes based on the real-time resource data corresponding to each of the storage nodes and the storage size of the target business data; updating the Markov chain based on the target transition probability; and determining the main storage node from at least one of the storage nodes based on the updated Markov chain.

[0088] In one implementation, determining the main storage node from at least one of the storage nodes based on the target business data and the Markov chain includes: when the storage partition corresponding to the target business data does not store historical business data, determining the storage node with the highest initial transfer probability from the Markov chain as the main storage node.

[0089] In one implementation, the data storage device 900 may further include: an updating module, configured to update the Markov chain.

[0090] In one implementation, the update module is specifically used to reduce the initial transfer probability from other storage nodes to the faulty node and the initial transfer probability of the faulty node selecting itself, and increase the initial transfer probability from the faulty node to other storage nodes when there is a faulty node between at least one of the storage nodes, wherein the sum of the initial transfer probability of the faulty node selecting itself and the initial transfer probability between the storage nodes is 1.

[0091] In one implementation, the updating module is specifically configured to update the Markov chain based on the resource data of each of the storage nodes.

[0092] A data storage device provided in an embodiment of the present application can implement each process implemented in the data storage method embodiment, and will not be described again here to avoid repetition.

[0093] The data storage device in the embodiment of the present application may be a device, or a component, an integrated circuit, or a chip in an electronic device. The embodiment of the present application does not specifically limit this.

[0094] The data storage device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0095] Optional, such as Fig.10As shown, an embodiment of the present application also provides an electronic device 1000, including a processor 1010, a memory 1020, and a program or instruction stored in the memory 1020 and executable on the processor 1010. When the program or instruction is executed by the processor 1010, each process of the above-mentioned data storage method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0096] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned data storage method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0097] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0098] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned data storage method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0099] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0100] An embodiment of the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer implements the various processes of the above-mentioned data storage method embodiment and can achieve the same technical effect. To avoid repetition, they are not repeated here.

[0101] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0103] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A data storage method, characterized in that: The method comprises: Acquire a storage partition corresponding to the target business data, wherein the storage partition corresponds to a Markov chain, and the Markov chain is used to represent the distribution state of the initial transition probability between the storage nodes included in the storage partition, and each of the initial transition probabilities is determined according to the resource data of the corresponding two storage nodes; Determine a primary storage node from at least one of the storage nodes according to the target service data and the Markov chain; The target business data is stored in the primary storage node.

2. The method according to claim 1, characterized in that Before obtaining the storage partition corresponding to the target business data, the method further includes: By analyzing historical business data, a target number is determined, wherein the target number is used to represent the number of partitions; Dividing the plurality of storage nodes into a target number of storage partitions; A Markov chain corresponding to each of the storage partitions is constructed.

3. The method according to claim 2, characterized in that The step of constructing a Markov chain corresponding to each storage partition includes: For each of the storage partitions, determining a transfer probability between the storage nodes according to resource data corresponding to each of the storage nodes; Based on the storage nodes and the transition probabilities between the storage nodes, a Markov chain corresponding to each storage partition is constructed.

4. The method according to claim 1, characterized in that: The obtaining of the storage partition corresponding to the target business data includes: A storage partition corresponding to the target business data is determined through a consistent hashing algorithm.

5. The method according to claim 1, characterized in that The determining a primary storage node from at least one storage node according to the target service data and the Markov chain comprises: In the case where the storage partition corresponding to the target business data has stored historical business data, obtaining a historical main storage node corresponding to the storage partition, wherein the historical main storage node is a main storage node corresponding to a previous storage operation performed on the storage partition; The main storage node is determined from at least one of the storage nodes according to the target business data, the Markov chain and the historical main storage node.

6. The method according to claim 5, characterized in that The determining the primary storage node from at least one of the storage nodes according to the target business data, the Markov chain, and the historical primary storage node comprises: Determine the target transfer probability between the historical main storage node and each storage node according to the real-time resource data corresponding to each storage node and the storage size of the target business data; Based on the target transition probability, updating the Markov chain; The main storage node is determined from at least one of the storage nodes according to the updated Markov chain.

7. The method according to claim 1, characterized in that The determining a primary storage node from at least one storage node according to the target service data and the Markov chain comprises: In the case that the storage partition corresponding to the target business data does not store historical business data, the storage node with the highest initial transfer probability is determined from the Markov chain as the primary storage node.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: The Markov chain is updated.

9. The method according to claim 8, characterized in that The updating of the Markov chain comprises: In the case where there is a faulty node between at least one of the storage nodes, the initial transfer probability from other storage nodes to the faulty node and the initial transfer probability of the faulty node selecting itself are reduced, and the initial transfer probability from the faulty node to other storage nodes is increased, wherein the sum of the initial transfer probability of the faulty node selecting itself and the initial transfer probability between the storage nodes is 1.

10. The method according to claim 8, characterized in that The updating of the Markov chain comprises: The Markov chain is updated based on the resource data of each storage node.

Images