Data storage method, data reading method, device, medium and electronic device

By determining the data storage node based on the first time information and hash code in the storage system, the problem of data migration and shard load uneven when adding the storage node is solved, and dynamic expansion and load balancing of the storage system are realized.

CN113779020BActive Publication Date: 2025-03-18BEIJING WODONG TIANJUN INFORMATION TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202110096331.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-25
Publication Date
2025-03-18
Estimated Expiration
2041-01-25

AI Technical Summary

Technical Problem

When adding new storage nodes, data needs to be migrated, and some shards have high loads.

Method used

By obtaining the first time information in the access data, searching for the corresponding storage configuration, performing hashing operations to obtain the hash code, and determining the target storage node for accessing the data based on the sum of the storage ratios and hash code, realizing dynamic storage and load balancing of the data.

Benefits of technology

It avoids the migration of data when adding new storage nodes, reduces the cost of expanding storage nodes, and evenly distributes the storage amount by adjusting the storage ratio, avoiding the problem of excessive load of some storage nodes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113779020B_ABST
    Figure CN113779020B_ABST
Patent Text Reader

Abstract

The present disclosure relates to the field of computers, and specifically to a data storage method, a data reading method and device, a computer-readable storage medium, and an electronic device. The data storage method includes: obtaining access data, the access data includes first time information and key value information; searching for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and the storage proportion corresponding to each candidate target storage node; performing a hash operation on the key value information to obtain a hash code; determining the storage node of the access data according to the sum of the storage proportions and the hash code; and storing the data information corresponding to the key value information in the storage node. Through the technical solution of the embodiment of the present disclosure, the storage node is determined by the time information and the key value information, and data migration is not required when adding a new storage node.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] With the rapid development of the Internet, the scale of data is getting larger and larger. Take shopping websites as an example. When the number of users reaches one million, the data related to personal users, such as in-site messages and system messages, will reach more than tens of millions.

[0003] Generally speaking, a single storage node cannot store such a large amount of data. Therefore, a distributed system can be used to divide the data into multiple pieces and store multiple pieces of data in the storage node to achieve the purpose of expanding the data scale. In the prior art, range sharding is usually used to divide the data. Range sharding divides the data into continuous ranges determined by key values, which allows efficient query of target data in the continuous range.

[0004] However, when using the above data storage method, in the face of continuous new data, data needs to be migrated when new storage nodes are added, and there are data hotspots in some shards, resulting in high load on some shards.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of the present disclosure is to provide a data storage method, a data reading method, a data storage device, a computer-readable storage medium and an electronic device, which can solve the problem that data needs to be migrated when a new storage node is added and the load of some shards is high.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to a first aspect of the present disclosure, a data storage method is provided, which is applied to a storage system, wherein the storage system includes multiple storage nodes, including: obtaining access data, wherein the access data includes first time information and key value information; searching for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage proportion corresponding to each of the candidate storage nodes; performing a hash operation on the key value information to obtain a hash code; determining a target storage node for the access data according to the sum of the storage proportions and the hash code; and storing data information corresponding to the key value information in the target storage node.

[0009] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, determining the target storage node for accessing the data according to the sum of the storage proportions and the hash code includes: obtaining the sum of the hash code and the storage proportions; using a hash modulus algorithm to perform a modulus operation on the sum of the storage proportions to obtain a hash modulus value; and determining the target storage node for accessing the data according to the hash modulus value.

[0010] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the storage configuration includes a storage weight interval, and determining the target storage node for accessing the data according to the hash modulus value includes: obtaining the position of the hash modulus value in the storage weight interval; and determining the target storage node for accessing the data according to the position.

[0011] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the first time information has a corresponding relationship with the storage configuration. After storing the data information in the target storage node, the method further includes: obtaining the amount of stored data and the storage threshold of the target storage node; and updating the corresponding relationship when the amount of stored data is greater than the storage threshold.

[0012] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, updating the storage configuration when the amount of stored data is greater than the storage threshold includes: obtaining the corresponding relationship when the amount of stored data is greater than the storage threshold; adding second time information and a second storage configuration corresponding to the second time information to the corresponding relationship.

[0013] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the storing of the data information in the target storage node includes: obtaining historical time information corresponding to the historical data information in the storage node; determining whether the difference between the historical time information and the time information is greater than or equal to a preset threshold; deleting the historical data information when the difference is greater than or equal to the preset threshold, and storing the data information in the target storage node; and storing the data information in the target storage node when the difference is less than the preset threshold.

[0014] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, searching for the storage configuration corresponding to the first time information according to the first time information includes: obtaining a time segment routing table, the time segment routing table including the correspondence between the first time information and the storage configuration; searching for the storage configuration corresponding to the first time information in the time segment routing table according to the first time information.

[0015] According to a second aspect of the present disclosure, a data reading method is provided, which is applied to a storage system, wherein the storage system includes multiple storage nodes, and includes: obtaining access data, wherein the access data includes first time information and key value information; searching for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage proportion corresponding to each of the candidate storage nodes; performing a hash operation on the key value information to obtain a hash code; determining a target storage node for the access data according to the sum of the storage proportions and the hash code; and reading data information corresponding to the key value information in the target storage node.

[0016] According to a third aspect of the present disclosure, there is provided a data storage device, which is applied to a storage system, wherein the storage system includes multiple storage nodes, and the device includes: an access data acquisition module, used to acquire access data, wherein the access data includes first time information and key value information; a storage configuration acquisition module, used to search for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage proportion corresponding to each of the candidate storage nodes; a key value hash operation module, used to perform a hash operation on the key value information to obtain a hash code; a storage node determination module, used to determine a target storage node for the access data according to the sum of the storage proportions and the hash code; and a data information storage module, used to store data information corresponding to the key value information in the target storage node.

[0017] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, determining the target storage node for accessing the data according to the sum of the storage proportions and the hash code includes: obtaining the sum of the hash code and the storage proportions; taking the modulus of the sum of the storage proportions using a hash modulus algorithm to obtain a hash modulus value; and determining the target storage node for accessing the data according to the hash modulus value.

[0018] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the storage configuration includes a storage weight interval, and determining the target storage node for accessing the data according to the hash modulus value includes: obtaining the position of the hash modulus value in the storage weight interval; and determining the target storage node for accessing the data according to the position.

[0019] According to a fourth aspect of the present disclosure, there is provided a data reading device, which is applied to a storage system, wherein the storage system includes multiple storage nodes, and the device includes: an access data acquisition module, used to acquire access data, wherein the access data includes first time information and key value information; a storage configuration acquisition module, used to search for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage proportion corresponding to each of the candidate storage nodes; a key value hash operation module, used to perform a hash operation on the key value information to obtain a hash code; a storage node determination module, used to determine a target storage node for the access data according to the sum of the storage proportions and the hash code; and a data information reading module, used to read data information corresponding to the key value information in the target storage node.

[0020] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the data storage method described in the first aspect of the above embodiment is implemented.

[0021] According to a sixth aspect of the present disclosure, there is provided an electronic device, including:

[0022] Processor; and

[0023] A memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the data storage method as described in the first aspect of the above embodiment.

[0024] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0025] In the data storage method provided in one embodiment of the present disclosure, after obtaining the access data, the storage configuration corresponding to the first time information in the access data can be searched, and then the key value information can be hashed to obtain a hash code, and the storage node of the access data can be determined based on the sum of the storage proportions and the hash code, and then the data information corresponding to the key value information is stored in the storage node. In the embodiment of the present disclosure, the storage node is determined based on the time information and the corresponding storage configuration. On the one hand, when adding a new storage node, the old node can take over the new data, so there is no need to migrate the data, thereby reducing the cost of expanding the storage node; on the other hand, the storage situation can be adjusted according to the storage proportion of each storage node, so that the storage capacity of each storage node is more uniform, avoiding the problem of excessive load on some storage nodes.

[0026] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0028] Figure 1 A schematic diagram schematically shows an exemplary system architecture to which a data storage method according to an embodiment of the present disclosure can be applied;

[0029] Figure 2 A flowchart schematically illustrates a data storage method in an exemplary embodiment of the present disclosure;

[0030] Figure 3 Schematically showing a flow chart of searching for storage configuration according to a time segment routing table in an exemplary embodiment of the present disclosure;

[0031] Figure 4 A flowchart schematically illustrates performing a modulus operation on the sum of storage weights to obtain a hash modulus value and determining a target storage node according to the hash modulus value in an exemplary embodiment of the present disclosure;

[0032] Figure 5 A flowchart schematically illustrates determining a target storage node for accessing data according to a location of a hash modulus value in a storage interval in an exemplary embodiment of the present disclosure;

[0033] Figure 6 A flowchart schematically illustrating updating the corresponding relationship when the amount of stored data is greater than the storage threshold in an exemplary embodiment of the present disclosure;

[0034] Figure 7 Schematically illustrating a flow chart of adding second time information and a second storage configuration corresponding to the second time information in the corresponding relationship when the amount of stored data is greater than the storage threshold in an exemplary embodiment of the present disclosure;

[0035] Figure 8 A flowchart schematically illustrates a method of determining whether a difference between the historical time information and the time information is greater than a preset threshold value in an exemplary embodiment of the present disclosure;

[0036] Fig. 9 A flowchart schematically illustrates a data reading method in an exemplary embodiment of the present disclosure;

[0037] Fig.10A schematic diagram schematically shows the composition of a data storage device in an exemplary embodiment of the present disclosure;

[0038] Fig.11 The structure diagram of a computer system suitable for implementing the electronic device of the exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In addition, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure.

[0040] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or these functional entities or parts of functional entities may be implemented in one or more software hardened modules, or these functional entities may be implemented in different networks and / or processor devices and / or microcontroller devices.

[0041] A distributed system uses multiple independent computers to solve storage and computing problems that cannot be handled by a single node. Data sharding divides a data set into independent and orthogonal data subsets according to certain rules, and then distributes the data subsets to different nodes.

[0042] Figure 1 A schematic diagram showing an exemplary system architecture to which the data storage method according to an embodiment of the present disclosure can be applied is shown.

[0043] like Figure 1 As shown, the system architecture 1000 may include one or more of terminal devices 1001, 1002, 1003, a network 1004, and a server 1005. The network 1004 is used to provide a medium for communication links between the terminal devices 1001, 1002, 1003 and the server 1005. The network 1004 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0044] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 1005 may be a server cluster composed of multiple servers.

[0045] Users can use terminal devices 1001, 1002, 1003 to interact with server 1005 through network 1004 to receive or send messages, etc. Terminal devices 1001, 1002, 1003 can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, portable computers, desktop computers, etc. In addition, server 1005 can be a server that provides various services.

[0046] In one embodiment, the execution subject of the data storage method of the present disclosure may be a server 1005, which may obtain access data sent by terminal devices 1001, 1002, and 1003, process the access data according to the data storage method of the present disclosure, and then return the storage result obtained after the processing to the terminal devices 1001, 1002, and 1003. In addition, the data storage method of the present disclosure may also be executed by the terminal devices 1001, 1002, and 1003 to realize the process of searching for a storage node according to the access data and storing the data information corresponding to the key value information in the storage node.

[0047] In addition, the implementation process of the data storage method disclosed in the present invention can also be implemented by the terminal devices 1001, 1002, 1003 and the server 1005. For example, the terminal devices 1001, 1002, 1003 can obtain access data, search for the storage configuration corresponding to the first time information according to the first time information, and then send the obtained storage configuration to the server 1005, so that the server 1005 can perform a hash operation on the key value information to obtain a hash code, and determine the target storage node for accessing the data according to the sum of the storage proportions in the storage configuration and the hash code, and then store the data information corresponding to the key value information in the target storage node.

[0048] The data storage solution disclosed in the present invention can be applied to various business scenarios. For different business scenarios, different numbers of storage nodes can be set according to needs. For example, in network applications, the session data running in the online advertising system, the session service center stores the click and exposure data of the advertisements, etc.; for a certain e-commerce business scenario, due to the increase in the number of users, the data storage solution disclosed in the present invention can be adopted; for another example, for a certain electronic game business scenario, due to the increase in the number of players, the data storage solution disclosed in the present invention can be adopted; for another example, for a certain travel service business scenario, due to the increase in the number of customers, the data storage solution disclosed in the present invention can be adopted. The solution disclosed in the present invention can be applied according to business needs, and the present disclosure does not specifically limit the application scenarios of data storage.

[0049] According to the data storage method provided in this exemplary embodiment, after the access data is obtained, the storage configuration corresponding to the first time information is searched according to the first time information, and then the key value information is hashed to obtain a hash code, and the target storage node for the access data is determined according to the sum of the storage proportions and the hash code, and then the data information corresponding to the key value information is stored in the target storage node. Figure 2 As shown, the data storage method may include the following steps:

[0050] Step S210, obtaining access data, the access data including first time information and key value information;

[0051] Step S220, searching for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage weight corresponding to each candidate storage node;

[0052] Step S230, performing a hash operation on the key value information to obtain a hash code;

[0053] Step S240, determining a target storage node for accessing the data according to the sum of the storage proportions and the hash code;

[0054] Step S250: storing the data information corresponding to the key value information in the target storage node.

[0055] In the data storage method provided in this example implementation, the storage configuration corresponding to the first time information can be searched based on the first time information in the access data, and then the key value information is hashed to obtain a hash code, and the storage node of the access data is determined based on the sum of the storage proportions and the hash code, and then the data information corresponding to the key value information is stored in the storage node. In the embodiment of the present disclosure, the storage node is determined based on the time information and the corresponding storage configuration. On the one hand, when a new storage node is added, the old node can take over the new data, so there is no need to migrate the data, thereby reducing the cost of expanding the storage node; on the other hand, the storage situation can be adjusted according to the storage proportion of each storage node, so that the storage capacity of each storage node is more uniform, avoiding the problem of excessive load on some storage nodes.

[0056] Next, the data storage steps S210 to S250 in this exemplary embodiment will be described in more detail with reference to the accompanying drawings and embodiments.

[0057] Step S210, obtaining access data, the access data including first time information and key value information;

[0058] In an exemplary embodiment of the present disclosure, when a user needs to store certain data in a corresponding storage node, access data may be input. Specifically, the access data may include first time information and key value information. The first time information is the time corresponding to the data to be input. The first time information may include the initial time when the data is generated, the end time when the data is generated, and the time during which the data is generated. The present disclosure does not specifically limit the setting standard of the first time information.

[0059] In an exemplary embodiment of the present disclosure, key-value information can be used to represent data. In other words, data can be searched based on the key-value information, which can be a part of the above data or derived from the data. Specifically, the key-value information can be a certain item in the data. For example, the data can include multiple users, and the user's ID number can be used as the key-value information; or, it can be a combination of other information in the data. For example, the data can include multiple users, and the users have user names and user ages. In this case, the combination of the user's name and age can be used as the key-value information. It should be noted that the present disclosure does not specifically limit the representation form of the key-value information.

[0060] For example, when a user creates an account on a shopping website, the creation completion time can be used as the first time information, and one of the account data (name, age, gender, phone number, account ID, etc.) can be selected as the key value information. At this time, the first time information and the key value information can be combined as the access data.

[0061] Step S220, searching for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage weight corresponding to each candidate storage node;

[0062] In an exemplary embodiment of the present disclosure, a storage configuration corresponding to the first time information may be set in advance, and the storage configuration may be set in a terminal device or a server. When the storage configuration is needed, it may be retrieved from the terminal device or the server. Specifically, a correspondence table between the first time information and the storage configuration corresponding to the first time information may be set in advance, and the correspondence table between the first time information and the storage configuration corresponding to the first time information includes the correspondence between the first time information and the storage configuration; the storage configuration corresponding to the first time information may also be searched based on the first time information, and the first time information may include a time index, and the storage configuration corresponding to the first time information may be searched based on the time index; the first time information may also be converted in form, and the storage configuration corresponding to the result may be searched based on the converted result.

[0063] In an exemplary embodiment of the present disclosure, a storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage weight corresponding to each candidate storage node. Specifically, the first time information may correspond to one or more candidate storage nodes, each storage node having a different storage weight, which represents the storage weight of the data in the candidate storage node between every two similar time information. For example, assuming that the first time information is T1, the storage configuration corresponding to the first time information is that the candidate storage nodes are n1, n2...nk, and the corresponding storage weights are w1, w2...wk. This storage configuration indicates that during the time period from T0 to T1, the acquired data is stored in storage nodes n1, n2...nk with storage weights of w1, w2...wk, respectively.

[0064] In an exemplary embodiment of the present disclosure, the storage ratio corresponding to the storage node only represents a certain

[0065] The storage status of the data acquired in a period of time in different storage nodes. That is, when storing data according to the storage configuration, the node may still have data stored according to the storage configuration corresponding to the previous time information. For example, the storage configuration corresponding to T1 is n1.25%; n2.25%; n3.50%, and the storage configuration corresponding to T2 is n1.25%; n2.25%; n3.25%; n4.25%. At T1, the data from T0 to T1 are stored in n1, n2, and n3 according to the storage configuration of n1.25%; n2.25%; n3.50%. At T2, the data from T1 to T2 are stored in n1, n2, n3, and n4 according to the storage configuration of n1.25%; n2.25%; n3.25%; n4.25%. That is, at T2, data has been stored in n1, n2, and n3.

[0066] Furthermore, the storage proportions corresponding to the same storage node in the storage configurations corresponding to different time information can be the same or different. For example, the storage proportion of n1 in the storage configuration corresponding to T1 is 25%, and the storage proportion of n1 in the storage configuration corresponding to T2 can be any proportion, for example, it can be set to 25%, it can also be set to 30%, and it can also be set to 50%. It can be adjusted in time according to the storage situation of the data in each node in the previous time period. For example, in the time period from T1 to T2, the amount of data stored in n2 is small. Therefore, when setting the storage configuration corresponding to T3, the storage proportion of n2 in the storage configuration corresponding to T3 can be increased a little, so that the data storage situation in each storage node is roughly the same.

[0067] In an exemplary embodiment of the present disclosure, a time segment routing table may be obtained, and a storage configuration corresponding to the first time information may be searched in the time segment routing table according to the first time information. Figure 3 As shown, searching for storage configuration according to the time segment routing table may include the following steps S310-S320:

[0068] Step S310, obtaining a time segment routing table, where the time segment routing table includes a correspondence between the first time information and the storage configuration;

[0069] In an exemplary embodiment of the present disclosure, a time segment routing table may be set in advance, and the time segment routing table may include a correspondence between the first time information and the storage configuration. Specifically, the time segment routing table may be stored in a terminal device or a server, and when it is necessary to search for a storage configuration corresponding to a time according to the time segment routing table, the time segment routing table may be retrieved in the terminal device or the server.

[0070] Step S320: searching a time segment routing table for a storage configuration corresponding to the first time information according to the first time information.

[0071] In an exemplary embodiment of the present disclosure, after obtaining the time segment routing table, the storage configuration corresponding to the first time information can be searched in the time segment routing table. The time segment routing table includes the correspondence between the first time information and the storage configuration. For example, when the first time information is T1, the storage configuration corresponding to T1 in the time segment routing table is n1.25%; n2.25%; n3.50%. It should be noted that the present disclosure does not specifically limit the form of the time segment routing table, as long as the corresponding storage configuration can be found in the time segment routing table according to the first time information. The time segment routing table is shown in the following table, where w is the storage proportion:

[0072] time Storage Configuration T0 {n1.w1, n2.w2} T1 {n1.w3, n2.w4, n3.w5} … … ∞ {n1.w6, n2.w7…nk.wu}

[0073] Through the above steps S310 to S320, the time segment routing table can be obtained, and the storage configuration corresponding to the first time information can be searched in the time segment routing table according to the first time information.

[0074] Step S230, performing a hash operation on the key value information to obtain a hash code;

[0075] In an exemplary embodiment of the present disclosure, the key value information obtained in the above steps can be hashed to obtain a hash code. Specifically, the hash operation is to accept an input of unlimited length and return an output of fixed length. The hash algorithm can convert a data into a symbol, which has a very close relationship with the source data. The hash algorithm is a generalized algorithm and can also be considered as an idea. The use of the hash algorithm can improve the utilization of storage space, improve the efficiency of data query, and can also make digital signatures to ensure the security of data transmission. Therefore, the hash algorithm is widely used in Internet applications.

[0076] For example, when the key value information input above is the name Zhang San, the obtained hash code can be 86, and when the key value information input above is the name Li Si, the obtained hash code can be 87. Hash operations include a variety of algorithms, such as direct addressing method, digital analysis method, square middle method, random number method, etc. Commonly used hash algorithms include MD4, MD5, SHA-1, etc. It should be noted that the present disclosure does not specifically limit the specific method of hash operation, as long as the hash code can be obtained according to the key value information.

[0077] Step S240, determining a target storage node for accessing the data according to the sum of the storage proportions and the hash code;

[0078] In an example embodiment of the present disclosure, after the storage configuration corresponding to the above-mentioned first time information is obtained, the sum of the storage proportions in the storage configuration can be calculated. For example, the storage configuration corresponding to T1 is n1.25%; n2.25%; n3.50%. At this time, the sum of the storage ratios can be calculated to be 100%. And the target storage node for accessing the data is calculated based on the sum of the storage proportions obtained in the above steps and the hash code. Specifically, the target storage node for accessing the data can be determined based on the corresponding relationship between the sum of the storage proportions and the hash code.

[0079] In an exemplary embodiment of the present disclosure, a modulus operation may be performed on the sum of the storage proportions obtained in the above steps to obtain a hash modulus value, and a target storage node for accessing data may be determined according to the hash modulus value. Figure 4 As shown, performing a modulus operation on the sum of the storage proportions to obtain a hash modulus value, and determining a target storage node according to the hash modulus value, includes the following steps S410-S420:

[0080] Step S410, obtaining the sum of the hash code and the storage ratio;

[0081] In an exemplary embodiment of the present disclosure, after obtaining the storage configuration corresponding to the first time information, the sum of the storage proportions in the storage configuration can be calculated. For example, the storage configuration corresponding to T2 is n1.25%; n2.25%; n3.25%; n4.25%. At this time, the sum of the storage proportions can be calculated to be 100%. And obtain the hash code after the hash operation in the above steps.

[0082] Step S420, using a hash modulus algorithm to perform a modulus operation on the sum of the stored proportions to obtain a hash modulus value;

[0083] In an exemplary embodiment of the present disclosure, the sum of the storage weights can be modulo-operated according to a hash modulo algorithm to obtain a hash modulo value. Specifically, the hash modulo algorithm can be a division and remainder method, and the remainder obtained after the keyword is divided by a number p that is not greater than the hash table length m is taken as the hash address. That is, H(key)=key MOD p, p<=m. Not only can the keyword be modulo-operated directly, but it can also be modulo-operated after folding, squaring, etc. In the scheme of the present disclosure, the sum of the storage weights can be taken as p, and the hash code can be used as the key to perform a modulo operation to obtain a hash modulo value. For example, when the obtained hash code is 86, the sum of the storage weights obtained is 100, then H(key)=86mod 100=86, and the hash modulo value at this time is 86.

[0084] Step S430, determining a target storage node for accessing data according to the hash modulus value.

[0085] In an exemplary embodiment of the present disclosure, after obtaining the hash modulus value obtained in the above steps, the target storage node for accessing the data can be determined according to the hash modulus value. Specifically, each storage node corresponds to a different storage interval, and the storage interval where the hash modulus value is located can be determined, and the target storage node can be determined according to the storage interval of the hash modulus value.

[0086] Through the above steps S410 to S430, a modulus operation can be performed on the sum of the storage proportions obtained in the above steps to obtain a hash modulus value, and a target storage node for accessing data can be determined according to the hash modulus value.

[0087] In an exemplary embodiment of the present disclosure, the position of the hash modulus value in the storage weight interval can be obtained, and the target storage node for accessing the data can be determined according to the position. Figure 5 As shown, determining the target storage node for accessing data according to the position of the hash modulus value in the storage interval may include the following steps S510-S520:

[0088] Step S510: obtaining the position of the hash modulus value in the storage weight interval;

[0089] In an exemplary embodiment of the present disclosure, after obtaining the hash modulus value obtained in the above steps, the storage weight interval of the storage configuration can be obtained. For example, the storage configuration corresponding to T2 is n1.25%; n2.25%; n3.25%; n4.25%, then the storage weight interval at this time is 1-25, 26-50, 51-75, 76-100, assuming that the hash modulus value obtained is 86, then 86 is within the storage weight interval of 76-100.

[0090] Step S520: Determine the target storage node for accessing data according to the location.

[0091] In an exemplary embodiment of the present disclosure, after obtaining the hash modulus value obtained in the above steps and the position of the hash modulus value in the storage weight interval, the target storage node for accessing the data can be determined based on the position. For example, the storage configuration corresponding to T3 is n1.20%; n2.20%; n3.20%; n4.20%; n5.20%, then the storage weight intervals at this time are 1-20, 21-40, 41-60, 61-80, 81-100. Assuming that the obtained hash modulus value is 86, then 86 is within the storage weight interval of 81-100, and the storage node corresponding to 81-100 is n5. Then the target storage node corresponding to the accessed data at this time is n5. As shown in the following table, the target storage node can be determined based on the storage weight interval:

[0092] Storage ratio range Target storage node w1 n1 w2 n2 … … wk n

[0093] Through the above steps S510 to S520, the position of the hash modulus value in the storage weight interval can be obtained, and the target storage node for accessing the data can be determined according to the position.

[0094] Step S250: storing the data information corresponding to the key value information in the target storage node.

[0095] In an exemplary embodiment of the present disclosure, after obtaining the target storage node obtained in the above steps, the data information corresponding to the key-value information can be stored in the target storage node. Specifically, the data information corresponding to the key-value information can be obtained according to the key-value information, and the data information can be stored in the target storage node. For example, in an online shopping business scenario, the key-value information is the user id. At this time, the user id also corresponds to all the data information of this account (such as name, phone number, age, order record, message information, etc.). All the data information corresponding to the user id can be obtained according to the key-value information, and the data information can be stored in the target storage node.

[0096] In an exemplary embodiment of the present disclosure, after storing data information in the target storage node, the amount of stored data and the storage threshold of the target storage node can be obtained, and the corresponding relationship is updated when the amount of stored data is greater than the storage threshold. Figure 6 As shown, when the amount of stored data is greater than the storage threshold, updating the corresponding relationship may include the following steps S610-S620:

[0097] Step S610, obtaining the amount of stored data and the storage threshold of the target storage node;

[0098] In an exemplary embodiment of the present disclosure, the amount of data stored in the target storage node can be obtained. Specifically, all data in the target storage node can be traversed, and the sum of all data can be calculated, and the sum of all data can be used as the amount of stored data, or the storage node can calculate the amount of stored data based on historical storage records, or the amount of stored data can be directly obtained.

[0099] In an exemplary embodiment of the present disclosure, the storage threshold of the target storage node can also be obtained. The storage threshold can include a specific value or a ratio threshold of the storage node. Specifically, the storage threshold of the target storage node can be set in a terminal device or a server. When the storage threshold needs to be obtained, the storage threshold can be obtained in the terminal device or the server. For example, the storage threshold can be a specific value of 100G or a ratio threshold of 50% of the storage node.

[0100] Step S620: When the amount of stored data is greater than the storage threshold, the corresponding relationship is updated.

[0101] In an exemplary embodiment of the present disclosure, there is a corresponding relationship between the first time information and the storage configuration, and the first time information can search for the storage configuration according to the corresponding relationship, the storage configuration and the one or more candidate storage nodes corresponding to the first time information, and the storage proportion corresponding to each candidate storage node. When the amount of stored data is greater than the storage threshold, the above-mentioned corresponding relationship can be updated, a new corresponding relationship can be added, or the original corresponding relationship can be adjusted.

[0102] For example, assuming that the storage threshold is 100 GB, the amount of data stored in the current storage node is 120 GB, that is, the amount of stored data is greater than the storage threshold. At this time, the corresponding relationship between the first time information and the storage configuration can be updated, and the storage proportion of the storage node can be adjusted so that the amount of data in the current storage node does not increase.

[0103] Through the above steps S610 to S620, the amount of stored data and the storage threshold of the target storage node can be obtained. When the amount of stored data is greater than the storage threshold, the corresponding relationship is updated, and the storage status of multiple storage nodes can be controlled in time.

[0104] In an exemplary embodiment of the present disclosure, when the amount of stored data is greater than a storage threshold, a corresponding relationship is acquired, and second time information and a second storage configuration corresponding to the second time information are added to the corresponding relationship. Figure 7 As shown, when the amount of stored data is greater than the storage threshold, adding the second time information and the second storage configuration corresponding to the second time information in the corresponding relationship may include the following steps S710-S720:

[0105] Step S710, when the amount of stored data is greater than the storage threshold, obtaining a corresponding relationship;

[0106] In an example embodiment of the present disclosure, after the amount of stored data and the storage threshold are obtained, it can be determined whether the amount of stored data is greater than the storage threshold. When the amount of stored data is greater than the storage threshold, the correspondence between the first time information and the storage configuration is obtained. The first time information can be used to search for the storage configuration based on the correspondence, and the storage configuration and one or more candidate storage nodes corresponding to the first time information and the storage proportion corresponding to each candidate storage node.

[0107] Step S720: Add the second time information and the second storage configuration corresponding to the second time information to the corresponding relationship.

[0108] In an exemplary embodiment of the present disclosure, the second time information and the second storage configuration corresponding to the second time information may be added to the corresponding relationship obtained in the above steps. Specifically, the second time information and the second storage configuration corresponding to the second time information may be added to the corresponding relationship, and the second storage configuration may include the old storage node and the new storage node and the storage proportions corresponding to these storage nodes.

[0109] For example, the storage configuration corresponding to T2 is n1.25%; n2.25%; n3.25%;

[0110] n4.25%, the amount of stored data of storage node n2 is 120GB, the storage threshold is 100GB, and the amount of stored data is greater than the storage threshold. At this time, T3 and the storage configuration corresponding to T3 can be added to the corresponding relationship as n1.20%; n2.10%; n3.20%; n4.20%; n5.30%.

[0111] Through the above steps S710 to S720, when the amount of stored data is greater than the storage threshold, the corresponding relationship is obtained, and the second time information and the second storage configuration corresponding to the second time information are added to the corresponding relationship. When the capacity of the storage node is insufficient to accommodate the current data, a new storage node can be added and the storage proportion can be reallocated, which facilitates capacity expansion without the need to migrate the original data.

[0112] In an exemplary embodiment of the present disclosure, the historical time information corresponding to the historical data information in the storage node can be obtained, and it is determined whether the difference between the historical time information and the time information is greater than a preset threshold. When the difference is greater than or equal to the preset threshold, the historical data information is deleted and the data information is stored in the target storage node. When the difference is less than the preset threshold, the data information is stored in the target storage node. Figure 8 As shown, judging whether the difference between the historical time information and the time information is greater than a preset threshold may include the following steps S810 to S840:

[0113] Step S810, obtaining historical time information corresponding to historical data information in the storage node;

[0114] In an exemplary embodiment of the present disclosure, after determining the target storage node according to the above steps, the historical data information stored in the target storage node can be obtained. Specifically, the historical data information is the data information that has been stored before the current data information is stored, and the historical data information is stored in the target storage node, and the historical data information corresponds to the historical time information of the historical data information being stored in the target storage node.

[0115] Step S820, determining whether the difference between the historical time information and the time information is greater than a preset threshold;

[0116] In an example embodiment of the present disclosure, a preset threshold value can be set in advance, and the preset threshold value is a time period. The preset threshold value can be stored in a terminal device or a server, and when the preset threshold value needs to be called, it can be retrieved in the terminal device or the server. Specifically, after obtaining the historical time information and the time information, the historical time information and the time information can be compared, and the difference between the historical time information and the time information can be obtained, and then it is determined whether the difference between the historical time information and the time information is greater than the preset threshold value. It should be noted that the difference between the historical time information and the time information can represent the length of time that the historical data information is stored in the target storage node.

[0117] Step S830, when the difference is greater than or equal to a preset threshold, the historical data information is deleted, and the data information is stored in the target storage node;

[0118] In an exemplary embodiment of the present disclosure, when it is determined according to the above steps that the difference between the historical time information and the time information is greater than or equal to a preset threshold, it can be indicated that the historical data information is data information that exceeds the preset storage time, and the preset storage time is the storage time that the user expects the historical data information to be stored in the storage node. The historical data information may include expired data information, redundant data information, etc., and may also include data information that is useless to the user. At this time, the historical data information in the target storage node can be deleted, and the current data information can be stored in the target storage node.

[0119] Step S840: When the difference is less than a preset threshold, the data information is stored in the target storage node.

[0120] In an exemplary embodiment of the present disclosure, when it is determined according to the above steps that the difference between the historical time information and the time information is less than a preset threshold, it can be indicated that the historical data information is data information that has not exceeded the preset storage time. Therefore, the current data information can be directly stored in the target node.

[0121] Through the above steps S810 to S840, the historical time information corresponding to the historical data information in the storage node can be obtained, and it is determined whether the difference between the historical time information and the time information is greater than the preset threshold. When the difference is greater than or equal to the preset threshold, the historical data information is deleted and the data information is stored in the target storage node. When the difference is less than the preset threshold, the data information is stored in the target storage node. The data in the storage node can be updated in time, so that the storage node can delete the historical data information and take over the new data information.

[0122] In an example embodiment of the present disclosure, storage nodes are determined based on time information and corresponding storage configurations. On the one hand, when adding new storage nodes, old nodes can take over new data, so there is no need to migrate data, thereby reducing the cost of expanding storage nodes; on the other hand, the storage situation can be adjusted according to the storage proportion of each storage node, so that the storage capacity of each storage node is more uniform, avoiding the problem of excessive load on some storage nodes.

[0123] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0124] In an exemplary embodiment of the present disclosure, a data reading method is also provided. Fig. 9 As shown, the data reading method may include the following steps:

[0125] Step S910, obtaining access data, the access data including first time information and key value information;

[0126] Step S920, searching for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage proportion corresponding to each candidate storage node;

[0127] Step S930, performing a hash operation on the key value information to obtain a hash code;

[0128] Step S940, determining a target storage node for accessing the data according to the sum of the storage proportions and the hash code;

[0129] Step S950: Read data information corresponding to the key value information in the target storage node.

[0130] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the target storage node for accessing data is determined according to the sum of the storage proportions and the hash code, including: obtaining the sum of the hash code and the storage proportion; taking the modulus of the sum of the storage proportions using a hash modulus algorithm to obtain a hash modulus value; and determining the target storage node for accessing data according to the hash modulus value.

[0131] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the storage configuration includes a storage weight interval, and determining the target storage node for accessing data according to the hash modulus value, including: obtaining the position of the hash modulus value in the storage weight interval; determining the target storage node for accessing data according to the position.

[0132] Since the steps of the data reading method of the example embodiment of the present disclosure correspond to the steps of the example embodiment of the data storage method described above, for details not disclosed in the data reading method embodiment of the present disclosure, please refer to the above-mentioned embodiment of the data storage method of the present disclosure.

[0133] In addition, in an exemplary embodiment of the present disclosure, a data storage device is also provided. Fig.10 As shown, a data storage device 1000 includes: an access data acquisition module 1010, a storage configuration acquisition module 1020, a key value hash operation module 1030, a storage node determination module 1040, and a data information storage module 1050.

[0134] Among them, an access data acquisition module is used to obtain access data, and the access data includes first time information and key value information; a storage configuration acquisition module is used to search for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and the storage proportion corresponding to each candidate storage node; a key value hash operation module is used to perform a hash operation on the key value information to obtain a hash code; a storage node determination module is used to determine a target storage node for access data according to the sum of the storage proportions and the hash code; a data information storage module is used to store the data information corresponding to the key value information in the target storage node.

[0135] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the target storage node for accessing data is determined according to the sum of the storage proportions and the hash code, including: obtaining the sum of the hash code and the storage proportion; using a hash modulus algorithm to perform a modulus operation on the sum of the storage proportions to obtain a hash modulus value; and determining the target storage node for accessing data according to the hash modulus value.

[0136] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the storage configuration includes a storage weight interval, and determining the target storage node for accessing data according to the hash modulus value, including: obtaining the position of the hash modulus value in the storage weight interval; determining the target storage node for accessing data according to the position.

[0137] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the first time information has a corresponding relationship with the storage configuration. After storing the data information in the target storage node, the method also includes: obtaining the amount of stored data and the storage threshold of the target storage node; and updating the corresponding relationship when the amount of stored data is greater than the storage threshold.

[0138] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the storage configuration is updated when the amount of stored data is greater than the storage threshold, including: obtaining a corresponding relationship when the amount of stored data is greater than the storage threshold; adding second time information and a second storage configuration corresponding to the second time information in the corresponding relationship.

[0139] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, data information is stored in a target storage node, including: obtaining historical time information corresponding to the historical data information in the storage node; determining whether the difference between the historical time information and the time information is greater than or equal to a preset threshold; deleting the historical data information when the difference is greater than or equal to the preset threshold, and storing the data information in the target storage node; and storing the data information in the target storage node when the difference is less than the preset threshold.

[0140] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, searching for a storage configuration corresponding to the first time information according to the first time information includes: obtaining a time segment routing table, the time segment routing table including the correspondence between the first time information and the storage configuration; searching for a storage configuration corresponding to the first time information in the time segment routing table according to the first time information.

[0141] Since the various functional modules of the data storage device of the example embodiment of the present disclosure correspond to the steps of the example embodiment of the data storage method described above, for details not disclosed in the embodiment of the device of the present disclosure, please refer to the embodiment of the data storage method described above in the present disclosure.

[0142] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0143] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above data storage method is also provided.

[0144] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware embodiments, complete software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0145] Refer to the following Fig.111100 according to this embodiment of the present disclosure is described. Fig.11 The electronic device 1100 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0146] like Fig.11 As shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 may include, but are not limited to: the at least one processing unit 1110, the at least one storage unit 1120, a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110), and a display unit 1140.

[0147] The storage unit stores program codes, which can be executed by the processing unit 1110, so that the processing unit 1110 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 1110 can execute the following steps: Figure 2 In step S220 shown in the figure, a storage configuration corresponding to the first time information is searched according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage proportion corresponding to each candidate storage node; in step S230, a hash operation is performed on the key value information to obtain a hash code; in step S240, a target storage node for accessing data is determined according to the sum of the storage proportions and the hash code; in step S250, data information corresponding to the key value information is stored in the target storage node.

[0148] For example, electronic devices can achieve Figure 2 The steps shown.

[0149] The storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1121 and / or a cache memory unit 1122 , and may further include a read-only memory unit (ROM) 1123 .

[0150] The storage unit 1120 may also include a program / utility 1124 having a set (at least one) of program modules 1125, such program modules 1125 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0151] Bus 1130 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0152] The electronic device 1100 may also communicate with one or more external devices 1170 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1100, and / or communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1150. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 1160. As shown, the network adapter 1160 communicates with other modules of the electronic device 1100 via a bus 1130. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0153] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0154] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0155] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0156] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0157] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0158] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0159] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A data storage method, characterized in that: Applied to a storage system, the storage system includes multiple storage nodes, including: Acquire access data, wherein the access data includes first time information and key value information; Searching for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage proportion corresponding to each of the candidate storage nodes; Performing a hash operation on the key value information to obtain a hash code; Obtaining the sum of the hash code and the storage proportion, performing a modulus operation on the sum of the storage proportion using a hash modulus algorithm to obtain a hash modulus value, and determining a target storage node for accessing the data according to the hash modulus value; The data information corresponding to the key value information is stored in the target storage node.

2. The method according to claim 1, characterized in that The storage configuration includes a storage weight interval, and determining a target storage node for accessing data according to the hash modulus value includes: Obtaining the position of the hash modulus value in the storage weight interval; A target storage node for accessing the data is determined according to the location.

3. The method according to claim 1, characterized in that The first time information has a corresponding relationship with the storage configuration. After storing the data information in the target storage node, the method further includes: Obtaining the amount of stored data and the storage threshold of the target storage node; When the amount of stored data is greater than the storage threshold, the corresponding relationship is updated.

4. The method according to claim 3, characterized in that When the amount of stored data is greater than the storage threshold, updating the storage configuration includes: When the amount of stored data is greater than the storage threshold, acquiring the corresponding relationship; The second time information and the second storage configuration corresponding to the second time information are added to the corresponding relationship.

5. The method according to claim 1, characterized in that The storing the data information in the target storage node includes: Obtaining historical time information corresponding to the historical data information in the storage node; Determine whether the difference between the historical time information and the time information is greater than or equal to a preset threshold; When the difference is greater than or equal to a preset threshold, the historical data information is deleted, and the data information is stored in the target storage node; When the difference is less than a preset threshold, the data information is stored in the target storage node.

6. The method according to claim 1, characterized in that The searching, according to the first time information, for a storage configuration corresponding to the first time information includes: Acquire a time segment routing table, wherein the time segment routing table includes a correspondence between the first time information and the storage configuration; According to the first time information, a storage configuration corresponding to the first time information is searched in a time segment routing table.

7. A data reading method, characterized in that: Applied to a storage system, the storage system includes multiple storage nodes, including: Acquire access data, wherein the access data includes first time information and key value information; Searching for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage proportion corresponding to each of the candidate storage nodes; Performing a hash operation on the key value information to obtain a hash code; Obtaining the sum of the hash code and the storage proportion, taking a modulus of the sum of the storage proportions using a hash modulus algorithm to obtain a hash modulus value, and determining a target storage node for accessing the data according to the hash modulus value; The data information corresponding to the key value information is read in the target storage node.

8. The method according to claim 7, characterized in that The storage configuration includes a storage weight interval, and determining a target storage node for accessing data according to the hash modulus value includes: Obtaining the position of the hash modulus value in the storage weight interval; A target storage node for accessing the data is determined according to the location.

9. A data storage device, characterized in that: Applied to a storage system, the storage system includes a plurality of storage nodes, and the device includes: An access data acquisition module, used to acquire access data, wherein the access data includes first time information and key value information; A storage configuration acquisition module, configured to search for a storage configuration corresponding to the first time information according to the first time information; wherein the storage configuration includes one or more candidate storage nodes corresponding to the first time information and a storage weight corresponding to each of the candidate storage nodes; A key value hash operation module, used for performing a hash operation on the key value information to obtain a hash code; A storage node determination module, used to obtain the sum of the hash code and the storage proportion, perform a modulus operation on the sum of the storage proportion using a hash modulus algorithm to obtain a hash modulus value, and determine a target storage node for accessing the data according to the hash modulus value; The data information storage module is used to store the data information corresponding to the key value information in the target storage node.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. An electronic device, characterized in that: include: processor; as well as A memory, used to store one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Data storage method and terminal equipment

    CN110209666A

  • Data processing method and device, equipment and storage medium

    CN110750529A

  • Client-based distributed storage load balancing method and device

    CN111475282A