Data storage method and device, computer equipment and readable storage medium

Abstract

The invention relates to a data storage method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring to-be-stored data, and determining a writing frequency of a data label corresponding to the to-be-stored data; determining an allocation priority for the to-be-stored data based on the write-in frequency and the capacity grade to which each candidate storage copy group belongs; and determining a target storage sub-book group from the candidate storage sub-book groups according to the allocation priority, and storing the to-be-stored data into the target storage sub-book group. By adopting the method, the storage space waste can be reduced while the data read-write operation efficiency is ensured.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data storage method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the advent of the big data era, the amount of data is growing explosively, and how to effectively store and access this data has become an urgent problem to be solved. Storage technology is one of the core technologies in the current cloud computing field. Users upload files to the cloud, and use reliable storage technology to persist them in the cloud, and provide users with a service of viewing and retrieving them, saving users the cost of purchasing storage devices and equipment maintenance.

[0003] However, current data storage methods cannot effectively balance the efficiency of data read and write operations and the efficient use of storage space when writing data into a storage replica group for storage. Summary of the invention

[0004] Based on this, it is necessary to provide a data storage method, device, computer equipment, computer-readable storage medium and computer program product that can reduce storage space waste while ensuring the efficiency of data reading and writing operations in order to address the above technical problems.

[0005] In a first aspect, the present application provides a data storage method, comprising:

[0006] Obtain the data to be stored, and determine the writing frequency of the data tag corresponding to the data to be stored;

[0007] Determine the allocation priority for the data to be stored based on the write frequency and the capacity classification of each candidate storage replica group;

[0008] A target storage replica group is determined from each candidate storage replica group according to the allocation priority, and the data to be stored is stored in the target storage replica group.

[0009] In a second aspect, the present application also provides a data storage device, comprising:

[0010] A writing frequency determination module is used to obtain the data to be stored and determine the writing frequency of the data tag corresponding to the data to be stored;

[0011] An allocation priority determination module, used to determine the allocation priority for the data to be stored based on the write frequency and the capacity classification to which each candidate storage replica group belongs;

[0012] The allocation processing module is used to determine the target storage replica group from each candidate storage replica group according to the allocation priority, and store the data to be stored in the target storage replica group.

[0013] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above data storage method are implemented.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above data storage method are implemented.

[0015] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the above data storage method are implemented.

[0016] The above-mentioned data storage method, apparatus, computer equipment, computer-readable storage medium and computer program product determine the write frequency of the data tag corresponding to the data to be stored, determine the allocation priority for the data to be stored based on the write frequency and the capacity classification to which each candidate storage replica group belongs, determine the target storage replica group from each candidate storage replica group according to the allocation priority, and store the data to be stored in the target storage replica group. During the data storage processing, the allocation priority is determined in combination with the write frequency of the data tag corresponding to the data to be stored and the capacity classification to which each candidate storage replica group belongs, and the target storage replica group is determined for data storage according to the allocation priority. The corresponding storage replica group can be adaptively allocated for storage of the data to be stored, ensuring the degree of aggregation during data storage, and can effectively balance the efficiency of data read and write operations and the efficient use of storage space, thereby reducing storage space waste while ensuring the efficiency of data read and write operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 An application environment diagram of a data storage method in one embodiment;

[0019] Figure 2 A schematic diagram of a flow chart of a data storage method in one embodiment;

[0020] Figure 3 A schematic diagram of a process for creating a storage replica group in one embodiment;

[0021] Figure 4 A schematic diagram of a process of adaptively allocating cg in one embodiment;

[0022] Figure 5 A schematic diagram of a process for adaptively creating a supplementary CG in one embodiment;

[0023] Figure 6 is a structural block diagram of a data storage device in one embodiment;

[0024] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0026] The data storage method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network, and the data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The user can send the data to be stored to the server 104 through the terminal 102. After the server 104 obtains the data to be stored, it can determine the write frequency of the data tag corresponding to the data to be stored. The server 104 can determine the allocation priority for the data to be stored based on the write frequency and the capacity classification of each candidate storage replica group, and determine the target storage replica group from each candidate storage replica group according to the allocation priority. The server 104 can then store the data to be stored in the target storage replica group, thereby realizing the storage processing of the data to be stored. In some embodiments, the server 104 can also return the storage result of the data to be stored to the terminal 102.

[0027] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0028] In an exemplary embodiment, Figure 2 As shown, a data storage method is provided, which is applied to Figure 1 The server in the example is used to illustrate, including the following steps 202 to 206. Among them:

[0029] Step 202, obtaining data to be stored, and determining the writing frequency of the data tag corresponding to the data to be stored.

[0030] The data to be stored refers to the data that needs to be stored, and specifically may include various business data generated in the business system, such as test scores, performance data, document data, and other business data. Data tags are used to identify data, and different data may have different data tags, so that each data can be distinguished by the data tags. The write frequency is the frequency of storage operations for data with the same data tag, and specifically may include the frequency of writing data with the same data tag to the storage replica group within a preset time range. The write frequency can be used to characterize the storage requirements of the data corresponding to the data tag. The higher the write frequency, the more frequent the storage operations for the corresponding data, and the stronger the storage requirements for the data.

[0031] Exemplarily, the server may obtain data to be stored, specifically, the data to be stored that needs to be stored and processed from the terminal. The server may determine the data tag of the data to be stored, and determine the write frequency corresponding to the data to be stored. In some embodiments, the server may count the write frequency of the data tag corresponding to each data to determine the write frequency of the corresponding data tag, so as to allocate the storage replica group according to the write frequency.

[0032] Step 204 : Determine the allocation priority for the data to be stored based on the write frequency and the capacity classification to which each candidate storage replica group belongs.

[0033] Among them, a storage copy group (chunkGroup, cg) is a storage unit composed of a group of storage copies. For example, when a chunkgroup with three copies stores data, three copies of the same data will be stored on each storage copy of the storage unit. For example, in the EC (Erasure Code) storage method, data shards and verification data will be stored on each storage copy, and the storage copies are scattered on different disks. A storage copy group is a storage strategy that stores multiple copies of data in different physical locations or storage devices to improve data reliability and fault tolerance. These copies are logically regarded as a whole and together constitute a storage copy group. Each copy in a storage copy group is a complete copy of the original data. These copies may be physically located in different storage devices, servers, or data centers. A candidate storage copy group is a storage copy group that has been created. The server can allocate storage copy groups from each candidate storage copy group to store the data to be stored.

[0034] Capacity grading is the grading of used capacity in candidate storage replica groups, that is, for each candidate storage replica group, each candidate storage replica group can be divided according to the used capacity of each candidate storage replica group to obtain different capacity grading. Through capacity grading, the amount of used capacity in the corresponding candidate storage replica group can be reflected. For example, capacity grading can include three levels: high, medium and low, where the high level indicates that the used capacity in the corresponding candidate storage replica group is high and the remaining storage space is small; the low level indicates that the used capacity in the corresponding candidate storage replica group is low and the remaining storage space is large. Allocation priority is the priority of assigning candidate storage replica groups to the data to be stored. The allocation priority is determined based on the write frequency of the data tag corresponding to the data to be stored and the capacity grading of each candidate storage replica group. Therefore, different allocation priorities can be adaptively determined for different data to be stored to balance the efficiency of data read and write operations and the efficient use of storage space.

[0035] Optionally, the server may determine each candidate storage replica group that has been created, and determine the capacity classification to which each candidate storage replica group belongs. In some embodiments, the server may pre-classify each candidate storage replica group into capacity levels to determine the capacity classification to which each candidate storage replica group belongs. The server may determine the allocation priority for the data to be stored based on the write frequency and the capacity classification. For example, when the write frequency of the data tag corresponding to the data to be stored is high frequency, the data may be stored in the corresponding storage replica group according to the allocation priority of the capacity classification from low level to medium level; when the write frequency of the data tag corresponding to the data to be stored is low frequency, the data may be stored in the corresponding storage replica group according to the allocation priority of the capacity classification from high level to medium level.

[0036] Step 206: determine a target storage replica group from each candidate storage replica group according to the allocation priority, and store the data to be stored in the target storage replica group.

[0037] The target storage replica group is a storage replica group selected from each candidate storage replica group for storing the data to be stored, that is, the target storage replica group is allocated for the data to be stored for data storage. Exemplarily, the server can screen from each candidate storage replica group according to the allocation priority to determine the target storage replica group, and store the data to be stored in the target storage replica group to implement storage processing for the data to be stored.

[0038] In the above data storage method, the write frequency of the data tag corresponding to the data to be stored is determined, and based on the write frequency and the capacity classification to which each candidate storage replica group belongs, the allocation priority for the data to be stored is determined, and the target storage replica group is determined from each candidate storage replica group according to the allocation priority, and the data to be stored is stored in the target storage replica group. In the data storage processing process, the allocation priority is determined in combination with the write frequency of the data tag corresponding to the data to be stored and the capacity classification to which each candidate storage replica group belongs, and the target storage replica group is determined for data storage according to the allocation priority, so that the corresponding storage replica group can be adaptively allocated for storage of the data to be stored, ensuring the degree of aggregation during data storage, and being able to effectively balance the efficiency of data read and write operations and the efficient use of storage space, thereby reducing storage space waste while ensuring the efficiency of data read and write operations.

[0039] In an exemplary embodiment, based on the write frequency and the capacity classification to which each candidate storage replica group belongs, the allocation priority for the data to be stored is determined, including: determining the frequency classification to which the write frequency corresponds; determining the capacity classification to which each candidate storage replica group belongs based on the used storage capacity of each candidate storage replica group; and determining the allocation priority for the data to be stored based on the frequency classification and the capacity classification.

[0040] Among them, frequency classification is the classification of the write frequency of each data tag, such as including three levels of high, medium and low; it can also be high and low. The number of levels of frequency classification can be flexibly configured according to actual needs. The used storage capacity refers to the capacity of the data stored in the candidate storage replica group. Based on the used storage capacity of each candidate storage replica group, the capacity level can be divided for each candidate storage replica group, so as to determine the capacity classification to which each candidate storage replica group belongs.

[0041] Exemplarily, the server can determine the corresponding frequency classification based on the write frequency, such as comparing the value of the write frequency with each preset frequency level threshold, so as to determine the frequency classification to which the write frequency belongs. The server can determine the used storage capacity of each candidate storage replica group, and compare the used storage capacity with each preset capacity level threshold, so as to determine the capacity classification to which each candidate storage replica group belongs. The server can determine the allocation priority for the data to be stored by combining the frequency classification and the capacity classification. For example, the server can determine the allocation priority of the storage replica group from each capacity classification according to the frequency classification.

[0042] In this embodiment, the server respectively determines the frequency classification corresponding to the write frequency and the capacity classification to which each candidate storage replica group belongs, and determines the allocation priority for the data to be stored based on the frequency classification and capacity classification. The corresponding storage replica group can be adaptively allocated for storage of the data to be stored according to the allocation priority, ensuring the degree of aggregation during data storage, and can effectively balance the efficiency of data read and write operations and the efficient use of storage space, thereby reducing storage space waste while ensuring the efficiency of data read and write operations.

[0043] In an exemplary embodiment, determining the write frequency of a data tag corresponding to data to be stored includes: determining the data tag corresponding to the data to be stored; when it is determined that the data tag is not assigned a corresponding storage replica group, obtaining the storage write data of the data tag; and obtaining the write frequency of the data tag based on the storage write data.

[0044] Among them, the storage write data is the data belonging to the same data tag written to each candidate storage replica group. Exemplarily, the server can determine the data tag corresponding to the data to be stored, such as performing tag extraction on the data to be stored to determine the corresponding data tag. The server can determine whether a corresponding storage replica group has been assigned for the data tag. When the storage replica group is not assigned, the server can obtain the storage write data of the data tag. In some embodiments, the server can perform a write query in each candidate storage replica group according to the data tag, so as to determine the storage write data belonging to the data tag in each candidate storage replica group. The server can determine the write frequency of the data tag based on the storage write data. Specifically, the server can calculate the write frequency of the data tag based on the amount of write data of the storage write data within a preset time range.

[0045] In this embodiment, the server determines the write frequency of the data tag based on the storage write data belonging to the same data tag, so that the corresponding storage replica group can be adaptively allocated for storage according to the write frequency corresponding to the data tag, ensuring the degree of aggregation during data storage, and can effectively balance the efficiency of data read and write operations and the efficient use of storage space, thereby reducing storage space waste while ensuring data read and write operation efficiency.

[0046] In an exemplary embodiment, the data storage method further includes: when it is determined that the data tag has been assigned a corresponding storage replica group, storing the data to be stored in the storage replica group that has been assigned the corresponding storage replica group.

[0047] Exemplarily, when it is determined that the data tag has been assigned a corresponding storage replica group, the server can directly store the data to be stored in the corresponding storage replica group that has been assigned, thereby making reasonable use of the assigned storage replica group and ensuring the degree of aggregation when storing data.

[0048] In an exemplary embodiment, Figure 3 As shown, the data storage method also includes the process of creating a storage replica group, specifically including steps 302 to 308. Among them:

[0049] Step 302: Determine a first storage replica group belonging to a target capacity level from among the candidate storage replica groups.

[0050] The target capacity classification is a capacity classification determined from various capacity classifications for determining whether to trigger the creation of a new storage replica group. The target capacity classification can be screened and determined from various capacity classifications according to actual needs, and can specifically include at least one capacity classification. The first storage replica group is a candidate storage replica group that belongs to the target capacity classification among the candidate storage replica groups.

[0051] Exemplarily, the server may determine a target capacity rating, such as when the capacity rating includes three ratings of high, medium, and low, the target capacity rating may include two ratings of medium and low. The server may filter each candidate storage replica group based on the target capacity rating to filter out the candidate storage replica groups belonging to the target capacity rating, thereby obtaining a first storage replica group.

[0052] Step 304: determine a capacity threshold based on the used storage capacity of the first storage replica group.

[0053] The capacity threshold is used to determine whether it is necessary to trigger the creation of a new storage replica group, and the capacity threshold can be determined based on the used storage capacity of the first storage replica group. Exemplarily, the server can determine the used storage capacity of the first storage replica group, and obtain the capacity threshold based on the used storage capacity of the first storage replica group. For example, the server can perform a weighted calculation based on the used storage capacity of the first storage replica group and the number of the first storage replica groups to obtain the capacity threshold.

[0054] Step 306: Determine, from each candidate storage replica group, a second storage replica group whose used storage capacity is not greater than a capacity threshold.

[0055] The second storage replica group is a candidate storage replica group whose used storage capacity is less than or equal to the capacity threshold among the candidate storage replica groups. Optionally, the server can determine the used storage capacity of each candidate storage replica group, and compare the numerical value of each used storage capacity with the capacity threshold to determine the candidate storage replica group whose used storage capacity is not greater than the capacity threshold, thereby obtaining the second storage replica group.

[0056] Step 308: When the number of the second storage replica groups is less than the preset number of reserved storage replica groups, create a supplementary storage replica group, and update the supplementary storage replica group to each candidate storage replica group.

[0057] The number of reserved storage replica groups is the preset number of storage replica groups that need to be reserved. The specific number of reserved storage replica groups can be flexibly set according to actual needs, such as 100 or 200.

[0058] Exemplarily, the server may determine the number of second storage replica groups, and compare the number of second storage replica groups with the preset number of reserved storage replica groups. When the number of second storage replica groups is less than the preset number of reserved storage replica groups, the server may determine that the triggering condition for creating a storage replica group is met, thereby creating a supplementary storage replica group, and updating the created supplementary storage replica group to each candidate storage replica group. In some embodiments, the server may determine the difference between the number of second storage replica groups and the number of reserved storage replica groups, determine the supplementary number based on the difference, and create a corresponding number of supplementary storage replica groups according to the supplementary number. For example, when the difference between the number of second storage replica groups and the number of reserved storage replica groups is 10, the server may determine that the supplementary number is 10, and the server may create 10 supplementary storage replica groups accordingly. For the created supplementary storage replica groups, the server may add the supplementary storage replica groups to the candidate storage replica groups, thereby dynamically supplementing and expanding the candidate storage replica groups.

[0059] In this embodiment, the server determines the capacity threshold based on the used storage capacity of the first storage replica group belonging to the target capacity level in the candidate storage replica group. When the number of second storage replica groups in the candidate storage replica group whose used storage capacity is not greater than the capacity threshold is less than the number of reserved storage replica groups, the server triggers the creation of a supplementary storage replica group to dynamically supplement and expand the candidate storage replica group, which can effectively balance the efficiency of data read and write operations and the efficient use of storage space, thereby reducing storage space waste while ensuring the efficiency of data read and write operations.

[0060] In an exemplary embodiment, determining a capacity threshold based on the used storage capacity of the first storage replica group includes: determining the number of first storage replica groups and statistical parameters of used capacity; and obtaining the capacity threshold by weighted calculation based on the number of first storage replica groups and statistical parameters of used capacity.

[0061] The used capacity statistical parameter may be obtained by statistically analyzing the used storage capacity of each of the first storage replica groups, and may include, for example, an arithmetic mean or a weighted mean of the used storage capacity of each of the first storage replica groups.

[0062] Exemplarily, the server may determine the number of first storage replica groups and the used storage capacity of each first storage replica group. The server may perform statistics based on the used storage capacity of each first storage replica group to obtain a used capacity statistical parameter, such as calculating the arithmetic mean of the used storage capacity. The server may perform weighted calculation based on the number of first storage replica groups and the used capacity statistical parameter to obtain a capacity threshold.

[0063] In this embodiment, the server performs weighted calculation to obtain the capacity threshold based on the number of first storage replica groups and the used capacity statistical parameters, thereby ensuring the accuracy of the capacity threshold, which is beneficial to balancing the efficiency of data read and write operations and the efficient use of storage space, thereby reducing storage space waste while ensuring data read and write operation efficiency.

[0064] The present application also provides an application scenario, which applies the above data storage method. Specifically, the application of the data storage method in the application scenario is as follows:

[0065] Storage technology is one of the core technologies in the current cloud computing field. Users upload files to the cloud, and through reliable storage technology, they are persisted in the cloud, and provide users with a service of "read and retrieve", saving users the cost of purchasing storage devices and equipment maintenance. In the process of data storage, fast writing and deletion of data are extremely important. Only by ensuring the speed of data writing and deletion can the user's experience be guaranteed to a large extent. At the same time, in order to provide cloud storage services, it is also necessary to ensure the full utilization of storage hardware. Only by fully utilizing the storage hardware can users use sufficient space and improve the resource utilization of cloud service providers.

[0066] The storage control center will create and supplement cg (storage copy group) in real time according to the quantity, and the storage agent will apply to the storage control center for cg in real time to write data. If the random write method is used to directly use cg to write data at will, it may cause the data to be randomly distributed. If the same label data is deleted in batches at the same time, the data deletion operation will be performed on multiple scattered disks, which will increase the IO (Input / Output) time of deleting data. At the same time, fragmented data holes will also increase the burden on the storage system when compacting the storage space. In order to solve this problem, data with the same label can be written to the same cg, but excessive concentration of the same label in the same cg will cause other problems. When there are many labels but not much data with the same label, the cg with the same label will not be fully used at all, and other labels cannot be written to cgs with different labels. They can only continuously create and supplement new cgs for use, which will cause space waste. Based on this, the data storage method proposed in this application can ensure the reasonable use of capacity while aggregating data as much as possible.

[0067] Specifically, in the data storage method provided in this embodiment, the storage center divides cg into four levels of capacity classification according to the used capacity: empty, low, medium, and high. When writing data, the storage agent performs a write frequency statistics on the storage tag to obtain the real-time write frequency of each tag, and initiates a cg application to the storage control center through the write frequency. The storage control center divides the frequency into three levels according to the frequency value, high frequency, medium frequency, and low frequency. For the high-frequency tag data written, empty cg (storage copy group with empty capacity classification) will be allocated first, followed by low cg (storage copy group with low capacity classification), and finally medium cg (storage copy group with medium capacity classification); for medium-frequency tag data, low cg will be allocated first, followed by empty cg; for low-frequency tag data, medium cg will be allocated first, followed by low cg, and finally empty cg.

[0068] Furthermore, in addition to allocating according to frequency and capacity classification, the data storage method provided in this embodiment also introduces a capacity threshold for dynamically replenishing the number of cgs, that is, calculating the average usage of low and medium cgs in real time, and taking the ratio of the number of low and medium cgs to the total number of cgs as the weight, a weighted average is calculated as the upper limit of dynamic capacity allocation. Assuming there are 100, each cg has a logical capacity of 4GB, the number of empty cgs is 20, the number of low cgs is 50, the number of medium cgs is 30, the boundary between low and medium is 20% of the total capacity of cg, that is, 819.2MB, and the boundary between medium and high is 80% of the total capacity of cg, that is, 3276.8MB. If the average usage value of low cg is 400MB and the average usage value of medium is 2800MB, then the capacity threshold for dynamically replenishing the number of cgs = 400*(50 / 100)+2800*(30 / 100)=1040MB. The capacity threshold for dynamically replenishing the number of cgs has a certain limiting effect on the creation of new cgs. With the capacity threshold as the upper limit, when the number of cgs with usage less than or equal to the capacity threshold is less than the number of reserved cgs, the creation of cgs will be triggered to replenish the shortage of cgs. For example, if the number of reserved cgs is 200, and the number of cgs less than or equal to 1040MB is 100, cg replenishment is required. If the number of cgs less than or equal to 1040MB is 300, there is no need to create a new cg. By dynamically adjusting the cg replenishment capacity threshold, the unlimited creation of new cgs can be avoided, so that cgs can be more fully used to avoid waste. If there is no dynamic threshold but a fixed threshold is used, it may cause new cgs to be continuously created when there are many cgs greater than the fixed threshold, so that these cgs cannot be fully used.

[0069] Specifically, Figure 4As shown, the data storage method provided in this embodiment provides a process for adaptively allocating cg according to the label frequency. Specifically, when obtaining the written labeled data, the server can determine whether the current label has a cg available. If so, the data is directly written to the available cg. If there is no available cg, the server can calculate the label frequency and apply for cg to the storage control center. The storage control center can make a judgment based on the label frequency to determine the corresponding allocation priority. Specifically, when the write frequency belongs to high frequency, the cg group allocation can be performed in the priority order of empty cg, low cg, and medium cg; when the write frequency belongs to medium frequency, the cg group allocation can be performed in the priority order of low cg, medium cg, and empty cg; when the write frequency belongs to low frequency, the cg group allocation can be performed in the priority order of medium cg, low cg, and empty cg. The server can write the data to the allocated cg group.

[0070] In an application, the data storage method provided in this embodiment can adaptively allocate CG according to the label frequency. Specifically, analyze the label of the written data, if the data has no label, it is classified as the same label; check whether the current label has a CG available, if there is an available CG, use it directly to write data; calculate the frequency of the label; initiate a CG application to the storage control center, and the CG application carries the label frequency information; the control center will determine whether the currently applied write data belongs to a high frequency / medium frequency / low frequency application based on the frequency; application requests of different frequencies will correspond to different CG usage groups (empty / low / medium) priorities, among which high is generally not allocated for use, and only when CG is not replenished in time will it consider allocating CG of the high usage group.

[0071] The allocation priority of different usage cgs is determined by estimating the frequency of tag data. Generally speaking, high-frequency data writing can quickly fully utilize the entire cg, so for data of this frequency, priority is given to allocating empty cg, followed by low cg and medium cg, because high-frequency data is likely to be able to fully utilize the space of cg; for medium-frequency data, its utilization rate and degree of sufficiency of cg space are relatively lower than those of high-frequency data, so for the allocation of cgs for medium-frequency data, priority is given to low cg, followed by medium cg and empty cg; for low-frequency data, it is difficult to completely monopolize an entire cg, and the utilization rate and degree of sufficiency of cg are the lowest, so priority is given to medium cg, followed by low cg and empty cg. Based on the characteristic that the higher the frequency of data, the higher the utilization rate of cg, the reasonable allocation of cg is arranged, which avoids the possible waste of cg space, and at the same time guarantees the aggregation effect of data to a large extent, and realizes tag aggregation, that is, the data stored on the storage replica group are all data with the same tag, which ensures the efficiency of data writing and deletion.

[0072] like Figure 5 As shown, the processing flow of adaptively creating a supplementary CG according to the current CG usage and quantity in the data storage method provided in this embodiment. Specifically, the server can initiate a CG quantity supplement check and calculate the average usage value of low CG and medium CG respectively. The server can calculate the weighted average of low CG and medium CG as the usage threshold of the number of dynamic supplementary CGs. The server can determine whether the number of CGs in each candidate CG whose used capacity is less than or equal to the usage threshold of the number of dynamic supplementary CGs is less than the number of reserved CGs. If so, a CG quantity supplement request is initiated to supplement the CGs.

[0073] Specifically, the data storage method provided in this embodiment can adaptively create a supplementary CG according to the current CG usage. Specifically, regularly initiate a CG quantity check; calculate the average usage of the low and medium CG groups; calculate the weighted average value as the usage threshold of the supplementary CG according to the percentage of the number of CGs in the low and medium CG groups to the total number of CGs; determine the number of CGs that are lower than the calculated usage threshold, and if the number of CGs that meet the conditions is lower than the reserved number of CGs, initiate the creation of a supplement.

[0074] By adjusting the cg usage in real time to create and supplement cgs, a threshold for the overall cg usage is obtained based on the average usage and quantity ratio of cgs in different usage groups. This threshold varies with the different cg usage distributions, so it can be determined whether cgs need to be supplemented based on the overall cg situation. For example, when low cgs account for a large proportion, it actually means that there are many cgs that have a large amount of free space that is not utilized. At this time, the calculated threshold will include a lot of lowcgs. In this way, the number of cgs below the threshold will be greater than the number of reserved cgs, and new cgs will not be created blindly. In this way, these idle cgs can be utilized to ensure the use of the created cg space.

[0075] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0076] Based on the same inventive concept, the embodiment of the present application also provides a data storage device for implementing the data storage method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more data storage device embodiments provided below can refer to the limitations on the data storage method above, and will not be repeated here.

[0077] In an exemplary embodiment, Figure 6 As shown, a data storage device 600 is provided, comprising: a write frequency determination module 602, an allocation priority determination module 604 and an allocation processing module 606, wherein:

[0078] A write frequency determination module 602 is used to obtain data to be stored and determine the write frequency of the data tag corresponding to the data to be stored;

[0079] An allocation priority determination module 604 is used to determine the allocation priority for the data to be stored based on the write frequency and the capacity classification to which each candidate storage replica group belongs;

[0080] The allocation processing module 606 is used to determine a target storage replica group from each candidate storage replica group according to the allocation priority, and store the data to be stored in the target storage replica group.

[0081] In an exemplary embodiment, the allocation priority determination module 604 is also used to determine the frequency classification to which the write frequency corresponds; determine the capacity classification to which each candidate storage replica group belongs based on the used storage capacity of each candidate storage replica group; and determine the allocation priority for the data to be stored based on the frequency classification and the capacity classification.

[0082] In an exemplary embodiment, the write frequency determination module 602 is also used to determine the data tag corresponding to the data to be stored; when it is determined that the data tag is not assigned a corresponding storage replica group, obtain the storage write data of the data tag; and obtain the write frequency of the data tag based on the storage write data.

[0083] In an exemplary embodiment, the write frequency determination module 602 is further configured to store the data to be stored in the storage replica group that has been allocated the corresponding storage replica group when it is determined that the data tag has been allocated the corresponding storage replica group.

[0084] In an exemplary embodiment, it also includes a storage replica group supplement module, which is used to determine a first storage replica group belonging to a target capacity level from various candidate storage replica groups; determine a capacity threshold based on the used storage capacity of the first storage replica group; determine a second storage replica group whose used storage capacity is not greater than the capacity threshold from various candidate storage replica groups; when the number of second storage replica groups is less than the preset number of reserved storage replica groups, create a supplementary storage replica group, and update the supplementary storage replica group to each candidate storage replica group.

[0085] In an exemplary embodiment, the storage replica group supplementation module is further used to determine the number of first storage replica groups and used capacity statistical parameters; based on the number of first storage replica groups and used capacity statistical parameters, weighted calculation is performed to obtain a capacity threshold.

[0086] Each module in the above data storage device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0087] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the data storage method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data storage method is implemented.

[0088] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0089] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0090] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0091] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0092] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0093] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0094] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0095] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data storage method, characterized in that: The method comprises: Acquire data to be stored, and determine the writing frequency of data tags corresponding to the data to be stored; Determining an allocation priority for the data to be stored based on the write frequency and the capacity classification to which each candidate storage replica group belongs; A target storage replica group is determined from each of the candidate storage replica groups according to the allocation priority, and the data to be stored is stored in the target storage replica group.

2. The method according to claim 1, characterized in that The step of determining the allocation priority for the data to be stored based on the write frequency and the capacity classification to which each candidate storage replica group belongs includes: Determine the frequency classification to which the writing frequency corresponds; Determining the capacity classification to which each candidate storage replica group belongs according to the used storage capacity of each candidate storage replica group; Based on the frequency classification and the capacity classification, an allocation priority for the data to be stored is determined.

3. The method according to claim 1, characterized in that The determining the writing frequency of the data tag corresponding to the data to be stored includes: Determine a data tag corresponding to the data to be stored; When it is determined that the data tag is not assigned a corresponding storage replica group, acquiring storage write data of the data tag; The writing frequency of the data tag is obtained based on the stored writing data.

4. The method according to claim 3, characterized in that The method further comprises: When it is determined that the data tag has been assigned a corresponding storage replica group, the data to be stored is stored in the storage replica group that has been assigned a corresponding storage replica group.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Determine a first storage replica group belonging to the target capacity level from each of the candidate storage replica groups; determining a capacity threshold based on the used storage capacity of the first storage replica group; Determine, from each of the candidate storage replica groups, a second storage replica group whose used storage capacity is not greater than the capacity threshold; When the number of the second storage replica groups is less than the preset number of reserved storage replica groups, a supplementary storage replica group is created, and the supplementary storage replica group is updated to each of the candidate storage replica groups.

6. The method according to claim 5, characterized in that The determining of the capacity threshold based on the used storage capacity of the first storage replica group includes: Determining the number of the first storage replica groups and used capacity statistical parameters; A capacity threshold is obtained by weighted calculation based on the number of the first storage replica groups and the used capacity statistical parameter.

7. A data storage device, characterized in that: The device comprises: A writing frequency determination module, used to obtain data to be stored and determine the writing frequency of data tags corresponding to the data to be stored; An allocation priority determination module, configured to determine an allocation priority for the data to be stored based on the write frequency and the capacity classification to which each candidate storage replica group belongs; The allocation processing module is used to determine a target storage replica group from each of the candidate storage replica groups according to the allocation priority, and store the data to be stored in the target storage replica group.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images