A data storage method, apparatus, device, and computer-readable storage medium

Abstract

The present application discloses a data storage method, apparatus, device, and computer-readable storage medium, which can be applied to a file storage system with Hadoop as the core, and can overcome the defects of the storage method provided by the existing Hadoop, meet the high availability of data in the file storage system, avoid a large consumption of data storage space, and ensure the read and write speed of data. Among them, the storage method includes: determining the access attributes of a target storage file in the file storage system, where the access attributes include: cold attributes and hot attributes; generating a target storage policy adapted to the access attributes for the target storage file; querying the historical storage policy of the target storage file in the file storage system; and performing a storage operation on the target storage file according to the comparison result between the target storage policy and the historical storage policy.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data storage method, apparatus, device, and computer-readable storage medium. Background Art

[0002] With the development of technology, in the field of big data, many big data components with Hadoop as the core have emerged, such as Hive, Spark, Presto, Impala, etc.

[0003] So-called Hadoop is a distributed system infrastructure. Through Hadoop, users can develop distributed programs without understanding the underlying details of distribution to make full use of the cluster for high-speed computing and storage.

[0004] In Hadoop, to achieve high availability of cluster storage, the HDFS file storage system of Hadoop provides two storage methods: Block replication (i.e., file block multiple replicas) and Erasure coding (i.e., erasure coding technology). However, the Block replication storage method not only improves data availability but also causes an additional 200% overhead in storage space and additional bandwidth consumption when writing data; while the Erasure coding storage method has greatly improved in terms of storage space usage and data writing speed compared to the Block replication storage method, but it does not improve the data reading speed. Therefore, in a file storage system with Hadoop as the core, it is a difficult choice to meet the high availability of data, avoid a large consumption of data storage space, and ensure the read and write speed of data. Summary of the Invention

[0005] This application provides a data storage method, apparatus, device, and computer-readable storage medium, which can be applied to a file storage system with Hadoop as the core, and can overcome the defects of the existing storage methods provided by Hadoop, meet the high availability of data in the file storage system, avoid a large consumption of data storage space, and ensure the read and write speed of data.

[0006] In view of this, a first aspect of this application provides a data storage method, including:

[0007] Determine the access attributes of the target storage file in the file storage system, where the access attributes include: cold attributes and hot attributes;

[0008] Generate a target storage strategy adapted to the access attributes for the target storage file;

[0009] Query the historical storage policy of the target storage file in the file storage system;

[0010] Perform a storage operation on the target storage file according to the comparison result between the target storage policy and the historical storage policy.

[0011] Optionally, determining the access attribute of the target storage file in the file storage system specifically includes:

[0012] Obtain the target storage file and the log file in the file storage system;

[0013] Filter out the access data corresponding to the target storage file from the log file;

[0014] Determine the access attribute of the target storage file according to the access data.

[0015] Optionally, the cold attribute includes: pre-cooling attribute; the hot attribute includes: potential hot spot attribute;

[0016] The determining the access attribute of the target storage file according to the access data specifically includes:

[0017] Calculate the access growth data according to the access data;

[0018] Obtain the reference access data and the reference access growth data corresponding to the pre-cooling attribute and the potential hot spot attribute respectively;

[0019] Compare the access data with the reference access data corresponding to the pre-cooling attribute and the potential hot spot attribute respectively to obtain a first comparison result;

[0020] Compare the access growth data with the reference access growth data corresponding to the pre-cooling attribute and the potential hot spot attribute respectively to obtain a second comparison result;

[0021] Combine the first comparison result and the second comparison result to determine that the access attribute of the target storage file is a cold attribute or a hot attribute.

[0022] Optionally, the combining the first comparison result and the second comparison result to determine that the access attribute of the target storage file is a cold attribute or a hot attribute specifically includes:

[0023] When the first comparison result is that the access data is greater than 0 and less than the reference access data corresponding to the pre-cooling attribute, and the second comparison result is that the access growth data is less than the reference access growth data corresponding to the pre-cooling attribute, determine the pre-cooling attribute as the access attribute of the target storage file;

[0024] When the first comparison result is that the access data is greater than 0 and less than the reference access data corresponding to the potential hot attribute, and the second comparison result is that the access growth data is greater than the reference access growth data corresponding to the potential hot attribute, the potential hot attribute is determined as the access attribute of the target storage file.

[0025] Optionally, the access data includes: the number of visitors and the number of accesses;

[0026] Calculating the access growth data according to the access data specifically includes:

[0027] Calculating the growth rate of the number of visitors in the access growth data based on the number of visitors;

[0028] Calculating the growth rate of the number of accesses in the access growth data based on the number of accesses.

[0029] Optionally, the cold attribute includes: the freezing attribute; the hot attribute includes: the current hot attribute, the thawing attribute, and the local hot attribute;

[0030] Determining the access attribute of the target storage file according to the access data specifically includes:

[0031] When the access data for the most recent N days is 0, the freezing attribute is determined as the access attribute of the target storage file, where N is a natural number;

[0032] When the access data is greater than the reference access data corresponding to the current hot attribute, the current hot attribute is determined as the access attribute of the target storage file;

[0033] When the access data for the most recent N days is not 0 and the access data for the previous N days is 0, the thawing attribute is determined as the access attribute of the target storage file;

[0034] When the number of visitors in the access data is greater than 0 and less than the reference number of visitors corresponding to the local hot attribute, and the number of accesses in the access data is greater than the reference number of accesses corresponding to the local hot attribute, the local hot attribute is determined as the access attribute of the target storage file.

[0035] Optionally, generating a target storage policy for the target storage file that is adapted to the access attribute specifically includes:

[0036] When the access attribute is a cold attribute, the object storage policy is determined as the target storage policy of the target storage file;

[0037] When the access attribute is a hot attribute, the replica storage policy is determined as the target storage policy of the target storage file.

[0038] Optionally, the replica storage policy includes: high replicas and normal replicas;

[0039] When the access attribute is a hot attribute, determining the replica storage policy as the target storage policy of the target storage file specifically includes:

[0040] When the access attribute is the current hot attribute or the potential hot attribute among the hot attributes, determining high replicas as the target storage policy of the target storage file;

[0041] When the access attribute is the thawed attribute or the local hot attribute among the hot attributes, determining normal replicas as the target storage policy of the target storage file.

[0042] Optionally, performing a storage operation on the target storage file according to the comparison result between the target storage policy and the historical storage policy specifically includes:

[0043] Comparing the target storage policy with the historical storage policy to obtain a comparison result;

[0044] When the comparison result is that the target storage policy is inconsistent with the historical storage policy, updating the historical storage policy to the target storage policy and performing a storage operation on the target storage file according to the target storage policy;

[0045] When the comparison result is that the target storage policy is consistent with the historical storage policy, continuing to perform a storage operation on the target storage file according to the historical storage policy.

[0046] The second aspect of the present application provides a data storage device, including:

[0047] A determination unit, configured to determine the access attribute of a target storage file in a file storage system, where the access attribute includes: cold attribute and hot attribute;

[0048] A generation unit, configured to generate a target storage policy adapted to the access attribute for the target storage file;

[0049] A query unit, configured to query the historical storage policy of the target storage file in the file storage system;

[0050] A storage unit, configured to perform a storage operation on the target storage file according to the comparison result between the target storage policy and the historical storage policy.

[0051] The third aspect of the present application provides a data storage device, where the storage device includes a processor and a memory;

[0052] The memory is used to store program code and transfer the program code to the processor;

[0053] The processor is used to execute the data storage method described in any of the above first aspects according to the instructions in the program code.

[0054] A fourth aspect of this application provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the data storage method described in any of the above first aspects.

[0055] From the above technical solutions, it can be seen that this application has the following advantages:

[0056] After the applicant studied the storage method of the existing HDFS file storage system, it was found that the root cause of waste of storage resources and high storage cost is that a "one-size-fits-all" storage strategy is adopted for all storage files, that is, each storage file is copied 3 times. Although this improves data availability, it also causes an additional 200% overhead in storage space, and at the same time, the storage cost is relatively high.

[0057] In order to avoid a large consumption of data storage space in the file storage system, the data storage method provided in this application first determines the access attributes of the target storage file in the file storage system. The access attributes include: cold attributes and hot attributes; a target storage strategy adapted to its access attributes is generated for the target storage file. This target storage strategy is generated after determining that the access attribute of the target storage file is a cold attribute or a hot attribute, and when the access attribute changes, the target storage strategy will also change accordingly; after formulating the currently adapted target storage strategy for the target storage file, query the historical storage strategy of the target storage file in the file storage system, compare the target storage strategy with the historical storage strategy, and perform a storage operation on the target storage file according to the comparison result of the target storage strategy and the historical storage strategy. Since this storage operation is an operation performed based on the comparison result of the current target storage strategy and the historical storage strategy of the target storage file, and this comparison result will change with different target storage strategies, the storage operation performed on the target storage file will also change accordingly. That is, for the same target storage file, the corresponding matching storage strategy and storage operation in the file storage system are different when its access attributes are different. This is beneficial to ensuring the high data availability of the target storage file at any moment of the system service, and can reduce the additional bandwidth consumption when accessing the target storage file while maintaining the data read and write speed. Description of the Drawings

[0058] To more clearly illustrate the technical methods in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0059] Figure 1 FIG. 4 is a schematic flowchart of Embodiment 1 of a data storage method in an embodiment of the present application;

[0060] Figure 2 FIG. 5 is a schematic flowchart of Embodiment 2 of a data storage method in an embodiment of the present application;

[0061] Figure 3 FIG. 6 is a schematic structural diagram of an embodiment of a data storage device in an embodiment of the present application;

[0062] Figure 4 FIG. 7 is a specific implementation structure diagram of a data storage device in an embodiment of the present application. Detailed Embodiments

[0063] The present application designs a data storage method, device, equipment, and computer-readable storage medium, which can be applied to a file storage system with Hadoop as the core, can overcome the defects of the existing storage methods provided by Hadoop, meet the high availability of data in the file storage system, avoid a large consumption of data storage space, and ensure the read and write speed of data.

[0064] To enable those skilled in the art to better understand the solution of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0065] For ease of understanding, please refer to Figure 1 , Figure 1 FIG. 4 is a schematic flowchart of Embodiment 1 of a data storage method in an embodiment of the present application.

[0066] As Figure 1 shown, a data storage method in this embodiment specifically includes:

[0067] Step 101: Determine the access attributes of the target storage file in the file storage system, where the access attributes include: cold attributes and hot attributes.

[0068] In this embodiment, the storage operation of the target storage file is based on the access attribute of the file. Thus, when the access attribute of the target storage file changes, the storage operation of the target storage file in the file storage system may also change. Therefore, in this embodiment, the access attribute of the target storage file in the file storage system is determined first.

[0069] It can be understood that the access attribute is determined according to the user's access behavior to the target storage file. Therefore, the access attribute can represent the access popularity of the user to the target storage file. The higher the access popularity, the more frequent the user's access behavior to the target storage file, and vice versa. In this embodiment, the access attribute is defined as a cold attribute and a hot attribute to distinguish different access popularities. When the target storage file has a hot attribute, it means that the user often accesses the target storage file, and the target storage file is frequently read and written. When the target storage file has a cold attribute, it means that the user rarely accesses the target storage file, and the target storage file is rarely read and written.

[0070] It should be understood that the target storage file can be any storage file in the file storage system, and this is not limited and elaborated in this embodiment.

[0071] Step 102: Generate a target storage policy adapted to the access attribute for the target storage file.

[0072] Different access attributes may correspond to different target storage policies. Therefore, after determining the access attribute corresponding to the target storage file, a target storage policy adapted to the access attribute can be generated for the target storage file. For example, when the access attribute of the target storage file is a cold attribute, a target storage policy adapted to the cold attribute is generated for the file. When the access attribute of the target storage file is a hot attribute, a target storage policy adapted to the hot attribute is generated for the file. Furthermore, when the access attribute changes, the target storage policy may also change. For example, when the access attribute changes from the historical cold attribute to the current hot attribute, the target storage policy also changes from the target storage policy adapted to the cold attribute to the target storage policy adapted to the hot attribute.

[0073] Step 103: Query the historical storage policy of the target storage file in the file storage system.

[0074] The storage operation of the target storage file in the file storage system is based on the comparison result between the target storage policy adapted to the current situation and the historical storage policy adapted to the history. Therefore, after formulating the target storage policy adapted to the current situation for the target storage file, query the historical storage policy of the target storage file in the file storage system.

[0075] It can be understood that in an alternative embodiment, the query for the historical storage policy can be as follows: the historical storage policy in the target storage file is recorded in the file storage system. When querying, the file information (such as file name or file number) of the target storage file is used as the query index to query the historical storage policy.

[0076] Step 104: Perform a storage operation on the target storage file according to the comparison result between the target storage policy and the historical storage policy.

[0077] The storage operation of the target storage file is an operation performed based on the comparison result between the current target storage policy and the historical storage policy. This comparison result will change with different target storage policies. Therefore, the storage operation performed on the target storage file may also change accordingly. And the target storage policy will also change with different access attributes. That is to say, for the same target storage file under different access attributes, the corresponding target storage policy and storage operation in the file storage system may also be different.

[0078] It can be understood that the comparison between the target storage policy and the historical storage policy can be achieved in various ways, which is not limited and elaborated in this embodiment.

[0079] In the data storage method of this embodiment, in order to avoid a large consumption of data storage space in the file storage system, the access attribute of the target storage file in the file storage system is determined first. The access attributes include: cold attribute and hot attribute. A target storage policy adapted to its access attribute is generated for the target storage file. This target storage policy is generated after determining that the access attribute of the target storage file is a cold attribute or a hot attribute. When the access attribute changes, the target storage policy will also change accordingly. After formulating the currently adapted target storage policy for the target storage file, query the historical storage policy of the target storage file in the file storage system, compare the target storage policy with the historical storage policy, and perform a storage operation on the target storage file according to the comparison result between the target storage policy and the historical storage policy. Since this storage operation is an operation performed based on the comparison result between the current target storage policy and the historical storage policy of the target storage file, and this comparison result will change with different target storage policies, the storage operation performed on the target storage file will also change accordingly. That is, for the same target storage file under different access attributes, the corresponding storage policy and storage operation in the file storage system are different. This is beneficial to ensuring the high data availability of the target storage file at any time during system service and can reduce the additional bandwidth consumption when accessing the target storage file while maintaining the data read and write speed.

[0080] The above is the first embodiment of a data storage method provided by the embodiments of the present application. The following is the second embodiment of a data storage method provided by the embodiments of the present application.

[0081] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of the second embodiment of a data storage method in the embodiments of the present application.

[0082] As Figure 2 shown, a data storage method in this embodiment specifically includes:

[0083] Step 201, obtain a target storage file and a log file in the file storage system.

[0084] In this embodiment, the access data of the target storage file can be obtained from the log file. Therefore, it is necessary to first obtain the target storage file and the log file in the file storage system. In the file storage system, the log file is a file type independent of the storage file, and the log data recorded in it includes some access data, bug information, download data, storage data, record data of certain processing operations completed by the system, and so on. It can be understood that the log file can be recorded and updated on a daily basis and the content can be stored regularly on a daily basis.

[0085] For the acquisition of the log file, it can be carried out in real time, that is, when configuring the storage operation for the target storage file, obtain the log file from the file storage system; it can also be obtained in advance, that is, obtain the log file from the file storage system in advance, and when configuring the storage operation for the target storage file, it can be directly used. When a more accurate storage operation is required, the former acquisition method can be selected, so that the obtained log file is the latest and most suitable for the current situation, and the subsequent storage operation is also applicable to the current situation. When shorter calculation time and faster calculation efficiency are required, the latter can be selected, and in order to further standardize the acquisition of the log file, timed acquisition can be set, such as 0:00, 6:00, etc. every day.

[0086] Step 202, filter out the access data corresponding to the target storage file from the log file.

[0087] In this embodiment, the access attribute is determined according to the user's access behavior to the target storage file, and the user's access behavior to the target storage file is reflected and recorded in the log file in the form of access data. Therefore, it is necessary to filter out the access data corresponding to the target storage file from the log file.

[0088] The data recorded in the log file is relatively diverse. Access data is just one type of data among them, and the access data recorded in the log file is the access data of all stored files in the file storage system. It is also necessary to filter out the access data corresponding to the target stored file from all the access data.

[0089] In a specific embodiment, filtering out the access data corresponding to the target stored file from the log file can be as follows: First, filter out all the log data corresponding to the target stored file from the log file, and then filter out the access data from all the log data. The corresponding implementation steps may include:

[0090] Step 2021: Using the file information of the target stored file as an index, filter out all the log data corresponding to the target stored file from the log file.

[0091] Step 2022: Using the access request as an index, query out the access data corresponding to the target stored file from all the log data.

[0092] In a specific embodiment, filtering out the access data corresponding to the target stored file from the log file can be as follows: First, filter out all the access data from the log file, and then filter out the access data corresponding to the target stored file from all the access data. The corresponding implementation steps may include:

[0093] Step 2021: Using the access request as an index, query out all the access data from the log data.

[0094] Step 2022: Using the file information of the target stored file as an index, filter out the log data corresponding to the target stored file from all the access data.

[0095] It can be understood that the above access request can be an access request name or an access request command, etc. As long as it can realize the discrimination of the access request / access behavior, it is not limited and elaborated in this embodiment. The file information of the target stored file can be a file name or a file number, etc., and it is also not limited in this embodiment.

[0096] Specifically, the access data may include: the number of access people and the number of access times. Among them, the number of access people represents the number of access users of the target stored file; the number of access times represents the number of access times of the target stored file without considering the differences of access users. When 1 user accesses the target stored file 3 times, the number of access people in the access data is 1 and the number of access times is 3; when 3 users each access the target stored file 1 time, the number of access people in the access data is 3 and the number of access times is 3.

[0097] For ease of understanding, in this embodiment, the implementation of the steps is described by way of specific examples. For example, the number of visitors to the target storage file A is 30, and the number of access times is 100 times.

[0098] Step 203: Determine the access attribute of the target storage file according to the access data.

[0099] It can be understood that there are multiple preset attributes in the file storage system, such as: current hot attribute, potential hot attribute, thawing attribute, local hot attribute, pre-cooling attribute, and freezing attribute. In this embodiment, the access attribute corresponding to the target storage file can be determined from the current hot attribute, potential hot attribute, thawing attribute, local hot attribute, pre-cooling attribute, and freezing attribute according to the access data.

[0100] Among them, the current hot attribute means that there are multiple users accessing the target storage file frequently currently; the potential hot attribute means that although there are a small number of users accessing the target storage file currently, there is a significant increase compared with historical accesses; the thawing attribute means that there are no users accessing the target storage file in the previous period of time, and there are users accessing the target storage file in the recent period of time; the local hot attribute means that only a few users access the target storage file frequently; the pre-cooling attribute means that there are almost no users accessing the target storage file historically and currently; the freezing attribute means that there are no users accessing the target storage file in the recent period of time. From the above description, the access behavior characteristics of the current hot attribute, potential hot attribute, thawing attribute, local hot attribute, pre-cooling attribute, and freezing attribute are shown in Table 1 below:

[0101] Table 1

[0102]

[0103]

[0104] It can be understood that the access behavior characteristics in Table 1 above are specifically manifested by the reference access data and reference access growth data corresponding to each attribute, and the specific values of the reference access data and reference access growth data corresponding to each attribute can be set as needed, and no specific limitation is made in this embodiment.

[0105] In a specific implementation manner, when determining whether the access attribute of the target storage file is a cold attribute or a hot attribute, it can be determined by combining the access data of the target storage file and the access growth data corresponding to the access data. In this implementation manner, the cold attribute can include: pre-cooling attribute; the hot attribute can include: potential hot attribute; determining the access attribute of the target storage file according to the access data specifically includes:

[0106] Step 2031: Calculate the access growth data based on the access data.

[0107] Specifically, in an alternative embodiment, when the access data includes the number of visitors and the number of visits, the access growth data may include the growth rate of the number of visitors and the growth rate of the number of visits. At this time, calculating the access growth data based on the access data specifically includes:

[0108] Based on the number of visitors, calculate the growth rate of the number of visitors in the access growth data;

[0109] Based on the number of visits, calculate the growth rate of the number of visits in the access growth data.

[0110] It can be understood that the growth rate of the number of visitors represents the growth speed of the number of visitors. The number of visitors and the number of visits obtained through step 202 are the access data currently adapted to the target storage file. It is also necessary to obtain the number of visitors and the number of visits that were historically adapted to the target storage file, and then compare the historical number of visitors with the current number of visitors to obtain the growth rate of the number of visitors. Similarly, the growth rate of the number of visits can be obtained.

[0111] Step 2032: Obtain the reference access data and the reference access growth data corresponding to the pre-cooling attribute and the potential hot spot attribute respectively.

[0112] In this embodiment, by comparing the access data, the access growth data of the target storage file with the reference access data and the reference access growth data corresponding to the pre-cooling attribute and the potential hot spot attribute respectively, the access attribute of the target storage file is determined. Therefore, after obtaining the access data and the access growth data of the target storage file, obtain the reference access data and the reference access growth data corresponding to the pre-cooling attribute and the potential hot spot attribute respectively.

[0113] It can be understood that when the access data is the number of visitors and the number of visits, the reference access data may be the reference number of visitors and the reference number of visits. When the access growth data is the growth rate of the number of visitors and the growth rate of the number of visits, the reference access growth data may be the reference growth rate of the number of visitors and the reference growth rate of the number of visits.

[0114] Step 2033: Compare the access data with the reference access data corresponding to the pre-cooling attribute and the potential hot spot attribute respectively to obtain the first comparison result.

[0115] Specifically, when comparing, the access data is compared with the reference access data, and the access data is respectively compared with the reference access data corresponding to the pre-cooling attribute and the potential hot-spot attribute. When the access data is the number of visitors and the number of accesses, and the reference access data is the reference number of visitors and the reference number of accesses, the number of visitors is compared with the reference number of visitors, and the number of accesses is compared with the reference number of accesses, that is: the number of visitors is respectively compared with the reference number of visitors corresponding to the pre-cooling attribute and the potential hot-spot attribute; the number of accesses is respectively compared with the reference number of accesses corresponding to the pre-cooling attribute and the potential hot-spot attribute.

[0116] Step 2034: Compare the access growth data with the reference access growth data corresponding to the pre-cooling attribute and the potential hot-spot attribute respectively to obtain a second comparison result.

[0117] Specifically, when comparing, the access growth data is compared with the reference access growth data, and the access growth data is respectively compared with the reference access growth data corresponding to the pre-cooling attribute and the potential hot-spot attribute. When the access growth data is the growth rate of the number of visitors and the growth rate of the number of accesses, and the reference access growth data is the reference growth rate of the number of visitors and the reference growth rate of the number of accesses, the growth rate of the number of visitors is compared with the reference growth rate of the number of visitors, and the growth rate of the number of accesses is compared with the reference growth rate of the number of accesses, that is: the growth rate of the number of visitors is respectively compared with the reference growth rate of the number of visitors corresponding to the pre-cooling attribute and the potential hot-spot attribute; the growth rate of the number of accesses is respectively compared with the reference growth rate of the number of accesses corresponding to the pre-cooling attribute and the potential hot-spot attribute.

[0118] Step 2035: Combine the first comparison result and the second comparison result to determine that the access attribute of the target storage file is a cold attribute or a hot attribute.

[0119] Specifically, after obtaining the first comparison result corresponding to the access data and the second comparison result corresponding to the access growth data, the first comparison result and the second comparison result can be combined to determine that the access attribute of the target storage file is a cold attribute or a hot attribute.

[0120] In an alternative embodiment, combining the first comparison result and the second comparison result to determine that the access attribute of the target storage file is a cold attribute or a hot attribute specifically includes:

[0121] When the first comparison result is that the access data is greater than 0 and less than the reference access data corresponding to the pre-cooling attribute, and the second comparison result is that the access growth data is less than the reference access growth data corresponding to the pre-cooling attribute, the pre-cooling attribute is determined as the access attribute of the target storage file;

[0122] When the first comparison result is that the access data is greater than 0 and less than the reference access data corresponding to the potential hot attribute, and the second comparison result is that the access growth data is greater than the reference access growth data corresponding to the potential hot attribute, the potential hot attribute is determined as the access attribute of the target storage file.

[0123] In a specific implementation manner, when determining whether the access attribute of the target storage file is a cold attribute or a hot attribute specifically, the access data of the target storage file can be used alone for determination. In this implementation manner, the cold attribute can include: a frozen attribute; the hot attribute can include: the current hot attribute, a thawed attribute, and a local hot attribute;

[0124] Determining the access attribute of the target storage file according to the access data may specifically include:

[0125] When the access data for the most recent N days is 0, the frozen attribute is determined as the access attribute of the target storage file, where N is a natural number;

[0126] When the access data is greater than the reference access data corresponding to the current hot attribute, the current hot attribute is determined as the access attribute of the target storage file;

[0127] When the access data for the most recent N days is not 0 and the access data for the previous N days is 0, the thawed attribute is determined as the access attribute of the target storage file;

[0128] When the number of access people in the access data is greater than 0 and less than the reference number of access people corresponding to the local hot attribute, and at the same time the number of access times in the access data is greater than the reference number of access times corresponding to the local hot attribute, the local hot attribute is determined as the access attribute of the target storage file.

[0129] It can be understood that the most recent N days and the previous N days respectively refer to a time period composed of N days. The most recent N days refer to a time period counting N days forward based on the current time, and the previous N days are the previous N-day time period of the most recent N days. For example, when the current time is the 15th and N is 3, the most recent N days are: a time period composed of the 15th, 14th, and 13th; the previous N days refer to: a time period composed of the 12th, 11th, and 10th.

[0130] It can be understood that when the above access data is compared, relationships such as less than, greater than, or less than or equal to mean that both the number of access people and the number of access times satisfy the above relationships. For example, the access data being greater than 0 means that both the number of access people and the number of access times are greater than 0, and the access data being less than the reference access data corresponding to the pre-cooling attribute means that the number of access people is less than the reference number of access people corresponding to the pre-cooling attribute, and at the same time the number of access times is also less than the reference number of access times corresponding to the pre-cooling attribute.

[0131] Similarly, when comparing access growth data, relationships such as less than, greater than, or less than or equal to mean that both the growth rate of the number of visitors and the growth rate of the number of accesses satisfy the above relationships. For example, the access growth data being less than the reference access growth data corresponding to the pre-cooling attribute means that the growth rate of the number of visitors is less than the reference growth rate of the number of visitors corresponding to the pre-cooling attribute, and at the same time, the growth rate of the number of accesses is also less than the reference growth rate of the number of accesses corresponding to the pre-cooling attribute.

[0132] Meanwhile, the access data being 0 or not 0 as referred to in the above description means that both the number of visitors and the number of accesses in the access data are 0 or not 0.

[0133] For example, when the number of visitors to the target storage file A is 30 and the number of accesses is 100, the number of visitors 30 is greater than the reference number of visitors 25 corresponding to the current hot attribute, and the number of accesses 100 is greater than the reference number of accesses 80 corresponding to the current hot attribute. At this time, the current hot attribute can be used as the access attribute of the target storage file.

[0134] Step 204: When the access attribute is a cold attribute, determine the object storage policy as the target storage policy of the target storage file.

[0135] In this embodiment, different access attributes may correspond to different storage policies. When the access attribute of the target storage file is a cold attribute, it means that the target storage file is rarely accessed by users in the recent time period. For the target storage file with a cold attribute, due to the low access demand of users, considering storage savings and reducing storage costs, the current target storage policy of the target storage file can be set to the object storage policy, that is, the target storage file can be stored using Object Storage Service (OSS), and the storage cost of this storage method is relatively low.

[0136] Step 205: When the access attribute is a hot attribute, determine the replica storage policy as the target storage policy of the target storage file.

[0137] When the access attribute of the target storage file is a hot attribute, it means that the target storage file will be accessed relatively frequently by users in the recent time period. For the target storage file with a hot attribute, due to the high access demand of users, in order to improve the access efficiency of the target storage file, meet the user's requirements for file read and write speed, and ensure the high data availability of the target storage file, the current target storage policy of the target storage file can be set to the replica storage policy. For example, the target storage file can be stored using the Block replication storage method.

[0138] The replica storage policy in this embodiment includes: high replicas and normal replicas; where the number of replicas corresponding to high replicas is greater than the number of replicas corresponding to normal replicas. The more the number of replicas, the higher the read performance and efficiency of the data, but the higher its storage cost.

[0139] In one implementation, when the access attribute is a hot attribute, the replica storage policy is determined as the target storage policy for the target storage file, specifically including:

[0140] When the access attribute is the current hot attribute or potential hot attribute among the hot attributes, the high replica is determined as the target storage policy for the target storage file;

[0141] When the access attribute is the thawed attribute or local hot attribute among the hot attributes, the normal replica is determined as the target storage policy for the target storage file.

[0142] It can be understood that, from the definition of the foregoing access attributes, the access heat of the current hot attribute and potential hot attribute is higher than that of the thawed attribute and local hot attribute. Therefore, when the access attribute is the current hot attribute or potential hot attribute, it indicates that the target storage file is frequently accessed. To ensure the read performance and efficiency of the target storage file at this time, the high replica with higher read performance and efficiency is determined as the target storage policy for the target storage file. When the access attribute is the thawed attribute or local hot attribute, it indicates that the access of the target storage file is not particularly frequent. Considering factors such as storage cost, file read and write performance, and read and write efficiency comprehensively, the normal replica in the replica storage policy with relatively low storage cost, relatively balanced storage performance and efficiency, and capable of ensuring user access requirements is determined as the target storage policy for the target storage file.

[0143] In summary, the target storage policies corresponding to different access attributes are as shown in Table 2 below:

[0144] Table 2

[0145] Access attribute Target storage policy Current hot attribute High replica Potential hot attribute High replica Thaw attribute Normal replica Local hot attribute Normal replica Pre-cooling attribute Object storage Freezing attribute Object storage

[0146] In a specific example: when the access attribute of the target storage file A is the current hot attribute, its corresponding target storage policy is: high replica.

[0147] Step 206, query the historical storage policy of the target storage file in the file storage system.

[0148] It can be understood that the description of step 206 is the same as the description of step 103 in Embodiment 1. For specific details, reference can be made to the above description and will not be elaborated in this embodiment.

[0149] Step 207, compare the target storage policy with the historical storage policy to obtain a comparison result.

[0150] After obtaining the historical storage policy of the target storage file and the target storage policy currently adapted to the target storage file, the two are compared for consistency to obtain the comparison result of whether they are consistent.

[0151] For example: The target storage policy of target storage file A is: high replicas, and the historical storage policy is normal replicas. Obviously, the two are inconsistent. At this time, the comparison result is that the target storage policy is inconsistent with the historical storage policy.

[0152] Step 208, when the comparison result is that the target storage policy is inconsistent with the historical storage policy, update the historical storage policy to the target storage policy, and perform a storage operation on the target storage file according to the target storage policy.

[0153] When the comparison result is that the target storage policy is inconsistent with the historical storage policy, it is necessary to change the historical storage policy to the target storage policy currently adapted, and perform a storage operation on the target storage file according to the target storage policy, so as to ensure that the storage policy of the target storage file in the file storage system is applicable to the current user's access requirements for the target storage file, so as to ensure the high data availability of the target storage file in the file storage system.

[0154] It can be understood that the storage operation can be to increase replicas of the storage file, or to reduce replicas of the storage file, or to change the storage policy of the storage file. For example: When the target storage policy of the target storage file is high replicas and the historical storage policy is normal replicas, the storage operation performed on the current target storage file is to increase replicas. When the target storage policy of the target storage file is normal replicas and the historical storage policy is high replicas, the storage operation performed on the current target storage file is to reduce replicas. When the target storage policy of the target storage file is normal replicas or high replicas and the historical storage policy is object storage, the storage operation performed on the current target storage file is to change the object storage policy to normal replicas or high replicas. When the target storage policy of the target storage file is the object storage policy and the historical storage policy is high replicas or normal replicas, the storage operation performed on the current target storage file is to change high replicas or normal replicas to the object storage policy.

[0155] For example, when the target storage policy of target storage file A is: high replicas and the historical storage policy is high replicas, the corresponding storage operation at this time is to increase replicas.

[0156] Step 209, when the comparison result is that the target storage policy is consistent with the historical storage policy, continue to perform a storage operation on the target storage file according to the historical storage policy.

[0157] When the comparison result shows that the target storage policy is the same as the historical storage policy, it indicates that the historical storage policy of the target storage file is also applicable to the current user requirements. Therefore, there is no need to allocate additional computing resources to update this historical storage policy, and the storage operation on the target storage file can continue to be performed according to the historical storage policy.

[0158] In this embodiment, the access attribute of the target storage file is distinguished as cold attribute or hot attribute according to the user's access situation of the target storage file. For the target storage file with cold attribute, a low-cost object storage policy is used for storage, which greatly reduces the storage cost; for the target storage file with hot attribute, a replica storage policy such as the Block replication storage method is used for storage, and the number of stored replicas is increased for the hot attribute file with a higher access heat, improving the data access performance and access efficiency of the target storage file, shortening the data query time, and the storage space optimization is more obvious, which can ensure high data availability and improve the user experience.

[0159] In the data storage method of this embodiment, in order to avoid a large consumption of data storage space in the file storage system, the access attribute of the target storage file in the file storage system is preferentially determined. The access attributes include: cold attribute and hot attribute; a target storage policy adapted to its access attribute is generated for the target storage file. This target storage policy is generated after determining that the access attribute of the target storage file is cold attribute or hot attribute, and when the access attribute changes, this target storage policy will also change accordingly; after formulating the currently adapted target storage policy for the target storage file, query the historical storage policy of the target storage file in the file storage system, compare the target storage policy with the historical storage policy, and perform a storage operation on the target storage file according to the comparison result of the target storage policy and the historical storage policy. This storage operation is an operation performed based on the comparison result of the current target storage policy and the historical storage policy of the target storage file, and this comparison result will change with different target storage policies. Therefore, the storage operation performed on the target storage file will also change accordingly, that is, for the same target storage file, the corresponding matching storage policy and storage operation in the file storage system are different under different access attributes. This is beneficial to ensuring the high data availability of the target storage file at any time in the system service, and can reduce the additional bandwidth consumption when accessing the target storage file while maintaining the data read and write speed.

[0160] The above is the second embodiment of a data storage method provided by this application. The following is an embodiment of a data storage device provided by this application.

[0161] Please refer to Figure 3 , Figure 3Schematic structural diagram of an embodiment of a data storage device in an embodiment of the present application.

[0162] In this embodiment, the data storage device includes:

[0163] A determination unit, configured to determine the access attribute of a target storage file in a file storage system, where the access attribute includes: a cold attribute and a hot attribute;

[0164] A generation unit, configured to generate a target storage policy adapted to the access attribute for the target storage file;

[0165] A query unit, configured to query the historical storage policy of the target storage file in the file storage system;

[0166] A storage unit, configured to perform a storage operation on the target storage file according to the comparison result between the target storage policy and the historical storage policy.

[0167] Optionally, the determination unit specifically includes:

[0168] A first determination subunit, configured to obtain the target storage file and the log file in the file storage system;

[0169] A filtering subunit, configured to filter out the access data corresponding to the target storage file from the log file;

[0170] A second determination subunit, configured to determine the access attribute of the target storage file according to the access data.

[0171] Optionally, the cold attribute includes: a precooling attribute; the hot attribute includes: a potential hot spot attribute; the second determination subunit specifically includes:

[0172] A calculation sub-subunit, configured to calculate access growth data according to the access data;

[0173] An obtaining sub-subunit, configured to obtain the reference access data and the reference access growth data corresponding to the precooling attribute and the potential hot spot attribute respectively;

[0174] A first comparison sub-subunit, configured to compare the access data with the reference access data corresponding to the precooling attribute and the potential hot spot attribute respectively to obtain a first comparison result;

[0175] A second comparison sub-subunit, configured to compare the access growth data with the reference access growth data corresponding to the precooling attribute and the potential hot spot attribute respectively to obtain a second comparison result;

[0176] A first determination sub-subunit, configured to combine the first comparison result and the second comparison result to determine that the access attribute of the target storage file is a cold attribute or a hot attribute.

[0177] Optionally, combining the first comparison result and the second comparison result, determine the access attribute of the target storage file as a cold attribute or a hot attribute, specifically including:

[0178] When the first comparison result is that the access data is greater than 0 and less than the reference access data corresponding to the pre-cooling attribute, and the second comparison result is that the access growth data is less than the reference access growth data corresponding to the pre-cooling attribute, determine the pre-cooling attribute as the access attribute of the target storage file;

[0179] When the first comparison result is that the access data is greater than 0 and less than the reference access data corresponding to the potential hot spot attribute, and the second comparison result is that the access growth data is greater than the reference access growth data corresponding to the potential hot spot attribute, determine the potential hot spot attribute as the access attribute of the target storage file.

[0180] Optionally, the access data includes: the number of visitors and the number of accesses;

[0181] According to the access data, calculate the access growth data, specifically including:

[0182] Based on the number of visitors, calculate the growth rate of the number of visitors in the access growth data;

[0183] Based on the number of accesses, calculate the growth rate of the number of accesses in the access growth data.

[0184] Optionally, the cold attribute includes: the freezing attribute; the hot attribute includes: the current hot spot attribute, the thawing attribute, and the local hot spot attribute; the second determination subunit specifically includes: specifically including:

[0185] The second determination sub-subunit is used to determine the freezing attribute as the access attribute of the target storage file when the access data in the most recent N days is 0, where N is a natural number;

[0186] The third determination sub-subunit is used to determine the current hot spot attribute as the access attribute of the target storage file when the access data is greater than the reference access data corresponding to the current hot spot attribute;

[0187] The fourth determination sub-subunit is used to determine the thawing attribute as the access attribute of the target storage file when the access data in the most recent N days is not 0 and the access data in the previous N days is 0;

[0188] The fifth determination sub-subunit is used to determine the local hot spot attribute as the access attribute of the target storage file when the number of visitors in the access data is greater than 0 and less than the reference number of visitors corresponding to the local hot spot attribute, and the number of accesses in the access data is greater than the reference number of accesses corresponding to the local hot spot attribute.

[0189] Specifically, the generation unit specifically includes:

[0190] The first generation subunit is used to determine the target storage policy of the target storage file as the target storage policy when the access attribute is a cold attribute;

[0191] The second generation subunit is used to determine the replica storage policy of the target storage file as the target storage policy when the access attribute is a hot attribute.

[0192] Optionally, the replica storage policy includes: high replicas and normal replicas;

[0193] The second generation subunit specifically includes:

[0194] The first generation sub-subunit is used to determine high replicas as the target storage policy of the target storage file when the access attribute is the current hot attribute or potential hot attribute in the hot attributes;

[0195] The second generation sub-subunit is used to determine normal replicas as the target storage policy of the target storage file when the access attribute is the thawed attribute or local hot attribute in the hot attributes.

[0196] Optionally, the storage unit specifically includes:

[0197] The comparison subunit is used to compare the target storage policy with the historical storage policy to obtain a comparison result;

[0198] The first storage subunit is used to update the historical storage policy to the target storage policy and perform a storage operation on the target storage file according to the target storage policy when the comparison result is that the target storage policy is inconsistent with the historical storage policy;

[0199] The second storage subunit is used to continue performing a storage operation on the target storage file according to the historical storage policy when the comparison result is that the target storage policy is consistent with the historical storage policy.

[0200] It can be understood that Figure 4 This is a specific implementation structure diagram of a data storage device in this embodiment.

[0201] The data storage device includes: a management server, a controller, a log collection module, a log storage module, a log analysis module, an analysis result module, a policy generation module, and a policy execution module. The working process of this system is introduced as follows:

[0202] 1. The user connects to the HDFS management server (i.e., NameNode) through the client and initiates an access request for the target storage file.

[0203] 2. While responding to the user request, NameNode saves the user's access behavior in a log file.

[0204] 3. The log collection module extracts log files in real time, writes them to the log storage module, and reports the write result to the controller after successfully writing the log files.

[0205] 4. After receiving the log write result, the controller controls the log analysis module to execute the log analysis task. The log analysis module obtains the log files from the log storage module, filters and counts the number of users accessing the target storage file and the number of access times, writes the statistical result to the analysis result module, and reports the write result to the controller.

[0206] 5. After receiving the analysis result write, the controller controls the policy generation module to execute the policy generation task. The policy generation module obtains the analysis results of the most recent N days from the analysis result module, analyzes the analysis results from four dimensions: the number of access people, the number of access times, the growth rate of the number of access people, and the growth rate of the number of access times, determines the access attributes of the target storage file and the target storage policy, and reports the generated target storage policy to the controller.

[0207] 6. After receiving the target storage policy, the controller controls the policy execution module to execute the policy at a predetermined time point. The policy execution module obtains the target storage policy and the historical storage policy from the policy generation module and makes a comparison to obtain the comparison result.

[0208] 7. Execute the storage operation through the NameNode, and report the change result to the controller after the change is completed.

[0209] In the data storage device of this embodiment, to avoid a large consumption of data storage space in the file storage system, the access attributes of the target storage file in the file storage system are preferentially determined. The access attributes include: cold attribute and hot attribute. A target storage policy adapted to its access attribute is generated for the target storage file. This target storage policy is generated after determining that the access attribute of the target storage file is a cold attribute or a hot attribute. When the access attribute changes, the target storage policy will also change accordingly. After formulating the currently adapted target storage policy for the target storage file, the historical storage policy of the target storage file in the file storage system is queried, and the target storage policy is compared with the historical storage policy. According to the comparison result of the target storage policy and the historical storage policy, a storage operation is performed on the target storage file. Since this storage operation is an operation performed based on the comparison result of the current target storage policy and the historical storage policy of the target storage file, and this comparison result will change with different target storage policies, the storage operation performed on the target storage file will also change accordingly. That is, for the same target storage file, when its access attributes are different, the corresponding matching storage policies and storage operations in the file storage system are also different. This is beneficial to ensuring the high data availability of the target storage file at any moment of the system service, and can reduce the additional bandwidth consumption when accessing the target storage file while maintaining the data read and write speed.

[0210] An embodiment of a data storage device is also provided in an embodiment of this application. The storage device in this embodiment includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the data storage method in the above embodiment according to the instructions in the program code.

[0211] An embodiment of a computer-readable storage medium is also provided in an embodiment of this application. This computer-readable storage medium is used to store program code, and the program code is used to execute the data storage method in the above embodiment.

[0212] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0213] In the description of the present application and the above-mentioned accompanying drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0214] It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist simultaneously. Here, A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or its similar expressions refer to any combination of these items, including any combination of single items (ones) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0215] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in an electrical, mechanical, or other form.

[0216] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0217] In addition, in each embodiment of the present application, each functional unit may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0218] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (English full name: Read-Only Memory, English abbreviation: ROM), random access memories (English full name: Random Access Memory, English abbreviation: RAM), magnetic disks, or optical discs that can store program codes.

[0219] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A data storage method, characterized in that, Including: Determine the access attributes of a target storage file in a file storage system, where the access attributes include: cold attributes and hot attributes; Generate a target storage policy adapted to the access attributes for the target storage file; Query the historical storage policy of the target storage file in the file storage system; Perform a storage operation on the target storage file according to the comparison result between the target storage policy and the historical storage policy; The performing a storage operation on the target storage file according to the comparison result between the target storage policy and the historical storage policy specifically includes: Compare the target storage policy with the historical storage policy to obtain a comparison result; When the comparison result is that the target storage policy is inconsistent with the historical storage policy, update the historical storage policy to the target storage policy and perform a storage operation on the target storage file according to the target storage policy; When the comparison result is that the target storage policy is consistent with the historical storage policy, continue to perform a storage operation on the target storage file according to the historical storage policy.

2. The data storage method according to claim 1, wherein The determining the access attributes of a target storage file in a file storage system specifically includes: Obtain the target storage file and the log file in the file storage system; Filter out the access data corresponding to the target storage file from the log file; Determine the access attributes of the target storage file according to the access data.

3. The data storage method according to claim 2, characterized in that, The cold attributes include: precooling attributes; the hot attributes include: potential hot spot attributes; The determining the access attributes of the target storage file according to the access data specifically includes: Calculate access growth data according to the access data; Obtain the reference access data and reference access growth data corresponding to the precooling attributes and the potential hot spot attributes respectively; Compare the access data with the reference access data corresponding to the precooling attributes and the potential hot spot attributes respectively to obtain a first comparison result; Compare the access growth data with the reference access growth data corresponding to the precooling attributes and the potential hot spot attributes respectively to obtain a second comparison result; Combine the first comparison result and the second comparison result to determine that the access attributes of the target storage file are cold attributes or hot attributes.

4. The data storage method according to claim 3, wherein The combining the first comparison result and the second comparison result to determine that the access attributes of the target storage file are cold attributes or hot attributes specifically includes: When the first comparison result is that the access data is greater than 0 and less than the reference access data corresponding to the precooling attributes, and the second comparison result is that the access growth data is less than the reference access growth data corresponding to the precooling attributes, determine the precooling attributes as the access attributes of the target storage file; When the first comparison result is that the access data is greater than 0 and less than the reference access data corresponding to the potential hot spot attributes, and the second comparison result is that the access growth data is greater than the reference access growth data corresponding to the potential hot spot attributes, determine the potential hot spot attributes as the access attributes of the target storage file.

5. The data storage method according to claim 3, wherein The access data includes: the number of access people and the number of access times; Calculating access growth data according to the access data specifically includes: Calculating the growth rate of the number of accesses in the access growth data based on the number of accesses; Calculating the growth rate of the number of access times in the access growth data based on the number of access times.

6. The data storage method according to claim 2, wherein The cold attributes include: freezing attributes; the hot attributes include: current hot attributes, thawing attributes, and local hot attributes; Determining the access attributes of the target storage file according to the access data specifically includes: When the access data for the most recent N days is 0, determining the freezing attribute as the access attribute of the target storage file, where N is a natural number; When the access data is greater than the reference access data corresponding to the current hot attribute, determining the current hot attribute as the access attribute of the target storage file; When the access data for the most recent N days is not 0 and the access data for the previous N days is 0, determining the thawing attribute as the access attribute of the target storage file; When the number of accesses in the access data is greater than 0 and less than the reference number of accesses corresponding to the local hot attribute, and the number of access times in the access data is greater than the reference number of access times corresponding to the local hot attribute, determining the local hot attribute as the access attribute of the target storage file.

7. The data storage method according to any one of claims 1 to 6, characterized in that, Generating a target storage strategy adapted to the access attribute for the target storage file specifically includes: When the access attribute is a cold attribute, determining the object storage strategy as the target storage strategy of the target storage file; When the access attribute is a hot attribute, determining the replica storage strategy as the target storage strategy of the target storage file.

8. The data storage method according to claim 7, characterized in that The replica storage strategy includes: high replicas and normal replicas; When the access attribute is a hot attribute, determining the replica storage strategy as the target storage strategy of the target storage file specifically includes: When the access attribute is the current hot attribute or potential hot attribute in the hot attributes, determining the high replicas as the target storage strategy of the target storage file; When the access attribute is the thawing attribute or local hot attribute in the hot attributes, determining the normal replicas as the target storage strategy of the target storage file.

9. A data storage device, characterized in that, Including: A determination unit for determining the access attributes of the target storage file in the file storage system, where the access attributes include: cold attributes and hot attributes; A generation unit for generating a target storage strategy adapted to the access attribute for the target storage file; A query unit for querying the historical storage strategy of the target storage file in the file storage system; A storage unit for performing a storage operation on the target storage file according to the comparison result between the target storage strategy and the historical storage strategy; The storage unit specifically includes: A comparison subunit for comparing the target storage strategy and the historical storage strategy to obtain a comparison result; A first storage subunit for, when the comparison result is that the target storage strategy is inconsistent with the historical storage strategy, updating the historical storage strategy to the target storage strategy and performing a storage operation on the target storage file according to the target storage strategy; A second storage subunit, configured to continue performing a storage operation on a target storage file according to a historical storage policy when a comparison result indicates that the target storage policy is consistent with the historical storage policy.

10. A data storage device, characterized in that, The storage device includes a processor and a memory; The memory is configured to store program code and transmit the program code to the processor; The processor is configured to execute the data storage method according to any one of claims 1 to 8 based on instructions in the program code.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store program code, and the program code is configured to execute the data storage method according to any one of claims 1 to 8.

Images