Data processing method, device, and storage medium

Abstract

Embodiments of the present invention provide a data processing method, a device, and a storage medium. The method comprises: acquiring first access log information of a block device in a first time period, wherein the block device corresponds to a plurality of logical block addresses; determining a target grouping length on the basis of the first access log information so as to group the plurality of logical block addresses on the basis of the target grouping length to obtain a plurality of groups, wherein the logical block addresses in a same group have similar access behaviors; determining access feature values respectively corresponding to the plurality of groups; and on the basis of the access feature values respectively corresponding to the plurality of groups, determining cold and hot data categories respectively corresponding to the plurality of groups.

Description

[0001] Data processing method, device and storage medium technical field

[0002]

[0001] The present disclosure relates to the field of cloud computing technology, and more particularly to a data processing method, device, and storage medium.

[0003] Block devices, commonly known as cloud disks, are components of cloud virtual machines. A block device has a certain storage capacity (e.g., 50GB, 100GB, etc.) and contains several consecutive logical block addresses (LBAs). An LBA is the smallest storage unit of a block device. Different LBAs of the same block device can be distributed across different physical devices.

[0004]

[0003] When writing data, a block device uses a log-structured structure to append data to the LBA in the form of a log. Afterwards, the garbage collection (GC) mechanism is used to perform offline data cleaning, such as deleting invalid data and performing data dumping. When writing data, the data is usually compressed and written using a compression algorithm with a certain compression ratio. When the GC dumps the data in the LBA, the data in the LBA is first decompressed and read into the memory, and then the data is recompressed and written to the block device using a compression algorithm with another compression ratio. During the above data dump process, all data is processed using the same compression algorithm. Moreover, different LBAs may be distributed in a physical machine using the same storage medium, which is not conducive to improving subsequent data access efficiency and reducing storage costs. SUMMARY OF THE INVENTION

[0005]

[0004] The embodiments of the present disclosure provide a data processing method, device, and storage medium for efficiently and accurately identifying hot and cold data corresponding to multiple LBAs in a block device.

[0006]

[0005] In a first aspect, an embodiment of the present disclosure provides a data processing method, the method comprising: obtaining first access log information of a block device within a first time period, the block device corresponding to a plurality of logical block addresses; determining a target group length according to the first access log information, and performing group processing on the plurality of logical block addresses based on the target group length to obtain a plurality of groups; wherein the logical block addresses within the same group have similar access behaviors; determining access feature values ​​corresponding to each of the plurality of groups; and determining, based on the access feature values ​​corresponding to each of the plurality of groups, the hot and cold data categories corresponding to each of the plurality of groups.

[0007]

[0006] In a second aspect, an embodiment of the present disclosure provides a data processing device, comprising: an acquisition module for acquiring first access log information of a block device within a first time period, wherein the block device corresponds to multiple logical block addresses; a grouping module for determining a target group length according to the first access log information, and performing grouping processing on the multiple logical block addresses based on the target group length to obtain multiple groups; wherein the logical block addresses in the same group have similar access behaviors; a classification module for determining access feature values ​​corresponding to each of the multiple groups; and determining the hot and cold data categories corresponding to each of the multiple groups according to the access feature values ​​corresponding to each of the multiple groups.

[0008]

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the data processing method described in the first aspect.

[0009]

[0008] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the data processing method described in the first aspect.

[0010]

[0009] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, comprising: a computer program, which, when executed by a processor of an electronic device, enables the processor to at least implement the data processing method described in the first aspect.

[0011] In the solution provided by the embodiments of the present disclosure, a block device corresponds to multiple logical block addresses. To accurately and efficiently determine the hot and cold data categories of the data corresponding to the multiple logical block addresses (hereinafter referred to as the hot and cold data categories corresponding to the logical block addresses), the multiple logical block addresses are grouped. This allows the hot and cold data categories corresponding to the groups to be determined based on the access characteristics reflected by each group. This is used as the hot and cold data category corresponding to the logical block addresses contained in the group, eliminating the need to determine the hot and cold data categories of each logical block address individually. In a specific implementation, first access log information for the block device within a first time period is obtained, and a target packet length is determined based on the first access log information. The target packet length is used to group the multiple logical block addresses to obtain multiple groups. In the multiple groups obtained based on the target group length, the logical block addresses within any group have similar access behaviors. That is, the access behaviors corresponding to the logical block addresses within the same group are largely consistent. For example, more than half of the logical block addresses within the same group have been accessed, or more than half of the logical block addresses within the same group have not been accessed. In other words, by determining the target group length, this solution ensures the rational grouping of the multiple logical block addresses contained in the block device. Next, the access characteristic values ​​corresponding to each of the multiple groups are determined. Finally, based on the access characteristic values ​​corresponding to the multiple groups, the corresponding hot and cold data categories are determined for each of the multiple groups. Given the rationality of the grouping results, the logical block addresses within each group are guaranteed to have similar access behaviors. Therefore, the ultimately determined hot and cold data categories corresponding to each group accurately represent the hot and cold data categories of the logical block addresses contained within the group. Based on the determination of the hot and cold data categories for different logical block addresses within the block device, different compression algorithms or storage strategies can be used to process the data within the logical block addresses of different hot and cold data categories, thereby improving subsequent data access efficiency and reducing storage costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0012]

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are some embodiments of the present disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0013] FIG1 is a flow chart of a data processing method provided in an embodiment of the present disclosure;

[0014] FIG2 is a schematic diagram of a data processing method according to an embodiment of the present disclosure;

[0015] FIG3 is a flow chart of a method for determining a target packet length according to an embodiment of the present disclosure;

[0016] FIG4 is a schematic diagram of an LBA grouping according to an embodiment of the present disclosure;

[0017] FIG5 is a flow chart of another data processing method provided in an embodiment of the present disclosure;

[0018] FIG6 is a schematic diagram of a target score ranking result provided by an embodiment of the present disclosure;

[0019] FIG7 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present disclosure;

[0020]

[0019] FIG8 and FIG7 are schematic diagrams showing the structure of an electronic device corresponding to the data processing device provided in the embodiment.

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present disclosure, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present disclosure without inventive effort are within the scope of protection of the present disclosure.

[0022]

[0021] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0023]

[0022] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0024] The data processing method provided in the embodiments of the present disclosure can be executed by an electronic device, which can be a terminal device such as a PC, a laptop, a smart phone, or a server. The server can be a physical server including an independent host, or a virtual server, or a cloud server or server cluster.

[0025]

[0024] FIG1 is a flow chart of a data processing method provided by an embodiment of the present disclosure. As shown in FIG1, the method may include the following steps 101 to 104.

[0025] 101. Obtain first access log information of a block device within a first time period. The block device corresponds to multiple logical block addresses.

[0026]

[0026] 102. Determine a target group length according to the first access log information, and perform grouping processing on multiple logical block addresses based on the target group length to obtain multiple groups; wherein the logical block addresses in the same group have similar access behaviors.

[0027]

[0027] 103. Determine access characteristic values ​​corresponding to each of the multiple groups.

[0028]

[0028] 104. Determine the hot and cold data categories corresponding to the multiple groups according to the access feature values ​​corresponding to the multiple groups.

[0029] The data processing method provided in this embodiment is used to identify hot and cold data categories for data corresponding to multiple logical block addresses (LBAs) contained in a block device. Specifically, it determines whether the data stored in each LBA is cold or hot data. The hotness or coldness of data is related to the number of times the data is accessed. Frequently accessed data is generally referred to as hot data, while infrequently accessed data is referred to as cold data. Data access includes, but is not limited to, operations such as reading and writing data. In this embodiment, for ease of description, the hot and cold data categories of data corresponding to an LBA are referred to as the hot and cold data categories of the LBA.

[0030] The number of LBAs in a block device is related to the capacity of the block device and the size of the LBAs. For example, if a block device has a capacity of 100GB and each LBA is 4KB, the number of LBAs it contains is 25*1024*1024. Therefore, a block device may contain tens of millions of different LBAs, a huge number. If the hot and cold data categories corresponding to each LBA were determined one by one, the amount of data required for analysis would be extremely large, making the analysis difficult and inefficient. Therefore, in this embodiment, when determining the hot and cold data categories corresponding to multiple LBAs in a block device, the multiple LBAs are first grouped. Then, the data access characteristics corresponding to each group are analyzed on a group-by-group basis to determine the hot and cold data categories corresponding to the group. These are used as the hot and cold data categories corresponding to the LBAs contained in the group. This eliminates the need to determine the hot and cold data categories of each logical block address one by one, thereby improving the efficiency of hot and cold data classification for multiple LBAs.

[0031] In practice, accesses to LBAs, such as writing data to an LBA or reading data from an LBA, are recorded in the form of logs, generating corresponding access logs. This embodiment obtains access log information for a block device before determining the grouping results of multiple LBAs and the data access characteristics reflected by each group. Since different LBAs may be distributed on different physical devices, the access log information for the block device may be obtained from different physical devices.

[0032] For ease of understanding, the present invention is described with reference to FIG2 . FIG2 is a schematic diagram illustrating a scenario of a data processing method provided by an embodiment of the present disclosure. As shown in FIG2 , in this embodiment, access log information of a block device is first obtained. This access log information can be used to group multiple LBAs contained in the block device and determine access characteristic values ​​corresponding to each of the multiple groups. For ease of distinction, the access log information within a first time period used for LBA grouping is referred to as first access log information, and the access log information within a second time period used for determining the access characteristics reflected by the grouping is referred to as second access log information. Optionally, the first access log and the second access log information can be the same or different.

[0033] During the LBA grouping stage, a target group length is determined for grouping the multiple LBAs based on the first access log information, so that the multiple LBAs are grouped to obtain multiple groups. The group length represents the number of LBAs contained in each group when the multiple LBAs are grouped. The target group length represents the group length that maximizes the ability to group LBAs with similar access behaviors. LBAs within the same group have similar access behaviors, meaning that the access behaviors corresponding to the LBAs within the same group are largely consistent. This includes, but is not limited to, more than half of the LBAs within the same group have been accessed, or more than half of the LBAs within the same group have not been accessed. For example, more than 90% of the LBAs within the same group have been accessed, or more than 90% of the LBAs within the same group have not been accessed. The user can customize the ratio of LBAs with the same access behavior within the same group to the total number of LBAs in the group. When the number of LBAs with the same access behavior within a group is greater than or equal to the customized ratio, the LBAs within the group are determined to have similar access behaviors. It is understood that when LBAs within the same group have similar access behaviors, the hot and cold data categories of the LBAs within the group are also consistent, i.e., the hot and cold data categories are the same. Therefore, by determining the hot and cold data categories corresponding to the group, the hot and cold data categories of the LBAs within the group can be determined.

[0034] During the access feature confirmation phase, the access feature values ​​corresponding to the multiple groups obtained by LBA grouping are determined based on the acquired second access log information. The access feature value corresponding to a particular group refers to the statistical results of the access behavior corresponding to the LBAs within the group during the second time period. Based on the access feature values ​​corresponding to the multiple groups, the hot and cold data categories corresponding to the multiple groups can be further determined. Given the rationality of the LBA grouping, the hot and cold data categories corresponding to the groups are, in other words, the hot and cold data categories corresponding to the multiple LBAs within the group.

[0035]

[0035] During specific implementation, optionally, the first time period and the second time period may be the same or different, that is, the first access log information and the second access log information may be the same or different.

[0036]

[0036] For example, in order to determine the hot and cold data categories corresponding to the multiple groups corresponding to the multiple LBAs of the block device in the past 24 hours, the past 24 hours can be used as the first time period and the second time period, and the access log information within these 24 hours is obtained as the first day access log information and the second access log information. That is, the first time period is the same as the second time period, and the first access log information is the same as the second access log information.

[0037]

[0037] For another example, in order to determine the hot and cold data categories corresponding to multiple groups corresponding to multiple LBAs of a block device within the past 24 hours, the one hour closest to the current time within the past 24 hours can be used as a first time period, and access log information within this one hour can be obtained as the first access log information. Since the first access log information has a small amount of information and thus a faster grouping speed, the efficiency of confirming the hot and cold data categories corresponding to the group can be improved. When analyzing the access characteristics of each group, in order to ensure the accuracy of the access characteristic value, access log information over a longer period of time can be combined. For example, the past 7 days (including the past 24 hours) can be used as a second time period, and access log information within the past 7 days can be obtained as the second access log information.

[0038] During specific implementation, the time lengths corresponding to the first and second time periods can be flexibly selected based on actual hot and cold data classification requirements. For example, when the accuracy requirement for hot and cold data classification is high, the first and second time periods with larger time coverage can be selected; when the accuracy requirement for hot and cold data classification is low, the first and second time periods with smaller time coverage can be selected. In this way, based on the first access log information within the first time period and the second access log information within the second time period, it can be determined whether multiple groups corresponding to multiple LBAs of a block device correspond to hot and cold data categories within any historical time period.

[0039] The hot and cold data categories of the same group in different historical time periods may be the same or different. The hot and cold data classification results of the group are related to its access behavior in the corresponding historical time period. For example, if a group is not accessed in the first historical time period, the corresponding hot and cold data is classified as cold data, but is frequently accessed in the second historical time period, the corresponding hot and cold data is classified as hot data.

[0039]

[0040] It should be noted that, when determining the hot and cold data categories corresponding to a plurality of groups within a certain historical time period, there is overlap between the first time period and the second time period and the historical time period.

[0040]

[0041] It is understandable that the access log information (including the first access log information and the second access log information) may contain multiple types of information describing access behavior. In this embodiment, to further improve the efficiency of determining the hot and cold data categories corresponding to each of the multiple groups, after obtaining the access log information, pre-processing such as data screening and format conversion can optionally be performed on the access log information. Subsequently, LBA grouping is performed based on the pre-processed access log information, and the access feature values ​​corresponding to the groups are determined.

[0041]

[0042] As an optional preprocessing method for access log information, for any access log information generated by an access, we can first filter out target access information related to access frequency from the access log information, and then organize the target access information into a pre-set data format. This ensures that the processed data has a unified data organization format when performing LBA grouping and determining the access characteristics corresponding to the grouping.

[0042]

[0043] Optionally, the target access information includes, but is not limited to, access operation type, offset address, access length, access time, etc.; the preset data organization format includes, but is not limited to, a tuple. For example, for the aforementioned types of target access information, a corresponding five-tuple (operation, offset, length, IO_start_time, IO_end_time) can be generated. Operation represents the operation type, indicating whether it is a write or read operation; offset represents the LBA offset address; length represents the length of the data address accessed; offset and length are used to determine which of the multiple LBAs of a block device the accessed LBA is; IO_start_time represents the start time of the access, and IO_end_time represents the end time of the access.

[0043]

[0044] As another optional preprocessing method for access log information, after obtaining the access log information, the access log information can be converted to a format that is convenient for computation and saves storage space. For example, access log information obtained from physical devices is generally in the Protocol Buffers (protobuf) format. This format can be converted to a stream format for storage to speed up parsing of the access log information and reduce memory usage during parsing.

[0044]

[0045] In summary, in this embodiment, first, based on the first access log information obtained from the block device during a first time period, a target group length for grouping multiple LBAs of the block device is determined. LBAs with similar access behavior are grouped together. Next, based on the second access log information obtained from the block device during a second time period, access characteristic values ​​for each group are determined. Based on the access characteristic values ​​corresponding to each of the multiple groups, the hot and cold data categories corresponding to each group are determined. Because LBAs within the same group have similar access behaviors, the hot and cold data categories determined based on the access characteristic values ​​corresponding to the group can be used to represent the hot and cold data categories corresponding to the LBAs within the group, ensuring the accuracy of the ultimately determined hot and cold data categories corresponding to the LBAs. Furthermore, by determining the hot and cold data categories corresponding to the group, the hot and cold data categories of multiple LBAs can be determined simultaneously. This significantly improves the efficiency of determining the hot and cold data categories of multiple LBAs in a block device compared to determining the hot and cold data categories of LBAs one by one. The results of determining the hot and cold data categories of different logical block addresses in a block device in this embodiment can be further applied to the data processing process in the logical block addresses. For example, different compression algorithms or storage strategies can be used to process the data in logical block addresses of different hot and cold data categories, thereby improving subsequent data access efficiency and reducing storage costs.

[0045]

[0046] The following describes in detail the LBA grouping process and the process of determining the access characteristics of the group in conjunction with other embodiments.

[0046]

[0047] FIG3 is a flow chart of a method for determining a target packet length provided by an embodiment of the present disclosure. As shown in FIG3 , the method includes the following steps 301 to 305.

[0047]

[0048] 30K determines multiple first packets obtained when multiple LBAs are grouped with a first packet length, and multiple second packets obtained when multiple LBAs are grouped with a second packet length, and the first packet length is smaller than the second packet length.

[0048]

[0049] 302. Determine access tags corresponding to each of the plurality of first groups and the plurality of second groups based on the first access log information, wherein if the target group contains an accessed LBA, the target group has a first access tag; if none of the logical block addresses contained in the target group have been accessed, the target group has a second access tag, and the target group is any one of the plurality of first groups and the plurality of second groups.

[0049]

[0050] 303. Determine the number of first groups corresponding to the second access label in the plurality of first groups, and the number of second groups corresponding to the second access label in the plurality of second groups.

[0050]

[0051] 304. If the ratio of the second number of packets to the first number of packets meets a set condition, determine the first packet length as the target packet length.

[0051]

[0052] 305. If the ratio of the second group number to the first group number does not meet the set condition, update the first group length and the second group length to determine a first group length that meets the set condition, wherein the updated first group length is greater than the first group length before the update, and the updated second group length is greater than the second group length before the update.

[0052]

[0053] When grouping multiple LBAs in a block device, a short group length results in a large number of groups, hindering the efficiency of determining the hot and cold data categories corresponding to the groups. A long group length, however, makes it impossible to ensure similar access behavior for LBAs within the same group. Therefore, it is necessary to determine a maximum group length as a target group length while ensuring similar access behavior for LBAs within the same group. This target group length is then used to group multiple LBAs when determining the hot and cold data categories corresponding to the LBAs.

[0053]

[0054] During the specific LBA grouping process, grouping is performed sequentially according to the order of the LBAs. For example, if the multiple LBAs of a block device are LBA1, LBA2, LBA3, and LBAn (n is a positive integer), and the group length is 3, then the multiple groups obtained by grouping the multiple LBAs are (LBA1, LBA2, LBA3), (LBA4, LBA5, LBA6), (LBA (n-2), LBA (n-1), LBAn). It can be seen that among the multiple groups obtained by grouping the multiple LBAs with a certain group length, a group may contain only unaccessed LBAs, only accessed LBAs, or both unaccessed and accessed LBAs. In this embodiment, a first access tag is set for a group containing accessed LBAs, and a second access tag is set for a group containing no accessed LBAs.

[0054]

[0055] It is understandable that as the packet length gradually increases, the number of LBAs contained in each packet gradually increases, thereby increasing the likelihood that unaccessed LBAs will be grouped into other packets containing accessed LBAs (i.e., packets with the first access tag). When the number of unaccessed LBAs in a packet with the first access tag exceeds an acceptable error range, the LBAs within the same packet can no longer be assumed to have similar access behavior. For example, for a packet with 100 packets, if only one of the 100 LBAs in the packet with the first access tag is unaccessed, the LBAs within the group can be assumed to have similar access behavior. However, if five of the 100 LBAs in the packet with the first access tag are unaccessed, the LBAs within the group can no longer be assumed to have similar access behavior.

[0055]

[0056] In this embodiment, the number of packets with second access tags corresponding to various packet lengths is used as a reference to determine whether, when performing LBA grouping at a certain packet length, multiple packets with similar access behaviors are grouped together. This is also the condition for determining the target packet length. When determining the target packet length, a recursive approach can be employed to gradually increase the packet length until the maximum packet length that meets the set condition is determined as the target packet length.

[0056]

[0057] Optionally, the set condition includes determining whether the ratio of the number of second packets corresponding to the second access label in the plurality of second packets obtained by grouping the plurality of LBA rows with the second packet length to the number of first packets corresponding to the second access label in the plurality of first packets obtained by grouping the plurality of LBA rows with the first packet length is less than or equal to a set threshold. The second packet length is greater than the first packet length, and the set threshold may be pre-set or determined based on historical LBA grouping results. When the ratio of the number of second packets to the number of first packets is less than or equal to the set threshold, it is determined that the LBAs within any second packet in the plurality of second packets obtained by grouping with the second packet length do not have similar access behavior, and the first packet length is determined as the target packet length. When the ratio of the number of second packets to the number of first packets is greater than the set threshold, it is determined that the LBAs within any second packet in the plurality of second packets obtained by grouping with the second packet length have similar access behavior. Therefore, the first packet length and the second packet length may be updated to further determine whether a packet length greater than the second packet length satisfies the set condition.

[0057]

[0058] Determining the set threshold based on historical LBA grouping results includes obtaining LBA grouping results corresponding to target group lengths determined based on different set thresholds, determining the target set threshold corresponding to the better grouping result, and using the target set threshold as the set threshold for currently determining the target group length. Optionally, the quality of the LBA grouping result can be evaluated based on the proportion of LBAs with the same access behavior within the same group. For example, in the LBA grouping result, the greater the proportion of LBAs with the same access behavior within the same group, the better the LBA grouping result.

[0058]

[0059] Optionally, the set condition also includes whether the packet length is less than or equal to an upper packet length limit, for example, whether the packet length is less than or equal to 100. The upper packet length limit can be flexibly set based on actual LBA grouping requirements. During specific implementations, when the second packet length is greater than the upper packet length limit and the ratio of the number of second packets to the number of first packets is greater than a set threshold, or when the second packet length is less than or equal to the upper packet length limit and the ratio of the number of second packets to the number of first packets is less than or equal to a set threshold, the first packet length less than or equal to the upper packet length limit is determined as the target packet length. When the second packet length is less than or equal to the upper packet length limit and the ratio of the number of second packets to the number of first packets is greater than a set threshold, the first and second packet lengths can be updated to further determine whether a packet length greater than the second packet length satisfies the set condition.

[0059]

[0060] For ease of understanding, for example, assume that the upper limit of the packet length is 10, the initial value of the first packet length k1 is 1, the initial value of the second packet length k2 is 2, and a threshold value s is set to 0.05. A block device contains 12 LBAs, namely LBA1, LBA2, LBA3, and LBA12. If "1" indicates access and "0" indicates non-access, the access status of LBA1 through LBA12 can be expressed as "001100110011", meaning that LBA3, LBA4, LBA7, LBA8, LBA1E, and LBA12 are accessed LBAs, and LBA1, LBA2, LBA5, LBA6, LBA9, and LBA10 are non-accessed LBAs. The access status of LBA1 through LBA12 is determined based on the first access log information.

[0060]

[0061] It is understandable that when k1=1, 12 first groups are obtained, one LBA corresponds to one group, and the LBAs in each group must have similar access behaviors. When k2=2, 6 second groups are obtained: (LBA1, LBA2),

[0061] (LBA3, LBA4), (LBA5, LBA6), (LBA7, LBA8), (LBA9, LBA10) and (LBA11, LBA12), each group contains two LBAs, and whether the LBAs in each group have similar access behaviors needs to be judged based on a set threshold.

[0062] Based on the aforementioned assumptions regarding access to LBA1 to LBA12, when k1=1, the number of first packets with second access labels in the 12 first packets is 6; when k2=2, the number of second packets with second access labels in the 6 second packets is (LBA1, LBA2), (LBA5, LBA6), and (LBA9, LBA10), i.e., the number of second packets with second access labels in the 6 second packets is 3. Since both packet lengths 1 and 2 are less than the packet upper limit of 10, and the ratio of the number of second packets, 3, to the number of first packets, 6, is 0.5, which is greater than the set threshold s=0.05, it is determined that when grouping with the second packet length k2=2, the LBAs within the 6 second packets have similar access behavior, a result consistent with the aforementioned assumption regarding LBA access. Furthermore, k1 and k2 can be updated to determine whether a packet length greater than 2 exists that meets the set conditions.

[0063] For example, k1 can be updated to 2 and k2 can be updated to 3. When k1=2, six first groups are obtained: (LBA1, LBA2), (LBA3, LBA4), (LBA5, LBA6), (LBA7, LBA8), (LBA9, LBA10), and (LBA11, LBA12). The above process has determined that the LBAs in each of these six first groups have similar access behaviors, and the number of first groups with second access labels in the six first groups is three. When k2=3, four second groups are obtained: (LBA1, LBA2, LBA3), (LBA4, LBA5, LBA6), (LBA7, LBA8), (LBA9, LBA10), and (LBA11, LBA12).

[0064] ( LBA7, LBA8, LBA9 ) and ( LBA10, LBA11, LBA12 ), each group contains three LBAs, and whether the LBAs in each group have similar access behaviors needs to be judged based on a set threshold.

[0065] Based on the above assumptions about the access conditions of LBA1 to LBA12, when k2=3, the number of second groups with second access tags in the four second groups is 0. Since both group lengths 2 and 3 are less than the grouping upper limit 10, and the ratio of the number of second groups 0 to the number of first groups 3 is 0.1, which is less than the set threshold value £=0.05, it is determined that among the four second groups obtained by grouping with k2=3, the LBAs in any second group do not have similar access behaviors. This result is consistent with the above assumptions about the LBA access conditions, for example, the group (LBA1, LBA2, LBA3) contains both accessed LBAs and unaccessed LBAs. Therefore, the first group length k1=2 is ultimately used as the target group length.

[0066]

[0065] It should be noted that the above examples are only for schematically illustrating the process of determining the target packet length. In the specific implementation process, the number of LBAs contained in the block device is determined by the capacity of the block device and the size of the LBA; the upper limit of the packet length and the set threshold can be flexibly set based on actual packet requirements; when updating the first number of packets and the second number of packets, different update strategies can be adopted, for example: the updated number of second packets can be N times the number of second packets before the update, where N is an integer greater than or equal to 1, and this is not limited in this embodiment.

[0067] In an optional embodiment, during the process of determining the target packet length, a plurality of time segments included in the first time period may be first determined. Then, a plurality of first packets corresponding to the plurality of time segments obtained when grouping the plurality of LBAs using the first packet length, and a plurality of second packets corresponding to the plurality of time segments obtained when grouping the plurality of LBAs using the second packet length, are determined. The plurality of first packets corresponding to different time segments are identical, and the plurality of second packets corresponding to different time segments are identical. Subsequently, the number of first packets corresponding to the second access label in the plurality of first packets within each time segment and the number of second packets corresponding to the second access label in the plurality of second packets within each time segment are determined. Finally, the number of first packets corresponding to the second access label in the plurality of first packets is determined based on the number of first packets corresponding to the plurality of time segments, and the number of second packets corresponding to the second access label in the plurality of second packets is determined based on the number of second packets corresponding to the plurality of time segments.

[0068]

[0067] Optionally, the sum of the numbers of first groups corresponding to the multiple time segments can be determined as the number of first groups corresponding to the second access label in the multiple first groups; and the sum of the numbers of second groups corresponding to the multiple time segments can be determined as the number of second groups corresponding to the second access label in the multiple second groups.

[0069] For ease of understanding, the process of determining the number of first groups is taken as an example and illustrated in conjunction with FIG4 . FIG4 is a schematic diagram of LBA grouping provided by an embodiment of the present disclosure. As shown in FIG4 , assuming that a block device contains nk LBAs and the first group length is k, the grouping results of nk LBAs grouped with the first group length k are: (LBA1, . . . , LBAk), (LBA(k+1), LBA2k), ..., (LBA(nk-k+1), LBAnk); the three time segments included in the first time period are: time segment 1 (time t0 to time t1), time segment 2 (time segment 3), and time segment 4 (time segment 5).

[0070] (time segment t1 to time segment t2), time segment 3 (time segment t2 to time segment t3). In any time segment, the grouping results are (LBA1, ..., LBAk), (LNA (k+1), LBA2k) (LBA

[0071] ( nk-k+1 ) , , LBAnk ) .

[0072] As shown in FIG4 , it is assumed that in time segment 1, (LBA1, LBAk) and (LNA(k+1), LBA2k) among the multiple first packets correspond to the second access label, that is, the number of first packets in time segment 1 is 2; in time segment 2, (LBA1, ..., LBAk) and (LNA(2k+1), LBA3k) among the multiple first packets correspond to the second access label, that is, the number of first packets in time segment 2 is 2; and in time segment 3, (LBA1, LBAk) among the multiple second packets corresponds to the second access label, that is, the number of first packets in time segment 3 is 1. Based on this assumption, it can be determined that the number of first packets corresponding to the second access label among the multiple first packets is 2+2+1=5.

[0073]

[0070] In this embodiment, by setting conditions, a target group length is determined from a plurality of different group lengths. On the one hand, this ensures that the LBAs contained in each group have similar access behaviors. On the other hand, this ensures that the number of groups obtained after grouping the plurality of LBAs of the block device is as small as possible, thereby achieving a high efficiency in identifying the categories of hot and cold data.

[0074]

[0071] FIG5 is a flow chart of another data processing method provided by an embodiment of the present disclosure. As shown in FIG5, the method includes at least the following steps 501 to 506.

[0075]

[0072] 501. Obtain the first access log information of the block device within the first time period, where the block device corresponds to multiple logical block addresses.

[0076]

[0073] 502. Determine a target group length according to the first access log information, and perform grouping processing on multiple logical block addresses based on the target group length to obtain multiple groups; wherein the logical block addresses in the same group have similar access behaviors.

[0077]

[0074] 503. Obtain second access log information of the block device within the second time period.

[0078]

[0075] 504. Determine, based on the second access log information, multiple access feature values ​​corresponding to each of the multiple groups.

[0079]

[0076] 505. Determine a target score corresponding to the target group based on multiple access feature values ​​corresponding to the target group and the extreme cold thresholds corresponding to the multiple access feature values. The target group is any one of the multiple groups. The extreme cold threshold is the access feature value corresponding to the target group when it has not been visited.

[0080]

[0077] 506. Determine the hot and cold data categories corresponding to the multiple groups according to the target scores corresponding to the multiple groups.

[0081]

[0078] The specific implementation process of steps 501 to 503 can refer to the above embodiment and will not be described in detail in this embodiment.

[0082]

[0079] In this embodiment, based on the second access log information, multiple access feature values ​​corresponding to each group are determined from multiple different dimensions, so that the access features corresponding to each group can be more comprehensively reflected, ensuring the accuracy of the hot and cold data categories corresponding to the determined groups.

[0083]

[0080] In an optional embodiment, the multiple access characteristic values ​​include a first access characteristic value, which is used to describe the number of accesses of the target group within a certain time period, such as the maximum number of accesses of the target group within a certain time period.

[0084] In step 504, multiple access characteristic values ​​corresponding to each of the multiple groups are determined based on the second access log information. This includes: determining multiple time segments included in the second time period; determining the number of accesses corresponding to each of the target groups in the multiple time segments based on the second access log information; and determining a statistical value of the number of accesses corresponding to each of the target groups as a first access characteristic value corresponding to the target group. The number of accesses corresponding to the target group is the sum of the number of accesses to each LBA within the target group. Each time an LBA within the target group is accessed, the number of accesses corresponding to the target group is incremented by one. Optionally, the statistical value of the number of accesses corresponding to each of the target groups includes the maximum value of the number of accesses corresponding to each of the target groups in the multiple time segments.

[0085]

[0082] For ease of understanding, for example, assuming that the second time period corresponds to a length of 7 days, and the second time period includes 7*24*60 / 10 time segments, that is, one time segment corresponds to every 10 minutes, then the number of visits corresponding to the target group in each of the 7*24*60 / 10 time segments can be first determined based on the second access log information; then, the maximum number of visits corresponding to the 7*24*60 / 10 time segments is determined as the first access feature value corresponding to the target group in the second time period. The first access feature value describes the maximum number of visits that may occur in a short period of time in the second time period (that is, within the time length corresponding to the time segment).

[0086]

[0083] Optionally, when determining the multiple time segments included in the second time period, multiple first time segments corresponding to longer time lengths may be first determined, and then the multiple second time segments included in each first time segment may be determined. Thereafter, based on the number of visits of the target group in each of the multiple second time segments, the first access characteristic value corresponding to the target group in each of the multiple first time segments may be determined. Finally, the first access characteristic value corresponding to the target group in each of the multiple first time segments may be determined as the first access characteristic value corresponding to the target group in the second time period.

[0087] For example, assuming the second time period is 7 days long, the second time period contains 7 first time segments, i.e., one first time segment per day, and each first time segment contains 24*60 / 10 second time segments, i.e., one second time segment every 10 minutes. For any first time segment, the number of visits corresponding to the target group in each of the 24*60 / 10 second time segments contained in the first time segment can be determined based on the second access log information, and the maximum value among the number of visits is used as the first access characteristic value corresponding to the first time segment. Based on the above assumptions, the first access characteristic values ​​corresponding to each of the 7 first time segments can be determined. Finally, these 7 first access characteristic values ​​are collectively used as the first access characteristic value corresponding to the target group in the second time period.

[0088]

[0085] In another optional embodiment, the multiple access characteristic values ​​include a second access characteristic value, which is used to describe the continuous access characteristics of the target group within a certain time period. The greater the proportion of the time that the target group is continuously accessed within a certain time period, the more likely it is that the target group is frequently accessed.

[0089]

[0086] In step 504, multiple access feature values ​​corresponding to each of the multiple groups are determined based on the second access log information, including: determining multiple time segments included in the second time period; determining access tag values ​​corresponding to the target group in the multiple time segments based on the second access log information; and determining the second access feature value corresponding to the target group based on the access tag values ​​corresponding to the target group in the multiple time segments.

[0090]

[0087] Among them, for any target time segment among the multiple time segments included in the second time period, if the target group has been accessed within the target time segment, the access mark value corresponding to the target group in the target time segment is a first bit value, such as 1; if the target group has not been accessed within the target time segment, the access mark value corresponding to the target group in the target time segment is a second bit value, such as 0.

[0091]

[0088] In practical applications, a bitmap or an access tag value sequence can be formed based on the access tag values ​​corresponding to the target group in multiple time segments to describe the continuous access characteristics of the target group in the second time period and obtain the second access feature value.

[0092] For example, assuming that the second time period is 7 days long and contains 7*24*60 / 10 time segments, i.e., one time segment every 10 minutes, the access tag value corresponding to the target group in each of the 7*24*60 / 10 time segments can be determined based on the second access log information to form an access tag value sequence. Assuming the access tag sequence is 1110001 10 (wherein this sequence indicates that the target group was accessed in the first to third time segments, and not in the fourth to sixth time segments, and so on), the second access feature value corresponding to the target group in the second time period can be determined based on the ratio of the number of 1s in the access tag sequence to the length of the entire access tag sequence. Optionally, if this ratio is greater than a certain threshold, it is determined that the target group was frequently accessed in the second time period; if this ratio is less than or equal to the certain threshold, it is determined that the target group was infrequently accessed in the second time period.

[0093]

[0090] In addition, according to the access mark sequence or bitmap, it is also possible to determine in which time segments of the target group within the second time period is continuously accessed, and whether it has the access feature of periodic access.

[0094]

[0091] Optionally, when determining the multiple time segments included in the second time period, multiple first time segments corresponding to longer time lengths may be first determined; then, multiple second time segments included in each first time segment may be determined. Thereafter, based on the second access log information, the access tag values ​​corresponding to the target group in each of the multiple second time segments within each first time segment may be determined as the second access feature values ​​corresponding to the target group in the first time segment. Finally, based on the second access feature values ​​corresponding to the target group in the multiple first time segments, the second access feature value corresponding to the target group in the second time period may be determined.

[0095]

[0092] In another optional embodiment, the multiple access characteristic values ​​include a third access characteristic value, which is used to describe the time difference between the last time the target group was accessed and the current time. The smaller the time difference, the more likely the target group is to be accessed again.

[0096]

[0093] In step 504, multiple access feature values ​​corresponding to each of the multiple groups are determined based on the second access log information, including: based on the second access log information, determining the time interval between the time when the second target group was last accessed within the second time period and the current time as the third access feature value corresponding to the second target group.

[0097]

[0094] In actual applications, optionally, the multiple access characteristic values ​​corresponding to the target group may include one or more of the first access characteristic value, the second access characteristic value, and the third access characteristic value.

[0098]

[0095] After determining the multiple access feature values ​​corresponding to each of the multiple groups, a target score for determining the hot and cold data categories corresponding to each group is determined based on the multiple access feature values ​​corresponding to each group.

[0099] In a specific implementation, multiple access feature values ​​corresponding to a group that has never been visited, i.e., an extreme cold threshold, can be first determined. Then, using the extreme cold threshold as a reference, a target score corresponding to the target group is determined based on the characteristic distances (e.g., Euclidean distance) between the multiple access feature values ​​corresponding to the target group and the extreme cold threshold. The first and second access feature values ​​in the extreme cold threshold are both 0, indicating that the group has never been visited, and the third access feature value is negative infinity.

[0100] As an optional method for determining the target score, multiple sub-scores corresponding to the target group can be determined based on characteristic distances between multiple access feature values ​​corresponding to the target group and corresponding extreme cold thresholds, where the multiple sub-scores correspond to the multiple access feature values. Then, the target score corresponding to the target group is determined based on the multiple sub-scores corresponding to the target group. Optionally, the target score can be determined by summing the multiple sub-scores corresponding to the target group, or by taking the logarithm of the sum of the multiple sub-scores corresponding to the target group.

[0101]

[0098] Optionally, the target score corresponding to the target group can be determined based on the characteristic distances between the various access characteristic values ​​corresponding to the target group and the corresponding extreme cold thresholds. For example, the target score can be positively correlated with the characteristic distance. Specifically, a smaller characteristic distance indicates that the hot and cold data category corresponding to the target group is more likely to be cold data, and the corresponding target score can be set smaller; a larger characteristic distance indicates that the hot and cold data category corresponding to the target group is more likely to be hot data, and the corresponding target score can be set larger.

[0102] After determining the target scores corresponding to the plurality of groups, a ranking result of the target scores corresponding to the plurality of groups is determined, for example, by ranking the target scores from largest to smallest or from smallest to largest. Subsequently, if the ranking result includes adjacent target scores whose target score difference is greater than a set score threshold, any one of the adjacent target scores is used as a first classification threshold to determine the hot and cold data categories corresponding to the plurality of groups. If no adjacent target scores exist, a preset second classification threshold is used to determine the hot and cold data categories corresponding to the plurality of groups.

[0103] For ease of understanding, for example, assume that LBA grouping results in 3 groups, wherein the target score corresponding to group 1 is Score1=10, the target score corresponding to group 2 is Score2=11, and the target score corresponding to group 3 is Score3=20. The score threshold is 5, and the target score is positively correlated with the above-mentioned feature distance. After sorting the target scores of the 3 groups from large to small, since the difference between Score2 and Score3 is greater than the score threshold 5, Score2 or Score3 can be used as the first classification threshold. For example, when Score2 is used as the first classification threshold, the group with a target score less than or equal to Score2 is sorted. The hot and cold data categories corresponding to group 1 and group 2 are cold data. The group with a target score greater than Score2 is determined as a hot and cold data category. The hot and cold data category corresponding to group 3 is hot data.

[0104]

[0101] Optionally, as shown in FIG6 , a curve corresponding to the sorted target scores may be determined, and then the slope corresponding to each point on the curve may be calculated. When the slope increases or decreases, the target score corresponding to the point where the slope increases or decreases is determined as the first classification threshold. FIG6 is a schematic diagram of a target score sorting result provided in an embodiment of the present disclosure.

[0105]

[0102] When there are no adjacent target scores whose target score difference is greater than a set score threshold in the sorting result, the hot and cold data categories corresponding to the multiple groups can be determined based on a preset second classification threshold. For example: when the score of the target group is greater than a certain classification threshold, its hot and cold data categories are determined to be hot data; when the score of the target group is less than or equal to a certain classification threshold, its hot and cold data categories are determined to be cold data.

[0106]

[0103] In addition, by using the first classification threshold, the second classification threshold and the target score corresponding to the target group, not only can the cold and hot data categories be classified into two categories, but the data can also be further classified into multiple cold and hot data categories such as cold data, warm data, and hot data.

[0107]

[0104] In this embodiment, by determining multiple access feature values ​​corresponding to each of the multiple groups based on the second access log information, the access features corresponding to each group are described from multiple dimensions, thereby ensuring the accuracy of the hot and cold data categories corresponding to the determined groups. In addition, based on the multiple access feature values ​​corresponding to the target group and the extremely cold thresholds corresponding to the multiple access feature values, a target score corresponding to the target group is determined, and based on the target score, the hot and cold data categories corresponding to the multiple groups are determined. On the one hand, this ensures the accuracy of the classification results, and on the other hand, it provides a solution for classifying multiple hot and cold data categories, that is, multiple different hot and cold data categories can be set and corresponding hot and cold data categories can be classified. The hot and cold data categories include, but are not limited to, cold data and hot data, and also warm and hot data.

[0105] In actual applications, when writing data, the block device uses a log-structured method to append data to the LBA in the form of a log. Afterwards, the garbage collection (GC) mechanism is used to perform offline data cleanup, such as deleting invalid data, performing data dumps, etc. When the GC dumps the data in the LBA, it usually first decompresses the data in the LBA and reads it into the memory, and then recompresses the data using a compression algorithm with a different compression ratio and writes it into the block device.

[0108]

[0106] In fact, during the data dump process, the higher the compression ratio of the compression algorithm used when the data is recompressed and written, the smaller the storage space occupied by the compressed and written data. However, when the data is subsequently read, the corresponding decompression time is longer, which can easily cause a long waiting time when accessing the data, affecting the user experience; the lower the compression ratio of the compression algorithm used when the data is recompressed and written, the shorter the decompression time is when the data is read. However, the storage space occupied by the compressed and written data is larger, which is not conducive to reducing storage costs.

[0109]

[0107] The data processing method provided in this embodiment can be applied to the data dump stage of GC. By determining the hot and cold data categories corresponding to different LBAs in the block device, and providing a compression algorithm or storage medium with an appropriate compression ratio for data of different hot and cold data categories, the data access efficiency is improved and the storage cost is reduced.

[0110] Specifically, when the GC needs to dump data within a certain time period, the data processing method provided in this embodiment can be used to determine the hot and cold data categories of each of the multiple groups corresponding to the multiple LBAs of the block device. The hot and cold data categories of each of the multiple groups are fed back to the GC in the format of, for example, a data table in a database, so that the GC can perform corresponding offline data dump operations based on the hot and cold data categories of each of the multiple groups. The hot and cold data categories corresponding to each group are also the hot and cold data categories corresponding to the LBAs contained in each group. When determining the hot and cold data categories of each of the multiple groups corresponding to the multiple LBAs of the block device, the first time period and the second time period match the time period corresponding to the data for which the GC needs to dump data. For example, the first time period and the second time period include the time period.

[0111] In an optional embodiment, the GC may determine compression algorithms corresponding to the multiple groups based on the cold and hot data categories corresponding to the multiple groups. The groups corresponding to cold data may use a compression algorithm with a first compression ratio, while the groups corresponding to hot data may use a compression algorithm with a second compression ratio, where the first compression ratio is higher than the second compression ratio. The GC then compresses the data written within the multiple groups using the compression algorithms corresponding to the multiple groups, that is, compresses the data written within each LBA. Because cold data has a lower access frequency and hot data has a higher access frequency, the first compression algorithm with a higher compression ratio may be used for cold data, while the second compression algorithm with a lower compression ratio may be used for hot data. This ensures that hot data can be quickly decompressed when frequently accessed, improves access efficiency, and saves storage space while ensuring a good user data access experience.

[0112] In another optional embodiment, the GC may further determine the storage medium type corresponding to each of the multiple groups based on the cold and hot data categories corresponding to each of the multiple groups, and store the data written to the multiple groups in the corresponding storage media based on the corresponding storage media types. The groups corresponding to the cold data utilize a storage medium with a first access performance, while the groups corresponding to the hot data utilize a second storage medium with a second access performance, where the second access performance is superior to the first access performance. The storage media include, but are not limited to, solid-state drives (SSDs) and mechanical hard disks. Descriptive metrics corresponding to the first and second access performances include, but are not limited to, input / output per second (IOPS), access latency, and throughput / bandwidth. In this solution, by using different storage media for storing cold and hot data, storage costs can be effectively reduced while ensuring access efficiency.

[0113]

[0111] The data processing devices of one or more embodiments of the present disclosure will be described in detail below. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0114]

[0112] FIG7 is a schematic structural diagram of a data processing device provided in an embodiment of the present disclosure. As shown in FIG7, the device includes: an acquisition module 11, a grouping module 12, and a classification module 13.

[0115]

[0113] The acquisition module 11 is used to obtain first access log information of a block device within a first time period, and the block device corresponds to multiple logical block addresses.

[0116]

[0114] The grouping module 12 is configured to determine a target group length according to the first access log information, and to group the plurality of logical block addresses based on the target group length to obtain a plurality of groups; wherein the logical block addresses within the same group have similar access behaviors.

[0117]

[0115] A classification module 13 is configured to determine the access feature values ​​corresponding to each of the plurality of groups; and to determine the hot and cold data categories corresponding to each of the plurality of groups based on the access feature values ​​corresponding to each of the plurality of groups.

[0118]

[0116] Optionally, the acquisition module 11 is further configured to acquire second access log information of the block device within a second time period.

[0119]

[0117] The classification module 13 is specifically configured to determine the access feature values ​​corresponding to each of the plurality of groups according to the second access log information.

[0120]

[0118] Optionally, the grouping module 12 is specifically configured to determine a plurality of first groups obtained when the plurality of logical block addresses are grouped using a first group length, and a plurality of second groups obtained when the plurality of logical block addresses are grouped using a second group length, wherein the first group length is smaller than the second group length; determine access tags corresponding to each of the plurality of first groups and the plurality of second groups based on the first access log information, wherein if a first target group includes an accessed logical block address, the first target group has a first access tag; if none of the logical block addresses included in the second target group has been accessed, the first target group has a second access tag, and the first target group is any one of the plurality of first groups and the plurality of second groups; determine the number of first groups corresponding to the second access tag in the plurality of first groups, and the number of second groups corresponding to the second access tag in the plurality of second groups; and determine the first group length to be the target group length if a ratio of the number of the second groups to the number of the first groups satisfies a set condition.

[0121]

[0119] Optionally, the grouping module 12 is further specifically configured to update the first group length and the second group length if the ratio of the second group number to the first group number does not satisfy the set condition, so as to determine a first group length that satisfies the set condition; wherein the updated first group length is greater than the first group length before the update, and the updated second group length is greater than the second group length before the update.

[0122]

[0120] Optionally, the grouping module 12 is further specifically configured to determine a plurality of time segments included in the first time period; determine a plurality of first groups corresponding to the plurality of time segments obtained when the plurality of logical block addresses are grouped using a first group length, and determine a plurality of second groups corresponding to the plurality of time segments obtained when the plurality of logical block addresses are grouped using a second group length, wherein the plurality of first groups corresponding to different time segments are the same, and the plurality of second groups corresponding to different time segments are the same.

[0123]

[0121] Optionally, the classification module 13 is specifically used to determine the multiple access feature values ​​corresponding to each of the multiple groups based on the second access log information; determine the target score corresponding to the second target group based on the multiple access feature values ​​corresponding to the second target group and the extremely cold threshold values ​​corresponding to the multiple access feature values, the second target group is any one of the multiple groups, and the extremely cold threshold value is the access feature value corresponding to the second target group when it has not been accessed; determine the hot and cold data categories corresponding to each of the multiple groups based on the target scores corresponding to the multiple groups.

[0124]

[0122] Optionally, the classification module 13 is further specifically used to determine a plurality of sub-scores corresponding to the second target group based on the characteristic distances between the plurality of access characteristic values ​​corresponding to the second target group and the corresponding extreme cold thresholds, wherein the plurality of sub-scores correspond to the plurality of access characteristic values; and determine the target score corresponding to the second target group based on the plurality of sub-scores corresponding to the second target group.

[0125]

[0123] Optionally, the classification module 13 is further specifically configured to determine a plurality of time segments included in the second time period; determine the number of visits corresponding to the second target group in the plurality of time segments according to the second access log information; and determine the statistical value of the number of visits corresponding to the second target group as the first access feature value corresponding to the second target group.

[0126]

[0124] Optionally, the classification module 13 is further specifically configured to determine a plurality of time segments included in the second time period; determine, based on the second access log information, the access tag values ​​corresponding to the second target group in the plurality of time segments; wherein, if the second target group has been accessed within a target time segment, the access tag value corresponding to the second target group in the target time segment is a first bit value; if the second target group has not been accessed within the target time segment, the access tag value corresponding to the second target group in the target time segment is a second bit value, and the target time segment is any one of the plurality of time segments; and determine, based on the access tag values ​​corresponding to the second target group in the plurality of time segments, a second access feature value corresponding to the second target group.

[0127]

[0125] Optionally, the classification module 13 is further configured to determine, based on the second access log information, a time interval between the last access time of the second target group within the second time period and the current time, as a third access feature value corresponding to the second target group.

[0128]

[0126] Optionally, the classification module 13 is further specifically configured to determine a ranking result of the target scores corresponding to each of the plurality of groups; if the ranking result includes adjacent target scores whose target score difference is greater than a set score threshold, using any one of the adjacent target scores as a first classification threshold to determine the cold and hot data categories corresponding to each of the plurality of groups; if the adjacent target scores do not exist, using a preset second classification threshold to determine the cold and hot data categories corresponding to each of the plurality of groups.

[0127] Optionally, the apparatus further includes a processing module configured to determine a compression algorithm corresponding to each of the plurality of groups based on the cold and hot data categories corresponding to each of the plurality of groups, wherein a compression algorithm having a first compression ratio is used for the group corresponding to cold data, and a compression algorithm having a second compression ratio is used for the group corresponding to hot data, wherein the first compression ratio is higher than the second compression ratio; and compressing the data written into the plurality of groups using the compression algorithm corresponding to each of the plurality of groups.

[0129]

[0128] Optionally, the processing module is further used to determine the storage medium type corresponding to each of the multiple groups according to the hot and cold data categories corresponding to each of the multiple groups; and store the data written in the multiple groups into the corresponding storage medium according to the storage medium type corresponding to each of the multiple groups.

[0130]

[0129] The device shown in Figure 7 can execute the steps introduced in the above embodiments. For detailed execution process and technical effects, please refer to the description in the above embodiments and will not be repeated here.

[0131] In one possible design, the structure of the data processing device shown in FIG. 7 can be implemented as an electronic device, as shown in FIG. 8 . The electronic device may include a memory 21, a processor 22, and a communication interface 23. The memory 21 stores executable code. When the executable code is executed by the processor 22, the processor 22 can at least implement the data processing method provided in the aforementioned embodiments.

[0132]

[0131] In addition, an embodiment of the present disclosure provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the data processing method provided in the aforementioned embodiment.

[0133]

[0132] An embodiment of the present disclosure provides a computer program product, comprising: a computer program, which, when executed by a processor of an electronic device, enables the processor to execute the data processing method provided in the aforementioned embodiment.

[0134] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0135]

[0134] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the existing technology can be embodied in the form of a computer product. The present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136]

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

Claims 1. A data processing method, wherein: include: Obtaining first access log information of a block device within a first time period, the block device corresponding to a plurality of logical block addresses; Determining a target group length based on the first access log information, and grouping the multiple logical block addresses based on the target group length to obtain multiple groups; wherein the logical block addresses within the same group have similar access behaviors; determining access feature values ​​corresponding to each of the multiple groups; and determining, based on the access feature values ​​corresponding to each of the multiple groups, hot and cold data categories corresponding to each of the multiple groups.

2. The method according to claim 1, wherein: The determining the access characteristic values ​​corresponding to each of the plurality of groups includes: obtaining second access log information of the block device within a second time period; and determining the access characteristic values ​​corresponding to each of the plurality of groups according to the second access log information.

3. The method according to claim 1, wherein: Determining a target packet length based on the first access log information includes: determining a plurality of first packets obtained when the plurality of logical block addresses are grouped using a first packet length, and a plurality of second packets obtained when the plurality of logical block addresses are grouped using a second packet length, wherein the first packet length is less than the second packet length; determining access tags corresponding to each of the plurality of first packets and the plurality of second packets based on the first access log information, wherein if a first target packet includes an accessed logical block address, the first target packet has a first access tag; if none of the logical block addresses included in the first target packet have been accessed, the first target packet has a second access tag, and the first target packet is any of the plurality of first packets and the plurality of second packets; determining the number of first packets corresponding to the second access tag in the plurality of first packets, and the number of second packets corresponding to the second access tag in the plurality of second packets; and determining the first packet length as the target packet length if a ratio of the number of the second packets to the number of the first packets satisfies a set condition.

4. The method according to claim 3, wherein: The method also includes: if the ratio of the second group number to the first group number does not meet the set condition, updating the first group length and the second group length to determine the first group length that meets the set condition; wherein, the updated first group length is greater than the first group length before the update, and the updated second group length is greater than the second group length before the update.

5. The method according to claim 3, wherein: The determining of the plurality of first groups obtained when the plurality of logical block addresses are grouped using the first group length, and the plurality of second groups obtained when the plurality of logical block addresses are grouped using the second group length, includes: Determine multiple time segments included in the first time period; determine multiple first groups corresponding to the multiple time segments obtained when the multiple logical block addresses are grouped with a first group length, and multiple second groups corresponding to the multiple time segments obtained when the multiple logical block addresses are grouped with a second group length, wherein the multiple first groups corresponding to different time segments are the same, and the multiple second groups corresponding to different time segments are the same.

6. The method according to claim 2, wherein: Determining the access feature values ​​corresponding to each of the multiple groups based on the second access log information includes: determining multiple access feature values ​​corresponding to each of the multiple groups based on the second access log information; determining the hot and cold data categories corresponding to each of the multiple groups based on the access feature values ​​corresponding to each of the multiple groups includes: determining the target score corresponding to the second target group based on the multiple access feature values ​​corresponding to the second target group and the extremely cold threshold values ​​corresponding to the multiple access feature values, wherein the second target group is any one of the multiple groups, and the extremely cold threshold value is the access feature value corresponding to the second target group when it has not been accessed; determining the hot and cold data categories corresponding to each of the multiple groups based on the target scores corresponding to the multiple groups.

7. The method according to claim 6, wherein: The method of determining the target score corresponding to the second target group based on the multiple access characteristic values ​​corresponding to the second target group and the extremely cold threshold values ​​corresponding to the multiple access characteristic values ​​includes: determining multiple sub-scores corresponding to the second target group based on the characteristic distances between the multiple access characteristic values ​​corresponding to the second target group and the corresponding extremely cold threshold values, the multiple sub-scores corresponding to the multiple access characteristic values; and determining the target score corresponding to the second target group based on the multiple sub-scores corresponding to the second target group.

8. The method according to claim 6, wherein: The method of determining the multiple access characteristic values ​​corresponding to each of the multiple groups based on the second access log information includes: determining the multiple time segments contained in the second time period; determining the number of visits corresponding to each of the second target groups in the multiple time segments based on the second access log information; and determining that the statistical value of the number of visits corresponding to each of the second target groups is the first access characteristic value corresponding to the second target group.

9. The method according to claim 6, wherein: The determining, based on the second access log information, the multiple access feature values ​​corresponding to each of the multiple groups, includes: determining multiple time segments included in the second time period; determining, based on the second access log information, access tag values ​​corresponding to the second target group in the multiple time segments; wherein, if the second target group has been accessed within a target time segment, the access tag value corresponding to the second target group in the target time segment is a first bit value; if the second target group has been accessed within a target time segment, the access tag value corresponding to the second target group in the target time segment is a first bit value; If the second target group has not been accessed within the target time segment, the access mark value corresponding to the second target group in the target time segment is a second bit value, and the target time segment is any one of the multiple time segments; according to the access mark values ​​corresponding to the second target group in the multiple time segments, the second access feature value corresponding to the second target group is determined.

10. The method according to claim 6, wherein: The method of determining the multiple access feature values ​​corresponding to each of the multiple groups based on the second access log information includes: determining the time interval between the last access time of the second target group within the second time period and the current time based on the second access log information, as the third access feature value corresponding to the second target group.

11. The method according to claim 6, wherein: The determining, based on the target scores corresponding to each of the multiple groups, the hot and cold data categories corresponding to each of the multiple groups includes: determining a ranking result of the target scores corresponding to each of the multiple groups; if there are adjacent target scores in the ranking result whose target score difference is greater than a set score threshold, using any one of the adjacent target scores as a first classification threshold to determine the hot and cold data categories corresponding to each of the multiple groups; if there are no adjacent target scores, using a preset second classification threshold to determine the hot and cold data categories corresponding to each of the multiple groups.

12. The method according to any one of claims 1 to 11, wherein: The method also includes: determining compression algorithms corresponding to each of the multiple groups based on the cold and hot data categories corresponding to each of the multiple groups, wherein the group corresponding to the cold data adopts a compression algorithm with a first compression ratio, and the group corresponding to the hot data adopts a compression algorithm with a second compression ratio, and the first compression ratio is higher than the second compression ratio; and compressing the data written in the multiple groups using the compression algorithms corresponding to each of the multiple groups.

13. The method according to any one of claims 1 to 11, wherein: The method further includes: determining the storage medium types corresponding to the multiple groups according to the hot and cold data categories corresponding to the multiple groups; and storing the data written in the multiple groups in the corresponding storage media according to the storage medium types corresponding to the multiple groups.

14. An electronic device, wherein: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the data processing method according to any one of claims 1 to 13.

15. A non-transitory machine-readable storage medium, wherein: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the data processing method according to any one of claims 1 to 13.

16. A computer program product, wherein: include: A computer program, when executed by a processor of an electronic device, causes the processor to execute the data processing method according to any one of claims 1 to 13.

Images