Data processing method and device for storage space, equipment and storage medium

Abstract

The embodiment of the invention discloses a data processing method and device for a storage space, equipment and a storage medium, and the related embodiments can be applied to various scenes such as artificial intelligence and intelligent transportation, and are used for improving the utilization rate of the storage space. The method comprises the steps that when a data writing request is received, to-be-processed data and the initial updating frequency of the to-be-processed data are obtained based on the data writing request; according to the updating frequency, the to-be-processed data is written into one of at least two groups in the storage space, the at least two groups comprise a first frequency group and a second frequency group, the first frequency group corresponds to the first updating frequency, the second frequency group corresponds to the second updating frequency, and the first updating frequency is higher than the second updating frequency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular, to a method, device, equipment and storage medium for data processing in a storage space. Background Art

[0002] In today's world where the amount of data is growing rapidly, Shingled Magnetic Recording (SMR) is a high-capacity disk that uses a new magnetic storage technology. SMR disks partially overlap the data tracks on the disk, just like tiles on a roof. This technology is called shingled magnetic recording technology, which can effectively reduce the disk storage cost per unit capacity and is the development trend of future high-density disk storage technology.

[0003] At present, the use of shingled magnetic recording mechanical hard disk SMR HDD is mainly to write the upper-level business data into the SMR HDD after serialization. Moreover, unlike the traditional mechanical hard disk HDD, if the data in certain locations is not needed, the data in these locations can be directly overwritten and written. After writing the data, if the SMR HDD needs to delete the data or delete part of the data, since the data or part of the data to be deleted is randomly distributed in each shingled magnetic recording zone (SMR zone), and the SMR zone cannot be directly overwritten, when deleting data, the data of the entire zone needs to be recovered before the data can be rewritten in the deleted position, that is, all the data in the zone is read out, the specified data is deleted in the cache, and then the data is rewritten in the zone or other zones. Such operation is likely to lead to low capacity utilization of the SMR zone, thereby leading to low utilization of the entire hard disk. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, device and storage medium for a storage space, which are used to group the storage space according to different update frequencies, so that the to-be-processed data corresponding to the different update frequencies can be written to the corresponding groups more specifically in the future, and the first frequency group corresponding to the first update frequency higher than the second update frequency can be recycled more frequently in a more targeted manner, while maintaining the second frequency group corresponding to the second update frequency to write as much data as possible, thereby improving the utilization rate of the disk storage space.

[0005] An embodiment of the present application provides a method for processing data in a storage space, including:

[0006] Obtaining the data to be processed based on the data processing request, and obtaining the update frequency of the data to be processed;

[0007] According to the update frequency, the data to be processed is written into one of at least two groups in the storage space, wherein the at least two groups include a first frequency group and a second frequency group, the first frequency group corresponds to a first update frequency, the second frequency group corresponds to a second update frequency, and the first update frequency is higher than the second update frequency.

[0008] On the other hand, the present application provides a storage space data processing method and device, including:

[0009] An acquisition unit, used to acquire the data to be processed based on the data processing request, and acquire the update frequency of the data to be processed;

[0010] The storage unit is used to write the data to be processed into one of at least two groups in the storage space according to the update frequency, wherein the at least two groups include a first frequency group and a second frequency group, the first frequency group corresponds to a first update frequency, the second frequency group corresponds to a second update frequency, and the first update frequency is higher than the second update frequency.

[0011] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0012] A processing unit, configured to, when detecting that the fragmentation rate in the first frequency group is greater than a fragmentation rate threshold, recycle the data in the first frequency group and continue to write data to the second frequency group;

[0013] The processing unit is further configured to recycle the data in the second frequency group when it is detected that the fragmentation rate in the second frequency group is greater than a fragmentation rate threshold.

[0014] In a possible design, in an implementation of another aspect of the embodiment of the present application, the acquisition unit may be specifically used to:

[0015] The statistical data identifies the historical read and write times in the storage space, and calculates the initial update frequency of the data to be processed based on the historical read and write times.

[0016] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0017] The acquiring unit is further used to take each to-be-processed data written into the first frequency group as the first frequency group data;

[0018] The processing unit is further used to count the number of times each first frequency group data is read and written from being written into the first frequency group to the present, and calculate the current update frequency of the first frequency group data based on the number of times it is read;

[0019] The storage unit is also used to convert the first frequency group data whose current update frequency is less than the first update frequency from the first group to other groups in the storage space for storage.

[0020] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0021] The acquiring unit is further used for taking each to-be-processed data written into the second frequency group as the second frequency group data;

[0022] The processing unit is further used to count the number of times each second frequency group data is read and written from being written into the second frequency group to the present, and calculate the current update frequency of the second frequency group data based on the number of times it is read;

[0023] The storage unit is also used to convert the second frequency group data whose current update frequency is greater than the second update frequency from the second group to other groups in the storage space for storage.

[0024] In a possible design, in an implementation of another aspect of the embodiment of the present application, the storage unit may be specifically used for:

[0025] Determine the grouping category corresponding to the data to be processed according to the update frequency of the data to be processed;

[0026] The data to be processed is written into one of at least two groups in the storage space according to the grouping category.

[0027] In a possible design, in an implementation of another aspect of the embodiment of the present application, the storage unit may be specifically used for:

[0028] If the update frequency of the data to be processed is higher than the first update frequency, determining that the grouping category corresponding to the data to be processed is a high-frequency group;

[0029] If the update frequency of the data to be processed is lower than the first update frequency and higher than the second update frequency, determining that the grouping category corresponding to the data to be processed is a low-frequency group;

[0030] If the update frequency of the data to be processed is lower than the second update frequency, it is determined that the grouping category corresponding to the data to be processed is a cold storage group.

[0031] In a possible design, in an implementation of another aspect of the embodiment of the present application, the storage unit may be specifically used for:

[0032] Writing the to-be-processed data corresponding to the high-frequency group into the first frequency group;

[0033] Writing the to-be-processed data corresponding to the low-frequency group into the second frequency group;

[0034] The data to be processed corresponding to the cold storage group is written into the third frequency group.

[0035] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0036] When the first frequency group data in the first frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the first frequency group;

[0037] Calculate the fragmentation rate of the first frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the first frequency group;

[0038] When the fragmentation rate of the first frequency group is greater than the fragmentation rate threshold, the data in the first frequency group is recycled, and data is continuously written to the second frequency group.

[0039] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0040] When the second frequency group data in the second frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the second frequency group;

[0041] Calculate the fragmentation rate of the second frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the second frequency group;

[0042] When the fragmentation rate of the second frequency group is greater than the fragmentation rate threshold, data in the second frequency group is recycled.

[0043] In a possible design, in an implementation of another aspect of the embodiment of the present application, the storage unit may be specifically used for:

[0044] If the update frequency of the data to be processed is lower than the first update frequency and higher than the fourth update frequency, determining that the grouping category corresponding to the data to be processed is a medium frequency group;

[0045] If the update frequency of the data to be processed is lower than the fourth update frequency and higher than the second update frequency, determining that the grouping category corresponding to the data to be processed is a low-frequency group;

[0046] The storage unit can be used for:

[0047] The data to be processed corresponding to the intermediate frequency group is written into the fourth frequency group.

[0048] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0049] The processing unit is further configured to, when it is detected that the fragmentation rate in the fourth frequency group is greater than a fragmentation rate threshold, recycle the data in the fourth frequency group and continue to write data to the second frequency group and the third frequency group;

[0050] The processing unit is further configured to recycle the data in the third frequency group when it is detected that the fragmentation rate in the third frequency group is greater than a fragmentation rate threshold.

[0051] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0052] When the fourth frequency group data in the fourth frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the fourth frequency group;

[0053] Calculate the fragmentation rate of the fourth frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the fourth frequency group;

[0054] When the fragmentation rate of the fourth frequency group is greater than the fragmentation rate threshold, the data in the fourth frequency group is recycled, and data is continuously written to the second frequency group and the third frequency group.

[0055] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0056] When the third frequency group data in the third frequency group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the third frequency group is accumulated;

[0057] Calculate the fragmentation rate of the third frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the third frequency group;

[0058] When the fragmentation rate of the third frequency group is greater than the fragmentation rate threshold, data in the third frequency group is recycled.

[0059] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0060] When the first frequency group data in the first group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the first group is accumulated;

[0061] When the first frequency group data in the second group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the second group is accumulated;

[0062] The processing unit can be used for:

[0063] Calculate the first fragmentation rate of the first group based on the accumulated memory occupied by the storage space fragments generated in the first group and the memory of the first group;

[0064] Calculate the second fragmentation rate of the second group based on the accumulated memory occupied by the storage space fragments generated in the second group and the memory of the second group;

[0065] The processing unit can be used for:

[0066] When the first fragmentation rate of the first group is greater than the fragmentation rate threshold and the second fragmentation rate of the second group is less than the fragmentation rate threshold, the data in the first group is recycled, and data is continuously written to the second group and the second frequency group;

[0067] When the second fragmentation rate of the second subgroup is greater than the fragmentation rate threshold, the data in the second subgroup is recycled, and data is continuously written to the second frequency group.

[0068] In a possible design, in an implementation of another aspect of the embodiment of the present application, the storage unit may be specifically used for:

[0069] If the memory required by the data to be processed is less than the remaining memory of the first group, the data to be processed is written into the first group;

[0070] If the memory required by the data to be processed is greater than the remaining memory of the first group, the data to be processed is written into the second group.

[0071] On the other hand, the present application provides a computer device, including: a memory, a processor, and a bus system;

[0072] Wherein, the memory is used to store programs;

[0073] The processor is used to implement the above-mentioned methods when executing the program in the memory;

[0074] The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

[0075] Another aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer is enabled to execute the above-mentioned methods.

[0076] It can be seen from the above technical solutions that the embodiments of the present application have the following beneficial effects:

[0077] Based on the data write request, the data to be processed is obtained, and the update frequency of the data to be processed is obtained. Then, according to the update frequency, the data to be processed is written into one of at least two groups in the storage space, wherein the at least two groups include a first frequency group and a second frequency group, the first frequency group corresponds to a first update frequency, the second frequency group corresponds to a second update frequency, and the first update frequency is higher than the second update frequency. By grouping the storage space according to different update frequencies to obtain at least two groups, the data to be processed with different update frequencies can be more specifically written into the corresponding groups such as the first frequency group or the second frequency group for storage, so that the memory management and data processing of different groups in the storage space can be more specifically performed based on different update frequencies. For example, the first frequency group corresponding to the first update frequency higher than the second update frequency is more specifically recycled more frequently, and the second frequency group corresponding to the second update frequency is maintained to write as much data as possible, so as to improve the utilization rate of the memory space of different groups, thereby improving the utilization rate of the storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 This is a schematic diagram of the architecture of a data object control system in an embodiment of the present application;

[0079] Figure 2 is a flow chart of an embodiment of a method for processing data in a storage space in an embodiment of the present application;

[0080] Figure 3 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0081] Figure 4 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0082] Figure 5 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0083] Figure 6 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0084] Figure 7 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0085] Figure 8 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0086] Fig. 9 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0087] Fig.10 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0088] Fig.11 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0089] Fig.12 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0090] Fig.13 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0091] Fig.14 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0092] Fig.15 is another embodiment flow chart of the data processing method of the storage space in the embodiment of the present application;

[0093] Fig.16 It is a schematic diagram of a principle flow of a data processing method of a storage space in an embodiment of the present application;

[0094] Fig.17 It is a track schematic diagram of the data processing method of the storage space in the embodiment of the present application;

[0095] Fig.18 This is a schematic diagram of the data processing effect of a storage space in the data processing method of the storage space in the embodiment of the present application;

[0096] Fig.19 2 is a schematic diagram of data processing effect of another storage space of the data processing method of the storage space in the embodiment of the present application;

[0097] Fig. 20 2 is a schematic diagram of data processing effect of another storage space of the data processing method of the storage space in the embodiment of the present application;

[0098] Fig.21 2 is a schematic diagram of data processing effect of another storage space of the data processing method of the storage space in the embodiment of the present application;

[0099] Fig. 22 2 is a schematic diagram of data processing effect of another storage space of the data processing method of the storage space in the embodiment of the present application;

[0100] Fig.23 is a schematic diagram of an embodiment of a data processing device for a storage space in an embodiment of the present application;

[0101] Fig.24 It is a schematic diagram of an embodiment of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0102] The embodiments of the present application provide a data processing method, apparatus, and device storage medium for a storage space, which are used to group the disk storage space according to different update frequencies, so that the to-be-processed data corresponding to the different update frequencies can be written to the corresponding groups more specifically in a subsequent manner, and the first frequency group corresponding to the first update frequency higher than the second update frequency can be more specifically recycled more frequently, while maintaining the second frequency group corresponding to the second update frequency to write as much data as possible, thereby improving the utilization rate of the disk storage space.

[0103] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0104] To facilitate understanding, some terms or concepts involved in the embodiments of the present application are first explained.

[0105] 1. Hard Disk Drive (HDD): refers to computer hardware used to permanently store different types of data on a computer, such as files, folders, images, videos, office documents, and so on.

[0106] 2. Conventional magnetic recording (CMR): The tracks are discrete, that is, the tracks where data is written to the disk never overlap.

[0107] 3. Shingled Magnetic Recording (SMR): The tracks are overlapping. SMR disks partially overlap the data tracks on the disk, just like tiles on a roof, which can effectively reduce the disk storage cost per unit capacity.

[0108] 4. Shingled Magnetic Recording Zone SMR zone: The SMR zone is an area in a shingled magnetic recording hard disk drive SMR HDD that can be operated independently and only supports sequential writes.

[0109] 5. Garbage Collection (GC): It refers to distinguishing which objects are still in use and which are not in use on the heap memory, and clearing the unused objects. So the garbage here actually refers to objects that can no longer be used in the memory and have no value but still occupy memory space, that is, storage space fragments.

[0110] For example, the unreachable objects, i.e., garbage, also called fragments, in each classification group can be counted by a counting algorithm or a reachability analysis algorithm. Then, when the fragmentation rate in each classification group is greater than the fragmentation threshold, that is, it exceeds the garbage load of the classification group, one or more of the clearing algorithm, mark-sweep algorithm, generational collection algorithm or copy algorithm can be used to recycle and clean up the garbage in the classification group, that is, to clean up the memory space occupied by the fragments, so that the fragments in the memory are cleaned up and the memory space occupied by the fragments is released.

[0111] For example, Fig.21 Assume that the fragmentation rate of group 1 of a high-frequency group is 51%, which is greater than the preset fragmentation threshold. Then, one or more of the clearing algorithm, mark-sweep algorithm, generational collection algorithm or replication algorithm can be used to recycle and clean up the fragments in group 1. The fragmentation rate of group 1 after cleaning can be 5% (as shown in FIG. Fig. 22 The fragmentation rate of group 1 of the high-frequency group shown in the figure), that is, the fragmentation rate of group 1 is significantly reduced, and more available space memory can be obtained.

[0112] It can be understood that in the specific implementation of the present application, related data such as the data to be processed, the first update frequency and the second update frequency are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0113] It is understandable that the data processing method of the storage space disclosed in the present application also involves cloud technology. The cloud technology is further introduced below. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize the calculation, storage, processing, and sharing of data. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application, which can form a resource pool, which is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0114] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides resources is called a "cloud". From the user's perspective, the resources in the "cloud" are infinitely scalable and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0115] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0116] It should be understood that the data processing method of the storage space provided in the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, maps, smart transportation, etc., and is used to group the storage space according to different update frequencies to write the to-be-processed data corresponding to different update frequencies into the corresponding groups more specifically, and then more specifically perform more frequent garbage collection on the first frequency group, and maintain the second group to write as much data as possible, so as to better realize the utilization of the disk storage space, and to be applied to scenarios such as live broadcast push or playback scenarios of live broadcast platforms, work display scenarios of social platforms, advertising or information push scenarios of web pages, advertising recommendation scenarios of game programs, map recommendation scenarios, and the like.

[0117] In order to solve the above problems, the present application proposes a data processing method for storage space, which is applied to Figure 1 The data object control system shown is shown in Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an architecture of a data object control system in an embodiment of the present application. Figure 1 As shown, the server obtains the data to be processed based on the data write request, and obtains the update frequency of the data to be processed, and then, according to the update frequency, writes the data to be processed into one of at least two groups in the storage space, wherein the at least two groups include a first frequency group and a second frequency group, the first frequency group corresponds to the first update frequency, the second frequency group corresponds to the second update frequency, and the first update frequency is higher than the second update frequency. By grouping the storage space according to different update frequencies to obtain at least two groups, the data to be processed with different update frequencies can be more specifically written into the corresponding groups such as the first frequency group or the second frequency group for storage, so that the memory management and data processing of different groups in the storage space can be more specifically performed based on different update frequencies. For example, the first frequency group corresponding to the first update frequency higher than the second update frequency is more specifically recycled more frequently, and the second frequency group corresponding to the second update frequency is maintained to write as much data as possible, so as to improve the utilization rate of the memory space of different groups, thereby improving the utilization rate of the storage space.

[0118] Understandably, Figure 1 Only one type of terminal device is shown in the figure. In actual scenarios, more types of terminal devices may participate in the data processing process. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, car terminals, etc. The specific number and type depend on the actual scenario and are not limited here. Figure 1 One server is shown in the figure, but in actual scenarios, multiple servers may also be involved, especially in the scenario of multi-model training interaction. The number of servers depends on the actual scenario and is not limited here.

[0119] It should be noted that in this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal device and the server can be directly or indirectly connected through wired or wireless communication, and the terminal device and the server can be connected to form a blockchain network, which is not limited in this application.

[0120] In combination with the above introduction, the data processing method of the storage space in this application will be introduced below. Figure 2 In one embodiment of the present application, a method for processing data in a storage space includes:

[0121] In step S101, the data to be processed is obtained based on the data processing request, and the update frequency of the data to be processed is obtained;

[0122] It is understandable that if Fig.17 As shown, SMR utilizes the overlapping of tracks (such as Fig.17 The right figure shows the overlapping effect of shingled data tracks), which is different from the discrete tracks of CMR (such as Fig.17 The left figure shows the discrete and non-overlapping effect of data tracks), which has a significant improvement in disk capacity (for example, there is a cost advantage of about 15%-20%). In order to better adapt to the characteristics of SMR shingled tracks, the concept of Zone is introduced in SMR HDD, and only sequential writing is supported by default in the Zone. It is understandable that there is also a part of the area in the SMR HDD that is the traditional magnetic recording area CMR Zone, which is used to support random writing, which is about 1% of the capacity.

[0123] For example, assuming that the memory size of a single zone can be designed to be 256MB, taking a 20TB SMR HDD as an example, there are about 80,000 SMR zones in the SMR HDD, where each zone can have about 100 tracks. The size of each track can be the same, such as 2.56MB, or different. The specific design can be based on actual business needs.

[0124] Furthermore, unlike CMR, in an SMR zone, if data is to be deleted from the front track or middle track of certain tracks with data written on them, it cannot be deleted directly according to the characteristics of the SMR zone. The data of the entire zone needs to be recovered before the data can be rewritten at the deleted location, that is, all data in the zone is read out and cached, and then the specified data is deleted, and then the cached data is rewritten to the zone or other zones. This will result in low utilization of the entire hard disk (for example, experiments have shown that in extreme cases the utilization rate may be less than 50%).

[0125] Therefore, in order to better improve the utilization rate of the entire hard disk, the present embodiment can first group the hard disk (i.e., disk storage space) according to different update frequencies, so that when the user object wants to write data to the disk, or to re-plan and group the data temporarily stored on the track, a corresponding data write request can be generated, so that when the system receives a data write request (i.e., a data processing request), it can obtain the pending data corresponding to the data write request (i.e., the data processing request) and the data identifier corresponding to the pending data, so that the pending data corresponding to different update frequencies can be better written to the corresponding group in a more targeted manner based on the data identifier.

[0126] Among them, the data to be processed (such as Fig.16 The business data indicated herein can specifically be expressed as video works or graphic works published by creators on social platforms, short videos published by popular authors or ordinary authors on short video interactive platforms, etc. It can also be expressed as other data, such as video works or graphic works published by public accounts, etc., and there is no specific limitation here.

[0127] Furthermore, after obtaining the data to be processed, the data identifier corresponding to the data to be processed can be obtained, and then the historical read and write times of the data identifier in the disk storage space can be counted, and then the initial update frequency of the data to be processed can be calculated based on the historical read and write times, so that the data to be processed can be written to the corresponding group more specifically based on the initial update frequency of the data to be processed.

[0128] In step S102, the data to be processed is written into one of at least two groups in the storage space according to the update frequency, wherein the at least two groups include a first frequency group and a second frequency group, the first frequency group corresponds to a first update frequency, the second frequency group corresponds to a second update frequency, and the first update frequency is higher than the second update frequency.

[0129] In an embodiment of the present application, after obtaining each data to be processed and the update frequency corresponding to each data to be processed, the data to be processed is written into different groups in the disk storage space, and the data to be processed corresponding to different update frequencies can be written into the corresponding groups more specifically, so that data processing and garbage collection can be performed more specifically according to different groups later, thereby improving the timeliness and accuracy of zone defragmentation (i.e., garbage collection).

[0130] Among them, Fig.18As shown, the hard disks (i.e., disk storage spaces) can be grouped according to different update frequencies based on actual application requirements. For example, the hard disks (i.e., disk storage spaces) can be divided into a high-frequency group and a low-frequency group. Alternatively, the hard disks (i.e., disk storage spaces) can be further divided into a first frequency group, a second frequency group, a third frequency group, and a fourth frequency group. For example, Fig.18 The high frequency group (i.e., the first frequency group), medium frequency group (i.e., the fourth frequency group), low frequency group (i.e., the second frequency group), ultra-low frequency group, and cold storage group (i.e., the third frequency group) shown can be further subdivided into other different groups, which are not specifically limited here.

[0131] For example, Fig.18 The high frequency group (i.e., the first frequency group), the medium frequency group (i.e., the fourth frequency group), the low frequency group (i.e., the second frequency group), the very low frequency group, and the cold storage group (i.e., the third frequency group) are shown. Assuming that in the initial state, the hard disk is empty, the hard disk can be divided into X groups (for example, X is 128, where 128 is only an example, which can be adjusted based on actual conditions and is not specifically limited here), where each group includes K zones (for example, K is 10, where 10 is only an example, which can be adjusted based on actual conditions and is not specifically limited here), then, based on the high frequency group (i.e., the first frequency group), the medium frequency group (i.e., the fourth frequency group), the low frequency group (i.e., the second frequency group), the very low frequency group, and the cold storage group (i.e., the third frequency group), the 128 groups (i.e., Fig.18 The groups (groups 0 to 127) shown in the figure are further divided into:

[0132] 1) The high frequency group (i.e. the first frequency group) includes: Fig.18 The illustrated group-0 to group-2 may be correspondingly set with a first update frequency (eg, 1 day), and may be used to store data with an update frequency less than the first update frequency (eg, 1 day);

[0133] 2) The intermediate frequency group (i.e., the fourth frequency group) includes: Fig.18 The illustrated group-3 to group-M-1 may be correspondingly set with a fourth update frequency (eg, 15 days), and may be used to store data with an update frequency less than the fourth update frequency (eg, 15 days);

[0134] 3) The low frequency group (i.e. the second frequency group) includes: Fig.18 The illustrated group-M to group-X-1 may be correspondingly set with a second update frequency (such as 30 days), and may be used to store data with an update frequency less than the second update frequency (such as 30 days), that is, the first update frequency is shorter than the second update frequency;

[0135] 4) The extremely low frequency group includes Fig.18 The illustrated group-X to group-124 may be correspondingly set with a second update frequency (such as 60 days), and may be used to store data with an update frequency less than the second update frequency (such as 60 days);

[0136] 5) The cold storage group (i.e. the third frequency group) includes: Fig.18 The illustrated group-125 to group-M-127 may be correspondingly set with a fifth update frequency (eg, 180 days), and may be used to store data with an update frequency less than the third update frequency (eg, 180 days).

[0137] It is understandable that the number of the above-mentioned grouping categories can be adjusted based on actual business conditions. Similarly, the number of zones in each category group can also be dynamically adjusted, and the update frequency corresponding to each of the above-mentioned groups can also be adjusted based on actual business conditions. No specific restrictions are made here.

[0138] Specifically, Fig.16 As shown, after obtaining each data to be processed and the initial update frequency corresponding to each data to be processed, the grouping category corresponding to each data to be processed (for example, a high-frequency group or a low-frequency group, etc.) can be determined according to the initial update frequency of each data to be processed, and then, each data to be processed can be written into different groups in the disk storage space according to the grouping category (for example, Fig.19 The data to be processed corresponding to the high-frequency group is written into group-0, group-1 and group-2, and the data to be processed corresponding to the low-frequency group is written into group-M, etc.).

[0139] For example, assuming that the initial update frequency corresponding to the data address A1 of a data to be processed is 0.3 days / time, and the first update frequency is 1 day / time, then a comparison shows that the initial update frequency corresponding to the data to be processed is less than the first update frequency, then the data to be processed can be stored in the first frequency group.

[0140] In an embodiment of the present application, a method for processing data of a storage space is provided. Through the above-mentioned method, the storage space is grouped and processed according to different update frequencies to obtain at least two groups, so that the to-be-processed data of different update frequencies can be more specifically written into the corresponding groups such as the first frequency group or the second frequency group for storage, so that the memory management and data processing of different groups in the storage space can be more specifically performed based on different update frequencies. For example, the first frequency group corresponding to the first update frequency higher than the second update frequency is more specifically recycled more frequently, and the second frequency group corresponding to the second update frequency is maintained to write as much data as possible, so as to improve the utilization rate of the memory space of different groups, thereby improving the utilization rate of the storage space.

[0141] Optionally, in the above Figure 2 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Figure 3As shown, after step S102 writes the data to be processed into one of at least two groups in the storage space according to the update frequency, the method includes:

[0142] In step S301, when it is detected that the fragmentation rate in the first frequency group is greater than the fragmentation rate threshold, the data in the first frequency group is recycled, and data is continuously written to the second frequency group;

[0143] It is understandable that when the data write request (i.e., data processing request) sent for the first time by each business object (for example, creators on social platforms, popular authors or ordinary authors on short video interactive platforms, etc.) is received, or when a memory rental request sent by each business object is received, or other business requests are received, and no specific restrictions are made here, a certain storage space can be allocated to each business object. For example, the corresponding storage space can be allocated to the business object according to the memory size required for the data to be processed corresponding to the data write request sent for the first time by each business object, for example, Y zones (for example, Y is 4, where 4 is only an example and can be adjusted based on actual conditions. No specific restrictions are made here, where the memory size of Y zones is greater than or equal to the memory size required for the data to be processed); or, the corresponding storage space can be allocated to the business object according to the leased storage space corresponding to the memory rental request sent by each business object.

[0144] Furthermore, the data to be processed can be temporarily stored in the track corresponding to the data identifier (i.e., data address) based on the data identifier of the data to be processed (i.e., data address), and then the initial update frequency of the data to be processed can be calculated based on the historical number of times the data identifier (i.e., data address) of the data to be processed has been read and written.

[0145] Furthermore, based on the initial update frequency of the data to be processed, the group in the disk storage space to which the data to be processed is to be written is determined, for example, the first frequency group (i.e., the high frequency group). Then, a corresponding storage space can be allocated for the business object in the group, for example, the first frequency group (i.e., the high frequency group). For example, a data to be processed is written to the three zones in group-1 of the first frequency group (i.e., the high frequency group) as shown in 18.

[0146] The fragmentation rate in the first frequency group may be obtained based on dividing the memory occupied by the storage space fragments in the first frequency group by the memory in the first frequency group.

[0147] Storage space fragmentation is also called file fragmentation, which is caused by the fact that files are stored in different places on the disk instead of being stored continuously in clusters on the disk. When the physical memory required by the application is insufficient, the operating system will generally generate a temporary swap file on the hard disk, and use the hard disk space occupied by the file to virtualize the memory. The virtual memory management program will frequently read and write to the hard disk, resulting in a large amount of fragmentation. Others, such as temporary files generated when browsing information by IE browser or the settings of temporary file directories, will also generate a large amount of fragmentation.

[0148] Further, it can be understood that after the data to be processed is written to the corresponding group in the disk storage space, for example, the first frequency group (i.e., the high-frequency group), the first frequency group data has been written. When the business object needs to delete all data, delete part of the data, or read and write frequently for a first frequency group data, it can directly index to the corresponding zone (for example, 3 zones in group-1 of the first frequency group (i.e., the high-frequency group)) according to the data identifier, and read the first frequency group data in the zone (for example, assuming that a zone is 256M) (for example, 100M data). During the reading process, it can be determined or read the data that needs to be deleted by the data deletion request (for example, 10M data out of 100M data), or the storage space fragments formed by continuously storing in continuous clusters of the disk (for example, 20M data).

[0149] Furthermore, the data to be deleted and the storage space fragments formed by continuously storing the data in the continuous clusters of the disk can be regarded as storage space fragments (such as Fig. 20 As shown, the high-frequency group has data that needs to be deleted or the fragments will increase accordingly when it is saved continuously). Then, the memory occupied by the storage space fragments can be divided by the memory of the first frequency group as the fragmentation rate (for example, Fig. 20 As shown, the fragmentation rate increased by 8%).

[0150] At the same time, the newly written data of the business object can be received, that is, a new data write request is received. According to the new data write request, the data obtained after deleting the data of the first frequency group read (that is, the remaining 90M data) can be spliced ​​with the new data to be processed (for example, 50M) to obtain the spliced ​​data (that is, 140M data). Since the zone has 156M of remaining storage space after consuming 100M of space, the spliced ​​data (that is, 140M data) can be overwritten to the data in the originally occupied 100M space, and 40M of the 156M remaining storage space can be sequentially occupied, that is, the occupied space is increased.

[0151] Therefore, since the update frequency of the first frequency group (i.e., the high-frequency group) is faster, it can be known that compared with the second frequency group (i.e., the low-frequency group), there are more data deletions or the data to be processed is continuously saved in the first frequency group (i.e., the high-frequency group) (for example, Group-1 and Group-2), and the corresponding storage space fragmentation will further increase. After continuous cumulative calculation, the fragmentation rate of the first frequency group (i.e., the high-frequency group) can be regularly detected at certain time intervals. For example, the fragmentation rate caused by the deletion of data or the continuous storage of data to be processed in the first frequency group (i.e., the high-frequency group) is detected at a first time interval (for example, 3 hours, etc.).

[0152] Further, if the fragmentation rate of the first frequency group (eg, Fig.21 The fragmentation rate of the first frequency group (i.e., the high frequency group) shown in the figure is 71%), which is greater than the fragmentation threshold (for example, 50%, and the fragmentation threshold can be set according to the actual application requirements and is not specifically limited here), that is, the business pressure or load of the first frequency group (i.e., the high frequency group) is heavy, then garbage collection (i.e., defragmentation) can be performed on the first frequency group (i.e., the high frequency group), and one or more of the clearing algorithm, the marking and sorting algorithm, the generational collection algorithm, or the replication algorithm can be used to recycle and clean up the storage space fragments in the classification group, that is, to clean up the memory space occupied by the fragments, so that the fragments of the first frequency group (i.e., the high frequency group) after recycling return to a healthy state (i.e., the fragments are reduced), that is, the remaining available space of the first frequency group (i.e., the high frequency group) increases (for example, from Fig.21 The remaining available space of the high frequency group shown is 29%, which is increased as follows Fig. 22 The remaining available space of the high-frequency group shown in FIG5 is 55%), and each zone can be defragmented more specifically, thereby improving the timeliness and accuracy of zone defragmentation. At the same time, since the update frequency of the second frequency group (i.e., the low-frequency group) is not high, the increase of fragments is relatively slow (for example, Fig.21 The fragmentation rate of the second frequency group (i.e., the low frequency group) shown is approximately 5%, which is much smaller than the fragmentation threshold. Frequent garbage collection is not required, so data can be continuously written to the second frequency group (i.e., the low frequency group) to write as much data as possible, thereby improving the utilization of disk storage space.

[0153] Among them, recycling processing is garbage collection (GC), which refers to distinguishing which objects are still in use and which objects are not in use on the heap memory, and clearing the unused objects, that is, storage space fragments. Recycling processing includes clearing algorithms, mark-sweep algorithms, generational collection algorithms or copy algorithms, etc., and also includes other algorithms, which are not specifically limited here.

[0154] Among them, the mark-and-sweep algorithm refers to traversing the objects in memory through references starting from the root node (that is, the starting object in the memory of each classification group) through a garbage collector (for example, a serial collector, a throughput-first collector, and a response-time-first collector, etc.), and marking all root-reachable objects. After that, the collector will traverse the objects in memory from beginning to end and recycle unmarked objects, that is, unmarked storage space fragments.

[0155] Among them, the mark-compact algorithm refers to the use of a garbage collector (for example, a serial collector, a throughput-first collector, and a response-time-first collector, etc.) to traverse objects in memory through references starting from the root node (that is, the starting object in memory) and mark all root-reachable objects. When the collector traverses for the second time, it will directly move the surviving objects forward. In this process, the space occupied by the garbage object will be directly overwritten, that is, its memory will be reclaimed.

[0156] Among them, the copy algorithm refers to dividing the memory into a surviving source area, namely the From area, and a surviving arrival area, namely the To area. The recycler will copy useful objects from the From area to the To area, so that the From area is recycled as a whole, and then the positions of From and To are exchanged. The To area is always empty.

[0157] Among them, the generational collection algorithm refers to dividing the memory into two large blocks, such as the new generation and the old generation. Among them, the new generation is divided into three small areas, such as the Garden of Eden, the survivor area From, and the survivor area To. Objects that are used for a long time are placed in the old generation, and objects that can be discarded after use are placed in the new generation. For example, when a new object is created, it will be placed in the Garden of Eden. When more and more new objects are created and the space in the Garden of Eden is full and cannot hold the next object, a garbage collection will be triggered. The root reachable algorithm will be used to find the living objects in the Garden of Eden, and then the replication algorithm will be used to put these objects into the survivor area To, and the lifespan of these objects will be increased by one, and then the positions of the From area and the To area will be swapped. Similarly, when the Garden of Eden is full again next time, a second garbage collection will be triggered. At this time, the Garden of Eden and the From area will be scanned, and these surviving objects will be placed in the To area, the lifespan will be increased by one, and then the garbage objects in the Garden of Eden and the From area will be recycled, and then the positions of the From area and the To area will be swapped, until an object is still alive after a certain number of garbage collections (such as 15 times), it will be promoted to the old generation.

[0158] In step S302, when it is detected that the fragmentation rate in the second frequency group is greater than the fragmentation rate threshold, data in the second frequency group is recycled.

[0159] In an embodiment of the present application, since the update frequency of the second frequency group (i.e., the low-frequency group) is slower, when it is detected that the fragmentation rate in the second frequency group is greater than the fragmentation rate threshold, the second frequency group (i.e., the low-frequency group) can be recycled to release the memory space of the second frequency group (i.e., the low-frequency group), thereby improving the utilization rate of the disk storage space.

[0160] Specifically, since the update frequency of the second frequency group (i.e., the low frequency group) is slower, it can be known that compared with the first frequency group (i.e., the high frequency group), when more data is deleted or continuously saved in the second frequency group (i.e., the low frequency group) (e.g., group-M), the corresponding storage space fragmentation will further increase. Therefore, the fragmentation rate of the second frequency group (i.e., the low frequency group) can be regularly detected at a certain time interval. For example, the fragmentation rate caused by deleting data or continuously saving data in the second frequency group (i.e., the low frequency group) is detected at a second time interval (e.g., 30 days, etc.). If the fragmentation rate (e.g., assuming that the second frequency group (i.e., the low frequency group) The fragmentation rate is 52%) greater than the fragmentation threshold (for example, 50%), that is, the business pressure or load of the second frequency group (that is, the low frequency group) is heavy, then garbage collection (that is, defragmentation) can be performed on the second frequency group (that is, the low frequency group), that is, one or more of the clearing algorithm, mark defragmentation, generational collection algorithm or replication algorithm can be used to recycle and clean up the storage space fragments in the classification group, that is, to clean up the memory space occupied by the storage space fragments, that is, to defragment each zone in a more targeted manner, thereby improving the timeliness and accuracy of the zone defragmentation.

[0161] Optionally, in the above Figure 2 On the basis of the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided by the embodiment of the present application, a data identifier is set corresponding to the data to be processed; and the update frequency of the data to be processed in step S101 includes:

[0162] The statistical data identifies the historical read and write times in the disk storage space, and calculates the initial update frequency of the data to be processed based on the historical read and write times.

[0163] In an embodiment of the present application, after obtaining the data identifier, the historical read and write times of the data identifier in the disk storage space can be counted, and then the initial update frequency of the data to be processed can be calculated based on the historical read and write times, so that the data to be processed can be written to the corresponding group more specifically based on the initial update frequency.

[0164] Among them, the data identifier of the data to be processed can be specifically expressed as a data address (for example, a logical address and a physical address, etc., which are not specifically limited here), which is used to indicate the data source location for reading the data to be processed and the track location for temporary storage of the data to be processed.

[0165] Among them, the logical address (LogicalAddress), also known as the virtual address, is an address space used by a program. It is the address used in the program code and is generated by the programmer or the operating system. The logical address refers to a segment of address space in the memory. It is an offset relative to the starting address (i.e., base address) of the program. When the program is executed, the logical address will be translated into a physical address. The logical address consists of two 16-bit address components, one for the segment base value and the other for the offset. Both components are encoded as unsigned numbers.

[0166] Among them, the physical address is the real address in the memory, which is generated by the central processing unit CPU and is used to access the actual memory unit. The physical address is the address passed by the CPU to the memory controller, which is passed to the memory module through the address bus to access the actual storage unit.

[0167] Specifically, after obtaining the data identifier, the number of times the data identifier (i.e., data address) has been read, written, or accessed (i.e., the historical number of read and write times) can be indexed or traversed in the disk storage space, and the timestamp of when the data identifier (i.e., data address) has been read, written, or accessed can be recorded. Then, based on the number of times it has been read, written, or accessed (i.e., the historical number of read and write times) and the timestamp of when it has been read, written, or accessed, the initial update frequency of the data to be processed can be calculated.

[0168] For example, the number of times a data address A1 of data to be processed has been read, written or accessed (i.e., the number of historical read and write times) is 10 times, and the timestamps recording that the data address A1 has been read, written or accessed are at four different times on a certain day (e.g., April 15), four different times on the second day (e.g., April 16), and two different times on the third day (e.g., April 17), that is, the data address A1 has been read, written or accessed 10 times within three days, and then, by dividing the number of days by the number of times, the initial update frequency corresponding to the data to be processed is obtained as 0.3 days / time.

[0169] Optionally, in the above Figure 2 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Figure 4 As shown, after step S102 writes the data to be processed into one of at least two groups in the storage space according to the update frequency, the method further includes:

[0170] In step S401, each to-be-processed data written into the first frequency group is taken as the first frequency group data;

[0171] In step S402, the number of times each first frequency group data is read and written from being written into the first frequency group to the current time is counted, and the current update frequency of the first frequency group data is calculated based on the number of times it is read;

[0172] In step S403, the first frequency group data whose current update frequency is less than the first update frequency is converted from the first group to other groups in the disk storage space for storage.

[0173] In the embodiment of the present application, after the data to be processed is written into the first frequency group, each data to be processed written into the first frequency group can be used as the first frequency group data, and then, the number of times each first frequency group data is read and written from the time of being written into the first frequency group to the present is counted, and based on the number of times it is read, the current update frequency of the first frequency group data is calculated, and then, the first frequency group data whose current update frequency is less than the first update frequency is converted from the first group to other groups in the disk storage space for storage. It can be understood that after the data to be processed is written into the first frequency group (i.e., the high frequency group), if there is the first frequency group data whose current update frequency is less than (i.e., slower than) the first update frequency, it can be understood that, The first frequency group data changes from high-frequency data to low-frequency data, that is, the first frequency group data gradually changes from hot data (for example, data that is accessed or used frequently, or popular data, etc.) to cold data (for example, data that is accessed or used less frequently, or unpopular data, etc.). The first frequency group data can be bidirectionally converted and stored between the first frequency group (that is, the high-frequency group) and other groups in the disk storage space, such as the second frequency group (that is, the low-frequency group). The data can be converted to a more suitable group for management and storage, so that the data of the updated group can be processed more specifically later, thereby improving the utilization rate of the disk storage space to a certain extent.

[0174] Specifically, Fig.16As shown, after the data to be processed is written into the first frequency group, each data to be processed written into the first frequency group can be taken as the first frequency group data, and then, at a certain time interval (for example, 2 days, which can be set according to actual application requirements and is not specifically limited here), the number of times each first frequency group data is read and written from the time it is written into the first frequency group to the present (i.e., at the current detection moment), as well as the timestamp corresponding to each reading and writing, can be counted, and then, the current update frequency of the first frequency group data can be calculated in accordance with the above-mentioned method of calculating the update frequency, and then, the current update frequency of each first frequency group data can be compared with the first update frequency, if there is a current update frequency that is less than (i.e., slower than) the first update frequency, for example, assuming that the update frequency of a first frequency group data is 0.5 days (for example, assuming that a popular creator on a social platform updates his creative dynamics every 0.5 days), at the current detection moment, assuming that the current update frequency of the first frequency group data is reduced to 10 days (for example, assuming that the popular creator on the social platform updates his works after a long period of time due to a creative bottleneck, such as updating every 10 days). The current update frequency of the first frequency group data is longer than the update frequency, that is, the current update frequency of the first frequency group data is reduced. It can be understood that the first frequency group data is gradually changing from hot data to cold data. Then, according to the current update frequency of the first frequency group data, the grouping category of the first frequency group data can be re-determined. For example, if the current update frequency of the first frequency group data (such as 10 days) is less than the fourth update frequency (such as 15 days) corresponding to the fourth frequency group (i.e., the medium frequency group), the first frequency group data can be re-determined from the first The first frequency group (i.e., high frequency group) is converted to other groups (e.g., the fourth frequency group (i.e., medium frequency group)) in the disk storage space for storage. Conversely, for example, when the current update frequency of the first frequency group data already stored in the fourth frequency group (i.e., medium frequency group) becomes higher (e.g., 15 hours), that is, the current update frequency (e.g., 15 hours) is less than the first update frequency (e.g., 1 day) corresponding to the first frequency group (i.e., high frequency group), the first frequency group data can be converted from the fourth frequency group (i.e., medium frequency group) to the first frequency group (i.e., high frequency group) for storage. For example, in the first frequency group (i.e., medium frequency group), the first frequency group data can be converted to the first frequency group (i.e., high frequency group) for storage. Fig.16 The high frequency group shown in FIG. 1 ) and other groups in the disk storage space, such as the fourth frequency group (i.e., Fig.16 The first frequency group data is bidirectionally converted and stored between the intermediate frequency groups shown in the figure.

[0175] Optionally, in the above Figure 2 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Figure 5 As shown, after step S102 writes the data to be processed into one of at least two groups in the storage space according to the update frequency, the method further includes:

[0176] In step S501, each to-be-processed data written into the second frequency group is taken as the second frequency group data;

[0177] In step S502, the number of times each second frequency group data is read and written from being written into the second frequency group to the current time is counted, and the current update frequency of the second frequency group data is calculated based on the number of times it is read;

[0178] In step S503, the second frequency group data whose current update frequency is greater than the second update frequency is converted from the second group to other groups in the storage space for storage.

[0179] In the embodiment of the present application, after the data to be processed is written into the second frequency group, each data to be processed written into the second frequency group can be used as the data of the second frequency group, and then, the number of times each data of the second frequency group is read and written from the time of being written into the second frequency group to the present is counted, and based on the number of times it is read, the current update frequency of the data of the second frequency group is calculated, and then, the data of the second frequency group whose current update frequency is greater than the second update frequency is converted from the second group to other groups in the disk storage space for storage. It can be understood that after the data to be processed is written into the second frequency group (i.e., the low-frequency group), if there is data of the second frequency group whose current update frequency is greater than (i.e., faster than) the second update frequency, it can be understood that, The second frequency group data changes from low-frequency data to high-frequency data, that is, the second frequency group data gradually changes from cold data (for example, data that is accessed or used less frequently, or unpopular data, etc.) to hot data (for example, data that is accessed or used more frequently, or popular data, etc.). The second frequency group data can be bidirectionally converted and stored between the second frequency group (i.e., the low-frequency group) and other groups in the disk storage space, such as the first frequency group (i.e., the high-frequency group). The data can be converted to a more appropriate group for management and storage, so that the data in the updated group can be processed more specifically later, thereby improving the utilization rate of the disk storage space to a certain extent.

[0180] Specifically, Fig.16As shown, after the data to be processed is written into the second frequency group, each data to be processed written into the second frequency group can be used as the second frequency group data, and then, at a certain time interval (for example, 2 days, which can be set according to actual application requirements and is not specifically limited here), the number of times each second frequency group data is read and written from the time it is written into the first frequency group to the present (i.e., the current detection moment), and the timestamp corresponding to each reading and writing, can be counted, and then, the current update frequency of the second frequency group data can be calculated according to the above-mentioned method of calculating the update frequency, and then, the current update frequency of each second frequency group data can be compared with the second update frequency, if there is a current update frequency greater than (i.e., faster than) the second update frequency, for example, assuming that the update frequency of one second frequency group data in the second frequency group (i.e., the low frequency group) is 20 days (for example, assuming that one of the social platform For example, if an ordinary creator on a social platform updates his / her creative dynamics once every 20 days, at the current detection moment, the current update frequency of the second frequency group data is increased to 0.5 days (for example, assuming that the work published by the ordinary creator on the social platform suddenly becomes the work corresponding to a hot topic, attracting a large number of visitors to visit the work, so that the update frequency of the work becomes higher, such as updating the work dynamics once every 0.5 days), it can be understood that the current update frequency of the second frequency group data has become shorter than the update frequency, that is, the current update frequency of the second frequency group data has become higher, and the second frequency group data is gradually changing from cold data to hot data. Then, according to the current update frequency of the second frequency group data, the grouping category of the second frequency group data can be re-determined. For example, the second update frequency corresponding to the second frequency group (i.e., the low frequency group) is less than the current update frequency of the second frequency group data (such as 0.5 days) < the first update frequency (e.g. 1 day) corresponding to the first frequency group (i.e. high frequency group), the second frequency group data can be converted from the second frequency group (i.e. low frequency group) to other groups in the disk storage space, such as the first frequency group (i.e. high frequency group) for storage. On the contrary, for example, when the update frequency of the second frequency group data already stored in the first frequency group (i.e. high frequency group) is lowered (e.g., the popularity of the topic decreases, the number of visits to the work decreases, so that the update frequency of the work becomes lower, such as updating the work dynamics once every 2 days), that is, the update frequency (e.g. 2 days) is higher than the fourth update frequency (e.g. 15 days) corresponding to the fourth frequency group (i.e. medium frequency group), the second frequency group data can be converted from the first frequency group (i.e. high frequency group) to the fourth frequency group. Similarly, if the update frequency of the second frequency group data that has been stored in the fourth frequency group (i.e., the medium frequency group) becomes lower (for example, the popularity of the topic decreases to a low popularity, the number of visits to the work is almost no longer visited, so that the creator of the work rarely updates the work, resulting in a lower update frequency of the written data corresponding to the work, such as updating the work dynamics once every 25 days), that is, the update frequency (such as 25 days) is lower than the fourth update frequency (such as 15 days) corresponding to the fourth frequency group (i.e., the medium frequency group) and higher than the second update frequency (such as 30 days) corresponding to the second frequency group (i.e., the low frequency group), then the second frequency group data can be converted from the fourth frequency group (i.e., the medium frequency group) to the second frequency group (i.e., the low frequency group) for storage. .

[0181] Optionally, in the above Figure 4 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Figure 6 As shown, step S102 writes the data to be processed into one of at least two groups in the storage space according to the update frequency, including:

[0182] In step S601, the grouping category corresponding to the data to be processed is determined according to the update frequency of the data to be processed;

[0183] In step S602, the data to be processed is written into one of at least two groups in the storage space according to the grouping category.

[0184] In an embodiment of the present application, after obtaining each data to be processed and the update frequency corresponding to each data to be processed, the grouping category corresponding to each data to be processed can be determined according to the update frequency corresponding to the data to be processed, and then, according to the grouping category, each data to be processed can be written into one of at least two groups in the disk storage space, so that the data to be processed corresponding to different update frequencies can be written to the corresponding groups more specifically, so that data processing and data recovery (for example, garbage collection) can be performed more specifically according to different groups later, thereby improving the timeliness and accuracy of zone defragmentation (i.e., garbage collection).

[0185] Specifically, Fig.18 As shown, after obtaining each data to be processed and the update frequency corresponding to each data to be processed, the grouping category corresponding to each data to be processed (for example, a high frequency group, a medium frequency group, a low frequency group, an extremely low frequency group or a cold storage group, etc., which are not specifically limited here) can be determined first according to the update frequency corresponding to the data to be processed, and then, each data to be processed can be written into one of the at least two groups in the disk storage space according to the grouping category (for example, Fig.19 The data to be processed corresponding to the high-frequency group is written into group-0, group-1 and group-2, and the data to be processed corresponding to the low-frequency group is written into group-M, etc.).

[0186] Optionally, in the above Figure 6 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Figure 7 As shown, the storage space also includes a third frequency group, and the third update frequency corresponding to the third group is lower than the second update frequency; the grouping categories at least include a high frequency group, a low frequency group, and a cold storage group; step S601 determines the grouping category corresponding to the data to be processed according to the update frequency of the data to be processed, including:

[0187] In step S701, if the update frequency of the data to be processed is higher than the first update frequency, the grouping category corresponding to the data to be processed is determined to be a high-frequency group;

[0188] In step S702, if the update frequency of the data to be processed is lower than the first update frequency and higher than the second update frequency, it is determined that the grouping category corresponding to the data to be processed is a low-frequency group;

[0189] In step S703, if the update frequency of the data to be processed is lower than the second update frequency, it is determined that the grouping category corresponding to the data to be processed is a cold storage group.

[0190] In an embodiment of the present application, after obtaining each data to be processed and the update frequency corresponding to each data to be processed, the grouping category corresponding to each data to be processed can be determined according to the different update frequencies corresponding to each data to be processed, that is, if the update frequency corresponding to the data to be processed is higher than the first update frequency, then the grouping category corresponding to the data to be processed can be determined to be a high-frequency group. Similarly, if the update frequency of the data to be processed is lower than the first update frequency and higher than the second update frequency, then the grouping category corresponding to the data to be processed is determined to be a low-frequency group. Similarly, if the update frequency of the data to be processed is lower than the second update frequency, then the grouping category corresponding to the data to be processed is determined to be a cold storage group, so that each data to be processed can be written to a different group in the disk storage space according to the grouping category, and data processing and data recovery (for example, garbage collection) can be performed more specifically according to different groups, thereby improving the timeliness and accuracy of zone defragmentation (i.e., garbage collection).

[0191] Specifically, Fig.18 As shown, after obtaining each data to be processed and the update frequency of each data to be processed, the update frequency of each data to be processed can be compared with the update frequency corresponding to each grouping category (for example, a high-frequency group, a medium-frequency group, a low-frequency group, an extremely low-frequency group or a cold storage group, etc., which are not specifically limited here). If the update frequency of a data to be processed (such as 0.5 days) is higher than the first update frequency (such as 1 day), it can be determined that the grouping category corresponding to the data to be processed is a high-frequency group; similarly, if the update frequency corresponding to a data to be processed (such as 20 days) is lower than the first update frequency (such as 1 day) and higher than the second update frequency (such as 30 days), it can be determined that the grouping category corresponding to the data to be processed is a low-frequency group. Similarly, if the update frequency corresponding to a data to be processed (such as 40 days) is lower than the second update frequency (such as 30 days) and higher than the third update frequency (such as 50 days), it can be determined that the grouping category corresponding to the data to be processed is a cold storage group.

[0192] Optionally, in the above Figure 7 On the basis of the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided by the embodiment of the present application, step S602 writes the to-be-processed data into one of at least two groups in the disk storage space according to the grouping category, preferably including:

[0193] In step S704, the to-be-processed data corresponding to the high-frequency group is written into the first frequency group;

[0194] In step S705, the to-be-processed data corresponding to the low frequency group is written into the second frequency group;

[0195] In step S706, the to-be-processed data corresponding to the cold storage group is written into the third frequency group.

[0196] In an embodiment of the present application, after obtaining the grouping category corresponding to each data to be processed, each data to be processed can be written into a different group in the disk storage space according to the grouping category, that is, the data to be processed corresponding to the high-frequency group is written into the first frequency group, and similarly, the data to be processed corresponding to the low-frequency group is written into the second frequency group, and similarly, the data to be processed corresponding to the cold storage group is written into the third frequency group. The data to be processed corresponding to different update frequencies can be written into the corresponding groups in a more targeted manner, so that data processing and data recovery (for example, garbage collection) can be performed more specifically according to different groups in the future, thereby improving the timeliness and accuracy of zone defragmentation (i.e., garbage collection).

[0197] Specifically, after obtaining the grouping category corresponding to each data to be processed, each data to be processed can be written into a different group in the disk storage space according to the grouping category, that is, the data to be processed corresponding to the high-frequency group is written into the first frequency group (i.e., the high-frequency group) (for example, Fig.19 The pending data corresponding to the high-frequency group shown is written into group-0, group-1 and group-2). Similarly, the pending data corresponding to the low-frequency group is written into the second frequency group (i.e., the low-frequency group) (for example, the pending data corresponding to the low-frequency group is written into group-M, etc.). Similarly, the pending data corresponding to the cold storage group is written into the third frequency group (i.e., the cold storage group) (for example, the pending data corresponding to the cold storage group is written into group-125, etc.).

[0198] Optionally, in the above Figure 3 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Figure 8 As shown, in step S301, when it is detected that the fragmentation rate in the first frequency group is greater than the fragmentation rate threshold, data in the first frequency group is recycled and data is continuously written to the second frequency group, including:

[0199] In step S801, when the first frequency group data in the first frequency group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the first frequency group is accumulated;

[0200] In step S802, a fragmentation rate of the first frequency group is calculated based on the accumulated memory occupied by the storage space fragments and the memory of the first frequency group;

[0201] In step S803, when the fragmentation rate of the first frequency group is greater than the fragmentation rate threshold, the data in the first frequency group is recycled, and data is continuously written to the second frequency group.

[0202] It can be understood that after the data to be processed is written to the corresponding group in the disk storage space, for example, the first frequency group (i.e., high frequency group), it is the first frequency group data. When the business object needs to delete or continuously save a certain first frequency group data in the first frequency group (i.e., high frequency group), it can directly index to the corresponding zone (for example, 3 zones in group-1 of the first frequency group (i.e., high frequency group)) according to the data identifier, and read the first frequency group data in the zone (for example, assuming that a zone is 256M) (for example, 100M data). In the reading process, it can be determined or read the data that needs to be deleted by the data deletion request (for example, 10M data in 100M data), or the storage space fragments (i.e., disk fragments) (for example, 20M data) formed by continuously saving in continuous clusters of the disk. Furthermore, the data that needs to be deleted and the storage space fragments formed by continuously saving in continuous clusters of the disk can be used as storage space fragments (i.e., disk fragments) (such as Fig. 20 As shown, the high-frequency group has data that needs to be deleted or the fragments will increase accordingly when it is saved continuously). Then, the memory occupied by the storage space fragments (i.e., disk fragments) can be divided by the memory of the first frequency group as the fragmentation rate (for example, Fig. 20 As shown, the fragmentation rate increased by 8%).

[0203] It should be noted that due to Fig.19 The group-0 shown is a CMR zone. According to the characteristics of CMR, data can be directly deleted.

[0204] Further, it can be understood that while determining the fragmented memory to be deleted in the high-frequency group according to the data deletion request, it is also possible to receive new data continuously written by the business object, that is, receive a new data write request. According to the new data write request, the data obtained after the data deletion of the read first frequency group data (for example, 100M) (that is, the remaining 90M data) can be spliced ​​with the new data to be processed (for example, 50M) to obtain the spliced ​​data (that is, 140M data). Since the zone (for example, 256M) has 156M of remaining storage space after consuming 100M of space, the spliced ​​data (that is, 140M data) can be overwritten to the data in the originally occupied 100M space, and sequentially occupy 40M of the 156M remaining storage space, that is, the occupied space is increased.

[0205] Therefore, since the update frequency of the first frequency group (ie, the high frequency group) is faster, it can be known that relative to the second frequency group (ie, the low frequency group), the first frequency group (ie, the high frequency group) (for example, Fig. 20If more data is deleted from the groups-1 and -2 shown in the figure, or the data to be processed is continuously saved, etc., the corresponding storage space fragments (i.e., disk fragments) will further increase. After continuous cumulative calculation, the fragmentation rate of the first frequency group (i.e., high frequency group) can be regularly detected at a certain time interval. For example, according to the first time interval (e.g., 3 hours, etc.), the memory occupied by the storage space fragments (i.e., disk fragments) generated in the first frequency group (i.e., high frequency group) corresponding to the current moment is counted (e.g., assuming it is 7.1G), and based on the memory occupied by the storage space fragments (i.e., disk fragments) generated in the first frequency group (i.e., high frequency group) corresponding to the current moment and the memory of the first frequency group, the fragmentation rate corresponding to the current moment is calculated. For example, the quotient between the memory occupied by the storage space fragments (i.e., disk fragments) generated in the first frequency group (i.e., high frequency group) corresponding to the current moment and the total memory of the first frequency group (e.g., 10G) is calculated, that is, the fragmentation rate corresponding to the current moment (e.g., 71%).

[0206] Furthermore, if Fig.21 As shown, if the fragmentation rate of the first frequency group (for example, Fig.21 The fragmentation rate of the first frequency group (i.e., the high frequency group) shown in the figure is 71%), which is greater than the fragmentation threshold (for example, 50%, and the fragmentation threshold can be set according to the actual application requirements and is not specifically limited here), that is, the business pressure or load of the first frequency group (i.e., the high frequency group) is heavy, then garbage collection (i.e., defragmentation) can be performed on the first frequency group (i.e., the high frequency group), and one or more of the clearing algorithm, the marking and sorting algorithm, the generational collection algorithm, or the replication algorithm can be used to recycle and clean up the storage space fragments (i.e., the disk fragments) in the classification group, that is, to clean up the memory space occupied by the fragments, so that the fragments of the first frequency group (i.e., the high frequency group) after recycling return to a healthy state (i.e., the fragments are reduced), and the remaining available space of the first frequency group (i.e., the high frequency group) increases (for example, from Fig.21 The remaining available space of the high frequency group shown is 29%, which is increased as follows Fig. 22 The remaining available space of the high-frequency group shown in FIG5 is 55%), and each zone can be defragmented more specifically, thereby improving the timeliness and accuracy of zone defragmentation. At the same time, since the update frequency of the second frequency group (i.e., the low-frequency group) is not high, the increase of fragments is relatively slow (for example, Fig.21 The fragmentation rate of the second frequency group (i.e., the low frequency group) shown is approximately 5%, which is much smaller than the fragmentation threshold. Frequent garbage collection is not required, so data can be continuously written to the second frequency group (i.e., the low frequency group) to write as much data as possible, thereby improving the utilization of disk storage space.

[0207] Optionally, in the above Figure 3Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Fig. 9 As shown, in step S302, when it is detected that the fragmentation rate in the second frequency group is greater than the fragmentation rate threshold, data in the second frequency group is recycled, including:

[0208] In step S901, when the second frequency group data in the second frequency group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the second frequency group is accumulated;

[0209] In step S902, the fragmentation rate of the second frequency group is calculated based on the accumulated memory occupied by the storage space fragments and the memory of the second frequency group;

[0210] In step S903, when the fragmentation rate of the second frequency group is greater than the fragmentation rate threshold, data in the second frequency group is recycled.

[0211] It can be understood that after the data to be processed is written to the corresponding group in the disk storage space, for example, the second frequency group (i.e., low-frequency group), it is the second frequency group data. When the business object needs to delete or continuously save a certain second frequency group data in the second frequency group (i.e., low-frequency group), it can directly index to the corresponding zone (for example, group-M of the second frequency group (i.e., low-frequency group)) according to the data identifier, and read the second frequency group data in the zone (for example, assuming that a zone is 256M) (for example, 50M data). In the reading process, it can be determined or read the data that needs to be deleted by the data deletion request (for example, 10M data in 50M data), or the storage space fragments (i.e., disk fragments) (for example, 20M data) formed by continuously saving in continuous clusters of the disk. Furthermore, the data that needs to be deleted and the disk fragments formed by continuously storing them in continuous clusters on the disk can be regarded as storage space fragments (for example, if the low-frequency group has data that needs to be deleted or there is a corresponding increase in fragmentation when it is continuously stored), and then the memory occupied by the storage space fragments can be divided by the memory of the second frequency group as the fragmentation rate (for example, the fragmentation rate has increased by 4%).

[0212] Further, it can be understood that while determining the fragmented memory to be deleted in the low-frequency group according to the data deletion request, it is also possible to receive new data continuously written by the business object, that is, receive a new data write request. According to the new data write request, the data obtained after the data deletion of the read second frequency group data (for example, 50M) (that is, the remaining 40M data) can be spliced ​​with the new data to be processed (for example, 20M) to obtain the spliced ​​data (that is, 60M data). Since the zone (for example, 256M) has 206M of remaining storage space after consuming 50M of space, the spliced ​​data (that is, 60M data) can be overwritten to the data in the originally occupied 50M space, and 10M of the remaining 206M storage space is sequentially occupied, that is, the occupied space is increased.

[0213] Therefore, since the update frequency of the second frequency group (i.e., the low frequency group) is slower, it can be known that compared with the first frequency group (i.e., the high frequency group), when more data is deleted or continuously saved in the second frequency group (i.e., the low frequency group) (e.g., group-M), the corresponding disk fragmentation will further increase. Therefore, the fragmentation rate of the second frequency group (i.e., the low frequency group) can be regularly detected at a certain time interval. For example, the fragmentation rate caused by deleting data or continuously saving data in the second frequency group (i.e., the low frequency group) can be detected at a second time interval (e.g., 30 days, etc.). If the fragmentation rate (e.g., assuming that the second frequency group (i.e., the low frequency group) is ) is 52%) is greater than the fragmentation threshold (for example, 50%), that is, the business pressure or load of the second frequency group (that is, the low frequency group) is heavy, then garbage collection (that is, defragmentation) can be performed on the second frequency group (that is, the low frequency group), that is, one or more of the clearing algorithm, mark defragmentation, generational collection algorithm or replication algorithm can be used to recycle and clean up the disk fragments in the classification group, that is, to clean up the memory space occupied by the disk fragments, that is, to defragment each zone in a more targeted manner, thereby improving the timeliness and accuracy of zone defragmentation.

[0214] Optionally, in the above Figure 7 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Fig.10 As shown, the disk storage space also includes a fourth frequency group, wherein the fourth update frequency corresponding to the fourth group is lower than the first update frequency and higher than the second update frequency; the grouping category also includes a medium frequency group; step S702 if the update frequency of the data to be processed is lower than the first update frequency and higher than the second update frequency, then determining that the grouping category corresponding to the data to be processed is a low frequency group, including: step S1001 to step S1002; step S705 also includes: step S1003;

[0215] In step S1001, if the update frequency of the data to be processed is lower than the first update frequency and higher than the fourth update frequency, it is determined that the grouping category corresponding to the data to be processed is a medium frequency group;

[0216] In step S1002, if the update frequency of the data to be processed is lower than the fourth update frequency and higher than the second update frequency, it is determined that the grouping category corresponding to the data to be processed is a low-frequency group;

[0217] In step S1003, the to-be-processed data corresponding to the intermediate frequency group is written into the fourth frequency group.

[0218] In an embodiment of the present application, after obtaining each data to be processed and the update frequency corresponding to each data to be processed, the grouping category corresponding to each data to be processed can be determined according to the different update frequencies corresponding to each data to be processed, that is, if the update frequency of the data to be processed is lower than the first update frequency and higher than the fourth update frequency, then the grouping category corresponding to the data to be processed is determined to be a medium-frequency group. Similarly, if the update frequency of the data to be processed is lower than the fourth update frequency and higher than the second update frequency, then the grouping category corresponding to the data to be processed is determined to be a low-frequency group, so that each data to be processed can be written to a different group in the disk storage space according to the grouping category, and data processing and data recovery (for example, garbage collection) can be performed more specifically according to different groups, thereby improving the timeliness and accuracy of zone defragmentation (i.e., garbage collection).

[0219] Specifically, Fig.18 As shown, after obtaining each data to be processed and the update frequency of each data to be processed, the update frequency of each data to be processed can be compared with the update frequency corresponding to each grouping category (for example, high-frequency group, medium-frequency group, low-frequency group, extremely low-frequency group or cold storage group, etc., not specifically limited here). If the update frequency corresponding to a data to be processed (such as 10 days) is lower than the first update frequency (such as 1 day) and higher than the fourth update frequency (such as 20 days), it can be determined that the grouping category corresponding to the data to be processed is the medium-frequency group. Similarly, if the update frequency corresponding to a data to be processed (such as 25 days) is lower than the fourth update frequency (such as 20 days) and higher than the second update frequency (such as 30 days), it can be determined that the grouping category corresponding to the data to be processed is the low-frequency group.

[0220] Further, after obtaining the grouping category corresponding to each data to be processed, each data to be processed can be written into a different group in the disk storage space according to the grouping category, that is, the data to be processed corresponding to the intermediate frequency group is written into the fourth frequency group (i.e., the intermediate frequency group) (for example, Fig.19 The data to be processed corresponding to the intermediate frequency groups shown are written into group-3 and group-M-1).

[0221] Optionally, in the above Fig.10 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Fig.11 As shown, after step S1003 writes the to-be-processed data corresponding to the intermediate frequency group into the fourth frequency group, the method further includes:

[0222] In step S1101, when it is detected that the fragmentation rate in the fourth frequency group is greater than the fragmentation rate threshold, the data in the fourth frequency group is recycled, and data is continuously written to the second frequency group and the third frequency group;

[0223] It can be understood that after the data to be processed is written to the corresponding group in the disk storage space, for example, the fourth frequency group (i.e., the medium frequency group), the fourth frequency group data has been written. When the business object needs to delete all data, delete part of the data, or read and write frequently for a fourth frequency group data, it can directly index to the corresponding zone (for example, 3 zones in group-3 of the fourth frequency group (i.e., the medium frequency group)) according to the data identifier, and read the fourth frequency group data in the zone (for example, assuming that a zone is 256M) (for example, 80M data). During the reading process, it can be determined or read the data that needs to be deleted by the data deletion request (for example, 20M data out of 80M data), or the storage space fragments (i.e., disk fragments) (for example, 10M data) formed by continuously storing in continuous clusters of the disk.

[0224] Furthermore, the data to be deleted and the disk fragments formed by continuously storing the data in the continuous clusters of the disk can be regarded as storage space fragments (such as Fig. 20 As shown, the fourth frequency group (i.e., the middle frequency group) has data that needs to be deleted or will have a corresponding increase in fragmentation when it is saved continuously). Then, the memory occupied by the storage space fragments can be divided by the memory of the fourth frequency group as the fragmentation rate (for example, the fragmentation rate increased by 4%).

[0225] At the same time, the newly written data of the business object can be received, that is, a new data write request is received. According to the new data write request, the data obtained after the data of the fourth frequency group is read and deleted (that is, the remaining 60M data) can be spliced ​​with the new data to be processed (for example, 30M) to obtain the spliced ​​data (that is, 90M data). Since the zone has 176M of remaining storage space after consuming 80M of space, the spliced ​​data (that is, 90M data) can be overwritten to the data in the originally occupied 80M space, and 10M of the remaining 176M storage space can be sequentially occupied, that is, the occupied space is increased.

[0226] Therefore, since the update frequency of the fourth frequency group (i.e., the medium frequency group) is not very fast, it can be seen that compared with the first frequency group (i.e., the high frequency group), there are more data deletions or the data to be processed is continuously saved in the fourth frequency group (i.e., the medium frequency group) (for example, Group-3 and Group-M-1), and the corresponding disk fragmentation will further increase. After continuous cumulative calculation, the fragmentation rate of the fourth frequency group (i.e., the medium frequency group) can be regularly detected at certain time intervals. For example, the fragmentation rate caused by the deletion of data or the continuous storage of data to be processed in the fourth frequency group (i.e., the medium frequency group) can be detected at certain time intervals (for example, one day, etc.).

[0227] Furthermore, if the fragmentation rate of the fourth frequency group (i.e., the medium frequency group) (for example, the fragmentation rate of the fourth frequency group (i.e., the medium frequency group) is 52%) is greater than the fragmentation threshold (for example, 50%, the fragmentation threshold can be set according to actual application requirements, and is not specifically limited here), that is, the business pressure or load of the fourth frequency group (i.e., the medium frequency group) is heavy, then the fourth frequency group (i.e., the medium frequency group) can be subjected to garbage collection (i.e., defragmentation), and one or more of a clearing algorithm, a marking and defragmenting algorithm, a generational collection algorithm, or a replication algorithm can be used to recycle and clean up the storage space fragments (i.e., disk fragments) in the classification group, that is, to use up the space occupied by the fragments. The memory space is cleaned up, so that the fragments of the fourth frequency group (i.e., the medium frequency group) return to a healthy state after recycling (i.e., the fragments are reduced), that is, the remaining available space of the fourth frequency group (i.e., the medium frequency group) increases (for example, the remaining available space of the fourth frequency group (i.e., the medium frequency group) is 48%, which increases to 61% of the available space of the fourth frequency group (i.e., the medium frequency group)), and each zone can be defragmented more specifically, thereby improving the timeliness and accuracy of the zone defragmentation. At the same time, since the update frequency of the second frequency group (i.e., the low frequency group) and the third frequency group (i.e., the cold storage group) is not high, that is, the fragments increase slowly (for example, Fig.21 The fragmentation rate of the second frequency group (i.e., the low frequency group) shown is approximately 5%, which is much smaller than the fragmentation threshold. Frequent garbage collection is not required, so data can be continuously written to the second frequency group (i.e., the low frequency group) and the third frequency group (i.e., the cold storage group) to write as much data as possible, thereby improving the utilization of the disk storage space.

[0228] In step S1102, when it is detected that the fragmentation rate in the third frequency group is greater than the fragmentation rate threshold, data in the third frequency group is recycled.

[0229] In an embodiment of the present application, since the update frequency of the third frequency group (i.e., the cold storage group) is very slow, when it is detected that the fragmentation rate in the third frequency group (i.e., the cold storage group) is greater than the fragmentation rate threshold, the third frequency group (i.e., the cold storage group) can be recycled to release the memory space of the third frequency group (i.e., the cold storage group), thereby improving the utilization rate of the disk storage space.

[0230] Specifically, since the update frequency of the third frequency group (i.e., the cold storage group) is very slow, it can be known that compared with the second frequency group (i.e., the low frequency group), when more data is deleted or continuously stored in the third frequency group (i.e., the cold storage group) (e.g., group-125), the corresponding disk fragmentation will further increase. Therefore, the fragmentation rate of the third frequency group (i.e., the cold storage group) can be regularly detected at a certain time interval. For example, the fragmentation rate generated by deleting data or continuously storing data in the third frequency group (i.e., the cold storage group) is detected at a second time interval (e.g., 30 days, etc.). If the fragmentation rate (e.g., assuming that the fragmentation rate of the third frequency group (i.e., the cold storage group) is If the fragmentation rate is 54%) and is greater than the fragmentation threshold (for example, 50%), that is, the business pressure or load of the third frequency group (that is, the cold storage group) is heavy, garbage collection (that is, defragmentation) can be performed on the third frequency group (that is, the cold storage group), that is, one or more of the clearing algorithm, mark defragmentation, generational collection algorithm, or replication algorithm can be used to recycle and clean up the disk fragments in the classification group, that is, to clean up the memory space occupied by the storage space fragments (that is, disk fragments), that is, to defragment each zone in a more targeted manner, thereby improving the timeliness and accuracy of the zone defragmentation.

[0231] Optionally, in the above Fig.11 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Fig.12 As shown, in step S1101, when it is detected that the fragmentation rate in the fourth frequency group is greater than the fragmentation rate threshold, data in the fourth frequency group is recycled, and data is continuously written to the second frequency group and the third frequency group, including:

[0232] In step S1201, when the fourth frequency group data in the fourth frequency group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the fourth frequency group is accumulated;

[0233] In step S1202, a fragmentation rate of the fourth frequency group is calculated based on the accumulated memory occupied by the storage space fragments and the memory of the fourth frequency group;

[0234] In step S1203, when the fragmentation rate of the fourth frequency group is greater than the fragmentation rate threshold, the data in the fourth frequency group is recycled, and data is continuously written to the second frequency group and the third frequency group.

[0235] It can be understood that after the data to be processed is written to the corresponding group in the disk storage space, for example, the fourth frequency group (i.e., the medium frequency group), the fourth frequency group data has been written. When the business object needs to delete all data, delete part of the data, or read and write frequently for a fourth frequency group data, it can directly index to the corresponding zone (for example, 3 zones in group-3 of the fourth frequency group (i.e., the medium frequency group)) according to the data identifier, and read the fourth frequency group data in the zone (for example, assuming that a zone is 256M) (for example, 80M data). During the reading process, it can be determined or read the data that needs to be deleted by the data deletion request (for example, 20M data out of 80M data), or the storage space fragments formed by continuously storing in continuous clusters of the disk (for example, 10M data).

[0236] Furthermore, the data to be deleted and the disk fragments formed by continuously storing the data in the continuous clusters of the disk can be regarded as storage space fragments (such as Fig. 20 As shown, the fourth frequency group (i.e., the middle frequency group) has data that needs to be deleted or will have a corresponding increase in fragmentation when it is saved continuously). Then, the memory occupied by the storage space fragments can be divided by the memory of the fourth frequency group as the fragmentation rate (for example, the fragmentation rate increased by 4%).

[0237] At the same time, the newly written data of the business object can be received, that is, a new data write request is received. According to the new data write request, the data obtained after the data of the fourth frequency group is read and deleted (that is, the remaining 60M data) can be spliced ​​with the new data to be processed (for example, 30M) to obtain the spliced ​​data (that is, 90M data). Since the zone has 176M of remaining storage space after consuming 80M of space, the spliced ​​data (that is, 90M data) can be overwritten to the data in the originally occupied 80M space, and 10M of the remaining 176M storage space can be sequentially occupied, that is, the occupied space is increased.

[0238] Therefore, since the update frequency of the fourth frequency group (i.e., the medium frequency group) is not very fast, it can be seen that compared with the first frequency group (i.e., the high frequency group), there are more data deletions or the data to be processed is continuously saved in the fourth frequency group (i.e., the medium frequency group) (for example, Group-3 and Group-M-1), and the corresponding disk fragmentation will further increase. After continuous cumulative calculation, the fragmentation rate of the fourth frequency group (i.e., the medium frequency group) can be regularly detected at certain time intervals. For example, the fragmentation rate caused by the deletion of data or the continuous storage of data to be processed in the fourth frequency group (i.e., the medium frequency group) can be detected at certain time intervals (for example, one day, etc.).

[0239] Furthermore, if the fragmentation rate of the fourth frequency group (i.e., the medium frequency group) (for example, the fragmentation rate of the fourth frequency group (i.e., the medium frequency group) is 52%) is greater than the fragmentation threshold (for example, 50%, the fragmentation threshold can be set according to actual application requirements, and is not specifically limited here), that is, the business pressure or load of the fourth frequency group (i.e., the medium frequency group) is heavy, then the fourth frequency group (i.e., the medium frequency group) can be subjected to garbage collection (i.e., defragmentation), and one or more of a clearing algorithm, a marking and defragmenting algorithm, a generational collection algorithm, or a replication algorithm can be used to recycle and clean up the disk fragments in the classification group, that is, to clear the memory space occupied by the fragments. The fourth frequency group (i.e., the middle frequency group) is cleaned up, so that the fragments of the fourth frequency group (i.e., the middle frequency group) return to a healthy state after recycling (i.e., the fragments are reduced), that is, the remaining available space of the fourth frequency group (i.e., the middle frequency group) increases (for example, the remaining available space of the fourth frequency group (i.e., the middle frequency group) is 48%, which increases to 61% of the available space of the fourth frequency group (i.e., the middle frequency group)), and each zone can be defragmented more specifically, thereby improving the timeliness and accuracy of the zone defragmentation. At the same time, since the update frequency of the second frequency group (i.e., the low frequency group) and the third frequency group (i.e., the cold storage group) is not high, that is, the fragments increase slowly (for example, Fig.21 The fragmentation rate of the second frequency group (i.e., the low frequency group) shown is approximately 5%, which is much smaller than the fragmentation threshold. Frequent garbage collection is not required, so data can be continuously written to the second frequency group (i.e., the low frequency group) and the third frequency group (i.e., the cold storage group) to write as much data as possible, thereby improving the utilization of the disk storage space.

[0240] Optionally, in the above Fig.11 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Fig.13 As shown, in step S1102, when it is detected that the fragmentation rate in the third frequency group is greater than the fragmentation rate threshold, data in the third frequency group is recycled, including:

[0241] In step S1301, when the third frequency group data in the third frequency group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the third frequency group is accumulated;

[0242] In step S1302, a fragmentation rate of the third frequency group is calculated based on the accumulated memory occupied by the storage space fragments and the memory of the third frequency group;

[0243] In step S1303, when the fragmentation rate of the third frequency group is greater than the fragmentation rate threshold, data in the third frequency group is recycled.

[0244] It can be understood that after the data to be processed is written to the corresponding group in the disk storage space, for example, the third frequency group (i.e., the cold storage group), it is the third frequency group data. When the business object needs to delete or continuously save a certain third frequency group data in the third frequency group (i.e., the cold storage group), it can directly index to the corresponding zone (e.g., group -125 of the third frequency group (i.e., the cold storage group)) according to the data identifier, and read the third frequency group data in the zone (e.g., assuming that a zone is 256M) (e.g., 60M data). During the reading process, it can be determined or read the data that needs to be deleted by the data deletion request (e.g., 10M data out of 60M data), or the storage space fragments (i.e., disk fragments) (e.g., 20M data) formed by continuously storing in continuous clusters of the disk. Furthermore, the data that needs to be deleted and the disk fragments formed by continuously storing them in continuous clusters on the disk can be regarded as storage space fragments (for example, if the low-frequency group has data that needs to be deleted or there is a corresponding increase in fragmentation when it is continuously stored), and then the memory occupied by the storage space fragments can be divided by the memory of the third frequency group as the fragmentation rate (for example, the fragmentation rate has increased by 4%).

[0245] Further, it can be understood that while determining the fragmented memory to be deleted in the low-frequency group according to the data deletion request, it is also possible to receive new data continuously written by the business object, that is, receive a new data write request. According to the new data write request, the data obtained after data deletion of the read third frequency group data (for example, 60M) (that is, the remaining 50M data) can be spliced ​​with the new data to be processed (for example, 30M) to obtain the spliced ​​data (that is, 80M data). Since the zone (for example, 256M) has 196M of remaining storage space after consuming 60M of space, the spliced ​​data (that is, 80M data) can be overwritten to the data in the originally occupied 60M space, and 20M of the remaining 206M storage space is sequentially occupied, that is, the occupied space is increased.

[0246] Therefore, since the update frequency of the third frequency group (i.e., the cold storage group) is very slow, it can be known that compared with the second frequency group (i.e., the low frequency group), when more data is deleted or continuously stored in the third frequency group (i.e., the cold storage group) (e.g., group-125), the corresponding disk fragmentation will further increase. Therefore, the fragmentation rate of the third frequency group (i.e., the cold storage group) can be regularly detected at a certain time interval. For example, the fragmentation rate caused by deleting data or continuously storing data in the third frequency group (i.e., the cold storage group) is detected at a second time interval (e.g., 30 days, etc.). If the fragmentation rate (e.g., assuming that the third frequency group (i.e., the cold storage group) The fragmentation rate is 54%) and is greater than the fragmentation threshold (for example, 50%), that is, the business pressure or load of the third frequency group (that is, the cold storage group) is heavy, then garbage collection (that is, defragmentation) can be performed on the third frequency group (that is, the cold storage group), that is, one or more of the clearing algorithm, mark defragmentation, generational collection algorithm or replication algorithm can be used to recycle and clean up the storage space fragments in the classification group, that is, to clean up the memory space occupied by the storage space fragments, that is, to defragment each zone in a more targeted manner, thereby improving the timeliness and accuracy of the zone defragmentation.

[0247] Optionally, in the above Figure 8 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Fig.14 As shown, the first frequency group includes at least a first subgroup and a second subgroup; the fragmentation rate of the first frequency group includes at least a first fragmentation rate and a second fragmentation rate; step S801 when the first frequency group data in the first frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the first frequency group, including: step S1401 to step S1402; step S802 includes: step S1403 to step S1404; step S803 includes: step S1405 to step S1406;

[0248] In step S1401, when the first frequency group data in the first subgroup is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the first subgroup is accumulated;

[0249] In step S1402, when the first frequency group data in the second group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the second group is accumulated;

[0250] Specifically, since the update frequency of the first frequency group (ie, the high frequency group) is relatively fast, and according to the characteristic that the zone supports sequential storage, it can be known that the first subgroup (eg, Fig. 20 The update frequency of the group-1) is higher than that of the second group (e.g., Fig. 202), when the first subgroup (e.g., Fig. 20 As more data is deleted in the group -1) shown in the figure, the corresponding fragmentation will further increase, so the fragmentation rate of the first frequency group (i.e., high frequency group) can be regularly detected at certain time intervals. For example, according to the first time interval (e.g., 3 hours, etc.), the fragmented memory to be deleted in the first group of the first frequency group (i.e., high frequency group) corresponding to the current moment is counted (e.g., assumed to be 5.1G).

[0251] Similarly, the fragmented memory to be deleted of the second small group in the first frequency group (i.e., high frequency group) corresponding to the current moment can be counted according to the first time interval (e.g., 3 hours, etc.) (e.g., assuming it is 2G), and based on the fragmented memory to be deleted of the second small group in the first frequency group (i.e., high frequency group) corresponding to the current moment.

[0252] In step S1403, a first fragmentation rate of the first group is calculated based on the accumulated memory occupied by the storage space fragments generated in the first group and the memory of the first group;

[0253] In step S1404, based on the accumulated memory occupied by the storage space fragments generated in the second group and the memory of the second group, a second fragmentation rate of the second group is calculated;

[0254] Specifically, based on the fragmented memory to be deleted of the first group in the first frequency group (i.e., high frequency group) corresponding to the current moment, the first fragmentation rate corresponding to the current moment is calculated. For example, the quotient between the fragmented memory to be deleted of the first group in the first frequency group (i.e., high frequency group) corresponding to the current moment (e.g., 5.1G) and the total memory of the first frequency group (i.e., high frequency group) (e.g., 10G) is calculated, that is, the first fragmentation rate corresponding to the current moment (e.g., Fig. 20 The indicated group-1 corresponds to a fragmentation rate of 51%).

[0255] Similarly, based on the fragmented memory to be deleted of the second group in the first frequency group (i.e., high frequency group) corresponding to the current moment, the second fragmentation rate corresponding to the current moment is calculated. For example, the quotient between the fragmented memory to be deleted of the second group in the high frequency group corresponding to the current moment (e.g., 2G) and the total memory of the first frequency group (i.e., high frequency group) (e.g., 10G) is calculated, that is, the second fragmentation rate corresponding to the current moment (e.g., Fig.21 The indicated group-2 corresponds to a fragmentation rate of 20%).

[0256] In step S1405, when the first fragmentation rate of the first group is greater than the fragmentation rate threshold and the second fragmentation rate of the second group is less than the fragmentation rate threshold, the data in the first group is recycled, and data is continuously written to the second group and the second frequency group;

[0257] In step S1406, when the second fragmentation rate of the second subgroup is greater than the fragmentation rate threshold, the data in the second subgroup is recycled, and data is continuously written to the second frequency group.

[0258] Specifically, if the first fragmentation rate of the first group (for example, the first fragmentation rate of the first group of the first frequency group (i.e., the high frequency group) is 51%) is greater than the fragmentation threshold (for example, 50%, the fragmentation threshold can be set according to actual application requirements, and is not specifically limited here), and the second fragmentation rate of the second group (for example, the second fragmentation rate of the second group of the first frequency group (i.e., the high frequency group) is 20%) is less than the fragmentation rate threshold, that is, the business pressure or load of the first group is heavy, then garbage collection (i.e., defragmentation) can be performed on the first group in the first frequency group (i.e., the high frequency group), and one or more of a clearing algorithm, a marking and sorting algorithm, a generational collection algorithm, or a replication algorithm can be used to recycle and clean up the storage space fragments (i.e., disk fragments) in the first group, that is, to clean up the memory space occupied by the fragments, so that the fragments of the first group after recycling return to a healthy state (i.e., the fragments are reduced), and each zone can be defragmented more specifically, thereby improving the timeliness and accuracy of the zone defragmentation. At the same time, since the fragmentation rate of the second group is not high and the update frequency of the second frequency group (i.e., the low frequency group) is not high, that is, the fragmentation increases slowly (for example, Fig.21 The fragmentation rate of the second frequency group (i.e., the low-frequency group) shown is approximately 5%, which is much smaller than the fragmentation threshold. Frequent garbage collection is not required, so data can be continuously written to the second small group and the second frequency group (i.e., the low-frequency group) to write as much data as possible, thereby improving the utilization of disk storage space.

[0259] Similarly, if the second fragmentation rate of the second group (for example, assuming that the second fragmentation rate of the second group of the first frequency group (i.e., high frequency group) is 52%) is greater than the fragmentation threshold (for example, 50%, the fragmentation threshold can be set according to actual application requirements, and no specific restrictions are made here), that is, the business pressure or load of the second group is heavy, then garbage collection (i.e., defragmentation) can be performed on the second group in the first frequency group (i.e., high frequency group), and one or more of the clearing algorithm, mark-defragmentation algorithm, generational collection algorithm, or replication algorithm can be used to recycle and clean up the storage space fragments (i.e., disk fragments) in the second group, that is, to clean up the memory space occupied by the fragments, so that the fragments of the second group return to a healthy state (i.e., the fragments are reduced) after recovery.

[0260] Optionally, in the above Figure 7 Based on the corresponding embodiment, in another optional embodiment of the data processing method of the storage space provided in the embodiment of the present application, as Fig.15As shown, the first frequency group includes at least a first subgroup and a second subgroup; step S704 writes the to-be-processed data corresponding to the high frequency group into the first frequency group, including:

[0261] In step S1501, if the memory required by the data to be processed is smaller than the remaining memory of the first group, the data to be processed is written into the first group;

[0262] In step S1502, if the memory required by the data to be processed is greater than the remaining memory of the first group, the data to be processed is written into the second group.

[0263] In an embodiment of the present application, when writing the data to be processed corresponding to the high-frequency group into the first frequency group (i.e., the high-frequency group), it can be based on the characteristic of the zone supporting sequential storage, that is, based on the order between the first small group and the second small group. If the memory required for the data to be processed is less than the remaining memory of the first small group, the data to be processed can be sequentially written into the first small group. Conversely, if the memory required for the data to be processed is greater than the remaining memory of the first small group, the data to be processed will be sequentially written into the second small group to facilitate subsequent accurate reading and access of the data.

[0264] Specifically, after obtaining the grouping category corresponding to each data to be processed, each data to be processed can be written into a different group in the disk storage space according to the grouping category. For example, the data to be processed corresponding to the high-frequency group is written into the first frequency group (i.e., the high-frequency group) (for example, Fig.19 When the to-be-processed data corresponding to the high-frequency group shown is written into group-1 and group-2, according to the first group (for example, Fig.19 The first frequency group (i.e., the high frequency group) is shown as group-1) and the second subgroup (e.g., Fig.19 The order between group -2) of the first frequency group (i.e., the high frequency group) shown in the figure is, first, the memory required for the current data to be processed is counted, and the remaining memory of the current first group and the remaining memory of the second group are obtained, and then the memory required for the current data to be processed is compared with the remaining memory of the current first group. If the memory required for the data to be processed is less than the remaining memory of the first group, the data to be processed can be written into the first group in priority order. On the contrary, if the memory required for the data to be processed is greater than the remaining memory of the first group, it can be understood that the remaining memory of the first group is insufficient, and the memory required for the current data to be processed can be compared with the remaining memory of the current second group. If the memory required for the data to be processed is less than the remaining memory of the second group, the data to be processed can be written into the second group in sequence.

[0265] It can be understood that, when data is continuously written to other groups (eg, the second frequency group (ie, the low frequency group) and the third frequency group (ie, the cold storage group)), for example, Fig.19Group-M to Group-N of the second frequency group (i.e., the low frequency group) shown continue to write data (i.e., data can be written to consecutive positions of the same zone in the group or to the next zone in the group). Similarly, the third frequency group (i.e., the cold storage group) continues to write data (i.e., data can be written to consecutive positions of the same zone in the group or to the next zone in the group).

[0266] The following is a detailed description of the data processing device of the storage space in this application. Fig.23 , Fig.23 This is a schematic diagram of an embodiment of a data processing device for storage space in an embodiment of the present application. The data processing device 20 for storage space includes:

[0267] An acquisition unit 201 is used to acquire the data to be processed based on the data processing request, and acquire the update frequency of the data to be processed;

[0268] The storage unit 202 is used to write the data to be processed into one of at least two groups in the storage space fragment according to the update frequency, wherein the at least two groups include a first frequency group and a second frequency group, the first frequency group corresponds to a first update frequency, the second frequency group corresponds to a second update frequency, and the first update frequency is higher than the second update frequency.

[0269] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the acquisition unit 201 can be specifically used for:

[0270] The statistical data identifies the historical read and write times in the storage space fragments, and calculates the initial update frequency of the to-be-processed data based on the historical read and write times.

[0271] Optionally, in the above Fig.23 Based on the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application,

[0272] The processing unit 203 is configured to recycle the data in the first frequency group and continue to write data to the second frequency group when it is detected that the fragmentation rate in the first frequency group is greater than the fragmentation rate threshold;

[0273] The processing unit 203 is further configured to recycle the data in the second frequency group when it is detected that the fragmentation rate in the second frequency group is greater than the fragmentation rate threshold.

[0274] Optionally, in the above Fig.23 Based on the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application,

[0275] The acquiring unit 201 is further configured to take each to-be-processed data written into the first frequency group as first frequency group data;

[0276] The processing unit 203 is further used to count the number of times each first frequency group data is read and written from being written into the first frequency group to the current time, and calculate the current update frequency of the first frequency group data based on the number of times it is read;

[0277] The storage unit 202 is further used to convert the first frequency group data whose current update frequency is less than the first update frequency from the first group to other groups in the storage space fragment for storage.

[0278] Optionally, in the above Fig.23 Based on the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application,

[0279] The acquiring unit 201 is further configured to take each to-be-processed data written into the second frequency group as the second frequency group data;

[0280] The processing unit 203 is further used to count the number of times each second frequency group data is read and written from being written into the second frequency group to the current time, and calculate the current update frequency of the second frequency group data based on the number of times it is read;

[0281] The storage unit 202 is further used to convert the second frequency group data whose current update frequency is greater than the second update frequency from the second group to other groups in the storage space fragment for storage.

[0282] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the storage unit 202 can be specifically used for:

[0283] Determine the grouping category corresponding to the data to be processed according to the initial update frequency of the data to be processed;

[0284] The data to be processed is written into one of at least two groups in the storage space fragment according to the grouping category.

[0285] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the storage unit 202 can be specifically used for:

[0286] If the initial update frequency of the data to be processed is higher than the first update frequency, determining that the grouping category corresponding to the data to be processed is a high-frequency group;

[0287] If the initial update frequency of the data to be processed is lower than the first update frequency and higher than the second update frequency, determining that the grouping category corresponding to the data to be processed is a low-frequency group;

[0288] If the initial update frequency of the data to be processed is lower than the second update frequency, it is determined that the grouping category corresponding to the data to be processed is a cold storage group.

[0289] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the storage unit 202 can be specifically used for:

[0290] Writing the to-be-processed data corresponding to the high-frequency group into the first frequency group;

[0291] Writing the to-be-processed data corresponding to the low-frequency group into the second frequency group;

[0292] The data to be processed corresponding to the cold storage group is written into the third frequency group.

[0293] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the processing unit 203 can be specifically used for:

[0294] When the first frequency group data in the first frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the first frequency group;

[0295] Calculate the fragmentation rate of the first frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the first frequency group;

[0296] When the fragmentation rate of the first frequency group is greater than the fragmentation rate threshold, the data in the first frequency group is recycled, and data is continuously written to the second frequency group.

[0297] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the processing unit 203 can be specifically used for:

[0298] When the second frequency group data in the second frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the second frequency group;

[0299] Calculate the fragmentation rate of the second frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the second frequency group;

[0300] When the fragmentation rate of the second frequency group is greater than the fragmentation rate threshold, data in the second frequency group is recycled.

[0301] Optionally, in the above Fig.23On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the storage unit 202 can be specifically used for:

[0302] If the initial update frequency of the data to be processed is lower than the first update frequency and higher than the fourth update frequency, determining that the grouping category corresponding to the data to be processed is a medium frequency group;

[0303] If the initial update frequency of the data to be processed is lower than the fourth update frequency and higher than the second update frequency, determining that the grouping category corresponding to the data to be processed is a low-frequency group;

[0304] The storage unit 202 can be specifically used for:

[0305] The data to be processed corresponding to the intermediate frequency group is written into the fourth frequency group.

[0306] Optionally, in the above Fig.23 Based on the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application,

[0307] The processing unit 203 is further configured to, when it is detected that the fragmentation rate in the fourth frequency group is greater than the fragmentation rate threshold, recycle the data in the fourth frequency group and continue to write data to the second frequency group and the third frequency group;

[0308] The processing unit 203 is further configured to recycle the data in the third frequency group when it is detected that the fragmentation rate in the third frequency group is greater than the fragmentation rate threshold.

[0309] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the processing unit 203 can be specifically used for:

[0310] When the fourth frequency group data in the fourth frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the fourth frequency group;

[0311] Calculate the fragmentation rate of the fourth frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the fourth frequency group;

[0312] When the fragmentation rate of the fourth frequency group is greater than the fragmentation rate threshold, the data in the fourth frequency group is recycled, and data is continuously written to the second frequency group and the third frequency group.

[0313] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the processing unit 203 can be specifically used for:

[0314] When the third frequency group data in the third frequency group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the third frequency group is accumulated;

[0315] Calculate the fragmentation rate of the third frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the third frequency group;

[0316] When the fragmentation rate of the third frequency group is greater than the fragmentation rate threshold, data in the third frequency group is recycled.

[0317] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the processing unit 203 can be specifically used for:

[0318] When the first frequency group data in the first group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the first group is accumulated;

[0319] When the first frequency group data in the second group is read and written to generate storage space fragments, the memory occupied by the storage space fragments generated in the second group is accumulated;

[0320] The processing unit 203 may be specifically configured to:

[0321] Calculate the first fragmentation rate of the first group based on the accumulated memory occupied by the storage space fragments generated in the first group and the memory of the first group;

[0322] Calculate the second fragmentation rate of the second group based on the accumulated memory occupied by the storage space fragments generated in the second group and the memory of the second group;

[0323] The processing unit 203 may be specifically configured to:

[0324] When the first fragmentation rate of the first group is greater than the fragmentation rate threshold and the second fragmentation rate of the second group is less than the fragmentation rate threshold, the data in the first group is recycled, and data is continuously written to the second group and the second frequency group;

[0325] When the second fragmentation rate of the second subgroup is greater than the fragmentation rate threshold, the data in the second subgroup is recycled, and data is continuously written to the second frequency group.

[0326] Optionally, in the above Fig.23 On the basis of the corresponding embodiment, in another embodiment of the data processing device for the storage space provided by the embodiment of the present application, the storage unit 202 can be specifically used for:

[0327] If the memory required by the data to be processed is less than the remaining memory of the first group, the data to be processed is written into the first group;

[0328] If the memory required by the data to be processed is greater than the remaining memory of the first group, the data to be processed is written into the second group.

[0329] On the other hand, the present application provides another schematic diagram of a computer device, such as Fig.24 As shown, Fig.24 3 is a schematic diagram of a computer device structure provided in an embodiment of the present application. The computer device 300 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) storing application programs 331 or data 332. Among them, the memory 320 and the storage medium 330 may be temporary storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the computer device 300. Furthermore, the central processing unit 310 may be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the computer device 300.

[0330] The computer device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 333, such as Windows Server 2000, Windows Server 2003, Windows Server 2003E, Windows Server 2003R ... TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0331] The computer device 300 is also used to execute the following steps: Figures 2 to 15 The steps in the corresponding embodiments.

[0332] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following Figures 2 to 15 The illustrated embodiments describe steps in a method.

[0333] Another aspect of the present application provides a computer program product comprising a computer program, which, when executed by a processor, implements the following Figures 2 to 15 The illustrated embodiments describe steps in a method.

[0334] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0335] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0336] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0337] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0338] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

Claims

1. A method for processing data in a storage space, characterized in that: include: Acquire the data to be processed based on the data processing request, and acquire the update frequency of the data to be processed; According to the update frequency, the data to be processed is written into one of at least two groups in the storage space, wherein the at least two groups include a first frequency group and a second frequency group, the first frequency group corresponds to a first update frequency, the second frequency group corresponds to a second update frequency, and the first update frequency is higher than the second update frequency.

2. The method according to claim 1, characterized in that After writing the to-be-processed data into one of the at least two groups in the storage space according to the update frequency, the method comprises: When it is detected that the fragmentation rate in the first frequency group is greater than the fragmentation rate threshold, the data in the first frequency group is recycled, and data is continuously written to the second frequency group; When it is detected that the fragmentation rate in the second frequency group is greater than a fragmentation rate threshold, data in the second frequency group is recycled.

3. The method according to claim 1, characterized in that The data to be processed is correspondingly provided with a data identifier; the updating frequency of obtaining the data to be processed includes: The number of times the data identifier has been read and written in the storage space in history is counted, and the initial update frequency of the to-be-processed data is calculated based on the number of times the data identifier has been read and written in history.

4. The method according to claim 1, characterized in that: After writing the to-be-processed data into one of the at least two groups in the storage space according to the update frequency, the method further comprises: each to-be-processed data written into the first frequency group as first frequency group data; Counting the number of times each of the first frequency group data is read and written from being written into the first frequency group to the present, and calculating the current update frequency of the first frequency group data based on the number of times it is read; The first frequency group data whose current update frequency is less than the first update frequency is converted from the first group to other groups in the storage space for storage.

5. The method according to claim 1, characterized in that After writing the to-be-processed data into one of the at least two groups in the storage space according to the update frequency, the method further comprises: each to-be-processed data written into the second frequency group as second frequency group data; Counting the number of times each of the second frequency group data is read and written from being written into the second frequency group to the present, and calculating the current update frequency of the second frequency group data based on the number of times it is read; The second frequency group data whose current update frequency is greater than the second update frequency is converted from the second group to other groups in the storage space for storage.

6. The method according to claim 4, characterized in that The step of writing the to-be-processed data into one of at least two groups in the storage space according to the update frequency includes: Determining the grouping category corresponding to the data to be processed according to the update frequency of the data to be processed; The to-be-processed data is written into one of at least two groups in the storage space according to the grouping category.

7. The method according to claim 6, characterized in that The storage space further includes a third frequency group, the third update frequency corresponding to the third group is lower than the second update frequency; the grouping categories include at least a high frequency group, a low frequency group and a cold storage group; The determining, according to the update frequency of the data to be processed, the grouping category corresponding to the data to be processed includes: If the update frequency of the data to be processed is higher than the first update frequency, determining that the grouping category corresponding to the data to be processed is a high-frequency group; If the update frequency of the data to be processed is lower than the first update frequency and higher than the second update frequency, determining that the grouping category corresponding to the data to be processed is a low-frequency group; If the update frequency of the data to be processed is lower than the second update frequency, it is determined that the grouping category corresponding to the data to be processed is a cold storage group.

8. The method according to claim 7, characterized in that Writing the to-be-processed data into one of at least two groups in the storage space according to the grouping category preferably includes: Writing the to-be-processed data corresponding to the high-frequency group into the first frequency group; Writing the to-be-processed data corresponding to the low-frequency group into the second frequency group; The to-be-processed data corresponding to the cold storage group is written into the third frequency group.

9. The method according to claim 2, characterized in that: When it is detected that the fragmentation rate in the first frequency group is greater than a fragmentation rate threshold, the data in the first frequency group is recycled and data is continuously written to the second frequency group, including: When storage space fragments are generated by reading and writing the first frequency group data in the first frequency group, accumulating the memory occupied by the storage space fragments generated in the first frequency group; Calculating a fragmentation rate of the first frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the first frequency group; When the fragmentation rate of the first frequency group is greater than a fragmentation rate threshold, data in the first frequency group is recycled, and data is continuously written to the second frequency group.

10. The method according to claim 2, characterized in that When it is detected that the fragmentation rate in the second frequency group is greater than a fragmentation rate threshold, recycling the data in the second frequency group includes: When storage space fragments are generated by reading and writing the second frequency group data in the second frequency group, accumulating the memory occupied by the storage space fragments generated in the second frequency group; Calculating a fragmentation rate of the second frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the second frequency group; When the fragmentation rate of the second frequency group is greater than a fragmentation rate threshold, data in the second frequency group is recycled.

11. The method according to claim 7, characterized in that The storage space further includes a fourth frequency group, wherein a fourth update frequency corresponding to the fourth group is lower than the first update frequency and higher than the second update frequency; the grouping category further includes a medium frequency group; If the update frequency of the data to be processed is lower than the first update frequency and higher than the second update frequency, determining that the grouping category corresponding to the data to be processed is a low-frequency group includes: If the update frequency of the data to be processed is lower than the first update frequency and higher than the fourth update frequency, determining that the grouping category corresponding to the data to be processed is a medium frequency group; If the update frequency of the data to be processed is lower than the fourth update frequency and higher than the second update frequency, determining that the grouping category corresponding to the data to be processed is a low-frequency group; Writing the to-be-processed data into one of at least two groups in the storage space according to the grouping category preferably includes: The to-be-processed data corresponding to the intermediate frequency group is written into the fourth frequency group.

12. The method according to claim 11, characterized in that After writing the to-be-processed data corresponding to the intermediate frequency group into the fourth frequency group, the method further includes: When it is detected that the fragmentation rate in the fourth frequency group is greater than the fragmentation rate threshold, the data in the fourth frequency group is recycled, and data is continuously written to the second frequency group and the third frequency group; When it is detected that the fragmentation rate in the third frequency group is greater than a fragmentation rate threshold, data in the third frequency group is recycled.

13. The method according to claim 12, characterized in that When it is detected that the fragmentation rate in the fourth frequency group is greater than a fragmentation rate threshold, the data in the fourth frequency group is recycled, and data is continuously written to the second frequency group and the third frequency group, including: When the fourth frequency group data in the fourth frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the fourth frequency group; Calculating a fragmentation rate of the fourth frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the fourth frequency group; When the fragmentation rate of the fourth frequency group is greater than the fragmentation rate threshold, the data in the fourth frequency group is recycled, and data is continuously written to the second frequency group and the third frequency group.

14. The method according to claim 12, characterized in that When it is detected that the fragmentation rate in the third frequency group is greater than a fragmentation rate threshold, recycling the data in the third frequency group includes: When the third frequency group data in the third frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the third frequency group; Calculating a fragmentation rate of the third frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the third frequency group; When the fragmentation rate of the third frequency group is greater than the fragmentation rate threshold, data in the third frequency group is recycled.

15. The method according to claim 9, characterized in that The first frequency group includes at least a first subgroup and a second subgroup; the fragmentation rate of the first frequency group includes at least a first fragmentation rate and a second fragmentation rate; when the first frequency group data in the first frequency group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the first frequency group includes: When the first frequency group data in the first group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the first group; When the first frequency group data in the second group is read and written to generate storage space fragments, accumulating the memory occupied by the storage space fragments generated in the second group; The calculating the fragmentation rate of the first frequency group based on the accumulated memory occupied by the storage space fragments and the memory of the first frequency group includes: Calculate a first fragmentation rate of the first group based on the accumulated memory occupied by the storage space fragments generated in the first group and the memory of the first group; Calculate a second fragmentation rate of the second group based on the accumulated memory occupied by the storage space fragments generated in the second group and the memory of the second group; When the fragmentation rate of the first frequency group is greater than a fragmentation rate threshold, recycling the data in the first frequency group and continuously writing data to the second frequency group includes: When the first fragmentation rate of the first group is greater than the fragmentation rate threshold and the second fragmentation rate of the second group is less than the fragmentation rate threshold, the data in the first group is recycled, and data is continuously written to the second group and the second frequency group; When the second fragmentation rate of the second subgroup is greater than the fragmentation rate threshold, the data in the second subgroup is recycled, and data is continuously written to the second frequency group.

16. The method according to claim 8, characterized in that The first frequency group includes at least a first subgroup and a second subgroup; and writing the to-be-processed data corresponding to the high-frequency group into the first frequency group includes: If the memory required by the data to be processed is smaller than the remaining memory of the first group, the data to be processed is written into the first group; If the memory required by the data to be processed is greater than the remaining memory of the first group, the data to be processed is written into the second group.

17. A method and device for processing data in a storage space, characterized in that: include: An acquisition unit, configured to acquire the data to be processed based on the data processing request, and acquire the update frequency of the data to be processed; A storage unit is used to write the data to be processed into one of at least two groups in the storage space according to the update frequency, wherein the at least two groups include a first frequency group and a second frequency group, the first frequency group corresponds to a first update frequency, the second frequency group corresponds to a second update frequency, and the first update frequency is higher than the second update frequency.

18. A computer device comprising a memory, a processor and a bus system, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 16 are implemented; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

20. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

Images