Object defragmentation method and device, equipment and storage medium

By dynamically adjusting the parameter values ​​of the object defragmentation task, using the space replacement frequency of the cache pool and the write-through ratio of the data pool, the problem of being unable to meet the burst task requirements of the front-end server in the existing technology is solved, and the response of the cloud storage device is improved. ability.

CN119336275AActive Publication Date: 2025-01-21ZHEJIANG UNIVIEW TECH CO LTD
View PDF 25 Cites 0 Cited by

Patent Information

Application Number
CN202411867050.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-18
Publication Date
2025-01-21
Estimated Expiration
2044-12-18

AI Technical Summary

Technical Problem

In the prior art, the space requirements and/or performance requirements of front-end server burst tasks cannot be met by preset time and timing.

Method used

By obtaining the space replacement frequency of the cache pool and the write-through ratio of the data pool, dynamically adjust the parameter values ​​of the object defragmentation task to match the current write business pressure.

Benefits of technology

When an emergency task of the front -end server, the data pool and the cache pool can meet their space requirements and performance needs, and improve the response capacity of cloud storage equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119336275A_ABST
    Figure CN119336275A_ABST
Patent Text Reader

Abstract

The invention provides an object defragmentation method, device and equipment and a storage medium, and is applied to the technical field of object storage, and the method comprises the steps: obtaining the number of update times of data in a preset address space in a cache pool in a previous unit time, determining the space replacement frequency of the cache pool in the previous unit time according to the update times in the previous unit time; obtaining a first data volume written through to the data pool by the front-end server in the previous unit time and a total second data volume written in, and determining a write-through proportion of the data pool according to the first data volume and the second data volume; determining a target value currently corresponding to each parameter of the object defragmentation task according to the space replacement frequency and the write-through proportion; the target value of each parameter is matched with the current writing service pressure condition of the front-end server; and executing an object defragmentation task according to the target value currently corresponding to each parameter. By adopting the technical scheme of the invention, the space requirement and / or the performance requirement required by the sudden task of the front-end server can be met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of object storage technology, and in particular to an object defragmentation method, device, equipment and storage medium. Background Art

[0002] In various storage application environments, there are more and more scenarios where cloud storage devices are used for storage (also known as object-based cloud storage). When storing, the front-end server generally sends files of various sizes to the cloud storage device for storage. Such application scenarios require both a large amount of storage space and high performance requirements, so cloud storage devices are required to take into account multiple requirements of performance, redundancy, security, and low cost. Based on this, in small file application scenarios, cloud storage systems usually add a cache pool composed of multiple SSD (Solid State Disk) disks, using the better write performance of SSD disks to temporarily store a large number of small file objects quickly sent by the upper front-end server. Then, through appropriate object merging strategies, small objects of various sizes are merged into larger objects and rewritten to a data pool composed of HDD (Hard Disk Drive) disks with larger capacity and lower cost to prepare for subsequent read and write needs of the front-end server.

[0003] However, when the cloud storage device encounters an exception during the process of merging small objects, or the cloud storage device itself is deleting expired data, or the front-end server may upload files with the same name or delete some of the written files, etc., it will cause object fragments of large objects to be generated in the data pool of the cloud storage device. The space occupied by these object fragments is still in use in the eyes of the front-end server, but it is actually invalid occupied space that can be released. To address this problem, it is usually necessary to sort the object fragments in the data pool to release the invalid occupied space in the data pool. In the related art, when sorting the object fragments in the data pool, it is generally necessary to pre-set the time and then sort the object fragments regularly.

[0004] However, the above technology may fail to meet the space requirements and / or performance requirements of the front-end server when burst tasks occur in some cases. Summary of the invention

[0005] The present invention provides an object defragmentation method, device, equipment and storage medium, which are used to solve the defect in the prior art that the space requirements and / or performance requirements required by the front-end server for sudden tasks may not be met in some cases by regularly defragmenting objects at a pre-set time. The parameter value of the object defragmentation task that matches the write business pressure of the data pool and the cache pool is determined by the replacement frequency of the cache pool in the previous unit time and the write-through ratio of the data pool in the previous unit time. The execution of the object defragmentation task can match the write business pressure of the current data pool and the cache pool, so that when the front-end server has a sudden task, the data pool and the cache pool can also meet its required space requirements and / or performance requirements.

[0006] The present invention provides an object fragmentation defragmentation method, comprising: Obtaining the number of updates of data in a preset address space in the cache pool in the previous unit time, and determining the space replacement frequency of the cache pool in the previous unit time according to the number of updates in the previous unit time; Obtaining a first data volume that is written through to the data pool by the front-end server in the previous unit time and a second data volume that is written in total by the front-end server in the previous unit time, and determining a write-through ratio of data to the data pool in the previous unit time based on the first data volume and the second data volume; According to the space replacement frequency of the cache pool and the write-through ratio of the data pool, the current target value corresponding to each parameter of the object defragmentation task is determined; the above-mentioned object defragmentation task is a task of arranging fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server; Executes the object defragmentation task according to the current target value corresponding to each parameter.

[0007] According to an object defragmentation method provided by the present invention, the target value currently corresponding to each parameter of the object defragmentation task is determined according to the space replacement frequency of the cache pool and the write-through ratio of the data pool, including: Determine whether the write-through ratio of the data pool exceeds a preset ratio threshold range to obtain a first determination result; and determine whether the space replacement frequency of the cache pool is less than a preset frequency threshold to obtain a second determination result; According to the first judgment result and the second judgment result, a target value currently corresponding to each parameter of the object defragmentation task is determined.

[0008] According to an object defragmentation method provided by the present invention, the preset ratio threshold range includes a lower limit value, and the current target value corresponding to each parameter of the object defragmentation task is determined according to the first judgment result and the second judgment result, including: If the first judgment result is that the write-through ratio of the data pool is equal to the lower limit value, and the second judgment result is that the space replacement frequency of the cache pool is less than the preset frequency threshold, then obtain the historical value corresponding to each parameter of the object defragmentation task; the above parameters include at least one of the expiration time of the object defragmentation task, the threshold for the fragmented object to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of the thread; Perform at least one of the following adjustment operations on the historical values ​​of the parameters of the object defragmentation task: reduce the historical value corresponding to the expiration time of the object defragmentation task to the target value, increase the historical value corresponding to the threshold of the fragmented object entering the defragmentation task queue to the target value, increase the historical value corresponding to the number of threads of the object defragmentation task to the target value, increase the historical value corresponding to the number of fragments processed in parallel by a single thread to the target value, and increase the historical value corresponding to the running time of the thread to the target value.

[0009] According to an object defragmentation method provided by the present invention, the preset ratio threshold range includes a lower limit value, and the current target value corresponding to each parameter of the object defragmentation task is determined according to the first judgment result and the second judgment result, including: If the first judgment result is that the write-through ratio of the data pool is equal to the lower limit value, and the second judgment result is that the space replacement frequency of the buffer pool is not less than the preset frequency threshold, then obtain the historical value corresponding to each parameter of the object defragmentation task; the above parameters include at least one of the first buffer pool usage rate threshold that triggers data to be written through to the data pool and the second buffer pool usage rate threshold that stops data from being written through to the data pool; At least one of the following adjustment operations is performed on the historical value of the parameter of the object defragmentation task: reducing the historical value of the first cache pool usage threshold to the target value, and increasing the historical value of the second cache pool usage threshold to the target value.

[0010] According to an object defragmentation method provided by the present invention, the preset ratio threshold range includes a lower limit value and an upper limit value, the upper limit value is greater than the lower limit value, and the current target value corresponding to each parameter of the object defragmentation task is determined according to the first judgment result and the second judgment result, including: If the first judgment result is that the write-through ratio of the data pool is greater than the lower limit value and less than the upper limit value, and the second judgment result is that the space replacement frequency of the cache pool is not less than the preset frequency threshold, then obtain the historical value corresponding to each parameter of the object defragmentation task; the above parameters include at least one of the expiration time of the object defragmentation task and the threshold value for the fragmented object to enter the defragmentation task queue; Perform at least one of the following adjustment operations on the historical value of the parameter of the object defragmentation task: increase the historical value corresponding to the expiration time of the object defragmentation task to the target value, and reduce the historical value corresponding to the threshold of the fragmented object entering the defragmentation task queue to the target value.

[0011] According to an object defragmentation method provided by the present invention, the preset ratio threshold range includes an upper limit value, and the current target value corresponding to each parameter of the object defragmentation task is determined according to the first judgment result and the second judgment result, including: If the first judgment result is that the write-through ratio of the data pool is greater than the upper limit value, and the second judgment result is that the space replacement frequency of the cache pool is less than the preset frequency threshold, then obtain the historical value corresponding to each parameter of the object defragmentation task; the above parameters include at least one of the expiration time of the object defragmentation task, the threshold for the fragmented object to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of the thread; Perform at least one of the following adjustment operations on the historical values ​​of the parameters of the object defragmentation task: increase the historical value corresponding to the expiration time of the object defragmentation task to the target value, reduce the historical value corresponding to the threshold of the fragmented object entering the defragmentation task queue to the target value, reduce the historical value corresponding to the number of threads of the object defragmentation task to the target value, reduce the historical value corresponding to the number of fragments processed in parallel by a single thread to the target value, and reduce the historical value corresponding to the running time of the thread to the target value.

[0012] According to an object defragmentation method provided by the present invention, the above-mentioned obtaining the update times of the data in the preset address space in the buffer pool in the previous unit time includes: Get the size of the target address space extracted from the buffer pool in the previous unit time; In the first address space, the second address space and the third address space of the buffer pool, sub-address spaces of the size of the target address space are selected in sequence; the first address space, the second address space and the third address space are address spaces corresponding to the front section, the middle section and the back section of the address code of the buffer pool respectively; The three sub-address spaces are determined as preset address spaces, and the number of updates of the data in the preset address spaces in the cache pool in the previous unit time is counted.

[0013] The present invention also provides an object fragmentation defragmentation device, comprising the following modules: A space replacement frequency determination module is used to obtain the number of updates of data in a preset address space in the cache pool in the previous unit time, and determine the space replacement frequency of the cache pool in the previous unit time according to the number of updates in the previous unit time; A write-through ratio determination module, used to obtain a first amount of data written through to the data pool by the front-end server in the previous unit time and a second amount of data written in total by the front-end server in the previous unit time, and determine a write-through ratio of data written through to the data pool in the previous unit time based on the first amount of data and the second amount of data; The defragmentation task parameter determination module is used to determine the current target value corresponding to each parameter of the object defragmentation task according to the space replacement frequency of the cache pool and the write-through ratio of the data pool; the above-mentioned object defragmentation task is a task for arranging fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server; The object defragmentation module is used to execute the object defragmentation task according to the target value currently corresponding to each parameter.

[0014] The present invention also provides a cloud storage device, which includes a cache pool, a data pool, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements any of the above-mentioned object defragmentation methods.

[0015] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the object defragmentation method described above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above object defragmentation methods.

[0017] The object defragmentation method, device, equipment and storage medium provided by the present invention obtain the number of updates of data in a preset address space in a cache pool in the previous unit time, and determine the space replacement frequency of the cache pool in the previous unit time according to the number of updates in the previous unit time, and simultaneously obtain the first data amount written to the data pool by the front-end server in the previous unit time and the second data amount written in total in the previous unit time, and determine the write-through ratio of data to the data pool in the previous unit time according to the first data amount and the second data amount, and then determine the current target value corresponding to each parameter of the object defragmentation task according to the space replacement frequency of the cache pool and the write-through ratio of the data pool, and then execute the object defragmentation task according to the current target value of each parameter; wherein the object defragmentation task is a task for defragmenting fragmented objects in the data pool, and the target value of each parameter thereof matches the current write business pressure of the front-end server. In this method, by calculating the replacement frequency of the cache pool in the previous unit time and the write-through ratio of the data pool in the previous unit time, the current write business pressure of the data pool and the cache pool, which are the cloud storage device, and the current write business pressure of the front-end server can be intuitively reflected, thereby calculating the parameter value of the object defragmentation task that matches the current write business pressure of the cloud storage device / front-end server. If the front-end server has a sudden task when executing the object defragmentation task according to this parameter value, the cloud storage device can adaptively meet the space requirements and / or performance requirements required by the front-end server. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a schematic diagram of the data flow of cloud storage devices.

[0020] Figure 2 This is a flowchart of object fragmentation generation.

[0021] Figure 3 This is a schematic diagram of object writing form and object fragmentation distribution.

[0022] Figure 4 The present invention is a schematic diagram of the object defragmentation process in the related art.

[0023] Figure 5 This is one of the flow charts of the object defragmentation method provided by the present invention.

[0024] Figure 6 This is the second flow chart of the object defragmentation method provided by the present invention.

[0025] Figure 7 It is a schematic diagram of a flow chart of dynamically adjusting parameters of an object defragmentation task provided by the present invention.

[0026] Figure 8 This is the third flow chart of the object defragmentation method provided by the present invention.

[0027] Fig. 9 This is a schematic diagram of the cache pool address coding provided by the present invention.

[0028] Fig.10 It is a structural schematic diagram of the object fragment defragmentation device provided by the present invention.

[0029] Fig.11 It is a structural schematic diagram of the cloud storage device provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] To facilitate understanding of the technical solution of the present invention, the following first describes the specific terms that may be involved in the embodiments of the present invention.

[0032] Storage pool: A logical storage container where front-end data is distributed in the storage system. It is usually composed of multiple OSDs (Object-Based Storage System) according to a certain strategy, including replica pools and erasure pools, which are invisible to front-end users. Among them, the read and write performance of the replica pool is better, the data has multiple backups and better security, but it occupies a large space, has low space utilization and is more expensive to use. The erasure pool is similar to the RAID (Redundant Arrays of Independent Disks) type of array storage. It ensures that the data has a certain degree of redundancy through the erasure algorithm, taking into account the performance requirements and the redundancy requirements of data security. It can improve space utilization and reduce usage costs.

[0033] Object cloud storage: also called object-based cloud storage, usually each small file or fragment of a large file is regarded as an object. After each object is written to the storage system, all objects are flatly distributed in the address space of the storage pool, and there is no hierarchical relationship between objects.

[0034] Object fragmentation: In small file application scenarios, object cloud storage usually adds a cache pool composed of multiple SSD disks, which usually uses a replica pool (three replicas by default). The better write performance of SSD disks than HDD disks is used to temporarily meet the write requirements of the front-end server for fast and large-scale delivery. Then formulate an object merging strategy. When the merging conditions are met, small objects of various sizes are merged into larger objects and written to a larger capacity data pool composed of HDD disks (default erasure pool). These large objects generated by merging will become object fragments when encountering abnormal merging process, upper layer duplication (duplicate objects of different or same size), expiration deletion, user deletion, etc. These fragments occupy real space and are not released. They cannot be used again when the front-end writes, resulting in a large waste of space, so the task of sorting object fragments is initiated regularly.

[0035] Bucket: Also called a storage bucket, it is the most basic logical unit for storing objects visible to users. Each user can have one or more buckets to store objects of different sizes.

[0036] The following is a description of the relevant technical background of the present invention.

[0037] In various storage application environments, there are more and more scenarios for using cloud storage devices. For example, in large hospitals, many departments are examining different patients at the same time. When each patient undergoes B-ultrasound, X-ray, CT (Computed Tomography), and MRI examinations, hundreds of high-definition images will be generated, and each outpatient clinic will generate a large number of various medical records, examination reports, etc., which will be written into the database, thus generating a large number of database files. Another example is public security or traffic police applications. During peak hours when the traffic volume is heavy, a large number of cameras on various roads will generate a large number of passing car images. All passing car images need to be stored, and then a large number of images need to be extracted to determine whether they are actually illegal or illegal. These analyses may be done manually or by AI (Artificial Intelligence) software. In the above application scenarios, whether it is an image file or a database file, the front-end server will eventually send files of various sizes to the cloud storage system. Such application scenarios require both a large amount of storage space and high performance requirements, so cloud storage devices are required to take into account multiple requirements of performance, redundant security, and low cost. Based on this, in small file application scenarios, cloud storage systems usually add a cache pool composed of multiple SSD disks, using the better write performance of SSD disks to temporarily store a large number of small file objects quickly sent down by the upper-level front-end servers. Then, through appropriate object merging strategies, small objects of various sizes are merged into larger objects and rewritten to the data pool composed of HDD disks with larger capacity and lower cost to prepare for subsequent read and write needs of the front-end servers.

[0038] When initializing the object cloud storage system / cloud storage device, see Figure 1 The data flow diagram of the cloud storage device shown in the figure generally creates a data pool (HDD) and a cache pool (SSD) respectively, and defines a small object threshold N (such as 256k, 1M, 4M, etc.). When the size of a single data object is larger than N (i.e., a large object), it is directly written to the data pool composed of HDD disks; when the data object is smaller than N, it is first written to the cache pool composed of SSD disks, and then the successful writing is returned to the front end, so as to respond to the high-performance writing requirements of the front end. At the same time, when the cloud storage system is initialized, the trigger strategy for merging small objects into large objects and the object defragmentation strategy for large objects are pre-set. See Figure 2The object fragmentation generation process diagram shown in FIG. 1 shows that when the cloud storage system receives an object written by the front-end, it first detects whether it is a small object. If it is a large object, it is directly written to the data pool; otherwise, it detects whether the cache pool is full. If the cache pool is not full, it is written to the cache pool, otherwise it is written to the data pool. When the usage rate of the cache pool reaches the set threshold or the number of objects written to a single Bucket reaches the threshold, it triggers the small object to be merged into a large object. If the merger is successful, the generated large object is written to the data pool and the small object in the cache pool is deleted; if an exception is encountered during the merger, the large object is added to the task queue of the object defragmentation to be executed, waiting for the defragmentation cycle to arrive. During the operation of the system, in addition to the front-end server constantly writing new data, the storage system itself has an expired deletion program to delete expired data. The front-end may also upload files with the same name or delete some of the written files, which will generate object fragments of large objects. The space occupied by these object fragments is still used in the front-end view, but it is actually invalid occupied space that can be released.

[0039] Continue to see Figure 2 as well as Figure 3 The object writing form and object fragmentation distribution diagram shown in the figure, assuming that the large object threshold is greater than 4M, all objects less than or equal to 4M will be judged as small objects and try to write to the cache pool first, while objects greater than 4M will be judged as non-small objects and written to the data pool. When the merging conditions are met to generate large objects, each merged large object can be limited to a maximum of 16M, for example. At this time, objects can be merged to generate large objects, and after the generation is successful, small objects in the cache pool are deleted and the corresponding space is released. When the cache pool is not full, only 16M large objects are stored in the data pool. If the cache pool is full, in addition to the 16M large objects, there will be objects of various sizes written down in the data pool (such as small objects such as pictures and databases). When a large number of object fragments are caused by merging failures, various deletions, front-end rewriting, etc., there will be a large number of object fragment spaces of various sizes in the data pool. These spaces are still actually occupied by 16M spaces one by one in the eyes of front-end users, but in fact, a large number of object fragments are in a state of being rewritten after being sorted. The sizes of these fragments may be different due to various reasons such as different deletion amounts, merge failures, front-end duplication, etc., such as 6M, 9M, 16M, etc., that is, the smallest size of these fragment spaces is the size of a small object, and the largest size is the space size of a 16M large object.

[0040] For the object fragments generated above, the cloud storage device can execute the object defragmentation task to defragment these object fragments and release the corresponding space. Figure 4The object defragmentation process diagram in the related technology shown in the figure sets a fragment expiration time T (i.e., the defragmentation cycle), triggers the object space usage threshold D (the default is 50%, and only when it is lower than D will it enter the task queue to be defragmented), and the object defragmentation task thread runs for t. If the time T has been reached, it is checked whether the space usage of each object fragment is lower than D. If it is lower than D, it is added to the task queue. If the task queue is not empty, the management process of each node will start a thread of the object defragmentation task and start to defragment all object fragments in the same bucket in the task queue one by one. When all object fragments in the bucket are defragmented, the object fragments of the next bucket will continue to be processed. Since for storage systems (such as cloud storage devices), responding to the read and write requests of front-end users as soon as possible is the highest priority task, the priority of the object defragmentation task is placed at a lower position in the system, and the speed of defragmentation is limited by default. If the number of fragments is large, and the cache pool is still merging to generate large objects and writing them to the data pool, or the cache pool is full, the front-end user data begins to be directly written to the data pool, or the front-end user is deleting old data, or the storage system itself is performing various read and write operations such as expired deletion, it will affect the speed of the object defragmentation task and the scheduling priority of each task, resulting in a long defragmentation time. It may have reached the time t to stop the defragmentation thread (that is, the defragmentation has timed out), and the object defragmentation task has not been completed. In this case, the current defragmentation task will be forced to stop, and a new round of defragmentation tasks will be restarted in the next cycle. If the defragmentation has not timed out, the defragmentation will continue until it is completed.

[0041] However, in the above-mentioned related technologies, when the system is initialized, all the defragmentation parameters are set to a small fixed value at one time. No matter how the front-end business pressure changes, no matter how many object fragments in the data pool enter the task queue to be defragmented, only one defragmentation thread is generated according to one management process, and only one fragment object in a bucket is processed per unit time. Although it is beneficial to not affect the response speed of business writes when the business pressure is high, it is impossible to complete the object defragmentation task quickly during idle time, and the speed of object fragment processing cannot be adjusted in real time to release the available space that is invalidly occupied by object fragments more quickly. If the system does not complete the object fragment defragmentation task as soon as possible during the idle period, and the traffic pressure sent by the front-end server increases later, it will further affect the speed of the object defragmentation task, and even encounter sudden pressure on the front end that causes the cache pool performance to be unable to meet, and then when a large number of small files are written through, the data pool cannot provide enough space and timely response speed to cope with temporary sudden write requirements. For example, assuming that the total available capacity of the data pool is 10T, theoretically, 10*1024*1024M / 16M=655360 16M large objects can be stored. According to the solution in the related technology, the fixed single-thread, single-object fragment processing method fails to process some object fragments in time. The space occupied by the fragment objects is 1000 6M, 500 16M, and 200 9M. The total number of fragments is 1700, accounting for 0.25% of the total number of objects 655360, resulting in invalid occupation of 15.43G (1000x6+500x16+200x9=15800M) of space, accounting for 0.15% of the total available space. At a certain moment, the front-end server suddenly has a burst write demand for a large number of small objects, each of which is 16k in size. Because the object fragmentation task cannot be completed in time, 15800*1024k / 16k=1011200 small objects cannot be written in time.

[0042] It can be seen that the relevant technology may in some cases be unable to meet the space requirements and / or performance requirements required by the front-end server for sudden tasks. Based on this, an embodiment of the present invention provides an object defragmentation method, device, equipment and storage medium, which can solve the above technical problems.

[0043] Combine the following Figure 5-Figure 9 An object defragmentation method according to an embodiment of the present invention is described.

[0044] It should be noted that the execution subject of the embodiment of the present invention may be an object defragmentation device, or may also be a cloud storage device or a cloud storage system. The following embodiments will be described by taking a cloud storage device as the execution subject.

[0045] Figure 5FIG. 1 is one of the flow charts of the object defragmentation method provided by the present invention, such as Figure 5 As shown, the method comprises the following steps: S102, obtaining the number of updates of data in a preset address space in the buffer pool in the previous unit time, and determining the space replacement frequency of the buffer pool in the previous unit time according to the number of updates in the previous unit time.

[0046] Among them, a cache pool and a data pool can be pre-set in the cloud storage device to store data or data objects. The write performance of the cache pool is better than the write performance of the data pool, but the memory space of the cache pool is smaller than the memory space of the data pool, and can be used to temporarily store data sent by the front-end server, while the data pool has a lower cost and a larger memory space, so it can store more data. In this embodiment, the cache pool is mainly used to store data of small objects, that is, data of objects that occupy a small memory space, and the data pool is mainly used to store data of large objects, that is, data of objects that occupy a large memory space. When the memory space of the cache pool is insufficient, the data of small objects can also be appropriately stored.

[0047] Since object fragments may exist in the data pool during the process of storing object data due to various reasons, it is necessary to start the object defragmentation task to defragment the fragments and release some memory space. However, the method of starting the object defragmentation task at a set time in the related technology may not be able to meet the performance and space requirements of the front-end server when sudden tasks occur in some cases. Therefore, this embodiment proposes an improved method for executing the object defragmentation task.

[0048] Specifically, in this step, the cloud storage device can first determine the previous unit time, and at the same time determine the preset address space of the cache pool, and then count the number of data updates in the preset address space of the cache pool in the previous unit time. The number of data updates here refers to the sum of the number of times from idle to data and the number of times from data to idle in the preset address space in the previous unit time. The previous unit time can be a unit time before the current moment, and the length of the previous unit time can be set according to actual conditions, such as 30 minutes, 1 hour, 24 hours, 2 days, etc. The preset address space of the cache pool refers to the preset memory space of the cache pool, which can be the entire address space corresponding to the cache pool address code, or a segment of address space selected from the entire address space, or a combination of multiple segments of address space selected from the entire address space, or other situations, which are not specifically limited here. In short, the number of data updates in the preset address space in the previous unit time can be counted.

[0049] After obtaining the number of data updates of the preset address space of the cache pool in the previous unit time, the number of updates can be directly used as the space replacement frequency of the cache pool in the previous unit time; or the space replacement frequency of the cache pool in the previous unit time can be determined by averaging or summing the number of data updates of the preset address space of the cache pool in multiple previous unit times (for example, counting once a day or 2 days before the current moment); or other methods can be used. In short, the space replacement frequency of the cache pool in the previous unit time can be counted, and the space replacement frequency can be denoted as P.

[0050] S104, obtaining a first amount of data that is written to the data pool by the front-end server in the previous unit time and a second amount of data that is written in total by the front-end server in the previous unit time, and determining a write-through ratio of data to the data pool in the previous unit time based on the first amount of data and the second amount of data.

[0051] In this step, the cloud storage device can count the amount of data that the front-end server has written to the data pool in the previous unit time that is the same as the cache pool, and record it as the first data amount. Insufficient memory space in the cache pool or the data generated by the front-end server is large object data may cause the front-end server to write data directly to the data pool when writing data, rather than writing it to the cache pool, that is, writing it to the data pool.

[0052] At the same time, the cloud storage device can count the total amount of data written to the data pool and the cache pool by the front-end server in the same previous unit time, and record it as the second data amount.

[0053] Afterwards, the first data volume can be directly compared with the second data volume, and the obtained ratio can be directly used as the write-through ratio of the data pool in the previous unit time. Alternatively, the first data volume and the second data volume of multiple previous unit time can be counted, and the final first data volume and the second data volume can be obtained after average processing, and then the ratio calculation can be performed to obtain the write-through ratio of the data pool in the previous unit time. Alternatively, other methods can be used for calculation, which are not specifically limited here, as long as the write-through ratio of the data pool in the previous unit time can be calculated, and the write-through ratio can be recorded as Q.

[0054] S106, determining the current target value corresponding to each parameter of the object defragmentation task according to the space replacement frequency of the cache pool and the write-through ratio of the data pool; the above-mentioned object defragmentation task is a task for defragmenting fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server.

[0055] Among them, during the operation of cloud storage devices, system resources (including CPU resources and memory resources, cache pool space, data pool space, etc.) need to respond in real time to various burst read and write tasks of many front-end servers at any time. These tasks occupy system resources including multiple subtasks such as calculation, scheduling, and waiting. Calculating the capacity required when reading or writing data sent by the front-end server to the memory, cache pool, data pool and other capacity spaces, calculating the space replacement frequency P of the cache pool, and calculating the write-through ratio Q are all computing subtasks; there are usually dozens or hundreds of different task processes (such as various write and read, various scheduled tasks, various exception handling including disk error handling, etc.) running at the same time in cloud storage devices. The priority of a process to obtain resources at different times is constantly changing. The CPU needs to apply for and allocate system resources in a timely manner according to system requirements. Scheduling tasks such as scheduling; after a task process obtains system resources, it completes the writing or reading of data, waits for the data to be written or read, and returns the completion result to the front-end server. During this period, it is all waiting time. Based on this, this embodiment proposes to adjust the values ​​of parameters such as the start conditions, defragmentation speed and defragmentation time of the object defragmentation task in time according to different business pressures through the space replacement frequency of the cache pool in the previous unit time and the write-through ratio of the data pool in the previous unit time, so as to complete the object defragmentation task in time and respond to the space and performance requirements of the burst write tasks of the front-end server in time.

[0056] Specifically, in this step, the object defragmentation task may correspond to a task queue, which may include object fragments to be defragmented in the data pool, and the data pool generally stores object data in a fixed-size space, such as storing an object data in a 16M space, so the memory space occupied by each object fragment in the task queue is generally less than 16M. In the related art, when executing the object defragmentation task, the expiration time and execution time of the object defragmentation task are generally set to fixed values ​​during initialization, which makes the data pool unable to flexibly respond to the performance and space requirements required by the burst write tasks of the front-end server.

[0057] Based on this, after obtaining the space replacement frequency of the cache pool in the previous unit time and the write-through ratio of the data pool in the previous unit time, this embodiment can dynamically adjust the values ​​of parameters related to the object defragmentation task through the space replacement frequency of the cache pool and the write-through ratio of the data pool to obtain the target value corresponding to each parameter, that is, adjust the value of each parameter from its historical value to the target value. Among them, the space replacement frequency of the cache pool in the previous unit time and the write-through ratio of the data pool in the previous unit time can comprehensively reflect the current write business situation of the front-end server. For example, if the space replacement frequency of the cache pool is low and the write-through ratio of the data pool is also low, it means that the current write business of the front-end server is small and the pressure is not great. The write pressure of the data pool and the cache pool are both low, that is, they are relatively idle. At this time, the data pool can complete the object defragmentation task as soon as possible, such as increasing the value of the parameter of the expiration time of the object defragmentation task to the target value; for another example, if the space replacement frequency of the cache pool and the write-through ratio of the data pool are gradually increasing, it means that the current write business of the front-end server is gradually increasing, and the pressure is also gradually increasing. There may be a sudden write business situation. The write pressure of the data pool and the cache pool is high. Therefore, the speed or time for the data pool to complete the object defragmentation task can be appropriately reduced, such as lowering the value of the parameter of the expiration time of the object defragmentation task to the target value, so that the memory space and performance of the data pool and the cache pool can meet the write business of the front-end server.

[0058] It can be understood that the above adjustment of the values ​​of the parameters related to the object defragmentation task reaches the target value, which can be understood as a target value that matches or adapts to the current write business pressure of the front-end server, and the write business pressure of the front-end server corresponds to the write pressure of both the data pool and the cache pool. Generally, the greater the write business pressure of the front-end server, the greater the write pressure of the data pool and the cache pool. Therefore, the target value here can also be understood as a target value that matches / adapts to the current data write pressure of the data pool and the cache pool, that is, the memory space and performance of the data pool and the cache pool can meet the current write business pressure of the front-end server. In addition, the adjustment of the value of the parameter of the object defragmentation task can be to adjust the value of one parameter, or to adjust the values ​​of multiple parameters or the values ​​of all parameters.

[0059] S108, executing the object defragmentation task according to the target value currently corresponding to each parameter.

[0060] In this step, after determining the target value of each parameter of the object defragmentation task, the object defragmentation can be performed according to the target value in the next defragmentation cycle so that the memory space and performance of the data pool and the cache pool can meet the current write business requirements of the front-end server.

[0061] In this embodiment, the number of updates of data in a preset address space in the cache pool in the previous unit time is obtained, and the space replacement frequency of the cache pool in the previous unit time is determined according to the number of updates in the previous unit time, and at the same time, the first data amount written to the data pool by the front-end server in the previous unit time and the second data amount written in the previous unit time are obtained, and the write-through ratio of data to the data pool in the previous unit time is determined according to the first data amount and the second data amount, and then the current target value corresponding to each parameter of the object defragmentation task is determined according to the space replacement frequency of the cache pool and the write-through ratio of the data pool, and then the object defragmentation task is executed according to the current target value of each parameter; wherein, the object defragmentation task is a task for defragmenting fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server. In this method, by calculating the replacement frequency of the cache pool in the previous unit time and the write-through ratio of the data pool in the previous unit time, the current write business pressure of the data pool and the cache pool, which are the cloud storage device, and the current write business pressure of the front-end server can be intuitively reflected, thereby calculating the parameter value of the object defragmentation task that matches the current write business pressure of the cloud storage device / front-end server. If the front-end server has a sudden task when executing the object defragmentation task according to this parameter value, the cloud storage device can adaptively meet the space requirements and / or performance requirements required by the front-end server.

[0062] The following embodiment describes a specific process of dynamically adjusting the values ​​of parameters related to the object defragmentation task through the space replacement frequency of the buffer pool and the write-through ratio of the data pool.

[0063] Figure 6 FIG. 2 is a flow chart of the object defragmentation method provided by the present invention. Figure 6 As shown, the above step 106 may include the following steps: Step 202, determine whether the space replacement frequency of the buffer pool is less than a preset frequency threshold, and obtain a first determination result; and determine whether the write-through ratio of the data pool exceeds a preset ratio threshold range, and obtain a second determination result.

[0064] Among them, the preset frequency threshold corresponds to the threshold of the spatial replacement frequency, and its size can be set according to actual conditions, for example, it can be set to 2, 3, 4, etc. The preset ratio threshold range corresponds to the threshold range of the write-through ratio of the data pool. The preset frequency threshold can include only one threshold, or a threshold range composed of multiple thresholds, such as an upper limit value and a lower limit value. The preset ratio threshold range can be a threshold range composed of a lower limit value and an upper limit value. The size of the preset ratio threshold range can also be set according to actual conditions, for example, it can be a threshold range [0~10%] composed of a lower limit value of 0 and an upper limit value of 10%. The specific upper limit value and lower limit value can be set according to actual business.

[0065] After obtaining the write-through ratio Q of the data pool in the previous unit time, it can be determined whether Q exceeds the preset ratio threshold range to obtain a first judgment result. At the same time, after obtaining the space replacement frequency P of the cache pool in the previous unit time, it can be determined whether P is less than the preset frequency threshold to obtain a second judgment result.

[0066] It should be noted that the preset frequency threshold and the preset ratio threshold range are both related to the data writing pressure of the cache pool and the data pool.

[0067] Step 204: Determine the target value currently corresponding to each parameter of the object defragmentation task according to the first judgment result and the second judgment result.

[0068] In this step, relevant parameters can be set in advance for the object defragmentation task. These relevant parameters may include, for example: the expiration time of the object defragmentation task, the threshold for fragmented objects to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, the running time of each thread, the first cache pool usage threshold, the second cache pool usage threshold, etc.

[0069] Among them, the expiration time of the object defragmentation task refers to the cycle of executing the object defragmentation task; the threshold for the fragmented object to enter the defragmentation task queue refers to the space utilization threshold of the object fragment. For example, if the space utilization of an object fragment is lower than the space utilization threshold, the object fragment can enter the object defragmentation task queue and wait for defragmentation; the number of fragments processed in parallel by a single thread refers to the number of fragmented objects merged by a single thread each time, for example, merging 5 object fragments into a large object each time; the running time of each thread refers to the length of time a single thread executes the object defragmentation task.

[0070] The first buffer pool usage threshold refers to the buffer pool usage threshold that triggers the data of the front-end server to be written through to the data pool; the second buffer pool usage threshold refers to the buffer pool usage threshold that stops the data of the front-end server from being written through to the data pool; the buffer pool usage refers to the ratio of the memory occupied by the data currently stored in the buffer pool to its total memory. When the write pressure of the front-end server is large, and the speed of releasing space after the buffer pool merges large objects successfully cannot keep up with the write speed of the front-end server, the buffer pool usage will continue to increase. When the buffer pool usage is greater than the first buffer pool usage threshold K, the buffer pool usage may continue to increase until it is completely full, causing some or all of the objects of the front-end server to be written through to the data pool. Therefore, when write-through occurs, the write-through ratio Q value increases from 0 to 100%. The larger the Q value, the greater the pressure on the front-end business. When the business pressure of the front-end server is reduced and the buffer pool usage is reduced to less than the second buffer pool usage threshold k, the data will be stopped from being written through to the data pool, reducing the data writing pressure of the data pool, so as to increase the priority of the object fragment processing task and complete the object fragment processing as soon as possible.

[0071] In addition, for the expiration time of the above-mentioned object defragmentation task and the threshold for the fragmented object to enter the defragmentation task queue, these two parameters are parameters related to the starting conditions of the object defragmentation task. The number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of each thread are parameters related to the defragmentation speed and defragmentation time of the object defragmentation task. The first cache pool utilization threshold and the second cache pool utilization threshold are parameters related to the space usage of the cache pool.

[0072] After obtaining the space replacement frequency judgment result of the cache pool and the write-through ratio judgment result of the data pool, one or more parameters of the relevant parameters of the above-mentioned object defragmentation task can be adjusted. The adjustment method can be to increase or decrease the historical value of the parameter (that is, the parameter value used in the previous execution cycle of the object defragmentation task). After increasing or decreasing the historical value of the parameter, a target value that matches the current write business pressure situation can be obtained.

[0073] In this embodiment, the space replacement frequency of the cache pool is compared with the corresponding threshold, and the write-through ratio of the data pool is compared with the corresponding threshold range. The parameter value that matches the object defragmentation task with the current write business pressure of the front end is comprehensively determined through these two judgment results. This quantitative method can refine the process of determining the parameter value of the object defragmentation task, so that the object defragmentation task can be executed under the optimal situation and the write business needs of the front end can be met.

[0074] The following embodiments describe several possible judgment results after judging the size of the space replacement frequency of the buffer pool and the corresponding threshold value and judging the size of the write-through ratio of the data pool and the corresponding threshold range.

[0075] In some embodiments, see Figure 7 The flowchart of dynamically adjusting the parameters of the object defragmentation task shown in the figure, the above step 204, according to the first judgment result and the second judgment result, determines the target value currently corresponding to each parameter of the object defragmentation task, which may include the following implementation methods: In order to facilitate a better understanding of the following implementation methods, here we first explain how the write-through ratio of the data pool and the space replacement frequency of the cache pool change as the amount of data written increases. When data is first written, the data is generally written to the cache pool first and will not be written through to the data pool, so the write-through ratio of the data pool is relatively low when data is first written, for example, equal to the lower limit value, and the space replacement frequency of the cache pool will gradually increase due to the continuous writing of data, but it will be less than the preset frequency threshold at the beginning. This scenario corresponds to the following implementation method one.

[0076] As the amount of data written continues to increase, if the capacity of the cache pool is still sufficient, the data will continue to be written to the cache pool. At this time, the write-through ratio of the data pool is still relatively low, for example, equal to the lower limit value. As the amount of data written to the cache pool continues to increase, the space replacement frequency will continue to increase, for example, it will increase to a frequency greater than the preset frequency threshold. This scenario corresponds to the following implementation method 2.

[0077] As the amount of written data continues to increase, the capacity of the cache pool may be insufficient. In this case, the written data will gradually be written through to the data pool, so the write-through ratio of the data pool will gradually increase, for example, it will be greater than the lower limit but less than the upper limit. At this time, since the amount of written data is still increasing, the space replacement frequency of the cache pool will continue to increase or remain unchanged, for example, it will still be greater than the preset frequency threshold. This scenario corresponds to the following implementation method three.

[0078] As the amount of write data continues to increase, more data will be written through to the data pool until the data pool is full. At this time, the write-through ratio of the data pool will continue to increase, for example, it will increase to a value greater than the upper limit. At this time, the data in the cache pool may be full, and the data in it will no longer be updated. Then its space replacement frequency will decrease, that is, the space replacement frequency of the cache pool may be less than the preset frequency threshold. This scenario corresponds to the following implementation mode 4.

[0079] The above are several scenarios that may exist in the process of continuously writing data into the data pool and the cache pool. The following describes the implementation methods of specifically adjusting the parameter values ​​of the object defragmentation task in these scenarios.

[0080] In the first implementation mode, the preset ratio threshold range includes a lower limit value. If the first judgment result is that the write-through ratio of the data pool is equal to the lower limit value, and the second judgment result is that the space replacement frequency of the cache pool is less than the preset frequency threshold value, then the historical value corresponding to each parameter of the object defragmentation task is obtained; the above parameters include at least one of the expiration time of the object defragmentation task, the threshold for the fragmented object to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of the thread; Perform at least one of the following adjustment operations on the historical values ​​of the parameters of the object defragmentation task: reduce the historical value corresponding to the expiration time of the object defragmentation task to the target value, increase the historical value corresponding to the threshold of the fragmented object entering the defragmentation task queue to the target value, increase the historical value corresponding to the number of threads of the object defragmentation task to the target value, increase the historical value corresponding to the number of fragments processed in parallel by a single thread to the target value, and increase the historical value corresponding to the running time of the thread to the target value.

[0081] Among them, the lower limit value of the preset ratio threshold range is assumed to be 0, the preset frequency threshold is assumed to be 2, the historical value of the expiration time of the object defragmentation task is assumed to be T, the historical value of the threshold for the fragmented object to enter the defragmentation task queue is assumed to be D (the default value of D is 50%), the historical value of the number of threads of the object defragmentation task is assumed to be m, the historical value of the number of fragments processed in parallel by a single thread is assumed to be n, the historical value of the running time of each thread is assumed to be t, the first cache pool usage threshold is recorded as K, and the second cache pool usage threshold is recorded as k.

[0082] Specifically, if the write-through ratio Q=0 and the space replacement frequency P is less than 2, it means that the data pool and the cache pool are relatively idle and the write business pressure of the front-end server is relatively small. Then, the following operations can be performed on the historical value of at least one parameter in the object defragmentation task to release the invalid space occupied by the object fragments as soon as possible during idle time: Reduce the expiration time of the object defragmentation task, specifically, reduce it from the historical value T to the target value T1 (T1 < T), so that the number of object fragmentation processing times can be increased within a certain period of time; Increase the threshold for triggering fragmented objects to enter the object defragmentation task queue. Specifically, it can be increased from the historical value D to the target value D1 (D1>D, such as D1=80%). This can lower the threshold for object fragments to enter the defragmentation task queue, allowing more object fragments to enter the task queue for defragmentation. Increase the number of defragmentation threads, specifically from the historical value m to the target value m1 (m1>m); Increase the number of object fragments processed in parallel by a single defragmentation thread, specifically from the historical value n to the target value n1 (n1>n), which can increase the speed of defragmentation; Increase the running time of a single defragmentation thread, specifically from the historical value t to the target value t1 (t1>t), which can extend the time of each fragment processing and release the space occupied by fragments faster; In addition, for the first buffer pool usage rate threshold and the second buffer pool usage rate threshold in the object defragmentation task related parameters, the target values ​​of these two parameters may remain unchanged at their default values.

[0083] Through the operation of the first implementation, the speed of object fragment processing can be adjusted in real time when the data pool and the cache pool are relatively idle, so that all object fragments in the data pool can be processed as soon as possible, so as to release the available space invalidly occupied by the object fragments more quickly.

[0084] In the second implementation mode, the preset ratio threshold range includes a lower limit value. If the first judgment result is that the write-through ratio of the data pool is equal to the lower limit value, and the second judgment result is that the space replacement frequency of the cache pool is not less than the preset frequency threshold value, then the historical value corresponding to each parameter of the object defragmentation task is obtained; the above parameters include at least one of the first cache pool usage threshold value for triggering data write-through to the data pool and the second cache pool usage threshold value for stopping data write-through to the data pool; At least one of the following adjustment operations is performed on the historical value of the parameter of the object defragmentation task: reducing the historical value of the first cache pool usage threshold to the target value, and increasing the historical value of the second cache pool usage threshold to the target value.

[0085] If the write-through ratio Q=0 and the space replacement frequency P is greater than or equal to 2, it means that the write business pressure of the front-end server is gradually increasing. Correspondingly, the business pressure of the cache pool is relatively large, and the pressure of the data pool temporarily meets the demand or is temporarily idle. In this case, the following operations can be performed on the historical value of at least one parameter in the object defragmentation task to appropriately reduce the data write pressure of the cache pool: Reduce the first buffer pool usage threshold K (the buffer pool usage threshold that triggers data write-through to the data pool). Specifically, you can reduce it from the historical value to the target value. For example, reduce K from 90% to 80%. This will allow more data from the front-end server to be written to the data pool, thereby appropriately alleviating the data writing pressure on the buffer pool. Increase the second cache pool usage threshold k (the cache pool usage threshold for stopping data write-through to the data pool). Specifically, you can increase it from the historical value to the target value. For example, increase k from 70% to 75%. This can relax the cache pool usage and allow more data to be written to the data pool, thereby appropriately alleviating the data write pressure on the cache pool.

[0086] In addition, for the expiration time of the object defragmentation task, the threshold for the fragmented object to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of each thread, the target values ​​of these parameters can all be set to the target values ​​after an increase or decrease in one implementation mode, that is, T1, D1, m1, n1, and t1 are set to remain unchanged. In this way, when the write business pressure of the front-end server gradually increases, it can be ensured that the existing object fragments are defragmented as soon as possible.

[0087] Through the operation of the first implementation mode described above, when the write service pressure of the front-end server gradually increases, it can be ensured that the existing object fragments are sorted out as soon as possible, and at the same time, the data writing pressure of the cache pool is appropriately reduced.

[0088] Implementation method three, the above-mentioned preset ratio threshold range includes a lower limit value and an upper limit value, the upper limit value is greater than the lower limit value, if the first judgment result is that the write-through ratio of the data pool is greater than the lower limit value and less than the upper limit value, and the second judgment result is that the space replacement frequency of the cache pool is not less than the preset frequency threshold, then the historical value corresponding to each parameter of the object defragmentation task is obtained; the above-mentioned parameters include at least one of the expiration time of the object defragmentation task and the threshold value for the fragmented object to enter the defragmentation task queue; Perform at least one of the following adjustment operations on the historical value of the parameter of the object defragmentation task: increase the historical value corresponding to the expiration time of the object defragmentation task to the target value, and reduce the historical value corresponding to the threshold of the fragmented object entering the defragmentation task queue to the target value.

[0089] The upper limit of the preset ratio threshold range is assumed to be 10%. If 0 < write-through ratio Q ≤ 10%, and the space replacement frequency P is greater than or equal to 2, it means that the business pressure of the front-end server continues to increase, the pressure of the cache pool is very large, and the data pool begins to directly bear the business write pressure of a small number of front-end servers. Then, the following operations can be performed on the historical value of at least one parameter in the object defragmentation task so that the data pool can meet the write business needs of the front-end server in a timely manner: Increase the expiration time of the object defragmentation task, specifically, increase it from the historical value T1 to the target value T2, where T1<T2<T, so that the number of object fragmentation processing can be appropriately reduced within a certain period of time; Reduce the threshold for fragmented objects to enter the defragmentation task queue. Specifically, it can be reduced from the historical value D1 to the target value D2, where D<D2<D1, for example, D2=60%. This can appropriately increase the threshold for object fragments to enter the defragmentation task queue, avoiding too many object fragments from entering the task queue for defragmentation, that is, reducing defragmentation; In addition, for the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of each thread, the target values ​​of these parameters can be set to the target values ​​increased or decreased in the first implementation mode, that is, m1, n1, and t1 are set to remain unchanged. At the same time, the target values ​​of the first cache pool usage threshold and the second cache pool usage threshold, these two parameters can be set to the target values ​​reduced or increased in the second implementation mode, that is, K remains unchanged at 80%, and k remains unchanged at 75%.

[0090] Through the operation of the third implementation mode described above, it is possible to facilitate the data pool to meet the business writing of the front-end server in a timely manner and extend the retention time of some object fragments.

[0091] Implementation method four, the above-mentioned preset ratio threshold range includes an upper limit value, if the first judgment result is that the write-through ratio of the data pool is greater than the upper limit value, and the second judgment result is that the space replacement frequency of the cache pool is less than the preset frequency threshold, then the historical value corresponding to each parameter of the object defragmentation task is obtained; the above-mentioned parameters include at least one of the expiration time of the object defragmentation task, the threshold for the fragmented object to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of the thread; Perform at least one of the following adjustment operations on the historical values ​​of the parameters of the object defragmentation task: increase the historical value corresponding to the expiration time of the object defragmentation task to the target value, reduce the historical value corresponding to the threshold of the fragmented object entering the defragmentation task queue to the target value, reduce the historical value corresponding to the number of threads of the object defragmentation task to the target value, reduce the historical value corresponding to the number of fragments processed in parallel by a single thread to the target value, and reduce the historical value corresponding to the running time of the thread to the target value.

[0092] Among them, if the write-through ratio Q is greater than 10%, and the space replacement frequency P is less than 2, it means that the write business pressure of the front-end server is very high. Correspondingly, the write pressure of both the cache pool and the data pool is very high. The small objects in the cache pool have no time to be merged into large objects, and then the space replacement frequency P will decrease. The data pool may not be able to respond to the merge write requirements of the cache pool and the write business pressure of the front-end server in time. In order to facilitate the data pool to fully respond to the business write or other read-write requirements of the front-end server, the priority of the object defragmentation task can be adjusted to the lowest. Specifically, the following operations can be performed on the historical value of at least one parameter in the object defragmentation task. Increase the expiration time of the object defragmentation task, specifically from the historical value T2 to the target value T, which can reduce the number of object fragmentation processing times within a certain period of time; Reduce the threshold for triggering fragmented objects to enter the object defragmentation task queue. Specifically, it can be reduced from the historical value D2 to the target value D. This can increase the threshold for object fragments to enter the defragmentation task queue, so that more object fragments can enter the task queue for defragmentation. Reduce the number of defragmentation threads, specifically from the historical value m1 to the target value m; Reduce the number of object fragments processed in parallel by a single defragmentation thread, specifically from the historical value n1 to the target value n, which can reduce the speed of defragmentation; Reduce the running time of a single defragmentation thread, specifically from the historical value t1 to the target value t, which can shorten the time of each fragmentation processing; In addition, for the first buffer pool usage rate threshold and the second buffer pool usage rate threshold in the object defragmentation task related parameters, the target values ​​of these two parameters can be adjusted to their corresponding default values.

[0093] Through the operation of the fourth implementation method above, when the write business pressure of the front-end server is large, the priority of the object defragmentation task can be adjusted to the lowest, ensuring that the data pool fully responds to the write business pressure of the front-end server or other read-write requirements (such as the merge write requirements of the cache pool), thereby improving the space utilization of the cloud storage device.

[0094] In this embodiment, when the cache pool space replacement frequency and the data pool write-through ratio are at different values, the corresponding write business pressure conditions of the front end, data pool and cache pool are determined in the corresponding situation. In this way, the parameter value of the object defragmentation task can be adjusted in time and dynamically, and the space invalidly occupied by object fragments can be released in time to meet the writing needs of new business data and improve the space utilization of cloud storage devices.

[0095] In the process of actually counting the space replacement frequency P of the cache pool, it is found that since the space of the cache pool is written by the front-end server with data of small objects, when its total capacity reaches a certain write ratio R, it will trigger the merger of small objects into large objects, and when the small objects written in each Bucket storage bucket of the cache pool reach a certain number S, it will also trigger the merger of large objects. After the large object is successfully merged, the small object will be deleted, and the released space will be used for front-end writing again. However, each address segment of the cache pool is not used by the same Bucket every time. The timing of merging small objects in each Bucket is different, and the business pressure of the front end will also be different in different periods, so the replacement frequency P of each address segment is different in different time periods. If the replacement frequency of all address spaces in the cache pool is counted, the amount of data that the system needs to calculate will be too large, which will not only cause the calculation time to be too long, affecting the calculation of other tasks, but also occupy too many CPU resources, affecting the application and scheduling of system resources by other tasks. Based on this, this embodiment proposes a solution for obtaining a part of the address space in the cache pool to count the space replacement frequency, and the following embodiment will illustrate this process.

[0096] Figure 8 FIG. 3 is a flow chart of the object defragmentation method provided by the present invention. Fig. 9 As shown, the "obtaining the number of updates of the data in the preset address space in the buffer pool in the previous unit time" in the above step 102 may include the following steps: Step 302, obtaining the size of the target address space corresponding to the extraction in the buffer pool in the previous unit time.

[0097] The target address space size extracted from the cache pool is smaller than the total address space size of the cache pool. As an optional embodiment, the method for determining the target address space size may be to determine the target address space size corresponding to the previous unit time according to the first space replacement frequency corresponding to the cache pool in the historical time; wherein the historical time is the historical time before the previous unit time, and the first space replacement frequency is proportional to the target address space size.

[0098] In other words, the greater the space replacement frequency of the cache pool in the historical time, the greater the write business pressure of the cache pool, and the larger the target address space size that needs to be extracted at present, so as to ensure the accurate statistics of the current space replacement frequency of the cache pool. It can be understood that even if the size of the extracted target address space is increased, the memory occupied by the process of calculating the space replacement frequency is very small, and it will not have a significant impact on the performance of the cloud storage device.

[0099] Step 304, in the first address space, the second address space and the third address space of the cache pool, select sub-address spaces of the size of the target address space in turn; the first address space, the second address space and the third address space are address spaces corresponding to the front, middle and back sections of the address code of the cache pool respectively.

[0100] In this step, the extracted target address space size obtained above can be the size of the entire extracted address space. Specifically, one or more segments of address space can be extracted from the cache pool to constitute the above target address space size. Taking the even extraction of three segments of address space as an example, a sub-address space corresponding to the front segment, the middle segment and the back segment of the address code of the cache pool can be extracted respectively, and the target address space is constituted by these three segments of address space.

[0101] For example, see Fig. 9 As shown in the schematic diagram of cache pool address coding, assuming that the cache pool capacity is 1T, assuming that the size of each sub-address space extracted above can be 3G, 10G, 50G, etc., taking 3G as an example, the address spaces corresponding to the front 3G, middle 3G, and rear 3G of the cache pool address coding are selected (the size of x can be determined according to the address segmentation situation). In total, only 9G of space needs to be counted for space replacement frequency, which is much faster than counting the space replacement frequency of 1T capacity and occupies much less resources. Therefore, it can ensure that the efficiency of counting the replacement frequency is improved on the basis of the accuracy of the space replacement frequency.

[0102] Step 306: determine the three sub-address spaces as the preset address spaces, and count the number of times the data in the preset address spaces in the cache pool are updated in the previous unit time.

[0103] In this step, after obtaining the three sub-address spaces extracted from the cache pool, the three sub-address spaces can be used as the preset address spaces extracted from the cache pool, and then the update times of the three sub-address spaces in the previous unit time can be counted respectively, and the sum of the three update times is used as the update times of the preset address space in the previous unit time. Then, the space replacement frequency of the preset address space in the previous unit time can be determined based on the update times of the preset address space of the cache pool in the previous unit time.

[0104] In this embodiment, a section of address space is selected from the front section, middle section and back section of the cache pool address space to count the number of updates and then count the space replacement frequency. In this way, fewer address spaces are selected for statistics. Therefore, the efficiency of counting the replacement frequency can be improved while ensuring the accuracy of the counted space replacement frequency.

[0105] The object defragmentation device provided by the present invention is described below. The object defragmentation device described below and the object defragmentation method described above can be referred to each other.

[0106] Fig.10 is a schematic diagram of the structure of the object fragmentation device provided by the present invention, see Fig.10 As shown, the device may include: The space replacement frequency determination module 410 is used to obtain the number of updates of data in a preset address space in the buffer pool in the previous unit time, and determine the space replacement frequency of the buffer pool in the previous unit time according to the number of updates in the previous unit time; The write-through ratio determination module 420 is used to obtain a first amount of data written to the data pool by the front-end server in the previous unit time and a second amount of data written in total by the front-end server in the previous unit time, and determine a write-through ratio of data written to the data pool in the previous unit time according to the first amount of data and the second amount of data; The defragmentation task parameter determination module 430 is used to determine the current target value corresponding to each parameter of the object defragmentation task according to the space replacement frequency of the cache pool and the write-through ratio of the data pool; the above-mentioned object defragmentation task is a task for arranging fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server; The object defragmentation module 440 is used to execute the object defragmentation task according to the target value currently corresponding to each parameter.

[0107] In some embodiments, the defragmentation task parameter determination module 430 includes: A judgment unit, used to judge whether the write-through ratio of the data pool exceeds a preset ratio threshold range to obtain a first judgment result; and to judge whether the space replacement frequency of the cache pool is less than a preset frequency threshold to obtain a second judgment result; The object defragmentation task parameter determination unit is used to determine the target value currently corresponding to each parameter of the object defragmentation task according to the first judgment result and the second judgment result.

[0108] In some embodiments, the preset ratio threshold range includes a lower limit value, and the object defragmentation task parameter determination unit is specifically used to obtain the historical value corresponding to each parameter of the object defragmentation task if the first judgment result is that the write-through ratio of the data pool is equal to the lower limit value, and the second judgment result is that the space replacement frequency of the cache pool is less than the preset frequency threshold; the above parameters include at least one of the expiration time of the object defragmentation task, the threshold for the fragmentation object to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of the thread; perform at least one of the following adjustment operations on the historical value of the parameter of the object defragmentation task: reduce the historical value corresponding to the expiration time of the object defragmentation task to the target value, increase the historical value corresponding to the threshold for the fragmentation object to enter the defragmentation task queue to the target value, increase the historical value corresponding to the number of threads of the object defragmentation task to the target value, increase the historical value corresponding to the number of fragments processed in parallel by a single thread to the target value, and increase the historical value corresponding to the running time of the thread to the target value.

[0109] In some embodiments, the preset ratio threshold range includes a lower limit value, and the object defragmentation task parameter determination unit is specifically used to obtain the historical value corresponding to each parameter of the object defragmentation task if the first judgment result is that the write-through ratio of the data pool is equal to the lower limit value, and the second judgment result is that the space replacement frequency of the cache pool is not less than the preset frequency threshold; the above-mentioned parameters include at least one of the first cache pool usage threshold that triggers data to be written through to the data pool and the second cache pool usage threshold that stops data from being written through to the data pool; perform at least one of the following adjustment operations on the historical values ​​of the parameters of the object defragmentation task: reduce the historical value of the first cache pool usage threshold to the target value, and increase the historical value of the second cache pool usage threshold to the target value.

[0110] In some embodiments, the above-mentioned preset ratio threshold range includes a lower limit value and an upper limit value, the upper limit value is greater than the lower limit value, and the above-mentioned object defragmentation task parameter determination unit is specifically used to obtain the historical value corresponding to each parameter of the object defragmentation task if the first judgment result is that the write-through ratio of the data pool is greater than the lower limit value and less than the upper limit value, and the second judgment result is that the space replacement frequency of the cache pool is not less than the preset frequency threshold; the above-mentioned parameters include at least one of the expiration time of the object defragmentation task and the threshold for the fragmentation object to enter the defragmentation task queue; perform at least one of the following adjustment operations on the historical value of the parameter of the object defragmentation task: increase the historical value corresponding to the expiration time of the object defragmentation task to the target value, and reduce the historical value corresponding to the threshold for the fragmentation object to enter the defragmentation task queue to the target value.

[0111] In some embodiments, the preset ratio threshold range includes an upper limit value, and the object defragmentation task parameter determination unit is specifically used to obtain the historical value corresponding to each parameter of the object defragmentation task if the first judgment result is that the write-through ratio of the data pool is greater than the upper limit value, and the second judgment result is that the space replacement frequency of the cache pool is less than the preset frequency threshold; the above parameters include at least one of the expiration time of the object defragmentation task, the threshold for the fragmentation object to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of the thread; perform at least one of the following adjustment operations on the historical value of the parameter of the object defragmentation task: increase the historical value corresponding to the expiration time of the object defragmentation task to the target value, reduce the historical value corresponding to the threshold for the fragmentation object to enter the defragmentation task queue to the target value, reduce the historical value corresponding to the number of threads of the object defragmentation task to the target value, reduce the historical value corresponding to the number of fragments processed in parallel by a single thread to the target value, and reduce the historical value corresponding to the running time of the thread to the target value.

[0112] In some embodiments, the above-mentioned space replacement frequency determination module 410 is specifically used to obtain the target address space size corresponding to the extraction in the cache pool in the previous unit time; in the first address space, the second address space and the third address space of the cache pool, select the sub-address space of the target address space size in turn; the above-mentioned first address space, the second address space and the third address space are respectively an address space corresponding to the front section, the middle section and the back section of the address code of the cache pool; the three sub-address spaces are determined as preset address spaces, and the number of updates of the data in the preset address space in the cache pool in the previous unit time is counted.

[0113] It should be noted here that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0114] Fig.11 The following is a schematic diagram of the physical structure of a cloud storage device. Fig.11As shown, the cloud storage device may include: a processor 510 , a communications interface 520 , a memory 530 and a communication bus 540 , wherein the processor 510 , the communications interface 520 , and the memory 530 communicate with each other via the communication bus 540 . The processor 510 can call the logic instructions in the memory 530 to execute the object defragmentation method, which includes: obtaining the number of updates of data in a preset address space in the cache pool in the previous unit time, and determining the space replacement frequency of the cache pool in the previous unit time based on the number of updates in the previous unit time; obtaining the first data volume written to the data pool by the front-end server in the previous unit time and the second data volume written in total by the front-end server in the previous unit time, and determining the write-through ratio of data to the data pool in the previous unit time based on the first data volume and the second data volume; determining the current target value corresponding to each parameter of the object defragmentation task based on the space replacement frequency of the cache pool and the write-through ratio of the data pool; the above-mentioned object defragmentation task is a task for sorting fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server; and executing the object defragmentation task according to the current target value corresponding to each parameter.

[0115] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0116] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the object defragmentation method provided by the above methods, which includes: obtaining the number of updates of data in a preset address space in a cache pool in the previous unit time, and determining the space replacement frequency of the cache pool in the previous unit time based on the number of updates in the previous unit time; obtaining a first data amount written to a data pool by a front-end server in the previous unit time and a second data amount written in total by the front-end server in the previous unit time, and determining the write-through ratio of data to the data pool in the previous unit time based on the first data amount and the second data amount; determining the current target value corresponding to each parameter of the object defragmentation task based on the space replacement frequency of the cache pool and the write-through ratio of the data pool; the above object defragmentation task is a task for sorting fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server; and executing the object defragmentation task based on the current target value corresponding to each parameter.

[0117] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the object defragmentation method provided by the above-mentioned methods, the method comprising: obtaining the number of updates of data in a preset address space in a cache pool in the previous unit time, and determining the space replacement frequency of the cache pool in the previous unit time based on the number of updates in the previous unit time; obtaining a first amount of data written to a data pool by a front-end server in the previous unit time and a second amount of data written in total by the front-end server in the previous unit time, and determining the write-through ratio of data to the data pool in the previous unit time based on the first amount of data and the second amount of data; determining the current target value corresponding to each parameter of the object defragmentation task based on the space replacement frequency of the cache pool and the write-through ratio of the data pool; the above-mentioned object defragmentation task is a task for sorting fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server; and executing the object defragmentation task based on the current target value corresponding to each parameter.

[0118] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for object defragmentation, characterized in that: include: Obtaining the number of updates of data in a preset address space in the cache pool in a previous unit time, and determining the space replacement frequency of the cache pool in the previous unit time according to the number of updates in the previous unit time; Obtaining a first amount of data that is written through to the data pool by the front-end server in the previous unit time and a second amount of data that is written in total by the front-end server in the previous unit time, and determining a write-through ratio of data to the data pool in the previous unit time according to the first amount of data and the second amount of data; Determining a target value currently corresponding to each parameter of the object defragmentation task according to the space replacement frequency of the cache pool and the write-through ratio of the data pool; The object defragmentation task is a task for defragmenting the fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server; The object defragmentation task is performed according to the target value currently corresponding to each parameter.

2. The object defragmentation method according to claim 1, characterized in that: Determining the target value currently corresponding to each parameter of the object defragmentation task according to the space replacement frequency of the cache pool and the write-through ratio of the data pool includes: Determine whether the write-through ratio of the data pool exceeds a preset ratio threshold range to obtain a first determination result; and determine whether the space replacement frequency of the buffer pool is less than a preset frequency threshold to obtain a second determination result; According to the first judgment result and the second judgment result, a target value currently corresponding to each parameter of the object defragmentation task is determined.

3. The object defragmentation method according to claim 2, characterized in that: The preset ratio threshold range includes a lower limit value, and determining the target value currently corresponding to each parameter of the object defragmentation task according to the first judgment result and the second judgment result includes: If the first judgment result is that the write-through ratio of the data pool is equal to the lower limit value, and the second judgment result is that the space replacement frequency of the cache pool is less than the preset frequency threshold, then obtaining the historical value corresponding to each parameter of the object defragmentation task; the parameters include at least one of the expiration time of the object defragmentation task, the threshold for the fragmented object to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of the thread; Perform at least one of the following adjustment operations on the historical values ​​of the parameters of the object defragmentation task: reduce the historical value corresponding to the expiration time of the object defragmentation task to the target value, increase the historical value corresponding to the threshold of the fragmented object entering the defragmentation task queue to the target value, increase the historical value corresponding to the number of threads of the object defragmentation task to the target value, increase the historical value corresponding to the number of fragments processed in parallel by a single thread to the target value, and increase the historical value corresponding to the running time of the thread to the target value.

4. The object defragmentation method according to claim 2, characterized in that: The preset ratio threshold range includes a lower limit value, and determining the target value currently corresponding to each parameter of the object defragmentation task according to the first judgment result and the second judgment result includes: If the first judgment result is that the write-through ratio of the data pool is equal to the lower limit value, and the second judgment result is that the space replacement frequency of the buffer pool is not less than the preset frequency threshold, then obtaining the historical value corresponding to each parameter of the object defragmentation task; the parameter includes at least one of a first buffer pool usage rate threshold for triggering data write-through to the data pool and a second buffer pool usage rate threshold for stopping data write-through to the data pool; At least one of the following adjustment operations is performed on the historical value of the parameter of the object defragmentation task: reducing the historical value of the first buffer pool usage threshold to a target value, and increasing the historical value of the second buffer pool usage threshold to a target value.

5. The object defragmentation method according to claim 2, characterized in that: The preset ratio threshold range includes a lower limit value and an upper limit value, the upper limit value is greater than the lower limit value, and determining the target value currently corresponding to each parameter of the object defragmentation task according to the first judgment result and the second judgment result includes: If the first judgment result is that the write-through ratio of the data pool is greater than the lower limit value and less than the upper limit value, and the second judgment result is that the space replacement frequency of the cache pool is not less than the preset frequency threshold, then obtaining the historical value corresponding to each parameter of the object defragmentation task; the parameter includes at least one of the expiration time of the object defragmentation task and the threshold value for the fragmented object to enter the defragmentation task queue; Perform at least one of the following adjustment operations on the historical value of the parameter of the object defragmentation task: increase the historical value corresponding to the expiration time of the object defragmentation task to the target value, and reduce the historical value corresponding to the threshold of the fragmented object entering the defragmentation task queue to the target value.

6. The object defragmentation method according to claim 2, characterized in that: The preset ratio threshold range includes an upper limit value, and determining the target value currently corresponding to each parameter of the object defragmentation task according to the first judgment result and the second judgment result includes: If the first judgment result is that the write-through ratio of the data pool is greater than the upper limit value, and the second judgment result is that the space replacement frequency of the cache pool is less than the preset frequency threshold, then obtaining the historical value corresponding to each parameter of the object defragmentation task; the parameters include at least one of the expiration time of the object defragmentation task, the threshold for the fragmented object to enter the defragmentation task queue, the number of threads of the object defragmentation task, the number of fragments processed in parallel by a single thread, and the running time of the thread; Perform at least one of the following adjustment operations on the historical values ​​of the parameters of the object defragmentation task: increase the historical value corresponding to the expiration time of the object defragmentation task to the target value, reduce the historical value corresponding to the threshold of the fragmented object entering the defragmentation task queue to the target value, reduce the historical value corresponding to the number of threads of the object defragmentation task to the target value, reduce the historical value corresponding to the number of fragments processed in parallel by a single thread to the target value, and reduce the historical value corresponding to the running time of the thread to the target value.

7. The object defragmentation method according to any one of claims 1 to 6, characterized in that: The obtaining of the number of updates of the data in the preset address space in the buffer pool in the previous unit time includes: Obtaining the size of the target address space corresponding to the extraction in the buffer pool in the previous unit time; In the first address space, the second address space and the third address space of the buffer pool, sub-address spaces of the size of the target address space are selected in sequence; the first address space, the second address space and the third address space are address spaces corresponding to the front section, the middle section and the back section of the address code of the buffer pool respectively; The three sub-address spaces are determined as the preset address spaces, and the number of updates of the data in the preset address spaces in the cache pool in the previous unit time is counted.

8. An object defragmentation device, characterized in that: include: A space replacement frequency determination module, used to obtain the number of updates of data in a preset address space in the cache pool in the previous unit time, and determine the space replacement frequency of the cache pool in the previous unit time according to the number of updates in the previous unit time; A write-through ratio determination module, used to obtain a first amount of data written through to the data pool by the front-end server in the previous unit time and a second amount of data written in total by the front-end server in the previous unit time, and determine a write-through ratio of data written through to the data pool in the previous unit time according to the first amount of data and the second amount of data; A defragmentation task parameter determination module is used to determine the target value currently corresponding to each parameter of the object defragmentation task according to the space replacement frequency of the cache pool and the write-through ratio of the data pool; the object defragmentation task is a task for arranging fragmented objects in the data pool, and the target value of each parameter matches the current write business pressure of the front-end server; The object defragmentation module is used to execute the object defragmentation task according to the target value currently corresponding to each parameter.

9. A cloud storage device, comprising a cache pool, a data pool, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the object defragmentation method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the object defragmentation method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Object-oriented access method and system

    CN102799583A

  • Storage fragmentation time adjustment method and system, terminal and storage medium

    CN112506428A

  • Method, system and device for controlling water level of buffer pool and medium

    CN112631521A

  • Read-write optimization method and device for distributed storage system

    CN113204536A

  • File system management method, electronic equipment and storage medium

    CN113961517A