A method, apparatus, device and medium for data object download

Abstract

The present application discloses a method, apparatus, device and medium for flushing data objects, which relates to the field of computer technologies, and includes: determining the data objects modified in the solid-state drive as dirty data objects and adding them to a dirty data object queue; dividing the dirty data objects according to a preset unit length of the mechanical hard drive to obtain a plurality of divided intervals; flushing the dirty data in the divided intervals where the total amount of dirty data is greater than a flushing threshold to the mechanical hard drive; adding the data objects after flushing to a target queue, and determining whether there is dirty data among the data objects after flushing. If there is dirty data, the dirty data objects are added to the dirty data object queue again. If there is no dirty data, the data objects after flushing in the target queue are released. The present application preferentially flushes the dirty data in the divided intervals where the total amount of dirty data is greater than the flushing threshold, avoiding flushing scattered dirty data. In this way, the degree of flushing aggregation is increased, and the pressure on the mechanical hard drive is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and particularly to a method, apparatus, device and medium for flushing data objects downward. Background Art

[0002] SSD (Solid State Drives, solid state drives) has high random IO (Input / Output, data reading and writing) performance. Therefore, many current storage systems use SSD as a cache disk, allowing service IO to be first written to the cache disk, and then the data is flushed downward from the SSD cache disk to the HDD (Hard Disk Drive, mechanical hard disk) asynchronously. Currently, most storage systems use data objects as the basic unit for system data processing. Therefore, currently, the entire object is usually used as the smallest unit for flushing. After a service IO modifies a certain data object, the data object is marked as a dirty data object and added to the dirty data object LRU (Least Recently Used, least recently used) queue. Then, the flushing module retrieves the dirty data object from the dirty data object LRU, calculates its dirty data segment and flushes it. After the flushing is completed, the data object is added to the target data object LRU queue, indicating that all data of the object has been written to the main memory, but the data in the cache has not been deleted and needs to release the target data of the object when reclaiming the cache space subsequently.

[0003] However, a data object is usually 4M or larger in size. In the business scenario of random small IO, there may be hundreds of dirty data segments on a data object. Among these dirty data segments, some of them may have been aggregated into a relatively large dirty data segment, but some other dirty data segments are still relatively scattered. Therefore, when flushing the entire data object, a large number of random IOs cause the HDD to frequently seek tracks, which will instantaneously generate a large pressure on the HDD, resulting in low flushing efficiency and increased latency due to disk busyness, affecting the performance of the service.

[0004] Therefore, how to avoid flushing scattered dirty data and reduce the number of HDD track seeks to reduce the pressure on the mechanical hard disk and improve service performance is an urgent problem to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method, apparatus, device and medium for flushing data objects downward, which can avoid flushing scattered dirty data and reduce the number of HDD track seeks to reduce the pressure on the mechanical hard disk and improve service performance. The specific solutions are as follows:

[0006] In a first aspect, the present application discloses a method for flushing data objects downward, including:

[0007] When it is detected that the client modifies a data object in the solid - state drive, determine that the data object is a dirty data object, and add the dirty data object to a preset dirty data object queue;

[0008] Determine the dirty data object from the dirty data object queue, and divide the dirty data object according to a preset unit length of the mechanical hard disk to obtain a number of divided intervals;

[0009] Judge whether the total amount of dirty data in each data segment of the dirty data object corresponding to each divided interval is greater than a preset dirty data amount flushing threshold, and flush the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data amount flushing threshold to the mechanical hard disk;

[0010] Add the flushed dirty data object to a preset target object queue, and judge whether there is dirty data in the flushed dirty data object. If there is, re - execute the step of adding the dirty data object to the preset dirty data object queue. If not, release the data of the flushed data object in the target object queue.

[0011] Optionally, the step of when it is detected that the client modifies a data object in the solid - state drive, determine that the data object is a dirty data object, and add the dirty data object to a preset dirty data object queue includes:

[0012] When it is detected that the client modifies a data segment in a data object in the solid - state drive, determine that the data object is a dirty data object, determine that the data segment is a dirty data segment, and then add the dirty data object to a preset dirty data object queue.

[0013] Optionally, before judging whether the total amount of dirty data in each data segment of the dirty data object corresponding to each divided interval is greater than a preset dirty data amount flushing threshold, it further includes:

[0014] Determine the total amount of dirty data and the number of dirty data objects in the dirty data object queue, and determine the quotient of the total amount of dirty data in the dirty data object queue and the number of dirty data objects in the dirty data object queue as a first parameter;

[0015] Determine the size of a single dirty data object in the dirty data object queue, and determine the quotient of the size of a single dirty data object in the dirty data object queue and the preset unit length as a second parameter;

[0016] Based on the quotient of the first parameter and the second parameter, determine the average amount of dirty data corresponding to the divided interval, and determine the average amount of dirty data as the preset dirty data amount flushing threshold.

[0017] Optionally, the method for flushing the data object further includes:

[0018] Calculating the average amount of dirty data at a preset time interval, and updating the preset threshold for flushing the amount of dirty data by using the average amount of dirty data.

[0019] Optionally, after flushing the dirty data in the division interval where the total amount of dirty data is greater than the preset threshold for flushing the amount of dirty data to the mechanical hard disk, the method further includes:

[0020] Marking the dirty data segment corresponding to the dirty data that has been flushed to the mechanical hard disk as the target data segment.

[0021] Optionally, the data release of the flushed data object in the target object queue includes:

[0022] Releasing the data of the target data segment in the flushed data object in the target object queue.

[0023] Optionally, after determining whether the total amount of dirty data of each data segment in the dirty data object corresponding to each division interval is greater than the preset threshold for flushing the amount of dirty data, the method further includes:

[0024] If the total amount of dirty data of each data segment in the dirty data object corresponding to the division interval is not greater than the preset threshold for flushing the amount of dirty data, then re-execute the step of adding the dirty data object to the preset dirty data object queue.

[0025] In a second aspect, the present application discloses a data object flushing device, including:

[0026] A dirty data object adding module, configured to determine the data object as a dirty data object when detecting a modification operation of the data object in the solid-state hard disk by the client, and add the dirty data object to a preset dirty data object queue;

[0027] A dirty data object dividing module, configured to determine the dirty data object from the dirty data object queue, and divide the dirty data object according to a preset unit length of the mechanical hard disk to obtain a plurality of division intervals;

[0028] A dirty data flushing module, configured to determine whether the total amount of dirty data of each data segment in the dirty data object corresponding to each division interval is greater than a preset threshold for flushing the amount of dirty data, and flush the dirty data in the division interval where the total amount of dirty data is greater than the preset threshold for flushing the amount of dirty data to the mechanical hard disk;

[0029] A data release module, configured to add the downbrushed dirty data objects to a preset target object queue, and determine whether there is dirty data in the downbrushed dirty data objects. If there is dirty data, the step of adding the dirty data objects to a preset dirty data object queue is executed again. If there is no dirty data, data release is performed on the downbrushed data objects in the target object queue.

[0030] In a third aspect, the present application discloses an electronic device, including:

[0031] A memory, configured to store a computer program;

[0032] A processor, configured to execute the computer program to implement the data object downbrushing method disclosed above.

[0033] In a fourth aspect, the present application discloses a computer-readable storage medium, configured to store a computer program; wherein, when the computer program is executed by a processor, the data object downbrushing method disclosed above is implemented.

[0034] It can be seen that the present application proposes a data object downbrushing method. When a modification operation on a data object in a solid-state drive is detected by a client, it is determined that the data object is a dirty data object, and the dirty data object is added to a preset dirty data object queue; the dirty data object is determined from the dirty data object queue, and the dirty data object is divided according to a preset unit length of a mechanical hard disk to obtain a plurality of divided intervals; it is determined whether the total amount of dirty data in each data segment of the dirty data object corresponding to each divided interval is greater than a preset dirty data amount downbrushing threshold, and the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data amount downbrushing threshold is downbrushed to the mechanical hard disk; the downbrushed dirty data objects are added to a preset target object queue, and it is determined whether there is dirty data in the downbrushed dirty data objects. If there is dirty data, the step of adding the dirty data objects to a preset dirty data object queue is executed again. If there is no dirty data, data release is performed on the downbrushed data objects in the target object queue. Thus, it can be seen that the present application divides the dirty data objects according to the preset unit length of the mechanical hard disk to obtain a plurality of divided intervals, and preferentially downbrushes the dirty data in the divided intervals where the total amount of dirty data is greater than the dirty data amount downbrushing threshold. In this way, by avoiding downbrushing scattered dirty data, the downbrushing aggregation degree is increased and the mechanical hard disk seek times are reduced, thereby reducing the pressure on the mechanical hard disk and improving the service performance. Description of the Drawings

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained according to the provided drawings.

[0036] Figure 1 Flowchart of a data object download method disclosed in this application;

[0037] Figure 2 Flowchart of a specific data object download method disclosed in this application;

[0038] Figure 3 Structural schematic diagram of a data object download method disclosed in this application;

[0039] Figure 4 Flowchart of a specific data object download method disclosed in this application;

[0040] Figure 5 Structural schematic diagram of a data object download device disclosed in this application;

[0041] Figure 6 Structural diagram of an electronic device disclosed in this application. Specific embodiments

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0043] When downloading the entire data object, a large number of random I / Os cause the HDD to frequently seek tracks, which will instantaneously generate a large pressure on the HDD. The download efficiency is low and the delay will increase due to the disk being busy, affecting the performance of the service.

[0044] Therefore, the embodiments of this application propose a data object download solution, which can avoid downloading scattered dirty data and reduce the number of seek operations of the mechanical hard disk, so as to reduce the pressure on the mechanical hard disk and improve the service performance.

[0045] The embodiments of this application disclose a data object download method. Refer to Figure 1 As shown, the method includes:

[0046] Step S11: When it is detected that the client modifies a data object in the solid-state drive, determine that the data object is a dirty data object, and add the dirty data object to a preset dirty data object queue.

[0047] In a specific embodiment, when it is detected that the client modifies a data object in the solid-state drive, determine that the data object is a dirty data object, and add the dirty data object to a preset dirty data object queue;

[0048] In another specific embodiment, when it is detected that the client newly writes a data object in the solid-state drive, determine that the data object is a dirty data object, and add the dirty data object to a preset dirty data object queue.

[0049] Step S12: Determine the dirty data object from the dirty data object queue, and divide the dirty data object according to a preset unit length of the mechanical hard disk to obtain a number of divided intervals.

[0050] In this embodiment, the preset unit length may be the minimum allocation unit length of the mechanical hard disk, for example, 256K. That is, assuming the dirty data object is 4M, then 0K - 256K is determined as the first divided interval, 256K - 512K is determined as the second divided interval, and so on. Since the storage system allocates the actual physical space according to the minimum allocation unit when writing data on the HDD, dividing the data object according to the minimum allocation unit of the HDD can ensure that the data objects within the same minimum allocation unit are also continuous on the HDD. Even if there is a certain degree of discontinuity during the write-back, it will not increase the seek time of the HDD, and thus will not cause too much pressure on the HDD.

[0051] Step S13: Determine whether the total amount of dirty data in each data segment of the dirty data object corresponding to each divided interval is greater than a preset dirty data write-back threshold, and write back the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data write-back threshold to the mechanical hard disk.

[0052] In this embodiment, if the total amount of dirty data in each data segment of the dirty data object corresponding to the divided interval is not greater than the preset dirty data write-back threshold, then re-execute the step of adding the dirty data object to the preset dirty data object queue.

[0053] It should be noted that the process of determining the underbrush threshold under the preset dirty data volume specifically includes: determining the total amount of dirty data and the number of dirty data objects in the dirty data object queue, and determining the quotient of the total amount of dirty data in the dirty data object queue and the number of dirty data objects in the dirty data object queue as the first parameter; determining the size of a single dirty data object in the dirty data object queue, and determining the quotient of the size of a single dirty data object in the dirty data object queue and the preset unit length as the second parameter; determining the average dirty data volume corresponding to the division interval based on the quotient of the first parameter and the second parameter, and determining the average dirty data volume as the underbrush threshold under the preset dirty data volume. Moreover, the average dirty data volume is calculated at preset time intervals, and the underbrush threshold under the preset dirty data volume is updated using the average dirty data volume. In this way, since the underbrush threshold under the preset dirty data volume is updated in real time according to the current dirty data statistics, the situation of not being able to find dirty data that can be underbrushed will not occur. At the same time, the present application can also set the minimum underbrush threshold of dirty data according to the actual application scenario to ensure that the dirty data is sufficiently aggregated before underbrushing.

[0054] In this embodiment, the dirty data in the division interval where the total amount of dirty data is greater than the underbrush threshold under the preset dirty data volume is underbrushed to the mechanical hard disk. In this way, by avoiding underbrushing scattered dirty data, the degree of underbrush aggregation is increased and the number of seek operations of the mechanical hard disk is reduced, thereby reducing the pressure on the mechanical hard disk and improving the service performance.

[0055] Step S14: Add the underbrushed dirty data object to a preset target object queue, and determine whether there is dirty data in the underbrushed dirty data object. If there is, re-execute the step of adding the dirty data object to the preset dirty data object queue. If not, release the data of the underbrushed data object in the target object queue.

[0056] In this embodiment, the underbrushed dirty data object is added to a preset target object queue, and it is determined whether there is dirty data in the underbrushed dirty data object. If there is, the dirty data object is re-added to the preset dirty data object queue. If not, the data of the underbrushed data object in the target object queue is released.

[0057] It can be seen that the present application proposes a data object flushing method. When it is detected that the client modifies a data object in the solid-state drive, it is determined that the data object is a dirty data object, and the dirty data object is added to a preset dirty data object queue; the dirty data object is determined from the dirty data object queue, and the dirty data object is divided according to a preset unit length of the mechanical hard disk to obtain a plurality of divided intervals; it is judged whether the total amount of dirty data of each data segment in the dirty data object corresponding to each divided interval is greater than a preset dirty data amount flushing threshold, and the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data amount flushing threshold is flushed to the mechanical hard disk; the flushed dirty data object is added to a preset target object queue, and it is judged whether there is dirty data in the flushed dirty data object. If there is, the step of adding the dirty data object to the preset dirty data object queue is executed again. If not, the flushed data object in the target object queue is released. Thus, it can be seen that the present application divides the dirty data object according to the preset unit length of the mechanical hard disk to obtain a plurality of divided intervals, and preferentially flushes the dirty data in the divided interval where the total amount of dirty data is greater than the dirty data amount flushing threshold. In this way, by avoiding flushing scattered dirty data, the flushing aggregation degree is increased and the mechanical hard disk seek times are reduced, thereby reducing the pressure on the mechanical hard disk and improving the service performance.

[0058] The embodiment of the present application discloses a specific data object flushing method. Compared with the previous embodiment, this embodiment further describes and optimizes the technical solution. Refer to Figure 2 as shown, specifically including:

[0059] Step S21: When it is detected that the client modifies a data segment in the data object in the solid-state drive, it is determined that the data object is a dirty data object, and the data segment is determined to be a dirty data segment, and then the dirty data object is added to a preset dirty data object queue.

[0060] In this embodiment, the modified data segment in the data object is determined as the dirty data segment.

[0061] Step S22: The dirty data object is determined from the dirty data object queue, and the dirty data object is divided according to a preset unit length of the mechanical hard disk to obtain a plurality of divided intervals.

[0062] Step S23: It is judged whether the total amount of dirty data of each data segment in the dirty data object corresponding to each divided interval is greater than a preset dirty data amount flushing threshold, and the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data amount flushing threshold is flushed to the mechanical hard disk.

[0063] Step S24: Mark the dirty data segment corresponding to the dirty data that has been flushed to the mechanical hard disk as the target data segment.

[0064] In this embodiment, after flushing the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data volume threshold to the mechanical hard disk, mark the dirty data segment corresponding to the dirty data that has been flushed to the mechanical hard disk as the target data segment.

[0065] Step S25: Add the flushed dirty data object to a preset target object queue, and determine whether there is dirty data in the flushed dirty data object. If there is, re - execute the step of adding the dirty data object to the preset dirty data object queue. If not, release the data of the target data segment in the flushed data object in the target object queue.

[0066] In this embodiment, add the flushed dirty data object to a preset target object queue, and determine whether there is dirty data in the flushed dirty data object. If not, release the data of the target data segment in the flushed data object in the target object queue. In this way, the cache space is released and the pressure on the solid - state drive is reduced.

[0067] Among them, for the more specific working processes of Step S22 and Step S23, refer to the previously disclosed embodiments, and no specific elaboration will be made here.

[0068] It can be seen that the present application proposes a method for flushing data objects, including: when detecting a modification operation on a data segment in a data object in a solid-state drive by a client, determining the data object as a dirty data object and determining the data segment as a dirty data segment, and then adding the dirty data object to a preset dirty data object queue; determining the dirty data object from the dirty data object queue, and dividing the dirty data object according to a preset unit length of a mechanical hard disk to obtain a plurality of divided intervals; determining whether the total amount of dirty data in each data segment in the dirty data object corresponding to each divided interval is greater than a preset dirty data amount flushing threshold, and flushing the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data amount flushing threshold to the mechanical hard disk; marking the dirty data segment corresponding to the dirty data that has been flushed to the mechanical hard disk as a target data segment; adding the flushed dirty data object to a preset target object queue, and determining whether there is dirty data in the flushed dirty data object. If there is, re-execute the step of adding the dirty data object to the preset dirty data object queue. If not, release the data of the target data segment in the flushed data object in the target object queue. It can be seen that the present application divides the dirty data object according to the preset unit length of the mechanical hard disk to obtain a plurality of divided intervals, and preferentially flushes the dirty data in the divided interval where the total amount of dirty data is greater than the dirty data amount flushing threshold. In this way, by avoiding flushing scattered dirty data, the flushing aggregation degree is increased and the mechanical hard disk seek times are reduced, thereby reducing the pressure on the mechanical hard disk and improving the service performance.

[0069] Figure 3 FIG. is a schematic structural diagram of a method for flushing a data object disclosed in the present application. Refer to Figure 3 As shown, the present application specifically can include the following content:

[0070] (1) Service IO read / write module: used to process the read / write IO of the client by the module. Whenever a new write or a modified write data object occurs, record the modified data segment in the data object as a dirty data segment, and add the data object to the dirty data object LRU queue.

[0071] (2) LRU management module: used to maintain the dirty data object LRU queue and the target data object LRU queue, and is responsible for enqueueing and dequeueing data objects. If a data object has unflushed data, it needs to be added to the dirty data object LRU queue. If there is flushed but unreleased data, it needs to be added to the target data object LRU queue. The same data object can also exist in these two LRU queues at the same time, that is, only part of the data object is flushed, that is, the data object has dirty data segments and also has clean data segments.

[0072] (3) Lower brush module: used to retrieve dirty data objects from the dirty data object LRU queue for lower brush. After the lower brush is completed, the object is passed to the LRU queue management module to be re-queued.

[0073] (4) Dirty data volume statistics module: used to update the total dirty data volume and dirty data objects when a data object is added to or retrieved from the dirty data object LRU, and to update the dirty data volume threshold of the data segment that can be lower-brushed to the lower brush module in real time.

[0074] See Figure 4 As shown, the lower brush process specifically includes the following:

[0075] When a newly written or modified data object is encountered, the newly written and modified data segments are updated to dirty data segments; and the data object is added to the dirty data object LRU queue; the lower brush module retrieves the dirty data object from the dirty data object LRU queue; taking the minimum allocation unit of the HDD as the division interval, traverse the dirty data object. If the dirty data volume of a certain interval is greater than the dirty data volume threshold of the data segment that can be lower-brushed, the dirty data in that interval is added to the lower brush request context, otherwise continue to judge the next segment. After the traversal is completed, if the dirty data volume of all intervals is not greater than the dirty data volume threshold of the data segment that can be lower-brushed, it means that the lower brush cannot be performed temporarily, and the data object is re-added to the dirty data object LRU queue, otherwise the lower brush request context is submitted; the dirty data in the lower brush request context is lower-brushed, and the data segment corresponding to the lower-brushed dirty data is marked as the target data segment; the lower-brushed dirty data object is added to the target data object LRU queue; judge whether there is still dirty data in the data object after the lower brush is completed. If there is, re-add it to the dirty data object LRU queue; if not, there is no need to re-queue, and the target data segment part in the target data object LRU queue is deleted from the cache device to release the cache space. In this way, this application increases the degree of lower brush aggregation and reduces the mechanical hard disk seek times by avoiding lower-brushing scattered dirty data, thereby reducing the pressure on the mechanical hard disk and improving the service performance.

[0076] Correspondingly, the embodiment of this application also discloses a data object lower brush device. See Figure 5 As shown, the device includes:

[0077] Dirty data object addition module 11, used to determine that the data object is a dirty data object when detecting a modification operation of the data object in the solid-state drive by the client, and add the dirty data object to a preset dirty data object queue;

[0078] Dirty data object division module 12, used to determine the dirty data object from the dirty data object queue, and divide the dirty data object according to the preset unit length of the mechanical hard disk to obtain several division intervals;

[0079] The dirty data down - brushing module 13 is used to determine whether the total amount of dirty data in each data segment of the dirty data objects corresponding to each of the divided intervals is greater than a preset dirty data amount down - brushing threshold, and down - brush the dirty data in the divided intervals where the total amount of dirty data is greater than the preset dirty data amount down - brushing threshold to the mechanical hard disk;

[0080] The data release module 14 is used to add the down - brushed dirty data objects to a preset target object queue, and determine whether there is dirty data in the down - brushed dirty data objects. If there is, the step of adding the dirty data objects to the preset dirty data object queue is re - executed. If not, the data in the down - brushed data objects in the target object queue is released.

[0081] Among them, for the more specific working processes of the above - mentioned various modules, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated herein.

[0082] It can be seen that the present application proposes a data object down - brushing method. When it detects a modification operation of a data object in the solid - state hard disk by the client, it determines that the data object is a dirty data object and adds the dirty data object to a preset dirty data object queue; determines the dirty data object from the dirty data object queue, divides the dirty data object according to the preset unit length of the mechanical hard disk to obtain several divided intervals; determines whether the total amount of dirty data in each data segment of the dirty data objects corresponding to each of the divided intervals is greater than a preset dirty data amount down - brushing threshold, and down - brush the dirty data in the divided intervals where the total amount of dirty data is greater than the preset dirty data amount down - brushing threshold to the mechanical hard disk; adds the down - brushed dirty data objects to a preset target object queue, and determines whether there is dirty data in the down - brushed dirty data objects. If there is, the step of adding the dirty data objects to the preset dirty data object queue is re - executed. If not, the data in the down - brushed data objects in the target object queue is released. Thus, the present application divides the dirty data object according to the preset unit length of the mechanical hard disk to obtain several divided intervals, and preferentially down - brushes the dirty data in the divided intervals where the total amount of dirty data is greater than the dirty data amount down - brushing threshold. In this way, by avoiding down - brushing scattered dirty data, the down - brushing aggregation degree is increased and the mechanical hard disk seek times are reduced, thereby reducing the pressure on the mechanical hard disk and improving the service performance.

[0083] Furthermore, the embodiment of the present application also provides an electronic device. Figure 6 It is a structural diagram of the electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation on the usage scope of the present application.

[0084] Figure 6Schematic diagram of the structure of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the following steps:

[0085] When it is detected that the client modifies a data object in the solid-state drive, determine that the data object is a dirty data object, and add the dirty data object to a preset dirty data object queue;

[0086] Determine the dirty data object from the dirty data object queue, and divide the dirty data object according to a preset unit length of the mechanical hard disk to obtain a number of divided intervals;

[0087] Judge whether the total amount of dirty data in each data segment of the dirty data object corresponding to each divided interval is greater than a preset dirty data amount flushing threshold, and flush the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data amount flushing threshold to the mechanical hard disk;

[0088] Add the flushed dirty data object to a preset target object queue, and judge whether there is dirty data in the flushed dirty data object. If there is, re-execute the step of adding the dirty data object to the preset dirty data object queue. If not, release the data of the flushed data object in the target object queue.

[0089] In some specific embodiments, the processor may specifically implement the following steps by executing the computer program stored in the memory:

[0090] When it is detected that the client modifies a data segment in a data object in the solid-state drive, determine that the data object is a dirty data object, determine that the data segment is a dirty data segment, and then add the dirty data object to a preset dirty data object queue.

[0091] In some specific embodiments, the processor may further include the following steps by executing the computer program stored in the memory:

[0092] Determine the total amount of dirty data and the number of dirty data objects in the dirty data object queue, and determine that the quotient of the total amount of dirty data in the dirty data object queue and the number of dirty data objects in the dirty data object queue is a first parameter;

[0093] Determine the size of a single dirty data object in the dirty data object queue, and determine the quotient of the size of a single dirty data object in the dirty data object queue and the preset unit length as the second parameter;

[0094] Based on the quotient of the first parameter and the second parameter, determine the average amount of dirty data corresponding to the division interval, and determine the average amount of dirty data as the preset threshold for flushing the dirty data.

[0095] In some specific embodiments, when the processor executes the computer program stored in the memory, the following steps may further be included:

[0096] Calculate the average amount of dirty data at preset time intervals, and update the preset threshold for flushing the dirty data using the average amount of dirty data.

[0097] In some specific embodiments, when the processor executes the computer program stored in the memory, the following steps may further be included:

[0098] Mark the dirty data segment corresponding to the dirty data that has been flushed to the mechanical hard disk as the target data segment.

[0099] In some specific embodiments, when the processor executes the computer program stored in the memory, the following steps may specifically be implemented:

[0100] Release the data of the target data segment in the data object after flushing in the target object queue.

[0101] In some specific embodiments, when the processor executes the computer program stored in the memory, the following steps may further be included:

[0102] If the total amount of dirty data in each data segment of the dirty data object corresponding to the division interval is not greater than the preset threshold for flushing the dirty data, then re-execute the step of adding the dirty data object to the preset dirty data object queue.

[0103] In this embodiment, the power supply 26 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of this application, and specific limitations are not imposed here; the input / output interface 24 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitations are made here.

[0104] In addition, as a carrier for storing resources, the memory 22 can be a read-only memory, a random access memory, a magnetic disk, an optical disk, etc. The resources stored thereon can include a computer program 221, and the storage method can be transient storage or permanent storage. Among them, in addition to the computer program that can be used to complete the data object download method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 221 can further include a computer program that can be used to complete other specific tasks.

[0105] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the data object download method disclosed above is implemented.

[0106] For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0107] The various embodiments in this application are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and reference can be made to the description in the method part for related parts.

[0108] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0109] The steps of the method or algorithm described in combination with the embodiments disclosed in this article can be directly implemented by hardware, a software module executed by a processor, or a combination of both. The software module can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0110] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0111] The above has introduced in detail a method, apparatus, device, and storage medium for downloading data objects provided by this application. Specific examples are used in this text to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A method for downloading a data object, characterized in that, Including: When it is detected that the client modifies a data object in the solid-state drive, determine that the data object is a dirty data object, and add the dirty data object to a preset dirty data object queue; Determine the dirty data object from the dirty data object queue, and divide the dirty data object according to a preset unit length of the mechanical hard drive to obtain a plurality of divided intervals; Judge whether the total amount of dirty data in each data segment of the dirty data object corresponding to each divided interval is greater than a preset dirty data amount flushing threshold, and flush the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data amount flushing threshold to the mechanical hard drive; Add the flushed dirty data object to a preset target object queue, and judge whether there is dirty data in the flushed dirty data object. If there is, re-execute the step of adding the dirty data object to the preset dirty data object queue. If not, release the data of the flushed data object in the target object queue.

2. The data object downloading method according to claim 1, wherein The step of when it is detected that the client modifies a data object in the solid-state drive, determine that the data object is a dirty data object, and add the dirty data object to a preset dirty data object queue includes: When it is detected that the client modifies a data segment in the data object in the solid-state drive, determine that the data object is a dirty data object, determine that the data segment is a dirty data segment, and then add the dirty data object to a preset dirty data object queue.

3. The data object downloading method according to claim 1, characterized in that Before judging whether the total amount of dirty data in each data segment of the dirty data object corresponding to each divided interval is greater than a preset dirty data amount flushing threshold, it further includes: Determine the total amount of dirty data and the number of dirty data objects in the dirty data object queue, and determine that the quotient of the total amount of dirty data in the dirty data object queue and the number of dirty data objects in the dirty data object queue is a first parameter; Determine the size of a single dirty data object in the dirty data object queue, and determine that the quotient of the size of a single dirty data object in the dirty data object queue and the preset unit length is a second parameter; Determine the average dirty data amount corresponding to the divided interval based on the quotient of the first parameter and the second parameter, and determine the average dirty data amount as the preset dirty data amount flushing threshold.

4. The data object downloading method according to claim 3, wherein It further includes: Calculate the average dirty data amount at a preset time interval, and update the preset dirty data amount flushing threshold using the average dirty data amount.

5. The data object download method according to claim 2, wherein After flushing the dirty data in the divided interval where the total amount of dirty data is greater than the preset dirty data amount flushing threshold to the mechanical hard drive, it further includes: Mark the dirty data segment corresponding to the dirty data flushed to the mechanical hard drive as a target data segment.

6. The data object download method according to claim 5, wherein The step of releasing the data of the flushed data object in the target object queue includes: Release the data of the target data segment in the flushed data object in the target object queue.

7. The data object downloading method according to any one of claims 1 to 6, characterized in that After determining whether the total amount of dirty data in each data segment of the dirty data object corresponding to each division interval is greater than a preset dirty data volume flushing threshold, the following steps are further included: If the total amount of dirty data in each data segment of the dirty data object corresponding to the division interval is not greater than the preset dirty data volume flushing threshold, the step of adding the dirty data object to a preset dirty data object queue is re-executed.

8. A data object downloading device, characterized in that It includes: A dirty data object adding module, configured to determine the data object as a dirty data object when detecting a modification operation of a data object in a solid-state drive by a client, and add the dirty data object to a preset dirty data object queue; A dirty data object dividing module, configured to determine the dirty data object from the dirty data object queue, and divide the dirty data object according to a preset unit length of a mechanical hard disk to obtain a plurality of division intervals; A dirty data flushing module, configured to determine whether the total amount of dirty data in each data segment of the dirty data object corresponding to each division interval is greater than a preset dirty data volume flushing threshold, and flush the dirty data in the division interval where the total amount of dirty data is greater than the preset dirty data volume flushing threshold to the mechanical hard disk; A data releasing module, configured to add the flushed dirty data object to a preset target object queue, and determine whether there is dirty data in the flushed dirty data object. If there is, the step of adding the dirty data object to the preset dirty data object queue is re-executed. If not, data of the flushed data object in the target object queue is released.

9. An electronic device, characterized in that, It includes: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the data object flushing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by a processor, the data object flushing method according to any one of claims 1 to 7 is implemented.

Images