Data storage method and device, computer equipment, storage medium and program product

Abstract

The invention relates to the technical field of computer storage, and discloses a data storage method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: polling a dirty cache number corresponding to each physical disk in a target disk group in a cache; for any physical disk in the target disk group, according to the number of dirty caches corresponding to the physical disk, triggering a disk falling circulation task of a priority level; the priority level is related to the number of dirty caches; and according to the priority level of the disk falling circulation task corresponding to each physical disk in the target disk group, scheduling each disk falling task and executing a data disk falling operation so as to fall dirty cache data corresponding to the disk falling task to the corresponding physical disk. According to the scheme, when the data disk falling function is achieved, the data disk falling efficiency and stability can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, and in particular to a method and device for storing data on a disk, a computer device, a storage medium and a program product. Background Art

[0002] At present, storage IO acceleration chips use physical memory with large storage space as on-chip cache for physical disks, where the cache consists of multiple cache lines. For the write operation of WB (Write Back) IO, the host writes the data to the corresponding cache line in the cache, and the data in the cache line is converted to dirty data. At this time, the write response can be completed without writing to the physical disk. The actual data flushing operation is completed by hardware to reduce software resource usage.

[0003] In the related technology, when executing the data drop operation through the hardware background, there is no priority scheduling for each data drop task, resulting in that more urgent data drop tasks cannot be processed with high priority. For example, a large number of data drop tasks triggered by a sudden increase in dirty data corresponding to a physical disk cannot be processed with priority, thereby occupying cache lines and blocking the normal execution of subsequent WB IO; processing the data drop tasks corresponding to the same physical disk at one time may take a long time due to the large number of data drop tasks corresponding to the physical disk, blocking the execution of other back-end IO (BIO) of the physical disk and causing imbalance in the back-end IO of other physical disks in the disk group, causing disk group performance fluctuations; the frequency and number of data drop tasks corresponding to some physical disks are low, and if the task is triggered immediately, it will compete with other physical disks with a large number of tasks and a high execution frequency for data drop resources, affecting the task execution efficiency; it is impossible to actively identify when some physical disks lose power or are unplugged, and there is a risk of data loss.

[0004] Therefore, there is an urgent need for a more efficient and stable method of storing data on disk. Summary of the invention

[0005] In view of this, an object of the present invention is to provide a method, apparatus, computer device, storage medium and program product for storing data on a disk, so as to solve the problems of low efficiency and poor stability when performing data storage tasks.

[0006] In a first aspect, the present invention provides a method for storing data on a disk, the method comprising:

[0007] Poll the number of dirty caches corresponding to each physical disk in the target disk group in the cache;

[0008] For any physical disk in the target disk group, triggering a disk flushing cycle task of a priority level according to the number of dirty caches corresponding to the physical disk; the priority level is related to the number of dirty caches;

[0009] According to the priority level of the disk placement cycle task corresponding to each physical disk in the target disk group, each disk placement task is scheduled and a data disk placement operation is performed to place the dirty cache data corresponding to the disk placement task on the corresponding physical disk.

[0010] In an optional implementation, the priority level includes a high priority and a low priority; and triggering the disk flushing cycle task of the priority level according to the number of dirty caches corresponding to the physical disk includes:

[0011] When the number of dirty caches corresponding to the target physical disk exceeds the high-priority disk drop start line, the high-priority disk drop cycle task is triggered to put the dirty cache data into the high-priority disk drop task queue;

[0012] When the number of dirty caches corresponding to the target physical disk exceeds the low priority disk drop start line, the low priority disk drop cycle task is triggered to put the dirty cache data into the low priority disk drop task queue; wherein the low priority disk drop start line is lower than the high priority disk drop start line.

[0013] In an optional implementation, the high priority disk placement cycle task includes:

[0014] When the dirty cache quantity corresponding to the target physical disk exceeds the high priority disk drop start line, dirty cache data equal to the target disk flushing quantity is put into the high priority disk drop task queue to wait for the nth round of data drop operation; the target disk flushing quantity corresponds to the target physical disk;

[0015] After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the high priority flushing task queue to wait for the n+1th round of data flushing; the flushing stop line is lower than the total flushing start line;

[0016] Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk drop stop line, and then stop the high priority disk drop cycle task.

[0017] In an optional implementation, the low-priority disk placement cycle task includes:

[0018] When the number of dirty caches corresponding to the target physical disk exceeds the low-priority disk flushing start line, dirty cache data equal to the target disk flushing amount is put into the low-priority disk flushing task queue to wait for the nth round of data flushing operation;

[0019] After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount will be put into the high-priority flushing task queue; if the number of dirty caches corresponding to the target physical disk is lower than the high-priority flushing start line and exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the low-priority flushing task queue to wait for the n+1th round of data flushing;

[0020] Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk drop stop line, and then stop the low priority disk drop cycle task.

[0021] In an optional embodiment, the method further includes:

[0022] When the total number of dirty caches corresponding to the physical disks in the target disk group exceeds the total disk drop start line, for each physical disk in the target disk group, the dirty cache data of the corresponding disk flushing amount will be put into the total disk drop task queue to wait for the nth round of data drop operation; wherein, the total disk drop start line is lower than the low priority disk drop start line; the priority level of the total disk drop task queue is lower than that of the low priority disk drop task queue.

[0023] In an optional embodiment, the method further includes:

[0024] Polling each physical disk according to a preset time period; wherein each physical disk is set with a corresponding timeout duration;

[0025] When it is detected that the number of dirty caches corresponding to the target physical disk is between the zero watermark and the stop line of the disk placement, the timing for the target physical disk is started; the zero watermark of the disk placement is lower than the stop line;

[0026] When the timing duration for the target physical disk exceeds the timeout duration corresponding to the target physical disk, the dirty cache data corresponding to the target physical disk, whose quantity is the target disk flushing quantity, will be delivered to the timeout disk drop task queue to wait for the nth round of data drop operation; wherein, the priority level of the timeout disk drop task queue is lower than that of the total disk drop task queue.

[0027] In an optional embodiment, the method further includes:

[0028] When it is detected that the target physical disk has lost power or been unplugged, the dirty cache data corresponding to the target physical disk is delivered to a special disk drop task queue to wait for the nth round of data drop operation; the priority level of the special disk drop task queue is higher than that of the high priority disk drop task queue.

[0029] In an optional embodiment, the method further includes:

[0030] The dirty cache data waiting to be written to the disk are mounted into the write buffer chain in the order of generation time; wherein, the dirty cache data are mounted in the order from the head to the tail of the write buffer chain;

[0031] For each dirty cache data on the write buffer chain, sort each dirty cache data corresponding to each physical disk according to the size of the logical block address, and mark the dirty cache data with continuous logical block addresses;

[0032] For each dirty cache data, aggregate the continuous blocks in the dirty cache data;

[0033] Aggregate dirty cache data with continuous logical block addresses to obtain aggregated dirty cache data;

[0034] Starting from the head of the chain, the dirty cache data after aggregation processing corresponding to the disk flushing amount is removed and stored in the corresponding physical disks in sequence.

[0035] In a second aspect, the present invention provides a data storage device, the device comprising:

[0036] A polling module, used to poll the dirty cache quantity corresponding to each physical disk in the target disk group in the cache;

[0037] A task triggering module, for triggering a disk drop cycle task of a priority level for any physical disk in the target disk group according to the number of dirty caches corresponding to the physical disk; the priority level is related to the number of dirty caches;

[0038] The disk placement module is used to schedule each disk placement task and perform data placement operations according to the priority level of the disk placement cycle task corresponding to each physical disk in the target disk group, so as to place the dirty cache data corresponding to the disk placement task to the corresponding physical disk.

[0039] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data storage method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data storage method of the first aspect or any corresponding embodiment thereof.

[0041] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the data storage method of the first aspect or any corresponding embodiment thereof.

[0042] The technical solution provided by the present invention may include the following beneficial effects:

[0043] The data drop method provided by the present invention first polls the number of dirty caches corresponding to each physical disk in the target disk group in the cache, then for any physical disk in the target disk group, triggers a drop disk cycle task of a priority level according to the number of dirty caches corresponding to the physical disk, and finally schedules each drop disk task and performs a data drop disk operation according to the priority level of the drop disk cycle task corresponding to each physical disk in the target disk group, so as to drop the dirty cache data corresponding to the drop disk task to the corresponding physical disk. By setting a drop disk cycle task with a priority level corresponding to the number of dirty caches, it is possible to set a priority for the drop disk task of dirty cache data corresponding to each physical disk in the target disk group, so that the drop disk tasks assigned to each physical disk are more balanced, and the drop disk tasks are cyclically executed in combination with the drop disk cycle task, so that the entire target disk group is more stable and the performance is more balanced, thereby improving the efficiency and stability of data drop disk. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic diagram showing data storage in the related art is shown;

[0046] Figure 2 is a flowchart of a method for storing data on a disk according to an embodiment of the present invention;

[0047] Figure 3 is a flow chart of another method for storing data on a disk according to an embodiment of the present invention;

[0048] Figure 4 is a schematic diagram of data storage processing according to an embodiment of the present invention;

[0049] Figure 5 is a schematic diagram of data disk quota management according to an embodiment of the present invention;

[0050] Figure 6 is a schematic diagram of a process of disk flushing time management according to an embodiment of the present invention;

[0051] Figure 7 is a schematic diagram of backend IO processing according to an embodiment of the present invention;

[0052] Figure 8 is a structural block diagram of a data storage device according to an embodiment of the present invention;

[0053] Fig. 9 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0055] Currently, storage IO acceleration chips (Storage IO Acceleration Chips) use physical memory (Physical Memory) with a larger storage space as an on-chip cache (Cache) of a physical disk (Physical Disk, PD). Figure 1 FIG. 4 shows a schematic diagram of data storage in the related art. Figure 1 As shown in the figure, the cache consists of multiple cache lines (CL). For the write operation of WB (Write Back) IO, the host writes the data to the corresponding cache line in the cache and then the data in the cache line is converted into dirty data (clean-CL becomes dirty-CL). At this time, there is no need to write to the physical disk (for example Figure 1 The write response can be completed by the hardware to reduce the software resource usage.

[0056] In the related technology, when executing the data drop operation through the hardware background, there is no priority scheduling for each data drop task, resulting in that more urgent data drop tasks cannot be processed with high priority. For example, a large number of data drop tasks triggered by a sudden increase in dirty data corresponding to a physical disk cannot be processed with priority, thereby occupying cache lines and blocking the normal execution of subsequent WB_IO; processing the data drop tasks corresponding to the same physical disk at one time may take a long time due to the large number of data drop tasks corresponding to the physical disk, blocking the execution of other back-end IO (BIO) of the physical disk and causing imbalance in the back-end IO of other physical disks in the disk group, causing disk group performance fluctuations; the frequency and number of data drop tasks corresponding to some physical disks are low, and if the task is triggered immediately, it will compete with other physical disks with a large number of tasks and a high execution frequency for data drop resources, affecting the task execution efficiency; it is impossible to actively identify when some physical disks lose power or are unplugged, and there is a risk of data loss.

[0057] According to an embodiment of the present invention, an embodiment of a method for transferring data to a disk is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] In this embodiment, a method for storing data on a disk is provided, which can be used for notebook computers, desktop computers, etc. Figure 2 is a flow chart of a method for storing data on a disk according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0059] Step S201, polling the dirty cache quantity corresponding to each physical disk in the target disk group in the cache.

[0060] In actual application scenarios, the target disk group contains multiple physical disks. The host will send the foreground IO corresponding to each physical disk in the target disk group to each cache line of the cache. After the foreground IO is stored in the cache line, the data in the cache line becomes dirty cache data. The number of dirty caches is the number of cache data that has been written to the cache but has not been written to the corresponding physical disk, that is, the number of dirty cache data. When implementing the data writing function, the present invention first polls the number of dirty cache data corresponding to each physical disk in the cache.

[0061] Step S202: for any physical disk in the target disk group, trigger a disk flushing cycle task of a priority level according to the number of dirty caches corresponding to the physical disk.

[0062] The priority level is related to the number of dirty caches, that is, when the number of dirty caches of the target physical disk is polled, the number of dirty caches is compared with the preset number threshold to determine the priority level of the disk drop cycle task corresponding to the target physical disk. For example, if the number of dirty caches corresponding to the target physical disk is greater than the preset number threshold, the priority level of the disk drop cycle task corresponding to the target physical disk is determined to be high, and the disk drop cycle task corresponding to the target physical disk is processed first. Among them, the disk drop cycle task can be a cyclic scheduling of the disk drop tasks corresponding to each dirty cache data corresponding to the physical disk, for example, only a certain amount of disk drop tasks are scheduled each time and cycled multiple times, or the number of cycles is set, and the quotient of the number of dirty caches and the number of cycles is used as the number of disk drop tasks scheduled each time.

[0063] Step S203, scheduling each disk placement task according to the priority level of the disk placement cycle task corresponding to each physical disk in the target disk group, and performing a data placement operation to place the dirty cache data corresponding to the disk placement task on the corresponding physical disk.

[0064] After determining the priority level of the disk drop cycle task corresponding to each physical disk in the target disk group according to the corresponding number of dirty caches, the disk drop cycle tasks corresponding to each physical disk can be arranged according to the priority level, and then the disk drop tasks corresponding to each dirty cache data can be scheduled according to the arrangement order, so as to drop each dirty cache data to the corresponding physical disk, thereby completing the process of dropping the foreground IO to the permanent storage medium (physical disk).

[0065] The data drop method provided in this embodiment first polls the number of dirty caches corresponding to each physical disk in the target disk group in the cache, and then triggers the drop disk cycle task with a priority level according to the number of dirty caches corresponding to the physical disk for any physical disk in the target disk group. Finally, according to the priority level of the drop disk cycle task corresponding to each physical disk in the target disk group, each drop disk task is scheduled and the data drop disk operation is performed to drop the dirty cache data corresponding to the drop disk task to the corresponding physical disk. By setting the drop disk cycle task with a priority level corresponding to the number of dirty caches, the drop disk task corresponding to each physical disk in the target disk group can be prioritized, so that the drop disk tasks assigned to each physical disk are more balanced, and the drop disk tasks are cyclically executed in combination with the drop disk cycle task, so that the entire target disk group is more stable and the performance is more balanced, thereby improving the efficiency and stability of data drop disk.

[0066] In this embodiment, a method for storing data on a disk is provided, which can be used for notebook computers, desktop computers, etc. Figure 3 is a flow chart of a method for storing data on a disk according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0067] Step S301, polling the dirty cache quantity corresponding to each physical disk in the target disk group in the cache.

[0068] The number of dirty caches is the number of cache data that has been written to the cache but has not been flushed to the corresponding physical disk.

[0069] For details, please see Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0070] Step S302: for any physical disk in the target disk group, trigger a disk flushing cycle task of a priority level according to the number of dirty caches corresponding to the physical disk.

[0071] The priority level is related to the number of dirty caches. Optionally, in this embodiment, the priority level includes a high priority and a low priority, and a target number of priority levels can be set according to actual needs, for example, 6 priority levels can be set. Figure 4 is a schematic diagram of data storage processing according to an embodiment of the present invention. Figure 5 The figure is a schematic diagram of quota management for data flushing to disk according to an embodiment of the present invention.

[0072] Specifically, the above step S302 includes:

[0073] Step S3021, when the number of dirty caches corresponding to the target physical disk exceeds the high priority disk flushing start line, the high priority disk flushing cycle task is triggered to put the dirty cache data into the high priority disk flushing task queue.

[0074] The target physical disk is any physical disk in the target disk group. Step S3021 is executed for each physical disk in the target disk group. Figure 5 The current Dirty-CL number of a single PD in refers to the number of dirty caches currently corresponding to a single physical disk.

[0075] The high priority disk drop cycle tasks include:

[0076] S1: When the number of dirty caches corresponding to the target physical disk exceeds the high priority flushing start line, dirty cache data equal to the target flushing amount is put into the high priority flushing task queue to wait for the nth round of data flushing operation, where n is a positive integer. The target flushing amount corresponds to the target physical disk, that is, the corresponding flushing amount is set for each physical disk, corresponding Figure 4 In the disk flushing task queue, you can set the disk flushing amount according to the importance of the physical disk function and daily workload. When the physical disk work changes, you can also update the disk flushing amount in time. Figure 4 The high queue in the high priority disk drop start line is Figure 5 High priority Flush start line in.

[0077] S2: After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the flushing stop line, the dirty cache data equal to the target flushing amount will be put into the high priority flushing task queue to wait for the n+1th round of data flushing. The flushing stop line is lower than the total flushing start line. Figure 5 The FLUSH (same concept as Flush) stop line in.

[0078] Repeat S1 to S2 until the number of dirty caches corresponding to the target physical disk is lower than the drop-to-disk stop line, and then stop the high-priority drop-to-disk cycle task. That is, after each round of data drop-to-disk operation, if the number of dirty caches corresponding to the target physical disk exceeds the drop-to-disk stop line, dirty cache data equal to the target flush amount will be put into the high-priority drop-to-disk task queue, waiting for the next round of data drop-to-disk operation until the number of dirty caches corresponding to the target physical disk is lower than the drop-to-disk stop line.

[0079] It should be noted that the disk flushing in the present invention refers to the processing of dirty cache data in the present invention, and the disk flushing generally refers to the reading and writing of each physical disk.

[0080] Step S3022: When the number of dirty caches corresponding to the target physical disk exceeds the low priority disk write start line, the low priority disk write cycle task is triggered to put the dirty cache data into the low priority disk write task queue.

[0081] The target physical disk is any physical disk in the target disk group, and step S3022 is performed for any physical disk in the target disk group. The low priority disk drop start line is lower than the high priority disk drop start line. The low priority disk drop start line corresponds to Figure 5 The low-priority Flush start line in the low-priority flush task queue corresponds to Figure 4 The low queue in .

[0082] The low priority disk drop cycle tasks include:

[0083] A1: When the number of dirty caches corresponding to the target physical disk exceeds the low-priority disk flushing start line, dirty cache data equal to the target disk flushing amount is put into the low-priority disk flushing task queue to wait for the nth round of data flushing operation.

[0084] A2: After the nth round of data drop operation, if the number of dirty caches corresponding to the target physical disk exceeds the high priority drop start line, the dirty cache data of the target flush amount will be put into the high priority drop task queue; if the number of dirty caches corresponding to the target physical disk is lower than the high priority drop start line and exceeds the drop stop line, the dirty cache data of the target flush amount will be put into the low priority drop task queue to wait for the n+1th round of data drop operation. In other words, after the nth round of data drop operation, if the number of dirty caches corresponding to the target physical disk increases suddenly to exceed the high priority drop start line, the priority level of the target physical disk is increased to high priority, and the dirty cache data of the target flush amount will be put into the high priority drop task queue; if the number of dirty caches corresponding to the target physical disk still does not exceed the high priority drop start line but has not decreased to less than the drop stop line, the priority level of the target physical disk is kept at low priority, and the dirty cache data of the target flush amount will be put into the low priority drop task queue.

[0085] Repeat A1 to A2 until the number of dirty caches corresponding to the target physical disk is lower than the drop-to-disk stop line, and then stop the low-priority drop-to-disk cycle task. That is, after each round of data drop-to-disk operation, if the number of dirty caches corresponding to the target physical disk exceeds the high-priority drop-to-disk start line, the dirty cache data equal to the target flush amount will be put into the high-priority drop-to-disk task queue; if the number of dirty caches corresponding to the target physical disk is lower than the high-priority drop-to-disk start line and exceeds the drop-to-disk stop line, the dirty cache data equal to the target flush amount will be put into the low-priority drop-to-disk task queue, waiting for the next round of data drop-to-disk operation until the number of dirty caches corresponding to the target physical disk is lower than the drop-to-disk stop line.

[0086] Optionally, if the total number of dirty caches corresponding to each physical disk in the target disk group (total number of dirty caches) increases suddenly within the target time, a total disk flushing task is set. Specifically, when the total number of dirty caches corresponding to each physical disk in the target disk group exceeds the total disk flushing start line ( Figure 5 (not shown in the figure), for each physical disk in the target disk group, the dirty cache data of the corresponding flushing amount is put into the total disk drop task queue to wait for the nth round of data drop operation. Among them, the total drop start line is lower than the low priority drop start line, the priority level of the total drop task queue is lower than the low priority drop task queue, and the total drop task queue corresponds to Figure 4 The total number of dirty caches corresponding to each physical disk in the target disk group corresponds to Figure 4 The current Dirty-CL number of all PDs in the target disk group. The flushing amount corresponding to each physical disk is set according to actual needs. The total disk drop task is not a cyclic task, that is, when the total number of dirty caches corresponding to each physical disk in the target disk group exceeds the total disk drop start line, the total disk drop task is executed once and the current total disk drop task is stopped until the next time the total number of dirty caches corresponding to each physical disk in the target disk group exceeds the total disk drop start line, the next total disk drop task is executed.

[0087] It should be noted that the high-priority disk placement cycle task, the low-priority disk placement cycle task and the total disk placement task correspond to Figure 4 CL quota management in .

[0088] Optionally, a special disk drop task can be set for the case where one or some physical disks in the target disk group suddenly lose power or are unplugged. Specifically, when it is detected that the target physical disk loses power or is unplugged (for example, the hardware is equipped with a power monitoring circuit, and when a power outage or voltage abnormality is detected on the target physical disk, it is determined as a power drop; the hardware determines whether the target physical disk is unplugged by detecting the change in the physical connection status between the target physical disk and the host or the interface signal), the dirty cache data corresponding to the target physical disk is delivered to the special disk drop task queue to wait for the nth round of data drop operation. The priority level of the special disk drop task queue is higher than that of the high-priority disk drop task queue, and the special disk drop task queue corresponds to Figure 4 That is to say, when the target physical disk triggers a special drop task, all dirty cache data corresponding to the target physical disk will be delivered to the special drop task queue, and the dirty cache data will be delivered at one time without considering the amount of flushing corresponding to the target physical disk. The hardware will automatically trigger the special drop task, and there is no need for software to participate in detecting power failure or unplugging the disk, which can minimize the loss of data on the target physical disk. Figure 4 Special scenes in.

[0089] Optionally, in order to solve the situation where the number of dirty caches corresponding to some physical disks does not trigger the high-priority disk flushing cycle task and the low-priority disk flushing cycle task, set the disk flushing time management. Figure 6 It is a flowchart of disk flushing time management according to an embodiment of the present invention, wherein the timeout value Time_out_value corresponds to the timeout duration, PD_i is the i-th physical disk, Start_time is the timing start time, Cur_time is the current time, and Cur_time-Start_time is the timing duration. Specifically, each physical disk is polled according to a preset time period, wherein each physical disk is set with a corresponding timeout duration, and the timeout duration of each physical disk can be set according to the functional type and workload of the physical disk. When it is detected that the number of dirty caches corresponding to the target physical disk is between the zero waterline and the stop line of the disk flushing, the timeout disk flushing task is triggered, and timing for the target physical disk is started. The zero waterline of the disk flushing is lower than the stop line, and the zero waterline of the disk flushing corresponds to Figure 5 When the duration of timing for the target physical disk exceeds the timeout duration corresponding to the target physical disk, the dirty cache data corresponding to the target physical disk, which is the target flushing amount, is delivered to the timeout disk drop task queue to wait for the nth round of data drop operation. Among them, the priority level of the timeout disk drop task queue is lower than that of the total disk drop task queue. The timeout disk drop task queue corresponds to Figure 4 TO (time out) queue in. When executing the timeout disk drop task for the target physical disk, the target physical disk will no longer be detected to avoid repeatedly triggering the timeout disk drop task for the target physical disk. It should be noted that when the number of dirty caches of the target physical disk is between the low priority start line and the stop line, you can wait for the total number of dirty caches corresponding to each physical disk in the target disk group to exceed the total disk drop start line to trigger the total disk drop task or the number of dirty caches corresponding to the target physical disk exceeds the low priority disk drop start line to trigger the low priority disk drop cycle task.

[0090] It should be noted that the high-priority disk drop cycle task, low-priority disk drop cycle task, total disk drop task and disk flushing time management process involve Figure 4The disk flushing volume management shown in the figure, specifically, when performing disk flushing volume management, a corresponding basic disk flushing volume is allocated to each physical disk, and then a corresponding disk flushing volume is allocated to each high-priority disk drop cycle task, a low-priority disk drop cycle task, a total disk drop task, and an overtime disk drop task. Exemplarily, a disk flushing coefficient is allocated to each priority level of the disk drop task, and the final disk flushing volume of the target physical disk is the product of the basic disk flushing volume and the disk flushing coefficient of the corresponding priority level. For example, the basic disk flushing volume of the target physical disk is 5, and the target physical disk is determined to be a high priority. The disk flushing coefficient corresponding to the high priority is 1.2, and the final disk flushing volume of the target physical disk is 6. The disk flushing volume will also be updated in real time according to the writing status of the front-end IO (change in the number of dirty caches), the execution status of the back-end IO (data drop operation status), etc. The disk flushing volume management can solve the problem of disk-side IO quantity and bandwidth limitations. Executing the disk drop task according to the disk flushing volume to generate the back-end IO can prevent a large amount of disk test IO from accumulating, thereby leaving processing space for the background IO generated by other subsequent foreground IOs. High-priority disk placement task queues, low-priority disk placement task queues, total disk placement task queues, special disk placement task queues, and timed disk placement task queues all participate in disk flushing task scheduling. Optionally, a weight-configurable WRR arbitrator is used to arbitrate each disk placement task queue according to priority level to schedule disk flushing tasks.

[0091] Step S303, according to the priority level of the disk placement cycle task corresponding to each physical disk in the target disk group, schedule each disk placement task and perform data placement operation to place the dirty cache data corresponding to the disk placement task on the corresponding physical disk.

[0092] Exemplarily, the order of priority levels from high to low is special disk drop task queue, high priority disk drop task queue, low priority disk drop task queue, total disk drop task queue and timeout disk drop task queue. In actual application scenarios, the order of priority levels can also be adjusted according to needs. For example, if the requirement is to process the timeout disk drop task as soon as possible, the priority level of the timeout disk drop task queue can be increased.

[0093] Optionally, back-end IO processing is performed on dirty cache data that needs to be written to disk, further improving the performance and efficiency of the disk-side physical link. Figure 7 Schematic diagram of backend IO processing according to an embodiment of the present invention. Specifically, each dirty cache data (eg Figure 7 The dirty cache queue WB_CL_0 to WB_CL_5 in the WB_CL_Queue in the write buffer chain are linked in the order of generation time (for example Figure 7Before sorting, the dirty cache data on each physical disk are sorted according to the size of the logical block address (LBA). Figure 7 After sorting, the order is LBA=0, LBA=4, LBA=5, LBA=8, LBA=11, LBA=12, and the order of dirty cache data becomes WB_CL_0, WB_CL_5, WB_CL_2, WB_CL_3, WB_CL_1, WB_CL_4), and the dirty cache data with continuous logical block addresses are marked (for example Figure 7 In the example, LBA=4 and LBA=5 are continuous and marked in yellow. For each dirty cache data, the continuous blocks in the dirty cache data are aggregated (for example, Figure 7 If there is data in Blk3-Blk9 and Blk15 of WB_CL_0 in the chain, and Blk3-Blk9 are continuous, then aggregation processing is performed. Finally, in the dirty cache data Flush_IO_0 after aggregation processing, there is data with address start_address0 corresponding to the aggregation of Blk3-Blk9 and data with address start_address1 corresponding to Blk15). Next, aggregate the dirty cache data with continuous logical block addresses to obtain the aggregated dirty cache data (for example, if LBA=11 and LBA=12 are continuous, the corresponding WB_CL_1 and WB_CL_4 are aggregated into Flush_IO_3). Finally, starting from the head of the chain, the aggregated dirty cache data of the corresponding flushing amount is removed and placed on the corresponding physical disk in sequence.

[0094] The data drop method provided in this embodiment first polls the number of dirty caches corresponding to each physical disk in the target disk group in the cache, and then triggers the drop disk cycle task with a priority level according to the number of dirty caches corresponding to the physical disk for any physical disk in the target disk group. Finally, according to the priority level of the drop disk cycle task corresponding to each physical disk in the target disk group, each drop disk task is scheduled and the data drop disk operation is performed to drop the dirty cache data corresponding to the drop disk task to the corresponding physical disk. By setting the drop disk cycle task with a priority level corresponding to the number of dirty caches, the drop disk task corresponding to each physical disk in the target disk group can be prioritized, so that the drop disk tasks assigned to each physical disk are more balanced, and the drop disk tasks are cyclically executed in combination with the drop disk cycle task, so that the entire target disk group is more stable and the performance is more balanced, thereby improving the efficiency and stability of data drop disk.

[0095] In this embodiment, a data disk device is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0096] This embodiment provides a data storage device, such as Figure 8 As shown, including:

[0097] The polling module 801 is used to poll the dirty cache quantity corresponding to each physical disk in the target disk group in the cache;

[0098] The task triggering module 802 is used to trigger a disk drop cycle task of a priority level for any physical disk in the target disk group according to the number of dirty buffers corresponding to the physical disk; the priority level is related to the number of dirty buffers;

[0099] The disk placement module 803 is used to schedule each disk placement task and perform data placement operations according to the priority level of the disk placement cycle task corresponding to each physical disk in the target disk group, so as to place the dirty cache data corresponding to the disk placement task to the corresponding physical disk.

[0100] In an optional implementation, the priority level includes a high priority and a low priority; the task triggering module is further used to:

[0101] When the number of dirty caches corresponding to the target physical disk exceeds the high-priority disk drop start line, the high-priority disk drop cycle task is triggered to put the dirty cache data into the high-priority disk drop task queue;

[0102] When the number of dirty caches corresponding to the target physical disk exceeds the low priority disk drop start line, the low priority disk drop cycle task is triggered to put the dirty cache data into the low priority disk drop task queue; wherein the low priority disk drop start line is lower than the high priority disk drop start line.

[0103] In an optional implementation, the high priority disk placement cycle task includes:

[0104] When the number of dirty caches corresponding to the target physical disk exceeds the high-priority disk drop start line, dirty cache data equal to the target disk flushing amount is put into the high-priority disk drop task queue to wait for the nth round of data drop operation; the target disk flushing amount corresponds to the target physical disk;

[0105] After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the high priority flushing task queue to wait for the n+1th round of data flushing; the flushing stop line is lower than the total flushing start line;

[0106] Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk drop stop line, and then stop the high priority disk drop cycle task.

[0107] In an optional implementation, the low priority disk placement cycle task includes:

[0108] When the number of dirty caches corresponding to the target physical disk exceeds the low-priority disk flushing start line, dirty cache data equal to the target disk flushing amount is put into the low-priority disk flushing task queue to wait for the nth round of data flushing operation;

[0109] After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount will be put into the high-priority flushing task queue; if the number of dirty caches corresponding to the target physical disk is lower than the high-priority flushing start line and exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the low-priority flushing task queue to wait for the n+1th round of data flushing;

[0110] Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk drop stop line, and then stop the low priority disk drop cycle task.

[0111] In an optional embodiment, the device further includes a total plate placement task module, which is used to:

[0112] When the total number of dirty caches corresponding to the physical disks in the target disk group exceeds the total disk drop start line, for each physical disk in the target disk group, the dirty cache data of the corresponding disk flushing amount will be put into the total disk drop task queue to wait for the nth round of data drop operation; wherein, the total disk drop start line is lower than the low priority disk drop start line; the priority level of the total disk drop task queue is lower than that of the low priority disk drop task queue.

[0113] In an optional implementation, the device further includes a timeout disk drop task module, which is used to:

[0114] Polling each physical disk according to a preset time period; wherein each physical disk is set with a corresponding timeout duration;

[0115] When it is detected that the number of dirty caches corresponding to the target physical disk is between the zero watermark and the stop line of the disk placement, the timing for the target physical disk is started; the zero watermark of the disk placement is lower than the stop line;

[0116] When the timing duration for the target physical disk exceeds the timeout duration corresponding to the target physical disk, the dirty cache data corresponding to the target physical disk, whose quantity is the target disk flushing quantity, will be delivered to the timeout disk drop task queue to wait for the nth round of data drop operation; wherein, the priority level of the timeout disk drop task queue is lower than that of the total disk drop task queue.

[0117] In an optional embodiment, the device further includes a special plate placement task module, which is used to:

[0118] When it is detected that the target physical disk has lost power or been unplugged, the dirty cache data corresponding to the target physical disk is delivered to a special disk drop task queue to wait for the nth round of data drop operation; the priority level of the special disk drop task queue is higher than the high priority disk drop task queue.

[0119] In an optional embodiment, the device further includes an aggregation module, configured to:

[0120] The dirty cache data waiting to be written to the disk are mounted into the write buffer chain in the order of generation time; wherein, the dirty cache data are mounted in the order from the head to the tail of the write buffer chain;

[0121] For each dirty cache data on the write buffer chain, sort each dirty cache data corresponding to each physical disk according to the size of the logical block address, and mark the dirty cache data with continuous logical block addresses;

[0122] For each dirty cache data, aggregate the continuous blocks in the dirty cache data;

[0123] Aggregate dirty cache data with continuous logical block addresses to obtain aggregated dirty cache data;

[0124] Starting from the head of the chain, the dirty cache data after aggregation processing corresponding to the disk flushing amount is removed and stored in the corresponding physical disks in sequence.

[0125] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0126] The data disk placement device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0127] The embodiment of the present invention also provides a computer device having the above Figure 8 The data disk-dropping device shown.

[0128] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig. 9 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0129] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0130] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0131] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0132] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0133] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0134] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0135] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0136] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0137] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations shall all fall within the scope of protection of the present invention.

Claims

1. A method for storing data on a disk, characterized in that: The method comprises: Poll the number of dirty caches corresponding to each physical disk in the target disk group in the cache; For any physical disk in the target disk group, triggering a disk flushing cycle task of a priority level according to the number of dirty caches corresponding to the physical disk; the priority level is related to the number of dirty caches; According to the priority level of the disk placement cycle task corresponding to each physical disk in the target disk group, each disk placement task is scheduled and a data disk placement operation is performed to place the dirty cache data corresponding to the disk placement task on the corresponding physical disk.

2. The method according to claim 1, characterized in that The priority levels include high priority and low priority; the disk flushing cycle task of the priority level is triggered according to the number of dirty caches corresponding to the physical disk, including: When the number of dirty caches corresponding to the target physical disk exceeds the high-priority disk drop start line, the high-priority disk drop cycle task is triggered to put the dirty cache data into the high-priority disk drop task queue; When the number of dirty caches corresponding to the target physical disk exceeds the low priority disk drop start line, the low priority disk drop cycle task is triggered to put the dirty cache data into the low priority disk drop task queue; wherein the low priority disk drop start line is lower than the high priority disk drop start line.

3. The method according to claim 2, characterized in that The high priority disk placement cycle tasks include: When the dirty cache quantity corresponding to the target physical disk exceeds the high priority disk drop start line, dirty cache data equal to the target disk flushing quantity is put into the high priority disk drop task queue to wait for the nth round of data drop operation; the target disk flushing quantity corresponds to the target physical disk; After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the high priority flushing task queue to wait for the n+1th round of data flushing; the flushing stop line is lower than the total flushing start line; Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk drop stop line, and then stop the high priority disk drop cycle task.

4. The method according to claim 3, characterized in that: The low priority disk placement cycle tasks include: When the number of dirty caches corresponding to the target physical disk exceeds the low-priority disk flushing start line, dirty cache data equal to the target disk flushing amount is put into the low-priority disk flushing task queue to wait for the nth round of data flushing operation; After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount will be put into the high-priority flushing task queue; if the number of dirty caches corresponding to the target physical disk is lower than the high-priority flushing start line and exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the low-priority flushing task queue to wait for the n+1th round of data flushing; Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk drop stop line, and then stop the low priority disk drop cycle task.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the total number of dirty caches corresponding to the physical disks in the target disk group exceeds the total disk drop start line, for each physical disk in the target disk group, the dirty cache data of the corresponding disk flushing amount will be put into the total disk drop task queue to wait for the nth round of data drop operation; wherein, the total disk drop start line is lower than the low priority disk drop start line; the priority level of the total disk drop task queue is lower than that of the low priority disk drop task queue.

6. The method according to claim 5, characterized in that The method further comprises: Polling each physical disk according to a preset time period; wherein each physical disk is set with a corresponding timeout duration; When it is detected that the number of dirty caches corresponding to the target physical disk is between the zero watermark and the stop line of the disk placement, the timing for the target physical disk is started; the zero watermark of the disk placement is lower than the stop line; When the timing duration for the target physical disk exceeds the timeout duration corresponding to the target physical disk, the dirty cache data corresponding to the target physical disk, whose quantity is the target disk flushing quantity, will be delivered to the timeout disk drop task queue to wait for the nth round of data drop operation; wherein, the priority level of the timeout disk drop task queue is lower than that of the total disk drop task queue.

7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When it is detected that the target physical disk has lost power or been unplugged, the dirty cache data corresponding to the target physical disk is delivered to a special disk drop task queue to wait for the nth round of data drop operation; the priority level of the special disk drop task queue is higher than that of the high priority disk drop task queue.

8. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The dirty cache data waiting to be written to the disk are mounted into the write buffer chain in the order of generation time; wherein, the dirty cache data are mounted in the order from the head to the tail of the write buffer chain; For each dirty cache data on the write buffer chain, sort each dirty cache data corresponding to each physical disk according to the size of the logical block address, and mark the dirty cache data with continuous logical block addresses; For each dirty cache data, aggregate the continuous blocks in the dirty cache data; Aggregate dirty cache data with continuous logical block addresses to obtain aggregated dirty cache data; Starting from the head of the chain, the dirty cache data after aggregation processing corresponding to the disk flushing amount is removed and stored in the corresponding physical disks in sequence.

9. A data disk device, characterized in that: The device comprises: A polling module, used to poll the dirty cache quantity corresponding to each physical disk in the target disk group in the cache; A task triggering module, for triggering a disk drop cycle task of a priority level for any physical disk in the target disk group according to the number of dirty caches corresponding to the physical disk; the priority level is related to the number of dirty caches; The disk placement module is used to schedule each disk placement task and perform data placement operations according to the priority level of the disk placement cycle task corresponding to each physical disk in the target disk group, so as to place the dirty cache data corresponding to the disk placement task to the corresponding physical disk.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data storage method according to any one of claims 1 to 8 by executing the computer instructions.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data storage method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the method for storing data in a disk according to any one of claims 1 to 8.

Images