Data storage method, device, computer equipment, storage medium and program product

Abstract

The present invention relates to the field of computer storage technology, and discloses a data disk write method, apparatus, computer equipment, storage medium, and program product. The method comprises: polling the number of dirty caches corresponding to each physical disk in a target disk group; for any physical disk in the target disk group, triggering a disk write cycle task with a priority level based on the number of dirty caches corresponding to the physical disk; the priority level is related to the number of dirty caches; and scheduling each disk write task and executing a data disk write operation according to the priority level of the disk write cycle task corresponding to each physical disk in the target disk group, so as to write the dirty cache data corresponding to the disk write task to the corresponding physical disk. The above scheme can improve the efficiency and stability of data disk write when implementing the data disk write function.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, and in particular to a method and apparatus for storing data on a disk, computer equipment, storage media, and program products. Background Art

[0002] Currently, storage IO acceleration chips utilize larger physical memory as on-chip cache for physical disks. The cache consists of multiple cache lines. For write-back (WB) IO write operations, the host writes data to the corresponding cache line in the cache. The data in the cache line becomes dirty data, completing the write response without writing to the physical disk. The actual flushing of data to disk is performed by hardware, reducing software resource usage.

[0003] In the related art, when executing the data drop operation through the hardware background, there is no priority scheduling for each data drop task, resulting in that more urgent data drop tasks cannot be processed with high priority. For example, a large number of data drop tasks triggered by a sudden increase in dirty data corresponding to a physical disk cannot be processed first, thereby occupying cache lines and blocking the normal execution of subsequent WB IO; processing the data drop tasks corresponding to the same physical disk at one time may take a long time due to the large number of data drop tasks corresponding to the physical disk, blocking the execution of other back-end IO (BIO) of the physical disk and causing imbalance in the back-end IO of other physical disks in the disk group, causing disk group performance fluctuations; the data drop tasks corresponding to some physical disks have low frequency and small number. If the task is triggered immediately, it will compete with other physical disks with a large number of tasks and high execution frequency for data drop resources, affecting task execution efficiency; it is impossible to actively identify when some physical disks lose power or are unplugged, and there is a risk of data loss.

[0004] Therefore, there is an urgent need for a more efficient and stable data storage method. Summary of the Invention

[0005] In view of this, an object of the present invention is to provide a method, apparatus, computer device, storage medium and program product for writing data to disk, so as to solve the problems of low efficiency and poor stability when executing the task of writing data to disk.

[0006] In a first aspect, the present invention provides a method for storing data on a disk, the method comprising:

[0007] Poll the number of dirty caches corresponding to each physical disk in the target disk group;

[0008] For any physical disk in the target disk group, triggering a disk flushing cycle task of a priority level according to the number of dirty buffers corresponding to the physical disk; the priority level is related to the number of dirty buffers;

[0009] According to the priority level of the disk flushing cycle task corresponding to each physical disk in the target disk group, each disk flushing task is scheduled and a data flushing operation is performed to flush the dirty cache data corresponding to the disk flushing task to the corresponding physical disk.

[0010] In an optional implementation, the priority levels include high priority and low priority; and triggering the disk flushing cycle task of the priority level according to the number of dirty caches corresponding to the physical disk includes:

[0011] When the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, the high-priority flushing cycle task is triggered to put the dirty cache data into the high-priority flushing task queue;

[0012] When the number of dirty caches corresponding to the target physical disk exceeds the low-priority disk write start line, the low-priority disk write cycle task is triggered to put the dirty cache data into the low-priority disk write task queue; wherein the low-priority disk write start line is lower than the high-priority disk write start line.

[0013] In an optional implementation, the high-priority disk flushing cycle task includes:

[0014] When the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount is put into the high-priority flushing task queue to wait for the nth round of data flushing operation; the target flushing amount corresponds to the target physical disk;

[0015] After the nth round of data flushing, if the number of dirty cached data corresponding to the target physical disk exceeds the flush stop line, dirty cached data equal to the target flush amount will be put into the high-priority flush task queue to wait for the n+1th round of data flushing. The flush stop line is lower than the total flush start line.

[0016] Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk flushing stop line, and then stop the high-priority disk flushing cycle task.

[0017] In an optional implementation, the low-priority disk flushing cycle task includes:

[0018] When the number of dirty caches corresponding to the target physical disk exceeds the low-priority flushing start line, dirty cache data equal to the target flushing amount will be put into the low-priority flushing task queue to wait for the nth round of data flushing operation;

[0019] After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount will be put into the high-priority flushing task queue. If the number of dirty caches corresponding to the target physical disk is lower than the high-priority flushing start line and exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the low-priority flushing task queue to wait for the n+1th round of data flushing.

[0020] Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk flushing stop line, and then stop the low-priority disk flushing cycle task.

[0021] In an optional embodiment, the method further includes:

[0022] When the total number of dirty caches corresponding to the physical disks in the target disk group exceeds the total disk drop start line, for each physical disk in the target disk group, the dirty cache data of the corresponding disk flushing amount will be put into the total disk drop task queue to wait for the nth round of data drop operation; wherein, the total disk drop start line is lower than the low-priority disk drop start line; the priority level of the total disk drop task queue is lower than that of the low-priority disk drop task queue.

[0023] In an optional embodiment, the method further includes:

[0024] Poll each physical disk according to a preset time period; each physical disk is set with a corresponding timeout period;

[0025] When it is detected that the number of dirty caches corresponding to the target physical disk is between the zero watermark and the stop line of the disk flushing, the timing for the target physical disk is started; the zero watermark of the disk flushing is lower than the stop line;

[0026] When the timing duration for the target physical disk exceeds the timeout duration corresponding to the target physical disk, the dirty cache data corresponding to the target physical disk, which is the target disk flushing amount, will be delivered to the timeout disk drop task queue to wait for the nth round of data disk drop operation; wherein, the priority level of the timeout disk drop task queue is lower than that of the total disk drop task queue.

[0027] In an optional embodiment, the method further includes:

[0028] When it is detected that the target physical disk has lost power or been unplugged, the dirty cache data corresponding to the target physical disk will be delivered to a special disk drop task queue to wait for the nth round of data drop operation; the priority level of the special disk drop task queue is higher than that of the high-priority disk drop task queue.

[0029] In an optional embodiment, the method further includes:

[0030] The dirty cache data waiting to be written to disk are added to the write buffer chain in the order of generation time; wherein, the dirty cache data are added in the order from the head to the end of the write buffer chain;

[0031] For each dirty cache data on the write buffer chain, sort the dirty cache data corresponding to each physical disk according to the logical block address size, and mark the dirty cache data with continuous logical block addresses;

[0032] For each dirty cache data, aggregate the continuous blocks in the dirty cache data;

[0033] Aggregate dirty cache data with continuous logical block addresses to obtain aggregated dirty cache data;

[0034] Starting from the chain head, the aggregated dirty cache data corresponding to the disk flushing amount is removed and written to the corresponding physical disks in sequence.

[0035] In a second aspect, the present invention provides a data storage device, comprising:

[0036] A polling module is used to poll the dirty cache count of each physical disk in the target disk group;

[0037] a task triggering module configured to trigger a disk flushing cycle task of a priority level for any physical disk in the target disk group according to the number of dirty buffers corresponding to the physical disk; the priority level is related to the number of dirty buffers;

[0038] The disk placement module is used to schedule each disk placement task and perform data placement operations according to the priority level of the disk placement cycle task corresponding to each physical disk in the target disk group, so as to place the dirty cache data corresponding to the disk placement task to the corresponding physical disk.

[0039] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data disk transfer method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data storage method of the first aspect or any corresponding embodiment thereof.

[0041] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the data storage method of the first aspect or any corresponding embodiment thereof.

[0042] The technical solution provided by the present invention can have the following beneficial effects:

[0043] The data drop method provided by the present invention first polls the number of dirty caches corresponding to each physical disk in the target disk group in the cache, then triggers a drop cycle task of a priority level according to the number of dirty caches corresponding to the physical disk for any physical disk in the target disk group, and finally schedules each drop task and executes a data drop operation according to the priority level of the drop cycle task corresponding to each physical disk in the target disk group, so as to drop the dirty cache data corresponding to the drop task to the corresponding physical disk. By setting a drop cycle task with a priority level corresponding to the number of dirty caches, it is possible to set a priority for the drop task of dirty cache data corresponding to each physical disk in the target disk group, so that the drop tasks assigned to each physical disk are more balanced, and the drop tasks are cyclically executed in combination with the drop cycle task, so that the entire target disk group is more stable and the performance is more balanced, thereby improving the efficiency and stability of data drop. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic diagram showing data storage in the related art is shown;

[0046] Figure 2 is a flowchart of a method for flushing data to a disk according to an embodiment of the present invention;

[0047] Figure 3 is a flowchart of another method for writing data to a disk according to an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of data storage processing according to an embodiment of the present invention;

[0049] Figure 5 2. This is a schematic diagram of data disk quota management according to an embodiment of the present invention;

[0050] Figure 6 is a schematic diagram of the process of disk flushing time management according to an embodiment of the present invention;

[0051] Figure 7 is a schematic diagram of backend IO processing according to an embodiment of the present invention;

[0052] Figure 8 is a structural block diagram of a data storage device according to an embodiment of the present invention;

[0053] Figure 9 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0055] Currently, storage IO acceleration chips (Storage IO Acceleration Chips) use physical memory (Physical Memory) with larger storage space as on-chip cache (Cache) of physical disks (Physical Disks, PDs). Figure 1 A schematic diagram of data storage in related technologies is shown. Figure 1 As shown in the figure, the cache consists of multiple cache lines (CL). For the write operation of WB (Write Back) IO, the host writes the data to the corresponding cache line in the cache and then the data in the cache line is converted into dirty data (clean-CL becomes dirty-CL). At this time, there is no need to write to the physical disk (for example Figure 1 The write response can be completed by reading PD_0 to PD_x in the memory. The actual data flush operation is completed by hardware to reduce software resource usage.

[0056] In the related art, when executing the data drop operation through the hardware background, there is no priority scheduling for each data drop task, resulting in that more urgent data drop tasks cannot be processed with high priority. For example, a large number of data drop tasks triggered by a sudden increase in dirty data corresponding to a physical disk cannot be processed first, thereby occupying cache lines and blocking the normal execution of subsequent WB_IO; processing the data drop tasks corresponding to the same physical disk at one time may take a long time due to the large number of data drop tasks corresponding to the physical disk, blocking the execution of other back-end IO (BIO) of the physical disk and causing imbalance in the back-end IO of other physical disks in the disk group, causing disk group performance fluctuations; the data drop tasks corresponding to some physical disks have low frequency and small number. If the task is triggered immediately, it will compete with other physical disks with a large number of tasks and high execution frequency for data drop resources, affecting task execution efficiency; it is impossible to actively identify when some physical disks lose power or are unplugged, and there is a risk of data loss.

[0057] According to an embodiment of the present invention, an embodiment of a method for storing data on a disk is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] In this embodiment, a method for writing data to a disk is provided, which can be used for notebook computers, desktop computers, etc. Figure 2 FIG. 1 is a flow chart of a method for storing data on a disk according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0059] Step S201: polling the dirty cache quantity corresponding to each physical disk in the target disk group.

[0060] In actual application scenarios, the target disk group contains multiple physical disks. The host sends the foreground IO corresponding to each physical disk in the target disk group to each cache line in the cache. After the foreground IO is stored in the cache line, the data in the cache line becomes dirty cache data. The number of dirty cache data is the amount of cache data that has been written to the cache but not flushed to the corresponding physical disk, i.e., the number of dirty cache data. When implementing the data flushing function, the present invention first polls the cache to see the number of dirty cache data corresponding to each physical disk.

[0061] Step S202 : For any physical disk in the target disk group, trigger a disk flushing cycle task of a priority level according to the number of dirty caches corresponding to the physical disk.

[0062] The priority level is related to the number of dirty caches. That is, after polling the number of dirty caches on the target physical disk, the dirty cache number is compared with a preset threshold to determine the priority level of the flushing task corresponding to the target physical disk. For example, if the number of dirty caches corresponding to the target physical disk is greater than the preset threshold, the priority level of the flushing task corresponding to the target physical disk is determined to be high, and the flushing task corresponding to the target physical disk is processed first. The flushing task can be a cyclic scheduling of the flushing tasks corresponding to each dirty cache data on the physical disk, for example, scheduling only a certain number of flushing tasks each time and looping multiple times, or setting a set number of loops and using the quotient of the number of dirty caches and the number of loops as the number of flushing tasks scheduled each time.

[0063] Step S203 , scheduling each disk flushing task according to the priority level of the disk flushing cycle task corresponding to each physical disk in the target disk group, and performing a data flushing operation to flush the dirty cache data corresponding to the disk flushing task to the corresponding physical disk.

[0064] After determining the priority level of the disk write cycle task corresponding to each physical disk in the target disk group according to the corresponding number of dirty caches, the disk write cycle tasks corresponding to each physical disk can be arranged according to the priority level, and then the disk write tasks corresponding to each dirty cache data can be scheduled in the order of arrangement to write each dirty cache data to the corresponding physical disk, thereby completing the process of writing the foreground IO to the permanent storage medium (physical disk).

[0065] The data drop method provided in this embodiment first polls the number of dirty caches corresponding to each physical disk in the target disk group in the cache, then triggers a drop task with a priority level according to the number of dirty caches corresponding to the physical disk for any physical disk in the target disk group, and finally schedules each drop task and executes the data drop operation according to the priority level of the drop task corresponding to each physical disk in the target disk group, so as to drop the dirty cache data corresponding to the drop task to the corresponding physical disk. By setting a drop task with a priority level corresponding to the number of dirty caches, it is possible to set a priority for the drop task of dirty cache data corresponding to each physical disk in the target disk group, so that the drop tasks assigned to each physical disk are more balanced, and the drop tasks are cyclically executed in combination with the drop task, so that the entire target disk group is more stable and the performance is more balanced, thereby improving the efficiency and stability of data drop.

[0066] In this embodiment, a method for writing data to a disk is provided, which can be used for notebook computers, desktop computers, etc. Figure 3 FIG. 1 is a flow chart of a method for storing data on a disk according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0067] Step S301: polling the dirty cache count corresponding to each physical disk in the target disk group.

[0068] The number of dirty caches is the number of cache data that has been written to the cache but has not yet been flushed to the corresponding physical disk.

[0069] For details, please see Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0070] Step S302 : For any physical disk in the target disk group, trigger a disk flushing cycle task of a priority level according to the number of dirty caches corresponding to the physical disk.

[0071] The priority level is related to the number of dirty caches. Optionally, in this embodiment, the priority level includes high priority and low priority, and a target number of priority levels can be set according to actual needs, for example, 6 priority levels. Figure 4 This is a schematic diagram of data disk processing according to an embodiment of the present invention. Figure 5 2 is a schematic diagram of data disk quota management according to an embodiment of the present invention.

[0072] Specifically, the above step S302 includes:

[0073] Step S3021: When the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, a high-priority flushing cycle task is triggered to put the dirty cache data into the high-priority flushing task queue.

[0074] The target physical disk is any physical disk in the target disk group. Step S3021 is executed for each physical disk in the target disk group. Figure 5 The current Dirty-CL number of a single PD refers to the number of dirty caches currently corresponding to a single physical disk.

[0075] The high-priority disk flushing cycle tasks include:

[0076] S1: When the number of dirty caches corresponding to the target physical disk exceeds the high priority flushing start line, the dirty cache data of the target flushing amount will be put into the high priority flushing task queue to wait for the nth round of data flushing operation, where n is a positive integer. The target flushing amount corresponds to the target physical disk, that is, the corresponding flushing amount is set for each physical disk. Figure 4 In the disk flushing task queue, you can set the disk flushing amount according to the importance of the physical disk function, daily workload, etc. When the physical disk work changes, you can also update the disk flushing amount in time. Figure 4 The high queue in the high priority disk drop start line is Figure 5 High priority Flush start line in.

[0077] S2: After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the high priority flushing task queue to wait for the n+1th round of data flushing. The flushing stop line is lower than the total flushing start line. Figure 5 The FLUSH (same concept as Flush) stops the line.

[0078] Repeat S1 to S2 until the number of dirty caches corresponding to the target physical disk falls below the drop-to-stop threshold, at which point the high-priority drop-to-disk task is terminated. That is, after each round of data drop-to-disk operations, if the number of dirty caches corresponding to the target physical disk exceeds the drop-to-stop threshold, dirty cache data equal to the target flush volume is placed in the high-priority drop-to-disk task queue, awaiting the next round of data drop-to-disk operations until the number of dirty caches corresponding to the target physical disk falls below the drop-to-stop threshold.

[0079] It should be noted that the disk flushing in the present invention refers to the processing of dirty cache data in the present invention, and the disk flushing generally refers to the reading and writing of each physical disk.

[0080] Step S3022: When the number of dirty caches corresponding to the target physical disk exceeds the low-priority flushing start line, a low-priority flushing cycle task is triggered to put the dirty cache data into the low-priority flushing task queue.

[0081] The target physical disk is any physical disk in the target disk group, and step S3022 is executed for any physical disk in the target disk group. The low priority disk drop start line is lower than the high priority disk drop start line. The low priority disk drop start line corresponds to Figure 5 The low-priority Flush start line in the low-priority flush task queue corresponds to Figure 4 The low queue in .

[0082] The low-priority disk flushing cycle tasks include:

[0083] A1: When the number of dirty caches corresponding to the target physical disk exceeds the low-priority flushing start line, dirty cache data equal to the target flushing amount is put into the low-priority flushing task queue to wait for the nth round of data flushing operation.

[0084] A2: After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount will be placed in the high-priority flushing task queue. If the number of dirty caches corresponding to the target physical disk is lower than the high-priority flushing start line but exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be placed in the low-priority flushing task queue to await the n+1th round of data flushing. In other words, after the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk suddenly increases to exceed the high-priority flushing start line, the priority level of the target physical disk will be raised to high priority, and dirty cache data equal to the target flushing amount will be placed in the high-priority flushing task queue. If the number of dirty caches corresponding to the target physical disk still does not exceed the high-priority flushing start line but has not decreased to below the flushing stop line, the priority level of the target physical disk will remain low priority, and dirty cache data equal to the target flushing amount will be placed in the low-priority flushing task queue.

[0085] Repeat A1 to A2 until the number of dirty caches corresponding to the target physical disk falls below the drop-to-disk stop line, at which point the low-priority drop-to-disk task is stopped. That is, after each round of data drop operations, if the number of dirty caches corresponding to the target physical disk exceeds the high-priority drop-to-disk start line, dirty cache data equal to the target flush amount will be placed in the high-priority drop-to-disk task queue. If the number of dirty caches corresponding to the target physical disk falls below the high-priority drop-to-disk start line and exceeds the drop-to-disk stop line, dirty cache data equal to the target flush amount will be placed in the low-priority drop-to-disk task queue, awaiting the next round of data drop operations until the number of dirty caches corresponding to the target physical disk falls below the drop-to-disk stop line.

[0086] Optionally, if the total dirty cache quantity (total dirty cache quantity) of each physical disk in the target disk group increases suddenly within the target time, a total disk flushing task is set. Specifically, when the total dirty cache quantity of each physical disk in the target disk group exceeds the total disk flushing start line ( Figure 5 (not shown in the figure), for each physical disk in the target disk group, the dirty cache data corresponding to the flush amount is put into the total disk flush task queue to wait for the nth round of data flush operation. Among them, the total flush start line is lower than the low priority flush start line, the priority level of the total flush task queue is lower than the low priority flush task queue, and the total flush task queue corresponds to Figure 4 The total number of dirty caches corresponding to each physical disk in the target disk group corresponds to Figure 4 The current Dirty-CL number of all PDs in the target disk group. The flushing amount corresponding to each physical disk is set according to actual needs. The total disk flushing task is not a cyclic task. That is, when the total number of dirty caches corresponding to each physical disk in the target disk group exceeds the total disk flushing start line, the total disk flushing task is executed once and then stopped. The next total disk flushing task is executed when the total number of dirty caches corresponding to each physical disk in the target disk group exceeds the total disk flushing start line.

[0087] It should be noted that the high priority disk drop cycle task, low priority disk drop cycle task and total disk drop task correspond to Figure 4 CL quota management in .

[0088] Optionally, a special disk drop task is set for the case where one or some physical disks in the target disk group suddenly lose power or are unplugged. Specifically, when it is detected that the target physical disk loses power or is unplugged (for example, the hardware is equipped with a power monitoring circuit, and it is determined to be a power drop when a power interruption or voltage abnormality is detected on the target physical disk; the hardware determines whether the target physical disk is unplugged by detecting the change in the physical connection status between the target physical disk and the host or the interface signal), the dirty cache data corresponding to the target physical disk is delivered to the special disk drop task queue to wait for the nth round of data drop operation. The priority level of the special disk drop task queue is higher than that of the high-priority disk drop task queue, and the special disk drop task queue corresponds to Figure 4 That is to say, when the target physical disk triggers a special drop task, all dirty cache data corresponding to the target physical disk will be delivered to the special drop task queue, completing the delivery of dirty cache data at one time, without considering the amount of flushing corresponding to the target physical disk, and the hardware will automatically trigger the special drop task, without the need for software to participate in detecting power failure or unplugging the disk, thus avoiding data loss on the target physical disk as much as possible. Special drop task corresponds to Figure 4 Special scenes in .

[0089] Optionally, to solve the problem that the number of dirty caches corresponding to some physical disks does not trigger the high-priority disk flushing cycle task and the low-priority disk flushing cycle task, set disk flushing time management. Figure 6 It is a flowchart of disk flushing time management according to an embodiment of the present invention, wherein the timeout value Time_out_value corresponds to the timeout duration, PD_i is the i-th physical disk, Start_time is the timing start time, Cur_time is the current time, and Cur_time-Start_time is the timing duration. Specifically, each physical disk is polled according to a preset time period, wherein each physical disk is set with a corresponding timeout duration, and the timeout duration of each physical disk can be set according to the functional type and workload of the physical disk. When it is detected that the number of dirty caches corresponding to the target physical disk is between the zero waterline and the stop line of the disk flushing, the timeout disk flushing task is triggered, and the timing for the target physical disk is started. The zero waterline of the disk flushing is lower than the stop line, and the zero waterline of the disk flushing corresponds to Figure 5 When the time duration for the target physical disk exceeds the timeout duration corresponding to the target physical disk, the dirty cache data corresponding to the target physical disk, which is the target flush amount, will be delivered to the timeout disk drop task queue to wait for the nth round of data drop operation. Among them, the priority level of the timeout disk drop task queue is lower than that of the total disk drop task queue. The timeout disk drop task queue corresponds to Figure 4 The TO (time out) queue in the target disk group. When executing the timeout disk drop task for the target physical disk, the target physical disk will no longer be checked to avoid repeatedly triggering the timeout disk drop task for the target physical disk. It should be noted that when the number of dirty caches of the target physical disk is between the low priority start line and the stop line, it is possible to wait for the total number of dirty caches corresponding to each physical disk in the target disk group to exceed the total disk drop start line to trigger the total disk drop task, or the number of dirty caches corresponding to the target physical disk exceeds the low priority disk drop start line to trigger the low priority disk drop cycle task.

[0090] It should be noted that the high-priority disk drop cycle task, low-priority disk drop cycle task, total disk drop task and disk flushing time management process involve Figure 4The disk flushing volume management shown, specifically, when performing disk flushing volume management, a corresponding basic disk flushing volume is allocated to each physical disk, and then corresponding disk flushing volumes are allocated to high-priority disk drop cycle tasks, low-priority disk drop cycle tasks, total disk drop tasks, and timed disk drop tasks. For example, a disk flushing coefficient is allocated to each priority level of disk drop tasks, and the final disk flushing volume of the target physical disk is the product of the basic disk flushing volume and the disk flushing coefficient of the corresponding priority level. For example, the basic disk flushing volume of the target physical disk is 5, and the target physical disk is judged to be high priority. The disk flushing coefficient corresponding to the high priority is 1.2, then the final disk flushing volume of the target physical disk is 6. The disk flushing volume will also be updated in real time based on the writing status of the front-end IO (changes in the number of dirty caches), the execution status of the back-end IO (data drop operation status), etc. The disk flushing volume management can solve the problem of disk-side IO number and bandwidth limitations. Executing the disk drop task according to the disk flushing volume to generate back-end IO can prevent a large amount of disk-test IO from accumulating, thereby leaving processing space for the background IO generated by other subsequent foreground IO. High-priority flushing task queues, low-priority flushing task queues, general flushing task queues, special flushing task queues, and timed flushing task queues all participate in the disk flushing task scheduling. Optionally, a weight-configurable WRR arbiter arbitrates each flushing task queue according to its priority level to schedule the flushing task.

[0091] Step S303 , according to the priority level of the disk flushing cycle task corresponding to each physical disk in the target disk group, schedule each disk flushing task and execute the data flushing operation to flush the dirty cache data corresponding to the disk flushing task to the corresponding physical disk.

[0092] For example, the order of priority levels from high to low is special disk drop task queue, high priority disk drop task queue, low priority disk drop task queue, total disk drop task queue and timeout disk drop task queue. In actual application scenarios, the order of priority levels can also be adjusted according to needs. For example, if the requirement is to process the timeout disk drop task as soon as possible, the priority level of the timeout disk drop task queue can be increased.

[0093] Optionally, back-end IO processing is performed on dirty cache data that needs to be written to disk to further improve the performance and efficiency of the disk-side physical link. Figure 7 Schematic diagram of back-end IO processing according to an embodiment of the present invention. Specifically, each dirty cache data (eg Figure 7 The dirty cache queues WB_CL_0 to WB_CL_5 in the WB_CL_Queue are linked to the write buffer chain (e.g. Figure 7Before sorting, the write buffer chains are mounted in the order of WB_CL_0 to WB_CL_5. Each physical disk corresponds to a write buffer chain. For each dirty cache data on the write buffer chain, the dirty cache data corresponding to each physical disk is sorted according to the size of the logical block address (LBA). Figure 7 After sorting, the order is LBA=0, LBA=4, LBA=5, LBA=8, LBA=11, LBA=12, and the order of dirty cache data becomes WB_CL_0, WB_CL_5, WB_CL_2, WB_CL_3, WB_CL_1, WB_CL_4), and the dirty cache data with continuous logical block addresses are marked (for example Figure 7 In the example, LBA=4 and LBA=5 are continuous and marked in yellow. For each dirty cache data, the continuous blocks in the dirty cache data are aggregated (for example, Figure 7 There is data in Blk3-Blk9 and Blk15 of WB_CL_0 in the chain. If Blk3-Blk9 are continuous, they are aggregated. Finally, in the aggregated dirty cache data Flush_IO_0, there is data at the address start_address0 corresponding to the aggregation of Blk3-Blk9 and data at the address start_address1 corresponding to Blk15. Next, the dirty cache data with continuous logical block addresses are aggregated to obtain the aggregated dirty cache data (for example, if LBA=11 and LBA=12 are continuous, the corresponding WB_CL_1 and WB_CL_4 are aggregated into Flush_IO_3). Finally, starting from the head of the chain, the aggregated dirty cache data corresponding to the flush amount is removed and placed on the corresponding physical disk in sequence.

[0094] The data drop method provided in this embodiment first polls the number of dirty caches corresponding to each physical disk in the target disk group in the cache, then triggers a drop task with a priority level according to the number of dirty caches corresponding to the physical disk for any physical disk in the target disk group, and finally schedules each drop task and executes the data drop operation according to the priority level of the drop task corresponding to each physical disk in the target disk group, so as to drop the dirty cache data corresponding to the drop task to the corresponding physical disk. By setting a drop task with a priority level corresponding to the number of dirty caches, it is possible to set a priority for the drop task of dirty cache data corresponding to each physical disk in the target disk group, so that the drop tasks assigned to each physical disk are more balanced, and the drop tasks are cyclically executed in combination with the drop task, so that the entire target disk group is more stable and the performance is more balanced, thereby improving the efficiency and stability of data drop.

[0095] This embodiment also provides a data storage device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0096] This embodiment provides a data storage device, such as Figure 8 As shown, including:

[0097] Polling module 801, used to poll the dirty cache quantity corresponding to each physical disk in the target disk group;

[0098] The task triggering module 802 is configured to trigger a disk flushing cycle task of a priority level for any physical disk in the target disk group according to the number of dirty buffers corresponding to the physical disk; the priority level is related to the number of dirty buffers;

[0099] The disk placement module 803 is used to schedule each disk placement task and perform data placement operations according to the priority level of the disk placement cycle task corresponding to each physical disk in the target disk group, so as to place the dirty cache data corresponding to the disk placement task to the corresponding physical disk.

[0100] In an optional implementation, the priority level includes a high priority and a low priority; the task triggering module is further configured to:

[0101] When the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, the high-priority flushing cycle task is triggered to put the dirty cache data into the high-priority flushing task queue;

[0102] When the number of dirty caches corresponding to the target physical disk exceeds the low-priority disk write start line, the low-priority disk write cycle task is triggered to put the dirty cache data into the low-priority disk write task queue; wherein the low-priority disk write start line is lower than the high-priority disk write start line.

[0103] In an optional implementation, the high-priority disk flushing cycle task includes:

[0104] When the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount is put into the high-priority flushing task queue to wait for the nth round of data flushing operation; the target flushing amount corresponds to the target physical disk;

[0105] After the nth round of data flushing, if the number of dirty cached data corresponding to the target physical disk exceeds the flush stop line, dirty cached data equal to the target flush amount will be put into the high-priority flush task queue to wait for the n+1th round of data flushing. The flush stop line is lower than the total flush start line.

[0106] Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk flushing stop line, and then stop the high-priority disk flushing cycle task.

[0107] In an optional implementation, the low-priority disk flushing cycle task includes:

[0108] When the number of dirty caches corresponding to the target physical disk exceeds the low-priority flushing start line, dirty cache data equal to the target flushing amount will be put into the low-priority flushing task queue to wait for the nth round of data flushing operation;

[0109] After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount will be put into the high-priority flushing task queue. If the number of dirty caches corresponding to the target physical disk is lower than the high-priority flushing start line and exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the low-priority flushing task queue to wait for the n+1th round of data flushing.

[0110] Repeat the above steps until the number of dirty caches corresponding to the target physical disk falls below the disk flushing stop line, and then stop the low-priority disk flushing cycle task.

[0111] In an optional embodiment, the device further includes a total disk placement task module, which is used to:

[0112] When the total number of dirty caches corresponding to the physical disks in the target disk group exceeds the total disk drop start line, for each physical disk in the target disk group, the dirty cache data of the corresponding disk flushing amount will be put into the total disk drop task queue to wait for the nth round of data drop operation; wherein, the total disk drop start line is lower than the low-priority disk drop start line; the priority level of the total disk drop task queue is lower than the low-priority disk drop task queue.

[0113] In an optional embodiment, the device further includes a timeout disk drop task module, which is configured to:

[0114] Poll each physical disk according to a preset time period; each physical disk is set with a corresponding timeout period;

[0115] When it is detected that the number of dirty caches corresponding to the target physical disk is between the zero watermark and the stop line of the disk flush, the timing for the target physical disk is started; the zero watermark of the disk flush is lower than the stop line;

[0116] When the timing duration for the target physical disk exceeds the timeout duration corresponding to the target physical disk, the dirty cache data corresponding to the target physical disk, which is the target disk flushing amount, will be delivered to the timeout disk drop task queue to wait for the nth round of data drop operation; wherein, the priority level of the timeout disk drop task queue is lower than that of the total disk drop task queue.

[0117] In an optional embodiment, the device further includes a special placement task module for:

[0118] When it is detected that the target physical disk has lost power or been unplugged, the dirty cache data corresponding to the target physical disk is delivered to a special disk drop task queue to wait for the nth round of data drop operation; the priority level of the special disk drop task queue is higher than the high-priority disk drop task queue.

[0119] In an optional embodiment, the device further includes an aggregation module, configured to:

[0120] The dirty cache data waiting to be written to disk are added to the write buffer chain in the order of generation time; wherein, the dirty cache data are added in the order from the head to the end of the write buffer chain;

[0121] For each dirty cache data on the write buffer chain, sort the dirty cache data corresponding to each physical disk according to the logical block address size, and mark the dirty cache data with continuous logical block addresses;

[0122] For each dirty cache data, aggregate the continuous blocks in the dirty cache data;

[0123] Aggregate dirty cache data with continuous logical block addresses to obtain aggregated dirty cache data;

[0124] Starting from the chain head, the aggregated dirty cache data corresponding to the disk flushing amount is removed and written to the corresponding physical disks in sequence.

[0125] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0126] The data disk-dropping device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0127] The embodiment of the present invention also provides a computer device having the above Figure 8 The data disk device shown.

[0128] See also Figure 9 , Figure 9 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 9 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 9 A processor 10 is taken as an example.

[0129] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0130] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0131] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0132] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0133] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 9 The bus connection is taken as an example.

[0134] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0135] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0136] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0137] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations shall fall within the scope of protection of the present invention.

Claims

1. A method for storing data on a disk, characterized in that: The method comprises: Poll the number of dirty caches corresponding to each physical disk in the target disk group; For any physical disk in the target disk group, triggering a disk flushing cycle task of a priority level according to the number of dirty buffers corresponding to the physical disk; the priority level is related to the number of dirty buffers; According to the priority level of the disk flushing cycle task corresponding to each physical disk in the target disk group, each disk flushing task is scheduled and a data flushing operation is performed to flush the dirty cache data corresponding to the disk flushing task to the corresponding physical disk; The priority levels include high priority and low priority; triggering the disk flushing cycle task of the priority level according to the number of dirty caches corresponding to the physical disk includes: When the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, the high-priority flushing cycle task is triggered to put the dirty cache data into the high-priority flushing task queue; When the number of dirty caches corresponding to the target physical disk exceeds the low-priority disk write start line, the low-priority disk write cycle task is triggered to put the dirty cache data into the low-priority disk write task queue; wherein the low-priority disk write start line is lower than the high-priority disk write start line.

2. The method according to claim 1, characterized in that The high-priority disk flushing cycle tasks include: When the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount is put into the high-priority flushing task queue to wait for the nth round of data flushing operation; the target flushing amount corresponds to the target physical disk; After the nth round of data flushing, if the number of dirty cached data corresponding to the target physical disk exceeds the flush stop line, dirty cached data equal to the target flush amount will be put into the high-priority flush task queue to wait for the n+1th round of data flushing. The flush stop line is lower than the total flush start line. Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk flushing stop line, and then stop the high-priority disk flushing cycle task.

3. The method according to claim 2, characterized in that The low-priority disk flushing cycle tasks include: When the number of dirty caches corresponding to the target physical disk exceeds the low-priority flushing start line, dirty cache data equal to the target flushing amount will be put into the low-priority flushing task queue to wait for the nth round of data flushing operation; After the nth round of data flushing, if the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, dirty cache data equal to the target flushing amount will be put into the high-priority flushing task queue. If the number of dirty caches corresponding to the target physical disk is lower than the high-priority flushing start line and exceeds the flushing stop line, dirty cache data equal to the target flushing amount will be put into the low-priority flushing task queue to wait for the n+1th round of data flushing. Repeat the above steps until the number of dirty caches corresponding to the target physical disk is lower than the disk flushing stop line, and then stop the low-priority disk flushing cycle task.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When the total number of dirty caches corresponding to the physical disks in the target disk group exceeds the total disk drop start line, for each physical disk in the target disk group, the dirty cache data of the corresponding disk flushing amount will be put into the total disk drop task queue to wait for the nth round of data drop operation; wherein, the total disk drop start line is lower than the low-priority disk drop start line; the priority level of the total disk drop task queue is lower than that of the low-priority disk drop task queue.

5. The method according to claim 4, characterized in that The method further comprises: Poll each physical disk according to a preset time period; each physical disk is set with a corresponding timeout period; When it is detected that the number of dirty caches corresponding to the target physical disk is between the zero watermark and the stop line of the disk flushing, the timing for the target physical disk is started; the zero watermark of the disk flushing is lower than the stop line; When the timing duration for the target physical disk exceeds the timeout duration corresponding to the target physical disk, the dirty cache data corresponding to the target physical disk, which is the target disk flushing amount, will be delivered to the timeout disk drop task queue to wait for the nth round of data disk drop operation; wherein, the priority level of the timeout disk drop task queue is lower than that of the total disk drop task queue.

6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When it is detected that the target physical disk has lost power or been unplugged, the dirty cache data corresponding to the target physical disk will be delivered to a special disk drop task queue to wait for the nth round of data drop operation; the priority level of the special disk drop task queue is higher than that of the high-priority disk drop task queue.

7. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The dirty cache data waiting to be written to disk are added to the write buffer chain in the order of generation time; wherein, the dirty cache data are added in the order from the head to the end of the write buffer chain; For each dirty cache data on the write buffer chain, sort the dirty cache data corresponding to each physical disk according to the logical block address size, and mark the dirty cache data with continuous logical block addresses; For each dirty cache data, aggregate the continuous blocks in the dirty cache data; Aggregate dirty cache data with continuous logical block addresses to obtain aggregated dirty cache data; Starting from the chain head, the aggregated dirty cache data corresponding to the disk flushing amount is removed and written to the corresponding physical disks in sequence.

8. A data storage device, characterized in that: The device comprises: A polling module is used to poll the dirty cache count of each physical disk in the target disk group; a task triggering module configured to trigger a disk flushing cycle task of a priority level for any physical disk in the target disk group according to the number of dirty buffers corresponding to the physical disk; the priority level is related to the number of dirty buffers; A disk flushing module is configured to schedule each flushing task and execute a data flushing operation according to the priority level of the flushing cycle task corresponding to each physical disk in the target disk group, so as to flush the dirty cache data corresponding to the flushing task to the corresponding physical disk; The priority levels include high priority and low priority; the task triggering module is further used to: When the number of dirty caches corresponding to the target physical disk exceeds the high-priority flushing start line, the high-priority flushing cycle task is triggered to put the dirty cache data into the high-priority flushing task queue; When the number of dirty caches corresponding to the target physical disk exceeds the low-priority disk write start line, the low-priority disk write cycle task is triggered to put the dirty cache data into the low-priority disk write task queue; wherein the low-priority disk write start line is lower than the high-priority disk write start line.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data storage method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data storage method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the data storage method according to any one of claims 1 to 7.

Images