A raid data synchronization method, device, equipment and medium

Abstract

The application discloses a data synchronization method and device of a raid, equipment and a medium, so that the maximum threshold of the synchronization speed can meet the read-write requirements of business data and improve the synchronization speed. When the i-th cycle of data synchronization of a redundant array of independent disks ends, i is an integer greater than 1, the following operations are performed: when it is determined that the amount of business data received in the i-th cycle is 0, the maximum threshold of the synchronization speed of the i-th cycle is increased, and the increased maximum threshold of the synchronization speed is taken as the maximum threshold of the synchronization speed of the (i+1)-th cycle; when it is determined that the amount of business data received in the i-th cycle is not 0, according to the size relationship between the amount of business data received in the i-th cycle and the amount of business data received in the (i-1)-th cycle, it is determined whether to perform first processing on the maximum threshold of the synchronization speed of the i-th cycle, and the first processed maximum threshold of the synchronization speed is taken as the maximum threshold of the synchronization speed of the (i+1)-th cycle.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a RAID data synchronization method, apparatus, device and medium. Background Technology

[0002] Redundant arrays of independent disks (RAID) are disk groups composed of multiple independent disks. They improve performance through disk concurrency and enhance data security through data redundancy. The working principle of RAID includes: constructing redundant data as check data according to certain rules; and when some data is lost, recalculating the lost data from the remaining data through inverse operations.

[0003] When a RAID array is first created or during its use, if a disk fails and a new disk is added, the internal data may not conform to the above rules. In this case, the RAID array will perform data synchronization to ensure the data meets the above rules. Data synchronization takes time, and it is desirable for the RAID synchronization process to complete as quickly as possible.

[0004] RAID has limited read / write bandwidth. Data synchronization and business data read / write operations both consume bandwidth. If the synchronization speed is too fast, the bandwidth consumed will be excessive, potentially impacting business data read / write operations. Conversely, if the synchronization speed is too slow, the synchronization cycle will be too long, and if a hard drive fails during synchronization, it may cause RAID failure and data loss.

[0005] The Linux kernel provides minimum and maximum synchronization speed thresholds to control the synchronization speed within a certain range. However, this approach has its limitations. Setting the maximum synchronization speed threshold too low can result in slow synchronization speeds and long synchronization cycles; setting it too high can lead to excessive bandwidth consumption during synchronization, impacting normal business operations. Therefore, how can the maximum synchronization speed threshold be set appropriately? Summary of the Invention

[0006] This application provides a data synchronization method, apparatus, device, and medium for an independent redundant disk array RAID, which provides a flexible way to determine the maximum threshold of synchronization speed, so that the determined maximum threshold of synchronization speed can both meet the read and write requirements of business data and improve the synchronization speed.

[0007] Firstly, a data synchronization method for an independent redundant disk array RAID is provided, including:

[0008] At the end of the i-th cycle of data synchronization in the independent redundant disk array RAID, where i is an integer greater than 1, perform the following operations:

[0009] When it is determined that the amount of service data received in the i-th period is 0, the maximum synchronization speed threshold of the i-th period is increased, and the increased maximum synchronization speed threshold is used as the maximum synchronization speed threshold of the (i+1)-th period.

[0010] When it is determined that the amount of service data received in the i-th period is not 0, the maximum synchronization speed threshold of the i-th period is determined according to the relationship between the amount of service data received in the i-th period and the amount of service data received in the (i-1)-th period, and the maximum synchronization speed threshold after the first processing is used as the maximum synchronization speed threshold of the (i+1)-th period.

[0011] In the (i+1)th cycle, data synchronization is performed based on the maximum synchronization speed threshold of the (i+1)th cycle.

[0012] In one possible implementation, increasing the maximum threshold for the synchronization speed of the i-th cycle includes:

[0013] If it is determined that the amount of service data received in the preset number of adjacent periods before the i-th period is 0, then the maximum threshold for the synchronization speed of the i-th period is increased to infinity.

[0014] If it is determined that the amount of service data received in the preset number of adjacent cycles before the i-th cycle is not all 0, then the maximum threshold of the synchronization speed in the (i-1)-th cycle is increased by a set step size.

[0015] In one possible implementation, based on the relationship between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle, a first process is determined to apply to the maximum synchronization speed threshold of the i-th cycle, including:

[0016] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing.

[0017] When it is determined that the amount of service data received in the i-th cycle is less than the amount of service data received in the (i-1)-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing.

[0018] When it is determined that the difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is within a preset range, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined based on the amount of service data read and written in the i-th cycle and the amount of service data received in the i-th cycle.

[0019] In one possible implementation, when it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing, including:

[0020] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is greater than or equal to a preset first threshold, the maximum threshold for the synchronization speed of the i-th cycle is reduced to a preset conservative speed threshold.

[0021] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is less than or equal to a preset first threshold, the maximum threshold for the synchronization speed of the i-th cycle is reduced by a preset step size.

[0022] In one possible implementation, based on the amount of service data completed for reading and writing in the i-th cycle and the amount of service data received in the i-th cycle, a first process for determining the maximum synchronization speed threshold of the i-th cycle is determined, including:

[0023] If it is determined that the amount of business data read and written in the i-th cycle is less than the amount of business data received in the i-th cycle, then the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing.

[0024] When it is determined that the amount of service data completed in the i-th cycle is equal to the amount of service data received in the i-th cycle, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing.

[0025] In one possible implementation, when determining the end of the i-th cycle, it also includes:

[0026] The synchronization speed within the i-th cycle is calculated.

[0027] Calculate the variance of the synchronization speed of the i-th cycle and a preset number of adjacent cycles preceding the i-th cycle;

[0028] When it is determined that the variance is less than a set variance threshold, the maximum threshold for the synchronization speed of the (i+1)th cycle is determined based on the synchronization speed of the i cycles and the synchronization speed of a preset number of adjacent cycles before the i-th cycle.

[0029] In one possible implementation, the maximum threshold for the synchronization speed of the (i+1)th cycle is determined based on the synchronization speeds of the i cycles and a preset number of adjacent cycles preceding the i-th cycle, including:

[0030] The average of the synchronization speeds of the i-th cycle and a preset number of adjacent cycles preceding the i-th cycle is determined as the maximum threshold for the synchronization speed of the (i+1)-th cycle.

[0031] Secondly, a data synchronization device for an independent redundant disk array RAID is provided, comprising:

[0032] The limit adjustment module is used to perform the following operations when the i-th cycle of data synchronization in the independent redundant disk array RAID ends, where i is an integer greater than 1: when it is determined that the amount of service data received in the i-th cycle is 0, the maximum synchronization speed threshold of the i-th cycle is increased, and the increased maximum synchronization speed threshold is used as the maximum synchronization speed threshold of the (i+1)-th cycle; when it is determined that the amount of service data received in the i-th cycle is not 0, based on the relationship between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle, the maximum synchronization speed threshold of the i-th cycle is determined to undergo a first processing, and the maximum synchronization speed threshold after the first processing is used as the maximum synchronization speed threshold of the (i+1)-th cycle.

[0033] The data synchronization module is used to perform data synchronization in the (i+1)th cycle according to the maximum threshold of the synchronization speed in the (i+1)th cycle.

[0034] In one possible implementation, when the limit adjustment module increases the maximum threshold of the synchronization speed for the i-th cycle, it is specifically used for:

[0035] If it is determined that the amount of service data received in the preset number of adjacent periods before the i-th period is 0, then the maximum threshold for the synchronization speed of the i-th period is increased to infinity.

[0036] If it is determined that the amount of service data received in the preset number of adjacent cycles before the i-th cycle is not all 0, then the maximum threshold of the synchronization speed in the (i-1)-th cycle is increased by a set step size.

[0037] In one possible implementation, when the limit adjustment module performs the first processing to determine the maximum threshold for the synchronization speed of the i-th cycle based on the relationship between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle, it is specifically used for:

[0038] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing.

[0039] When it is determined that the amount of service data received in the i-th cycle is less than the amount of service data received in the (i-1)-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing.

[0040] When it is determined that the difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is within a preset range, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined based on the amount of service data read and written in the i-th cycle and the amount of service data received in the i-th cycle.

[0041] In one possible implementation, when the limit adjustment module determines that the first processing of the maximum synchronization speed threshold of the i-th period is a reduction processing when it is determined that the amount of service data received in the i-th period is greater than the amount of service data received in the (i-1)-th period, the specific processing is as follows:

[0042] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is greater than or equal to a preset first threshold, the maximum threshold for the synchronization speed of the i-th cycle is reduced to a preset conservative speed threshold.

[0043] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is less than or equal to a preset first threshold, the maximum threshold for the synchronization speed of the i-th cycle is reduced by a preset step size.

[0044] In one possible implementation, when the limit adjustment module performs the first processing to determine the maximum synchronization speed threshold for the i-th cycle based on the amount of service data read and written in the i-th cycle and the amount of service data received in the i-th cycle, it is specifically used for:

[0045] If it is determined that the amount of business data read and written in the i-th cycle is less than the amount of business data received in the i-th cycle, then the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing.

[0046] When it is determined that the amount of service data completed in the i-th cycle is equal to the amount of service data received in the i-th cycle, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing.

[0047] In one possible implementation, the limit adjustment module is further configured to: when the end of the i-th cycle is determined, calculate the synchronization speed within the i-th cycle; calculate the variance of the synchronization speed of the i-th cycle and a preset number of adjacent cycles preceding the i-th cycle; and when the variance is determined to be less than a set variance threshold, determine a maximum threshold for the synchronization speed of the (i+1)-th cycle based on the synchronization speed of the i-th cycle and the preset number of adjacent cycles preceding the i-th cycle.

[0048] In one possible implementation, when the limit adjustment module determines the maximum threshold of the synchronization speed of the (i+1)th cycle based on the synchronization speed of the i cycles and the preset number of adjacent cycles before the i-th cycle, it is specifically used to: determine the average value of the synchronization speed of the i cycles and the preset number of adjacent cycles before the i-th cycle as the maximum threshold of the synchronization speed of the (i+1)th cycle.

[0049] Thirdly, this application provides an electronic device, including: a processor, optionally further including a memory; the processor and the memory are coupled; the memory is used to store computer programs or instructions; the processor is used to execute part or all of the computer programs or instructions in the memory, and when the part or all of the computer programs or instructions are executed, it is used to implement the function in any of the above methods.

[0050] In one possible implementation, the apparatus may further include a transceiver for transmitting signals processed by the processor or receiving signals input to the processor. The transceiver may perform either the transmitting or receiving action of any of the methods.

[0051] Fourthly, a computer-readable storage medium is provided for storing a computer program, the computer program including instructions for implementing any of the functions.

[0052] Alternatively, a computer-readable storage medium for storing a computer program, which, when executed by a computer, causes the computer to perform any of the methods described above.

[0053] Fifthly, a computer program product is provided, the computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform any of the methods described above.

[0054] During RAID data synchronization, the synchronization speed always attempts to reach the maximum threshold; this is the operating mechanism of RAID. If the amount of business data received in a certain period is 0, it indicates that the business is completed or about to be completed, and the synchronization speed can be increased. In this case, this application proposes to increase the maximum synchronization speed threshold of a certain period, which will then serve as the maximum synchronization speed threshold for the next period. Since the maximum synchronization speed threshold for the next RAID period has increased, the synchronization speed for the next period will also increase. Because the business is completed or about to be completed, this not only does not affect the reading and writing of business data but also improves the synchronization speed.

[0055] If the amount of business data received in a given period is not zero, it indicates that the business is still ongoing. By analyzing the amount of business data received between two consecutive periods, the trend of business data volume change is determined. Based on this trend, the trend of the maximum synchronization speed threshold is determined. This ensures that the maximum synchronization speed threshold for the next period meets both business requirements and data synchronization speed requirements. Attached Figure Description

[0056] To more clearly illustrate the implementation methods in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0057] Figure 1 This illustration shows a data synchronization process diagram of an independent redundant disk array RAID provided in an embodiment of this application;

[0058] Figure 2 This illustration shows a data synchronization process diagram of an independent redundant disk array RAID provided in an embodiment of this application;

[0059] Figure 3 A schematic diagram of a process for adjusting the maximum threshold of synchronization speed according to an embodiment of this application is shown;

[0060] Figure 4This paper shows a structural diagram of a RAID data synchronization device according to an embodiment of this application;

[0061] Figure 5 A structural diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0062] To make the objectives and implementation methods of this application clearer, exemplary embodiments of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments. Obviously, the described exemplary embodiments are only a part of the embodiments of this application, and not all of the embodiments. The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms can be used interchangeably where appropriate.

[0063] It should be noted that the embodiments of this application are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0064] The synchronization speed mentioned in this application can be understood as the bandwidth used. The synchronization speed of the i-th cycle can be understood as the bandwidth used for synchronization in the i-th cycle. The maximum threshold for the synchronization speed of the i-th cycle can be understood as the maximum bandwidth allowed to be used for the data synchronization process in the i-th cycle.

[0065] The business data volume mentioned in this application can be understood as business I / O. The data involved in data synchronization can be understood as synchronization I / O.

[0066] Figure 1 This illustration shows a schematic diagram of a data synchronization process for an independent redundant disk array RAID according to an embodiment of this application. The process includes the following steps:

[0067] Step 11: At the end of the i-th cycle of data synchronization in the independent redundant disk array RAID, determine whether the amount of service data received in the i-th cycle is 0.

[0068] Where i is an integer greater than 1.

[0069] If yes, proceed to step 12; otherwise, proceed to step 13.

[0070] Step 12: Increase the maximum synchronization speed threshold of the i-th cycle, and use the increased maximum synchronization speed threshold as the maximum synchronization speed threshold of the (i+1)-th cycle.

[0071] During RAID data synchronization, the synchronization speed always attempts to reach the maximum threshold; this is the operating mechanism of RAID. If the amount of business data received in a certain period is 0, it indicates that the business is completed or about to be completed, and the synchronization speed can be increased. In this case, this application proposes to increase the maximum synchronization speed threshold of a certain period, which will then serve as the maximum synchronization speed threshold for the next period. Since the maximum synchronization speed threshold for the next RAID period has increased, the synchronization speed for the next period will also increase. Because the business is completed or about to be completed, this not only does not affect the reading and writing of business data but also improves the synchronization speed.

[0072] Step 13: Based on the relationship between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle, determine the maximum synchronization speed threshold of the i-th cycle and perform a first processing, then use the maximum synchronization speed threshold after the first processing as the maximum synchronization speed threshold of the (i+1)-th cycle.

[0073] The first process can be either an increase process, a decrease process, or a unchanged process.

[0074] If the amount of business data received in a given period is not zero, it indicates that the business is still ongoing. By analyzing the amount of business data received between two consecutive periods, the trend of business data volume change is determined. Based on this trend, the trend of the maximum synchronization speed threshold is determined. This ensures that the maximum synchronization speed threshold for the next period meets both business requirements and data synchronization speed requirements.

[0075] Step 14: In the (i+1)th cycle, perform data synchronization based on the maximum synchronization speed threshold of the (i+1)th cycle.

[0076] The maximum synchronization speed threshold of the i-th cycle is used to: pause data synchronization in the i-th cycle when the synchronization speed is greater than or equal to the maximum synchronization speed threshold of the i-th cycle.

[0077] Figure 2 The illustration shows a schematic diagram of a RAID data synchronization process provided by an embodiment of this application.

[0078] Step 21: Preset the cycle value and conservative speed threshold (sync_speed_min).

[0079] The period value should not be too long, such as 1 second, 2 seconds, 3 seconds, etc.

[0080] Set a conservative speed threshold (sync_speed_min) in advance. This threshold should not be too large. The requirement for the conservative speed threshold is that when synchronizing data at the conservative speed threshold, the remaining bandwidth of the RAID should meet the read / write requirements of the highest-performing service. The highest-performing service can be understood as the service that requires the most bandwidth among all services in the RAID. For example, the conservative speed threshold could be 0, or 2MB / s, 5MB / s, or 10MB / s.

[0081] Step 22: The default maximum threshold for synchronization speed in the first cycle is the pre-set conservative speed threshold (sync_speed_min).

[0082] Step 23: Begin periodic testing.

[0083] Step 24: Determine if the current cycle has ended.

[0084] If no, proceed to step 25. If yes, proceed to step 27.

[0085] Step 25: Calculate the synchronization speed for the current cycle.

[0086] Step 26: Determine whether the synchronization speed of the current cycle is greater than the maximum threshold for the synchronization speed of the current cycle.

[0087] If yes, then data synchronization will not be performed in the current cycle; if no, then data synchronization will continue in the current cycle.

[0088] Step 27: Statistically analyze and save multiple parameters from the just-ended cycle.

[0089] The parameters include: the amount of business data received in this cycle (the amount of business data is simply referred to as business data volume), the amount of business data read and written in this cycle, and the synchronization speed of this cycle.

[0090] Step 28: Based on statistical parameters, determine the maximum threshold for synchronization speed in the next cycle.

[0091] The process in step 28 can be referred to above. Figure 1 The process of introduction will not be repeated here.

[0092] When the next cycle begins as a new cycle, steps 24 to 28 above are repeated based on the determined maximum threshold of the synchronization speed of the new cycle.

[0093] Taking the end of the i-th cycle as an example, the process of determining the maximum threshold of the synchronization speed in the (i+1)-th cycle is introduced in various scenarios.

[0094] Scenario 1: The amount of business data received in a certain period (i.e. the i-th period) is 0.

[0095] When it is determined that the amount of service data received in the i-th period is 0, the maximum synchronization speed threshold of the i-th period is increased, and the increased maximum synchronization speed threshold is used as the maximum synchronization speed threshold of the (i+1)-th period (i.e., step 12).

[0096] Based on scenario 1, we will further divide the process into several examples and provide a detailed explanation of how to increase the maximum threshold of the synchronization speed in the i-th cycle.

[0097] In one example, if it is determined that the amount of service data received in a preset number of adjacent periods before the i-th period is 0, then the maximum threshold for the synchronization speed in the i-th period is increased to infinity. That is, in the (i+1)-th period, the synchronization speed is no longer limited. For example, the preset number could be 1, 2, 3, 4, or 5, etc.

[0098] In another example, if it is determined that the amount of service data received in a preset number of adjacent periods before the i-th period is not all zero, the maximum synchronization speed threshold for the i-th period can be increased by a set step size. That is, the maximum synchronization speed threshold can be increased slightly. For example, the set step size could be 2MB / s, 3MB / s, etc.

[0099] Scenario 2: The amount of business data received in a certain period (i.e. the i-th period) is not 0.

[0100] When it is determined that the amount of service data received in the i-th cycle is not 0, the maximum synchronization speed threshold of the i-th cycle is determined to be processed first based on the relationship between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle. The maximum synchronization speed threshold after the first processing is then used as the maximum synchronization speed threshold of the (i+1)-th cycle. The first processing can be either an increase processing, a decrease processing, or no change (i.e., step 13).

[0101] Based on scenario 2, it is further divided into multiple scenarios. The process of determining the maximum threshold of the synchronization speed for the i-th period and performing the first processing based on the relationship between the amount of service data received in the i-th period and the amount of service data received in the (i-1)-th period is described in detail.

[0102] Scenario 2.1: If the amount of business data in a certain period is larger than that in the previous period, then the maximum threshold for synchronization speed should be reduced.

[0103] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing. The reduction processing can be reducing to a conservative speed threshold or reducing by a set step size.

[0104] In scenario 2.1, optionally, a first threshold can be set to determine whether the increase in business data volume is significant, thereby determining how to reduce the maximum threshold of synchronization speed in the i-th cycle.

[0105] Scenario 2.1.1: The amount of business data in a certain period is significantly larger than that in the previous period.

[0106] In one example, when it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is greater than or equal to a preset first threshold, the maximum threshold for the synchronization speed of the i-th cycle is reduced to a preset conservative speed threshold.

[0107] For example, the first threshold could be 10MB / s, 15MB / s, or the amount of service data received in the (i-1)th cycle. Alternatively, the step size could be set to 1MB / s, 2MB / s, 3MB / s, etc.

[0108] If the detected traffic volume is larger than before and the increase exceeds the set first threshold, it may be that the business has suddenly increased IO. In order to prevent insufficient business bandwidth, it is necessary to significantly reduce the synchronization speed. One approach is to directly reduce the maximum synchronization speed threshold to the pre-set conservative speed threshold sync_speed_min.

[0109] Scenario 2.1.2: The increase in business data volume in a certain period compared to the previous period is not significant.

[0110] In one example, when it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is less than or equal to a preset first threshold, the maximum synchronization speed threshold for the i-th cycle is reduced by a set step size. That is, the maximum synchronization speed threshold is reduced slightly. For example, the set step size could be 2MB / s, 3MB / s, etc.

[0111] Scenario 2.2: If the amount of business data in a certain period is less than that in the previous period, increase the maximum threshold for synchronization speed.

[0112] When it is determined that the amount of service data received in the i-th cycle is less than the amount of service data received in the (i-1)-th cycle, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing. For example, the set step size is increased, that is, the maximum synchronization speed threshold is increased slightly.

[0113] Alternatively, the first processing step for determining the maximum threshold of the synchronization speed in the i-th cycle can be to keep it unchanged.

[0114] In scenario 2.2, optionally, a second threshold can be set to determine whether the reduction in business data volume is significant, thereby determining how to handle the maximum threshold of synchronization speed.

[0115] In one example, if the amount of service data in a certain period is significantly less than that in the previous period, the set step size is increased. For example, when it is determined that the amount of service data received in the i-th period is less than the amount of service data received in the (i-1)-th period, and the second difference between the amount of service data received in the (i-1)-th period and the amount of service data received in the (i-1)-th period is greater than or equal to a preset second threshold, the set step size for increasing the maximum synchronization speed threshold of the i-th period is increased, i.e., the maximum synchronization speed threshold is increased slightly.

[0116] For example, the second threshold could be 10MB / s, 15MB / s, etc. Or, the step size could be set to 1MB / s, 2MB / s, 3MB / s, etc.

[0117] If the amount of business data is detected to be smaller than before, and the reduction exceeds the set second threshold, it may be that the business has suddenly reduced IO. You can slightly increase the maximum threshold for synchronization speed.

[0118] In another example, if the amount of business data in a certain period does not decrease significantly compared to the previous period, then it remains unchanged.

[0119] For example, when it is determined that the amount of service data received in the i-th cycle is less than the amount of service data received in the (i-1)-th cycle, and the second difference between the amount of service data received in the (i-1)-th cycle and the amount of service data received in the (i-1)-th cycle is less than or equal to a preset second threshold, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to remain unchanged.

[0120] Scenario 2.3: The amount of business data in a certain period is not significantly different from that in the previous period.

[0121] In one example, when it is determined that the difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is within a preset range, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to remain unchanged.

[0122] The preset range is related to the first threshold and the second threshold, and the first threshold and / or the second threshold are the boundary values ​​of the preset range.

[0123] When considering whether the amount of business data has changed, scenarios 2.1, 2.2, and 2.3 above consider the difference between the amount of business data received in the i-th cycle and the amount of business data received in the (i-1)-th cycle. In addition, the average amount of business data received in at least two cycles prior to the i-th cycle can also be considered. The amount of business data received in the i-th cycle is compared with the average amount of business data received in the at least two cycles prior to the i-th cycle. The first processing step for the maximum synchronization speed threshold of the i-th cycle is determined by whether the difference between the two is within a preset range. The principle is similar to scenarios 2.1, 2.2, and 2.3 above, except that the "difference between the amount of business data received in the (i-1)-th cycle" mentioned in scenarios 2.1, 2.2, and 2.3 is replaced with "the average amount of business data received in at least two cycles prior to the i-th cycle".

[0124] To further optimize synchronization speed, when the amount of business data in a given period does not change significantly from the previous period, the trend of the maximum synchronization speed threshold can be determined based on whether the reading and writing of business data has been fully completed. This will be explained in detail below.

[0125] When it is determined that the difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is within a preset range, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined based on the amount of service data read and written in the i-th cycle and the amount of service data received in the i-th cycle.

[0126] In one example, when it is determined that the amount of service data completed in the i-th cycle is less than the amount of service data received in the i-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing.

[0127] For example, reduce the set step size, that is, slightly reduce the maximum threshold of the synchronization speed. For example, set the step size to 1MB / s, 2MB / s, 3MB / s, etc.

[0128] The word "less than" here has two meanings. One meaning is: less than in an absolute sense. The other meaning is: the amount of business data read and written in the i-th cycle is less than the amount of business data received in the i-th cycle, and the difference between the two is greater than or equal to a set third threshold. If the difference between the two is less than the set third threshold, they can be considered to be the same.

[0129] If the amount of business data in a certain period does not change significantly compared to the previous period, but the amount of completed business data decreases, one possible reason is that the synchronization speed is too high. You can slightly reduce the maximum threshold of the synchronization speed.

[0130] In another example, when it is determined that the amount of service data completed in the i-th cycle is equal to the amount of service data received in the i-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing.

[0131] For example, increase the set step size, which means slightly increasing the maximum threshold of the synchronization speed. For example, set the step size to 1MB / s, 2MB / s, 3MB / s, etc.

[0132] The word "equal to" here has two meanings. One meaning is: equal in an absolute sense. The other meaning is: the difference between the amount of business data read and written in the i-th cycle and the amount of business data received in the i-th cycle is less than or equal to a set fourth threshold. If the difference between the two is less than or equal to the set fourth threshold, they can be considered to be the same.

[0133] If the amount of business data in a certain period does not change significantly compared to the previous period, and the received business data can be completed, it means that the total bandwidth is still sufficient. The maximum threshold of the synchronization speed can be increased slightly to see if there is still room for improvement in the synchronization speed.

[0134] Figure 3 The illustration shows a process for adjusting the maximum threshold of synchronization speed according to an embodiment of this application.

[0135] Step 301: At the end of the i-th cycle, determine whether the amount of service data received in the i-th cycle is 0; if yes (corresponding to scenario 1 described above), then proceed to step 302. If no (corresponding to scenario 2 described above), then proceed to step 303.

[0136] Step 302: Determine whether the amount of service data received in the preset number of adjacent periods before the i-th period is also 0.

[0137] If so, then the synchronization speed of the (i+1)th cycle is determined to be unlimited. That is, the maximum threshold for the synchronization speed of the (i+1)th cycle is infinite.

[0138] If not, slightly increase the maximum synchronization speed threshold based on the maximum synchronization speed threshold of the i-th cycle.

[0139] Step 303: Determine whether the amount of business data has increased. Specifically, determine whether the amount of business data received in the i-th period is greater than the amount of business data received in the (i-1)-th period.

[0140] If yes, proceed to step 304; otherwise, proceed to step 305.

[0141] Step 304: Determine whether the increase in business data volume exceeds the first threshold. Specifically, determine whether the first difference between the business data volume received in the i-th period and the business data volume received in the (i-1)-th period is greater than or equal to the preset first threshold.

[0142] If so, the maximum threshold for synchronization speed in the (i+1)th cycle is determined to be a pre-set conservative speed threshold.

[0143] If not, then slightly reduce the maximum synchronization speed threshold based on the maximum synchronization speed threshold of the i-th cycle.

[0144] Step 305: Determine whether the amount of business data has decreased. Specifically, determine whether the amount of business data received in the i-th period is less than the amount of business data received in the (i-1)-th period.

[0145] If so, the maximum synchronization speed threshold is increased slightly based on the maximum synchronization speed threshold in the i-th cycle.

[0146] If not, proceed to step 306.

[0147] Step 306: Determine whether all business data for the i-th period has been completed.

[0148] If so, the maximum synchronization speed threshold is increased slightly based on the maximum synchronization speed threshold in the i-th cycle.

[0149] If not, then slightly reduce the maximum synchronization speed threshold based on the maximum synchronization speed threshold of the i-th cycle.

[0150] In one possible implementation, when the end of the i-th cycle is determined, the synchronization speed within the i-th cycle can also be statistically analyzed; the variance of the synchronization speed of the i-th cycle and a preset number of adjacent cycles preceding the i-th cycle can be calculated (for example, the variance of the synchronization speed of 3, 4, or 5 consecutive cycles can be calculated).

[0151] When it is determined that the variance is less than the set variance threshold, it indicates that the current business situation has basically reached a balanced state. At this time, the maximum threshold of the synchronization speed of the (i+1)th cycle can be determined based on the synchronization speed of the i-th cycle and the synchronization speed of the adjacent cycles that are a preset number of cycles before the i-th cycle.

[0152] For example, the average or median of the synchronization speeds of the i-th cycle and a preset number of adjacent cycles preceding the i-th cycle can be determined as the maximum synchronization speed threshold for the (i+1)-th cycle. Alternatively, it can be set as the maximum synchronization speed threshold for cycles after the (i+1)-th cycle, meaning the maximum synchronization speed threshold is no longer adjusted.

[0153] This application intelligently adjusts the RAID synchronization speed based on business pressure. By periodically analyzing business I / O, synchronization I / O, and the I / O completion status of previous periods, the maximum threshold of the synchronization speed is dynamically adjusted to achieve the following objectives: maximizing hard drive performance while meeting business I / O performance requirements, adjusting the synchronization speed to its maximum as much as possible, shortening synchronization time, and improving data reliability.

[0154] Based on the same technical concept, this application also provides a RAID data synchronization device. Figure 4 A schematic diagram of a RAID data synchronization device is shown, including:

[0155] The limit adjustment module 41 is used to perform the following operations when the i-th cycle of data synchronization in the independent redundant disk array RAID ends, where i is an integer greater than 1: when it is determined that the amount of service data received in the i-th cycle is 0, the maximum synchronization speed threshold of the i-th cycle is increased, and the increased maximum synchronization speed threshold is used as the maximum synchronization speed threshold of the (i+1)-th cycle; when it is determined that the amount of service data received in the i-th cycle is not 0, based on the relationship between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle, the maximum synchronization speed threshold of the i-th cycle is determined to undergo a first processing, and the maximum synchronization speed threshold after the first processing is used as the maximum synchronization speed threshold of the (i+1)-th cycle.

[0156] The data synchronization module 42 is used to perform data synchronization in the (i+1)th cycle according to the maximum threshold of the synchronization speed in the (i+1)th cycle.

[0157] In one possible implementation, when the limit adjustment module 41 increases the maximum threshold of the synchronization speed in the i-th cycle, it is specifically used for:

[0158] If it is determined that the amount of service data received in the preset number of adjacent periods before the i-th period is 0, then the maximum threshold for the synchronization speed of the i-th period is increased to infinity.

[0159] If it is determined that the amount of service data received in the preset number of adjacent cycles before the i-th cycle is not all 0, then the maximum threshold of the synchronization speed in the (i-1)-th cycle is increased by a set step size.

[0160] In one possible implementation, when the limit adjustment module 41 performs the first processing to determine the maximum threshold of the synchronization speed for the i-th period based on the relationship between the amount of service data received in the i-th period and the amount of service data received in the (i-1)-th period, it is specifically used for:

[0161] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing.

[0162] When it is determined that the amount of service data received in the i-th cycle is less than the amount of service data received in the (i-1)-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing.

[0163] When it is determined that the difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is within a preset range, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined based on the amount of service data read and written in the i-th cycle and the amount of service data received in the i-th cycle.

[0164] In one possible implementation, when the limit adjustment module 41 determines that the first processing of the maximum synchronization speed threshold of the i-th cycle is a reduction processing when it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, the specific processing is as follows:

[0165] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is greater than or equal to a preset first threshold, the maximum threshold for the synchronization speed of the i-th cycle is reduced to a preset conservative speed threshold.

[0166] When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is less than or equal to a preset first threshold, the maximum threshold for the synchronization speed of the i-th cycle is reduced by a preset step size.

[0167] In one possible implementation, when the limit adjustment module 41 performs the first processing to determine the maximum synchronization speed threshold for the i-th cycle based on the amount of service data read and written in the i-th cycle and the amount of service data received in the i-th cycle, it is specifically used for:

[0168] If it is determined that the amount of business data read and written in the i-th cycle is less than the amount of business data received in the i-th cycle, then the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing.

[0169] When it is determined that the amount of service data completed in the i-th cycle is equal to the amount of service data received in the i-th cycle, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing.

[0170] In one possible implementation, the limit adjustment module 41 is further configured to: when the end of the i-th cycle is determined, calculate the synchronization speed within the i-th cycle; calculate the variance of the synchronization speed of the i-th cycle and a preset number of adjacent cycles preceding the i-th cycle; and when the variance is determined to be less than a set variance threshold, determine the maximum threshold of the synchronization speed for the (i+1)-th cycle based on the synchronization speed of the i-th cycle and the preset number of adjacent cycles preceding the i-th cycle.

[0171] In one possible implementation, when the limit adjustment module 41 determines the maximum threshold of the synchronization speed of the (i+1)th cycle based on the synchronization speed of the i cycles and the preset number of adjacent cycles before the i-th cycle, it is specifically used to: determine the average value of the synchronization speed of the i cycles and the preset number of adjacent cycles before the i-th cycle as the maximum threshold of the synchronization speed of the (i+1)th cycle.

[0172] Based on the same technical concept, this application also provides an electronic device. Figure 5 A schematic diagram of an electronic device structure is shown, such as Figure 5 As shown, it includes: processor 51, and optionally, it also includes: communication interface 52, memory 53 and communication bus 54, wherein the processor 51, communication interface 52 and memory 53 communicate with each other through communication bus 54.

[0173] The memory 53 stores a computer program, which, when executed by the processor 51, causes the processor 51 to complete the steps of the RAID data synchronization method described above.

[0174] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0175] Communication interface 52 is used for communication between the above-mentioned electronic device and other devices.

[0176] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0177] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0178] Based on the same technical concept and the above embodiments, this application provides a computer-readable storage medium storing a computer program executable by an electronic device, wherein computer-executable instructions are used to cause a computer to perform the steps of the above-described RAID data synchronization method.

[0179] The aforementioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in an electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), optical storage such as CDs, DVDs, BDs, HVDs, etc., and semiconductor storage such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs), etc.

[0180] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0181] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0182] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0183] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0184] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A data synchronization method for an independent redundant disk array RAID, characterized in that, include: At the end of the i-th cycle of data synchronization in the independent redundant disk array RAID, where i is an integer greater than 1, perform the following operations: When it is determined that the amount of service data received in the i-th period is 0, the maximum synchronization speed threshold of the i-th period is increased, and the increased maximum synchronization speed threshold is used as the maximum synchronization speed threshold of the (i+1)-th period. When it is determined that the amount of service data received in the i-th period is not 0, the maximum synchronization speed threshold of the i-th period is determined according to the relationship between the amount of service data received in the i-th period and the amount of service data received in the (i-1)-th period, and the maximum synchronization speed threshold after the first processing is used as the maximum synchronization speed threshold of the (i+1)-th period. In the (i+1)th cycle, data synchronization is performed based on the maximum synchronization speed threshold of the (i+1)th cycle.

2. The method as described in claim 1, characterized in that, Increasing the maximum threshold of the synchronization speed for the i-th cycle includes: If it is determined that the amount of service data received in the preset number of adjacent periods before the i-th period is 0, then the maximum threshold for the synchronization speed of the i-th period is increased to infinity. If it is determined that the amount of service data received in the preset number of adjacent cycles before the i-th cycle is not all 0, then the maximum threshold of the synchronization speed in the i-th cycle is increased by a set step size.

3. The method as described in claim 1, characterized in that, Based on the relationship between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle, a first process is determined to apply to the maximum synchronization speed threshold of the i-th cycle, including: When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing. When it is determined that the amount of service data received in the i-th cycle is less than the amount of service data received in the (i-1)-th cycle, the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing. When it is determined that the difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is within a preset range, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined based on the amount of service data read and written in the i-th cycle and the amount of service data received in the i-th cycle.

4. The method as described in claim 3, characterized in that, When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing, including: When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is greater than a preset first threshold, the maximum threshold for the synchronization speed of the i-th cycle is reduced to a preset conservative speed threshold. When it is determined that the amount of service data received in the i-th cycle is greater than the amount of service data received in the (i-1)-th cycle, and the first difference between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle is less than or equal to a preset first threshold, the maximum threshold for the synchronization speed of the i-th cycle is reduced by a preset step size.

5. The method as described in claim 3, characterized in that, Based on the amount of service data completed for reading and writing in the i-th cycle and the amount of service data received in the i-th cycle, a first process is determined for the maximum synchronization speed threshold in the i-th cycle, including: If it is determined that the amount of business data read and written in the i-th cycle is less than the amount of business data received in the i-th cycle, then the first processing of the maximum synchronization speed threshold of the i-th cycle is determined to be a reduction processing. When it is determined that the amount of service data completed in the i-th cycle is equal to the amount of service data received in the i-th cycle, the first processing for the maximum synchronization speed threshold of the i-th cycle is determined to be an increase processing.

6. The method as described in claim 1, characterized in that, When it is determined that the i-th cycle has ended, the following is also included: The synchronization speed within the i-th cycle is calculated. Calculate the variance of the synchronization speed of the i-th cycle and a preset number of adjacent cycles preceding the i-th cycle; When it is determined that the variance is less than a set variance threshold, the maximum threshold of the synchronization speed of the (i+1)th cycle is determined based on the synchronization speed of the i-th cycle and the synchronization speed of a preset number of adjacent cycles before the i-th cycle.

7. The method as described in claim 6, characterized in that, Based on the synchronization speeds of the i-th cycles and a preset number of adjacent cycles preceding the i-th cycle, the maximum threshold for the synchronization speed of the (i+1)-th cycle is determined, including: The average of the synchronization speeds of the i-th cycle and a preset number of adjacent cycles preceding the i-th cycle is determined as the maximum threshold for the synchronization speed of the (i+1)-th cycle.

8. A data synchronization device for an independent redundant disk array RAID, characterized in that, include: The limit adjustment module is used to perform the following operations when the i-th cycle of data synchronization in the independent redundant disk array RAID ends, where i is an integer greater than 1: when it is determined that the amount of service data received in the i-th cycle is 0, the maximum synchronization speed threshold of the i-th cycle is increased, and the increased maximum synchronization speed threshold is used as the maximum synchronization speed threshold of the (i+1)-th cycle; when it is determined that the amount of service data received in the i-th cycle is not 0, based on the relationship between the amount of service data received in the i-th cycle and the amount of service data received in the (i-1)-th cycle, the maximum synchronization speed threshold of the i-th cycle is determined to undergo a first processing, and the maximum synchronization speed threshold after the first processing is used as the maximum synchronization speed threshold of the (i+1)-th cycle. The data synchronization module is used to perform data synchronization in the (i+1)th cycle according to the maximum threshold of the synchronization speed in the (i+1)th cycle.

9. An electronic device, characterized in that, include: Processor and memory; The memory is used to store computer programs or instructions; The processor is configured to execute some or all of the computer programs or instructions in the memory, and when the some or all of the computer programs or instructions are executed, to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, the computer program including instructions for implementing the method of any one of claims 1-7.

Images