Method and system for operation optimization of a high performance storage device

Abstract

The application relates to a high-performance storage device operation optimization method and system, and belongs to the technical field of data reading and writing. The method comprises the following steps: acquiring the response time of an I / O port of a high-performance storage device within a preset historical time period and calculating a defragmentation rate through historical defragmentation data of the high-performance storage device; based on the relationship between the response time of the I / O port and the defragmentation rate, a comparison table is constructed, and a target defragmentation rate range with a response time less than a preset time threshold is screened out; if the current disk fragmentation rate is greater than a preset fragmentation rate threshold, the current I / O port queue depth is acquired; based on the current I / O port queue depth, the current disk fragmentation rate and the target defragmentation rate range, the target defragmentation rate is determined, and defragmentation is performed. Through the above mode, the adaptation capability and operation stability of the high-performance storage device in a high-load and high-real-time scene can be significantly enhanced, and the core performance advantage of the high-performance storage device can be fully exerted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data reading and writing, and particularly relates to a running optimization method and system of a high-performance storage device. BACKGROUND

[0002] With the rapid development of data-intensive technologies such as cloud computing, big data and artificial intelligence, the performance of data storage systems is highly required in scenarios such as financial transactions, real-time industrial control and high-definition video stream processing, and high-performance storage devices have become the core infrastructure to support the stable operation of the above scenarios.

[0003] However, the current defragmentation scheme of the high-performance storage device generally has the defects of single factor driving or fixed strategy execution, and it is difficult to balance the defragmentation effect and the real-time I / O performance of the storage device. On the one hand, some schemes only use the disk fragmentation rate as the only basis for triggering defragmentation, that is, when the fragmentation rate exceeds the preset threshold, defragmentation is directly performed. If the amount of defragmentation is large, it will occupy a large amount of computing resources and I / O bandwidth of the storage controller, causing the queuing waiting time of I / O requests of concurrent business data to be prolonged, the I / O port response time to be significantly increased, and even causing the business request to be timed out. If defragmentation is only performed during idle time, the defragmentation period is too long, the fragmentation rate is high for a long time, and the I / O delay problem cannot be timely alleviated.

[0004] It can be seen that the current defragmentation operation and real-time business demand of the storage device will cause resource conflicts, and it is difficult to achieve the goal of both efficiently reducing the fragmentation rate and guaranteeing the stable I / O performance, which restricts the performance of the high-performance storage device in high-load and high-real-time scenarios. SUMMARY

[0005] To solve the above technical problems, the present application provides a running optimization method and system of a high-performance storage device.

[0006] In a first aspect, the present application provides a running optimization method of a high-performance storage device, comprising: obtaining the response time of an I / O port of the high-performance storage device in a preset historical time period and calculating a defragmentation rate by historical defragmentation data of the high-performance storage device; based on the relationship between the response time of the I / O port and the defragmentation rate, constructing a comparison table, and screening a target defragmentation rate range with a response time less than a preset time threshold; if the current disk fragmentation rate is greater than a preset fragmentation rate threshold, obtaining the current I / O port queue depth; based on the current I / O port queue depth, the current disk fragmentation rate and the target defragmentation rate range, determining a target defragmentation rate; and performing defragmentation based on the target defragmentation rate.

[0007] Optionally, the N time period defragmentation rates are calculated based on historical defragmentation data of the high-performance storage device; N is a positive integer greater than or equal to 2; and the relationship between the I / O port response time and the defragmentation rate is used to construct a comparison table, including: obtaining the I / O port response time corresponding to each time period, and averaging the I / O port response time corresponding to each time period; and constructing the comparison table based on the average of the I / O port response time of the N time periods and the defragmentation rate of the N time periods.

[0008] Optionally, the target defragmentation rate is determined based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range, including: determining a current load interval based on the current I / O port queue depth; different load intervals correspond to different intervals of the target defragmentation rate range; determining a fragmentation rate level based on the current disk fragmentation rate; different fragmentation rate levels correspond to different values of the target defragmentation rate range; and determining the target defragmentation rate from the interval of the target defragmentation rate corresponding to the current load interval based on the fragmentation rate level.

[0009] Optionally, the target defragmentation rate is determined based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range, including: determining a first coefficient based on the current I / O port queue depth; determining a first defragmentation value based on the first coefficient and the target defragmentation rate range; determining a second coefficient based on the current disk fragmentation rate; determining a second defragmentation value based on the second coefficient and the target defragmentation rate range; determining a load weight and a fragmentation processing weight according to a current business scenario; and determining the target defragmentation rate based on the first defragmentation value, the load weight, the second defragmentation value, and the fragmentation processing weight.

[0010] Optionally, the defragmentation is performed based on the target defragmentation rate, including: performing heat classification on the fragmented data to determine hot data, warm data, and cold data in the fragmented data; different heat classification fragmented data is pre-configured with different rate adjustment coefficients; determining actual defragmentation rates of the hot data, the warm data, and the cold data based on the rate adjustment coefficients of the different heat classification fragmented data and the target defragmentation rate; and performing defragmentation based on the actual defragmentation rates of the hot data, the warm data, and the cold data.

[0011] Optionally, the method further comprises: continuously monitoring the I / O port queue depth during the defragmentation process; in response to monitoring that the I / O port queue depth exceeds a first preset threshold, adjusting the rate adjustment coefficient of the different heat level graded fragmented data by a first preset proportion; or, in response to monitoring that the I / O port queue depth is lower than a second preset threshold, adjusting the rate adjustment coefficient of the different heat level graded fragmented data by a second preset proportion; wherein the first preset threshold is greater than the second preset threshold.

[0012] Optionally, the determination of the actual defragmentation rate of the hot data, the actual defragmentation rate of the warm data and the actual defragmentation rate of the cold data based on the rate adjustment coefficient of the different heat level graded fragmented data and the target defragmentation rate further comprises: determining the fragmented form distribution of the different heat level graded fragmented data based on the fragmented form quantitative detection.

[0013] determining the respective fine tuning coefficient based on the fragmented form distribution of the different heat level graded fragmented data; determining the actual defragmentation rate of the hot data, the actual defragmentation rate of the warm data and the actual defragmentation rate of the cold data based on the rate adjustment coefficient of the different heat level graded fragmented data, the fine tuning coefficient of the different heat level graded fragmented data and the target defragmentation rate.

[0014] Optionally, the method further comprises: executing different defragmentation strategies based on the different heat level graded fragmented data; wherein logical address remapping is performed for the hot data; incremental physical migration is performed for the warm data; and full physical migration is performed for the cold data.

[0015] Optionally, the method further comprises: if the current disk fragmentation rate is not greater than the preset fragmentation rate threshold, determining the idle time based on the historical data of the high performance storage device.

[0016] In the idle time, defragmentation is started.

[0017] In a second aspect, the embodiments of the present application provide a running optimization system of a high-performance storage device, comprising: a first acquisition module configured to acquire a response time of an I / O port of the high-performance storage device in a preset historical time period and a defragmentation rate calculated based on historical defragmentation data of the high-performance storage device; a construction module configured to construct a comparison relationship table based on a relationship between the response time of the I / O port and the defragmentation rate, and screen a target defragmentation rate range with a response time less than a preset time threshold; a second acquisition module configured to acquire a current I / O port queue depth if a current disk fragmentation rate is greater than a preset fragmentation rate threshold; a determination module configured to determine a target defragmentation rate based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range; and a defragmentation module configured to perform defragmentation based on the target defragmentation rate.

[0018] The beneficial effects of the present application include: first, the present application screens a target defragmentation rate range with a response time less than a preset time threshold by constructing a comparison relationship table representing a relationship between the response time of the I / O port and the defragmentation rate, thereby setting a performance safety boundary for the defragmentation rate from the source, and ensuring that the subsequent defragmentation process will not affect the processing requirement for the I / O response time.

[0019] Meanwhile, in actual application, the current I / O port queue depth, a real-time load indicator, is introduced, and the specific defragmentation rate is dynamically determined from the target defragmentation rate range in combination with the current disk fragmentation rate, thereby enabling the final defragmentation rate to adapt to real-time business pressure and match the defragmentation requirement. The above-mentioned manner breaks the limitation of the prior art relying on a single indicator, such as triggering only by the fragmentation rate, and finally realizes the dynamic adaptation of the defragmentation operation and the running of the storage device. While rapidly reducing the negative impact of disk fragmentation on storage performance, improving the storage throughput and resource utilization, and always ensuring that the I / O port response time is stable within the business acceptable range, the adaptability and running stability of the high-performance storage device in a high-load and high real-time scenario can be significantly enhanced, and the core performance advantage thereof can be fully exerted.

[0020] In addition, in actual application, the current disk fragmentation rate greater than the preset fragmentation rate threshold is taken as a triggering condition, thereby avoiding unnecessary defragmentation operation when the disk fragmentation rate is low, and reducing resource consumption. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A step flowchart of a running optimization method of a high-performance storage device provided by the embodiments of the present application;

[0022] Figure 2 A step flowchart of another running optimization method of a high-performance storage device provided by the embodiments of the present application;

[0023] Figure 3 A module block diagram of a running optimization system of a high-performance storage device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0025] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0026] At present, resource conflicts between the defragmentation operation and the real-time business needs of the storage device can occur, and it is difficult to achieve the goal of both efficiently reducing the fragmentation rate and guaranteeing the stable I / O performance, which restricts the performance of the high-performance storage device in high-load and high-real-time scenarios.

[0027] In view of the above problems, the present application proposes the following embodiments to solve the above technical problems.

[0028] Please refer to Figure 1 The present application provides a running optimization method of a high-performance storage device, comprising steps 101-105.

[0029] Step 101: Obtain the response time of the I / O port of the high-performance storage device within a preset historical time period and calculate the defragmentation rate by using the historical defragmentation data of the high-performance storage device.

[0030] The preset historical time period can be the past one month, one week, etc., and then the response time of the I / O port of the high-performance storage device within the preset historical time period is obtained.

[0031] The response time of the I / O port is a key indicator directly related to the core performance of the high-performance storage device, specifically referring to the total time length from the initiation of the I / O request to the completion of the request processing and the return of the result.

[0032] At the same time, the historical data when the defragmentation operation is performed in the past, such as the data volume and the defragmentation time length when a defragmentation is performed, are needed to calculate the defragmentation rate.

[0033] Step 102: Construct a comparison table based on the relationship between the response time of the I / O port and the defragmentation rate, and screen out the target defragmentation rate range with a response time less than a preset time threshold.

[0034] The comparison table here includes a one-to-one correspondence between the response time of the I / O port and the defragmentation rate, such as defragmentation rate 1 corresponding to response time 1, and defragmentation rate 2 corresponding to response time 2.

[0035] Then, by setting a preset time threshold (such as 100 microseconds, 1 millisecond), etc., all defragmentation rates that can make the response time lower than the preset time threshold are screened out from the comparison table, thereby forming the above-mentioned target defragmentation rate range.

[0036] Step 103: If the current disk fragmentation rate is greater than a preset fragmentation rate threshold, obtain the current I / O port queue depth.

[0037] The precondition before this step is that if the current disk fragmentation rate is greater than a preset fragmentation rate threshold. That is, in the process of the embodiment, the current disk fragmentation rate is monitored in real time, and only when the current disk fragmentation rate is greater than the preset fragmentation rate threshold, the current I / O port queue depth is obtained.

[0038] The above-mentioned preset fragmentation rate threshold can be set according to the needs, such as 20%, 30%, etc. The threshold can also be determined based on the performance decay inflection point of the high-performance storage device, and at the same time, the threshold can also indicate the node that needs to be defragmented.

[0039] The current I / O port queue depth is obtained to introduce a key indicator reflecting the real-time load pressure of the high-performance storage device, such as a high queue depth (such as a queue length > 50), which means that the current business I / O request is intensive, and the storage controller resource is tight, and vice versa, which means that the resource is relatively idle.

[0040] Step 104: Determine the target defragmentation rate based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range.

[0041] Then, combined with the dynamic decision of the three types of parameters, a unique target defragmentation rate is determined from the target defragmentation rate range based on the current I / O port queue depth and the current disk fragmentation rate.

[0042] Among them, the target defragmentation rate range mainly reflects the performance boundary, the current I / O port queue depth mainly reflects the real-time load, and the current disk fragmentation rate mainly reflects the defragmentation demand. The three are comprehensively decided to output the target defragmentation rate that meets the conditions.

[0043] Step 105: defragmentation is performed based on the target defragmentation rate.

[0044] Finally, the high-performance storage device is controlled to perform the defragmentation operation according to the determined target defragmentation rate.

[0045] In summary, the method for optimizing the operation of a high-performance storage device provided by the embodiments has the following beneficial effects:

[0046] Firstly, the application filters out a target defragmentation rate range with a response time less than a preset time threshold by constructing a comparison table representing the relationship between the response time of the I / O port and the defragmentation rate, thereby setting a performance safety boundary for the defragmentation rate from the source, and ensuring that the subsequent defragmentation process will not affect the processing requirement for the I / O response time.

[0047] Meanwhile, in actual application, the current I / O port queue depth, a real-time load indicator, is introduced, and the specific defragmentation rate is dynamically determined from the target defragmentation rate range in combination with the current disk fragmentation rate, thereby making the final defragmentation rate adapt to the real-time business pressure and match the fragmentation removal requirement. The above-mentioned method breaks the limitation of relying on a single indicator in the prior art, such as triggering only by the fragmentation rate, and finally realizes the dynamic adaptation of the defragmentation operation and the business operation of the storage device. While rapidly reducing the negative impact of disk fragmentation on storage performance and improving the storage throughput and resource utilization, the I / O port response time is always stabilized in the business acceptable range, which can significantly enhance the adaptation capability and operation stability of the high-performance storage device in high-load and high-real-time scenarios, and fully exert its core performance advantages.

[0048] In addition, in actual application, the current disk fragmentation rate greater than a preset fragmentation rate threshold is taken as the triggering condition, which avoids unnecessary defragmentation operation when the disk fragmentation rate is low and reduces resource consumption.

[0049] In addition, it should be noted that the embodiments of the application mainly focus on the relationship between the response time of the I / O port and the defragmentation rate, and the remaining influencing factors can be controlled within an acceptable range by existing means. Of course, in some scenarios, a plurality of comparison tables of the response time of the I / O port and the defragmentation rate under different influencing factors (such as hardware level factors and system configuration level factors) can also be constructed. In actual application, the comparison table matched with the current factor is selected for cooperative optimization, and the application does not make any limitation in this regard.

[0050] Optionally, the defragmentation rates of N time periods are calculated through historical defragmentation data of the high-performance storage device; N is a positive integer greater than or equal to 2.

[0051] Correspondingly, the step of constructing the comparison table based on the relationship between the response time of the I / O port and the defragmentation rate includes: obtaining the response time of the I / O port corresponding to each time period, and averaging the response time of the I / O port corresponding to each time period; and constructing the comparison table based on the average value of the response time of the I / O port in the N time periods and the defragmentation rate in the N time periods.

[0052] That is, all the response time data of the I / O port in each time period is collected through the N time periods divided by the historical defragmentation data, and then the average value of the response time of the I / O port in each time period can be calculated by means of arithmetic average or weighted average (according to the importance of the business).

[0053] Then, the N groups of segmented defragmentation rates are corresponded to the segmented average response time to form a structured comparison table.

[0054] Since the segmented data covers the rate and response time correlation under different load scenarios, the constructed comparison table can more comprehensively reflect the influence law of different rates on the response time.

[0055] Next, the determination method of the target defragmentation rate is described.

[0056] Optionally, the step of determining the target defragmentation rate based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range includes: determining the current load interval based on the current I / O port queue depth; wherein different load intervals correspond to different intervals of the target defragmentation rate range; determining the fragmentation rate level based on the current disk fragmentation rate; wherein different fragmentation rate levels correspond to different values of the target defragmentation rate range; and determining the target defragmentation rate from the interval of the target defragmentation rate corresponding to the current load interval based on the fragmentation rate level.

[0057] For example, it is assumed that the target defragmentation rate range determined by the foregoing embodiment is 50MB / s~150MB / s.

[0058] Different load intervals correspond to different intervals of the target defragmentation rate range. For example, the I / O port queue depth less than or equal to 20 is a low load interval, the I / O port queue depth greater than 20 and less than or equal to 50 is a medium load interval, and the I / O port queue depth greater than 50 is a high load interval.

[0059] Based on the division of the foregoing load intervals, the low load interval corresponds to 100MB / s~150MB / s, the medium load corresponds to 70MB / s~100MB / s, and the high load corresponds to 50MB / s~70MB / s.

[0060] If the current I / O port queue depth is 40, the corresponding low load interval is 100MB / s~150MB / s.

[0061] Then, based on the current disk fragmentation rate, the fragmentation rate level is determined. For example, the disk fragmentation rate less than or equal to 22% is set as a low level, the disk fragmentation rate greater than 22% and less than or equal to 30% is set as a medium level, and the disk fragmentation rate greater than 30% is set as a high level.

[0062] It should be noted that the higher the fragmentation rate level, the more urgent the need for consolidation, and the value is more biased towards the upper limit of the sub-interval of the current load interval (to speed up the consolidation efficiency); the lower the fragmentation rate level, the value is more biased towards the lower limit of the sub-interval (to reduce resource consumption). That is, each interval range can be divided into three sub-intervals, and according to the fragmentation rate level, it is determined which sub-interval the final target defragmentation rate falls into. For example, the low level corresponds to 100MB / s~120MB / s; the medium level corresponds to 121MB / s~135MB / s; and the high level corresponds to 136MB / s~150MB / s.

[0063] Finally, based on the fragmentation rate level, the target defragmentation rate is determined from the interval corresponding to the target defragmentation rate of the current load interval. The final target defragmentation rate can be determined by randomly selecting a value in the sub-interval, or by taking the average value of the sub-interval, or by taking the maximum value of the sub-interval, which is not limited by the present application.

[0064] The above-mentioned determination method of the target defragmentation rate realizes real-time load fine adaptation through load interval division, converts I / O queue depth into explicit load interval, and corresponds to the sub-interval of the target rate range, so that the association between load and rate is more concrete and can be implemented. It can avoid resource competition caused by high load rate, and avoid resource waste caused by low load rate, so that the rate adjustment is more suitable for the real-time running pressure of the storage device, further ensuring the stability of the I / O port response time. Secondly, the accurate matching of the consolidation demand is realized through the fragmentation rate level division. The disk fragmentation rate is upgraded to multi-level division, so that the defragmentation rate can be dynamically adjusted according to the demand urgency, i.e. the consolidation efficiency and the fragmentation demand are accurately matched. That is, by combining the current load interval and the fragmentation rate level, the accuracy, operability and scene adaptability of the target defragmentation rate determination can be further improved.

[0065] Optionally, the step of determining the target defragmentation rate based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range comprises: determining a first coefficient based on the current I / O port queue depth; determining a first defragmentation value based on the first coefficient and the target defragmentation rate range; determining a second coefficient based on the current disk fragmentation rate; determining a second defragmentation value based on the second coefficient and the target defragmentation rate range; determining a load weight and a defragmentation weight according to the current business scenario; and determining the target defragmentation rate based on the first defragmentation value, the load weight, the second defragmentation value, and the defragmentation weight.

[0066] wherein the first defragmentation value = the minimum value of the target defragmentation rate range + the first coefficient x (the maximum value of the target defragmentation rate range - the minimum value of the target defragmentation rate range).

[0067] The first coefficient is a quantitative representation of real-time load pressure, and has a value range of [0, 1] and is negatively correlated with the I / O port queue depth, i.e., the larger the queue depth, the smaller the first coefficient, and the smaller the queue depth, the larger the first coefficient. For example, when the queue depth is less than or equal to 20, the first coefficient is 0.8; when the queue depth is greater than 20 and less than or equal to 50, the first coefficient is 0.5; and when the queue depth is greater than 50, the first coefficient is 0.2.

[0068] For example, when the target defragmentation rate range is 100 MB / s~150 MB / s and the queue depth is 10, the first defragmentation value = 100 + 0.8 x (150-100) = 140 MB / s.

[0069] The second defragmentation value = the minimum value of the target defragmentation rate range + the second coefficient x (the maximum value of the target defragmentation rate range - the minimum value of the target defragmentation rate range).

[0070] The second coefficient is a quantitative representation of the urgency of defragmentation demand, and also has a value range of [0, 1] and is positively correlated with the disk fragmentation rate, i.e., the higher the disk fragmentation rate, the larger the second coefficient, and the lower the disk fragmentation rate (the lower the defragmentation demand), the smaller the second coefficient. For example, when the disk fragmentation rate is less than or equal to 22%, the second coefficient is 0.3; when the disk fragmentation rate is greater than 22% and less than or equal to 30%, the second coefficient is 0.6; and when the disk fragmentation rate is greater than 30%, the second coefficient is 0.8.

[0071] For example, when the target defragmentation rate range is 100 MB / s~150 MB / s and the disk fragmentation rate is 24%, the second defragmentation value = 100 + 0.6 x (150-100) = 130 MB / s.

[0072] Then, according to the current business scenario, the load weight and the fragment processing weight are determined; for example, in a high real-time scenario (such as financial high-frequency trading, industrial real-time control): load weight > fragment processing weight, then the load weight can be 0.7, and the fragment processing weight is 0.3. Of course, in a normal scenario, the load weight and the fragment processing weight can be 0.5. Finally, the target fragment consolidation rate is calculated in combination with the weights, such as in a high real-time scenario, the target fragment consolidation rate = 0.7*140MB / s + 0.3*130MB / s = 137MB / s.

[0073] The above-mentioned determination mode of the target fragment consolidation rate provides a more refined, scenario-based and standardized mode through the design of the coefficient and the weight balance, and significantly optimizes the running adaptation capability of the high-performance storage device under multi-dimensional requirements. The real-time I / O queue depth and the current disk fragmentation rate are respectively converted into continuous coefficients in the [0, 1] interval, and the rate reference value corresponding to the load adaptation and the consolidation requirement is obtained through linear operation, which effectively avoids the boundary rigidity problem of discrete grading, makes the rate adjustment more smoothly fit the real running state of the storage device, and realizes the refined quantitative balance of the load pressure and the consolidation requirement. In addition, through the configurable load weight and the fragment processing weight, the priority can be flexibly allocated according to the core demand of different business scenarios, solving the defect that the one-size-fits-all fixed strategy cannot adapt to diversified scenarios.

[0074] The fragment consolidation process provided by the embodiments of the present application will be described in detail below.

[0075] Please refer to Figure 2 The above-mentioned step of performing fragment consolidation based on the target fragment consolidation rate includes steps 201-203.

[0076] Step 201: The hotness of the fragment data is graded to determine hot data, warm data and cold data in the fragment data.

[0077] Different rate adjustment coefficients are pre-set for different hotness-graded fragment data.

[0078] The hot data, the warm data and the cold data are determined by screening according to the access frequency.

[0079] The rate adjustment coefficient is negatively correlated with the data hotness, that is, the higher the hotness, the smaller the coefficient, and the lower the hotness, the larger the coefficient. For example, the rate adjustment coefficient of the hot data adjustment coefficient is 0.1, the rate adjustment coefficient of the warm data adjustment coefficient is 0.8, and the rate adjustment coefficient of the cold data adjustment coefficient is 1.2.

[0080] Step 202: Based on the rate adjustment coefficients of different hotness-graded fragment data and the target fragment consolidation rate, the actual consolidation rate of the hot data, the actual consolidation rate of the warm data and the actual consolidation rate of the cold data are determined.

[0081] If the target defragmentation rate is 100 MB / s, the actual defragmentation rate of hot data = 100 MB / s x 0.1 = 10 MB / s, the actual defragmentation rate of warm data = 100 MB / s x 0.8 = 80 MB / s, and the actual defragmentation rate of cold data = 100 MB / s x 1.2 = 120 MB / s.

[0082] Here, the target defragmentation rate is used as the overall constraint reference, and the rate is fine-tuned by adjusting the coefficients, which ensures that the defragmentation rates of all data fluctuate around the overall reference and do not exceed the set I / O performance safety boundary, and allocates differentiated defragmentation resources according to the access requirement differences of different heat data.

[0083] Step 203: Defragmentation is performed based on the actual defragmentation rate of hot data, the actual defragmentation rate of warm data, and the actual defragmentation rate of cold data.

[0084] Finally, the defragmentation of different heat data is performed in a differentiated manner.

[0085] In summary, by designing the defragmentation strategy of the data, the defragmentation strategy is further optimized from the data dimension, and by the above-mentioned manner, the real-time access performance of hot data can be maximized, and the defragmentation efficiency of cold data can be significantly improved. Specifically, hot data is the core data of the business, and its access delay directly determines the business continuity, therefore, the defragmentation rate of hot data is adjusted to be much lower than the target rate, which greatly reduces the occupation of hot data access resources by the defragmentation operation. And the cold data has a much lower sensitivity to the defragmentation rate than the hot data because of its extremely low access frequency, therefore, the defragmentation rate of cold data is adjusted to be slightly higher than the target rate, which can fully utilize the resources of the storage device to quickly remove cold data fragments, avoid long-term accumulation of cold data fragments occupying storage space, and affect the service life of the storage medium, and at the same time, there is no need to worry about the impact of the high rate on the read-write business.

[0086] Optionally, under the above-mentioned embodiment, the method further comprises: continuously monitoring the I / O port queue depth during the defragmentation process; in response to monitoring that the I / O port queue depth exceeds a first preset threshold, the rate adjustment coefficients of the defragmentation data of different heat levels are adjusted by a first preset proportion; or, in response to monitoring that the I / O port queue depth is lower than a second preset threshold, the rate adjustment coefficients of the defragmentation data of different heat levels are adjusted by a second preset proportion; wherein the first preset threshold is greater than the second preset threshold.

[0087] That is, the I / O port queue depth is monitored in real time during the defragmentation process, so as to capture the dynamic changes of the load in time.

[0088] It should be noted that the first preset threshold is a critical value for defining a high load risk, which can be set based on the I / O processing upper limit and the business tolerance. When the queue depth exceeds the first preset threshold, the surface I / O request is intensive, and the resource is in short supply, and the resource occupation of the defragmentation needs to be reduced to avoid performance conflicts. Therefore, the rate adjustment coefficients of the fragmented data of different heat levels are collectively reduced by the first preset proportion. For example, the first preset proportion can be 20%. If the original adjustment coefficient of hot data is 0.3, the original adjustment coefficient of warm data is 0.8, and the original adjustment coefficient of cold data is 1.4, the first preset proportion is 20%, and the adjustment coefficient of hot data after reduction is 0.24, the adjustment coefficient of warm data is 0.64, and the adjustment coefficient of cold data is 1.12.

[0089] The second preset threshold can be a critical value for defining a low load. When the queue depth is lower than the second preset threshold, it indicates that the current I / O request is less, and the resource is idle, and the defragmentation rate can be improved to speed up the efficiency. For example, the second preset proportion can be 15%. If the original adjustment coefficient of hot data is 0.3, the original adjustment coefficient of warm data is 0.8, and the original adjustment coefficient of cold data is 1.4, the first preset proportion is 20%, and the adjustment coefficient of hot data after increase is 0.34, the adjustment coefficient of warm data is 0.92, and the adjustment coefficient of cold data is 1.61.

[0090] It can be seen that through the real-time load monitoring and dynamic adjustment mechanism of overall scaling of the coefficient, the performance conflict or resource waste problem caused by load mutation in the defragmentation process can be further solved, and the defragmentation strategy is more flexible and robust. Specifically, it can dynamically respond to load fluctuations in the defragmentation process and strengthen the reliability of I / O performance guarantee. During the execution of the defragmentation, the high-performance storage device can face a sudden high load. At this time, by continuously monitoring the queue depth, the coefficient is collectively reduced when the load exceeds the first preset threshold, which can quickly reduce the resource occupation of the defragmentation, avoid the extension of the response time caused by the competition for resources between the defragmentation and the business I / O, and resist the performance risk brought by the sudden load. When the load is lower than the second preset threshold, the coefficient is increased, which can fully utilize the idle resources to speed up the defragmentation progress, avoid resource idling, and realize the dynamic balance of performance protection during load fluctuation and efficiency improvement during load idling.

[0091] Optionally, the actual defragmentation rate of the hot data, the actual defragmentation rate of the warm data, and the actual defragmentation rate of the cold data are determined based on the rate adjustment coefficients of the fragmented data of different heat levels and the target defragmentation rate, and the method further comprises: determining the fragmented data distribution of different heat levels based on the fragmented data of different heat levels; determining the respective fine-tuning coefficients based on the fragmented data distribution of different heat levels; and determining the actual defragmentation rate of the hot data, the actual defragmentation rate of the warm data, and the actual defragmentation rate of the cold data based on the rate adjustment coefficients of the fragmented data of different heat levels, the fine-tuning coefficients of the fragmented data of different heat levels, and the target defragmentation rate.

[0092] The fragmented data distribution can include the fragmented data size and the fragmented data dispersion degree.

[0093] It should be noted that the fine-tuning coefficient is a quantitative correction index for the fragmented data distribution defragmentation cost, and the core logic is negatively correlated with the defragmentation difficulty of the fragmented data distribution. For example, the higher the proportion of small fragmented data and high dispersion in certain heat data, the greater the defragmentation difficulty and resource consumption, and the smaller the fine-tuning coefficient, which needs to further reduce the defragmentation rate to avoid excessive resource occupation. The higher the proportion of large fragmented data and low dispersion in certain heat data, the smaller the defragmentation difficulty and resource consumption, and the larger the fine-tuning coefficient, which can appropriately increase the rate to improve the defragmentation efficiency.

[0094] The value range of the fine-tuning coefficient is 0.8-1.2.

[0095] In an embodiment, the actual defragmentation rate can be equal to the target defragmentation rate multiplied by the rate adjustment coefficient multiplied by the fine-tuning coefficient.

[0096] It can be seen that, through the design of the fragmentation morphology quantization and the double-coefficient collaborative calculation, the defragmentation rate mismatch problem caused by different fragmented data distributions of the same heat data is further optimized, and the defragmentation strategy of the high-performance storage device is more suitable for the actual operation and maintenance scene.

[0097] Optionally, the method further comprises: executing different defragmentation strategies based on the fragmented data of different heat levels; wherein, the logical address remapping is performed on the hot data; the incremental physical migration is performed on the warm data; and the full-amount physical migration is performed on the cold data.

[0098] The logical address remapping refers to not moving the physical storage location of the hot data, but only modifying the logical address and physical address mapping table in the storage controller to associate the originally dispersed logical addresses corresponding to the physical addresses to the continuous virtual logical address segment.

[0099] The incremental physical migration refers to performing physical migration and merging only on the fragmented data in the warm data that has not been modified or newly added in the recent period of time.

[0100] And the full physical migration refers to the defragmentation of all cold data.

[0101] It can be seen that the above-mentioned mode further differentiates the processing from the operation mode level on the basis of rate differentiation, constructs a full-dimension differentiated system of heat level grading and strategy matching, and further improves the scene adaptability and practical value.

[0102] Optionally, under the above-mentioned embodiment, the method further comprises: if the current disk fragmentation rate is not greater than the preset fragmentation rate threshold, determining the idle time based on the historical data of the high-performance storage device; and starting the defragmentation at the idle time.

[0103] That is, if the current disk fragmentation rate is not greater than the preset fragmentation rate threshold, no resource needs to be occupied for emergency defragmentation, but the idle time can be determined based on the historical data to filter out the period with the lowest storage device load and the least business access for defragmentation.

[0104] Please refer to Figure 3 Based on the same inventive concept, the application also provides a high-performance storage device operation optimization system 30, comprising:

[0105] The first acquisition module 301 is configured to acquire the response time of the I / O port of the high-performance storage device within a preset historical time period and calculate the defragmentation rate based on the historical defragmentation data of the high-performance storage device.

[0106] The construction module 302 is configured to construct a comparison relationship table based on the relationship between the response time of the I / O port and the defragmentation rate, and filter out a target defragmentation rate range with a response time less than a preset time threshold.

[0107] The second acquisition module 303 is configured to acquire the current I / O port queue depth if the current disk fragmentation rate is greater than the preset fragmentation rate threshold.

[0108] The determination module 304 is configured to determine a target defragmentation rate based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range.

[0109] The defragmentation module 305 is configured to perform defragmentation based on the target defragmentation rate.

[0110] It should be noted that the above-mentioned system and the like are based on the same concept as the method embodiments, and the steps performed by the modules of the system and the technical effects brought by the modules can be referred to the method embodiment part, which will not be described here.

[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software functional unit. In addition, the specific names of the functional units and modules are only for easy distinction, and do not limit the protection scope of the application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0112] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments.

[0113] The embodiments of the present application provide a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal is caused to implement the steps in the above method embodiments.

[0114] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above embodiment methods, which can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium. The computer program is executed by a processor to implement the steps in the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0115] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0116] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0117] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division, and there can be another division in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or in other forms.

[0118] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0119] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for optimizing the operation of a high-performance storage device, characterized in that, include: The response time of the I / O ports of the high-performance storage device within a preset historical time period is obtained, and the defragmentation rate is calculated using the historical defragmentation data of the high-performance storage device. Based on the relationship between I / O port response time and defragmentation rate, a comparison table was constructed, and the target defragmentation rate range with response time less than a preset time threshold was selected. If the current disk fragmentation rate is greater than the preset fragmentation rate threshold, obtain the current I / O port queue depth; The target defragmentation rate is determined based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range; wherein the target defragmentation rate is inversely proportional to the current I / O port queue depth and directly proportional to the current disk fragmentation rate. Fragmentation is performed based on the target fragmentation rate; The step of determining the target defragmentation rate based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range includes: determining a first coefficient based on the current I / O port queue depth; determining a first defragmentation value based on the first coefficient and the target defragmentation rate range; determining a second coefficient based on the current disk fragmentation rate; determining a second defragmentation value based on the second coefficient and the target defragmentation rate range; determining a load weight and a fragmentation processing weight according to the current business scenario; and determining the target defragmentation rate based on the first defragmentation value, the load weight, the second defragmentation value, and the fragmentation processing weight.

2. The method for optimizing the operation of a high-performance storage device according to claim 1, characterized in that, The defragmentation rate for N time periods is calculated using historical defragmentation data from the high-performance storage device. N is a positive integer greater than or equal to 2; The relationship between I / O port response time and defragmentation rate is described in a comparison table, including: Obtain the response time of the I / O port for each time period, and take the average response time of the I / O port for each time period; The comparison table is constructed based on the average response time of the I / O ports over N time periods and the defragmentation rate over N time periods.

3. The method for optimizing the operation of a high-performance storage device according to claim 1, characterized in that, The determination of the target defragmentation rate based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range includes: Based on the current I / O port queue depth, the current load range is determined; where different load ranges correspond to different intervals of the target defragmentation rate range. Based on the current disk fragmentation rate, a fragmentation rate level is determined; where different fragmentation rate levels correspond to different values ​​within the target defragmentation rate range. Based on the fragmentation rate level, the target fragmentation rate is determined from the interval corresponding to the target fragmentation rate of the current load interval.

4. The method for optimizing the operation of a high-performance storage device according to claim 1, characterized in that, The defragmentation based on the target defragmentation rate includes: Fragmented data is classified by heat level to determine hot data, warm data, and cold data in the fragmented data; wherein, different rate adjustment coefficients are preset for fragmented data of different heat levels; Based on the rate adjustment coefficients of fragmented data with different heat levels and the target fragment sorting rate, the actual sorting rate of the hot data, the actual sorting rate of the warm data, and the actual sorting rate of the cold data are determined. Fragmentation is performed based on the actual processing rate of the hot data, the actual processing rate of the warm data, and the actual processing rate of the cold data.

5. The method for optimizing the operation of a high-performance storage device according to claim 4, characterized in that, The method further includes: During defragmentation, the I / O port queue depth is continuously monitored; In response to the detection that the I / O port queue depth exceeds a first preset threshold, the rate adjustment coefficient of fragmented data of different heat levels is reduced by a first preset ratio; or, In response to the detection that the I / O port queue depth is lower than the second preset threshold, the rate adjustment coefficient of fragment data of different heat levels is increased by the second preset ratio as a whole. Wherein, the first preset threshold is greater than the second preset threshold.

6. The method for optimizing the operation of a high-performance storage device according to claim 4, characterized in that, The method of determining the actual processing rate of hot data, the actual processing rate of warm data, and the actual processing rate of cold data based on the rate adjustment coefficients of fragmented data with different heat levels and the target fragment processing rate further includes: The distribution of fragment morphology is determined based on fragment morphology quantification detection for fragment data of different heat levels; Based on the morphological distribution of fragment data at different heat levels, the fine-tuning coefficients for each are determined. Based on the rate adjustment coefficients of fragmented data with different heat levels, the fine-tuning coefficients of fragmented data with different heat levels, and the target fragment sorting rate, the actual sorting rate of the hot data, the actual sorting rate of the warm data, and the actual sorting rate of the cold data are determined.

7. The method for optimizing the operation of a high-performance storage device according to claim 4, characterized in that, The method further includes: Different fragmentation strategies are implemented based on fragmented data with different popularity levels. Specifically, logical address remapping is performed on the hot data; incremental physical migration is performed on the warm data; and full physical migration is performed on the cold data.

8. The method for optimizing the operation of a high-performance storage device according to claim 1, characterized in that, The method further includes: If the current disk fragmentation rate is not greater than the preset fragmentation rate threshold, then the idle time is determined based on the historical data of the high-performance storage device; During the idle time, defragmentation is initiated.

9. A system for optimizing the operation of a high-performance storage device, characterized in that, include: The first acquisition module is used to acquire the response time of the I / O port of the high-performance storage device within a preset historical time period and to calculate the defragmentation rate using the historical defragmentation data of the high-performance storage device. The module is used to build a comparison table based on the relationship between I / O port response time and defragmentation rate, and to filter out the target defragmentation rate range where the response time is less than a preset time threshold. The second acquisition module is used to acquire the current I / O port queue depth if the current disk fragmentation rate is greater than a preset fragmentation rate threshold. The determination module is used to determine the target defragmentation rate based on the current I / O port queue depth, the current disk fragmentation rate, and the target defragmentation rate range; wherein the target defragmentation rate is inversely proportional to the current I / O port queue depth and directly proportional to the current disk fragmentation rate. The defragmentation module is used to defragment fragments based on the target defragmentation rate. The determining module is further specifically configured to: determine a first coefficient based on the current I / O port queue depth; determine a first defragmentation value based on the first coefficient and the target defragmentation rate range; determine a second coefficient based on the current disk fragmentation rate; determine a second defragmentation value based on the second coefficient and the target defragmentation rate range; determine a load weight and a fragmentation processing weight according to the current business scenario; and determine the target defragmentation rate based on the first defragmentation value, the load weight, the second defragmentation value, and the fragmentation processing weight.

Images