Data copy adjustment method and device, equipment, medium and program product

Abstract

The invention provides a data copy adjustment method and device, equipment, a medium and a program product. Relates to the technical field of distributed storage. The method comprises the following steps: for target storage equipment, determining relative performance data of the target storage equipment relative to all storage equipment; determining descriptive statistics of the relative performance data of the target storage device detected in a preset time period, and determining a performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device; and according to the performance evaluation result of the target storage device, determining an adjustment mode of the data copy in the target storage device. According to the technical scheme, the relative performance of the target storage device can be accurately evaluated according to the descriptive statistics of the relative performance data of the target storage device and all the storage devices, so that the data copy can be adjusted in time under the condition that the relative performance is poor, and the user experience is improved. And normal access of data is prevented from being affected by faults of the target storage device.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a data copy adjustment method, device, equipment, medium and program product. Background Art

[0002] With the development of cloud computing and big data technology, distributed storage systems have been widely used in various fields. Distributed storage systems improve the reliability and access speed of data storage by splitting and storing data on multiple devices at multiple nodes. However, as the use time of servers and devices increases, servers and devices will gradually age and enter sub-healthy and faulty states, thus affecting the reliability and availability of copy data. How to effectively manage data copies during the aging process of servers and devices to avoid data storage reliability degradation and uneven data distribution has become a key issue for distributed storage systems.

[0003] Existing intelligent replica selection and management strategies, including adjustment methods based on single factors such as access frequency, node load, and network latency, have solved the load balancing problem to a certain extent, but still have the following limitations: the number of replicas is fixed, the storage location is fixed, and it is impossible to flexibly and effectively respond to the impact of equipment aging; the impact of server and equipment aging on data access performance, reliability, availability, and data distribution is not considered. Summary of the invention

[0004] The present application provides a data copy adjustment method, apparatus, device, medium and program product to evaluate storage devices with relatively poor performance from a global perspective of storage devices, and adjust data copies in a timely manner to avoid affecting normal access.

[0005] In a first aspect, the present application provides a data replica adjustment method, comprising:

[0006] For a target storage device, determine relative performance data of the target storage device relative to all storage devices; wherein the performance data includes at least one of bandwidth, processing latency, data write volume, storage device temperature, and number of data write errors;

[0007] Determine descriptive statistics of relative performance data of a target storage device detected within a preset time period, and determine a performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device;

[0008] According to the performance evaluation result of the target storage device, an adjustment method for the data copy in the target storage device is determined.

[0009] In a second aspect, the present application provides a data replica adjustment device, comprising:

[0010] A relative performance data determination module, used to determine, for a target storage device, relative performance data of the target storage device relative to all storage devices; wherein the performance data includes at least one of bandwidth, processing latency, data write volume, storage device temperature, and number of data write errors;

[0011] A descriptive statistics determination module, used to determine the descriptive statistics of the relative performance data of the target storage device detected within a preset time period, and determine the performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device;

[0012] The adjustment mode determination module is used to determine the adjustment mode of the data copy in the target storage device according to the performance evaluation result of the target storage device.

[0013] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement a data copy adjustment method as described in any embodiment of the present application.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data copy adjustment method described in any embodiment of the present application when executed.

[0015] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the data copy adjustment method described in any embodiment of the present application.

[0016] The data copy adjustment method, apparatus, device, medium and program product provided by the present application, for a target storage device, determines the relative performance data of the target storage device relative to all storage devices; wherein the performance data includes at least one of bandwidth, processing delay, data write volume, storage device temperature and number of data write errors; determines the descriptive statistics of the relative performance data of the target storage device detected within a preset time period, and determines the performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device; determines the adjustment method for the data copy in the target storage device according to the performance evaluation result of the target storage device. The technical solution of the embodiment of the present application can accurately evaluate the relative performance of the target storage device according to the descriptive statistics of the relative performance data of the target storage device and all storage devices, so as to adjust the data copy in time when the relative performance is poor, and avoid the failure of the target storage device affecting the normal access of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] Figure 1 The process of the data copy adjustment method provided in the embodiment of the present application Figure 1 ;

[0019] Figure 2 The process of the data copy adjustment method provided in the embodiment of the present application Figure 2 ;

[0020] Figure 3 The process of the data copy adjustment method provided in the embodiment of the present application Figure 3 ;

[0021] Figure 4 A schematic diagram of the structure of a data replica adjustment device provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0023] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0024] The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. The acquisition, transmission, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0025] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0026] The embodiment of the present application also provides a data copy adjustment method. Figure 1 The process of the data copy adjustment method provided in the embodiment of the present application Figure 1. The embodiment of the present application may be applicable to the case where the account transaction of the target account is processed. Typically, the embodiment of the present application may be applicable to the case where the account transaction of the target account across transaction scenarios is processed. The method may be executed by a data copy adjustment device, which may be implemented in the form of hardware and / or software, and the data copy adjustment device may be configured in an electronic device. Figure 1 As shown, the data copy adjustment method includes:

[0027] S110. Determine, for a target storage device, relative performance data of the target storage device relative to all storage devices; wherein the performance data includes at least one of bandwidth, processing latency, data write volume, storage device temperature, and number of data write errors.

[0028] Among them, the storage device can be a device used to store data copies in distributed storage. The specific number of storage devices is not limited and can be set according to actual conditions. The target storage device can be any storage device in the storage device. For example, at least one storage device of all storage devices can be selected as the target storage device for analysis, and storage devices with a relatively long application date and storage devices with older versions can be selected, that is, it is easier to judge from the basic information that the probability of performance degradation is relatively high. Each storage device can also be used as a target storage device for performance analysis. Bandwidth is the network bandwidth when the storage device transmits data. The processing delay is the delay of the storage device's corresponding data copy processing request, reflecting the time required for the storage device to process the data copy processing request. The data write amount is the amount of data copies written to the storage device, and the storage device temperature is the temperature of the hardware device in the operation of the storage device. The number of data write errors is the number of times the storage device has errors when writing data.

[0029] For example, the absolute performance of the target storage device is difficult to reflect the performance of the target storage device in actual applications. For example, in some scenarios with heavy business volume, the performance of all storage devices may be lower than the performance of storage devices in scenarios with light business pressure. However, this does not mean that all storage devices in scenarios with heavy business volume are abnormal. Therefore, it is necessary to evaluate the relative performance of the target storage device in the current business scenario to accurately reflect whether the target storage device is normal in processing the current business.

[0030] Specifically, all storage devices can be used as a reference. Determine the relative performance data of the target storage device relative to all storage devices. If the relative performance data of the target storage device relative to all storage devices is better, it means that the performance of the target storage device is better among all storage devices. If the relative performance data of the target storage device relative to all storage devices is worse, it means that the performance of the target storage device is worse among all storage devices, that is, the performance in processing the current business is poor. The data copy of the target storage device should be adjusted first to avoid the target storage device with poor performance affecting the normal processing of the business.

[0031] S120: Determine descriptive statistics of relative performance data of a target storage device detected within a preset time period, and determine a performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device.

[0032] Descriptive statistics are. Descriptive statistics is a basic method in data analysis. It is a summary statistic used to quantitatively describe or summarize the characteristics of information sets. It is an effective way to grasp the distribution of data and reveal the laws behind it. It mainly includes indicators such as central quantity, difference quantity, skewness and kurtosis of distribution, and correlation coefficient between variables, such as mean, variance, standard deviation, etc.

[0033] Exemplarily, in order to improve the credibility and reliability of the relative performance data and avoid the misjudgment of the performance of the target storage device due to accidental events, the relative performance data of the target storage device may be continuously detected within a preset time period to determine the descriptive statistics of the relative performance data of the target storage device within the preset time period. For example, the mean, variance, standard deviation, etc. of the relative performance data of the target storage device within the preset time period may be determined.

[0034] Exemplarily, the descriptive statistics of the relative performance data of the target storage device can reflect the performance of the target storage device within a preset time period, and the performance evaluation result of the target storage device can be determined based on the descriptive statistics of the target storage device. The descriptive statistics of the relative performance data of the target storage device are quantitative data, and the performance evaluation result is the obtained performance evaluation conclusion, which can be described as excellent, good, acceptable, poor, normal, abnormal, healthy, sub-healthy, etc.

[0035] Exemplarily, the correspondence between the numerical range of the descriptive statistics of the relative performance data and the performance evaluation result may be predetermined, and the performance evaluation result of the target storage device may be determined according to the numerical range of the descriptive statistics of the relative performance data of the target storage device.

[0036] S130: Determine an adjustment method for the data copy in the target storage device according to the performance evaluation result of the target storage device.

[0037] Exemplarily, the adjustment method of the data copy in the target storage device can be determined according to the performance evaluation result of the target storage. For example, if the performance evaluation result of the target storage device is abnormal, the data copy in the target storage device can be adjusted, and if the performance evaluation result of the target storage device is normal, the data copy in the target storage device can be not adjusted. Alternatively, the performance evaluation results of all storage devices can be determined, and the data copy of the target storage device with a poor performance evaluation result can be adjusted, while the data copies of other storage devices are not adjusted, so as to avoid the performance degradation of some storage devices in the storage device affecting the normal processing of the current business.

[0038] The scheme in the embodiment of the present application determines, for a target storage device, relative performance data of the target storage device relative to all storage devices; wherein the performance data includes at least one of bandwidth, processing latency, data write volume, storage device temperature, and number of data write errors; determines descriptive statistics of the relative performance data of the target storage device detected within a preset time period, and determines a performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device; and determines an adjustment method for the data copy in the target storage device according to the performance evaluation result of the target storage device. The technical scheme in the embodiment of the present application can accurately evaluate the relative performance of the target storage device according to the descriptive statistics of the relative performance data of the target storage device and all storage devices, so as to adjust the data copy in time when the relative performance is poor, and avoid failure of the target storage device affecting normal data access.

[0039] Figure 2 The process of the data copy adjustment method provided in the embodiment of the present application Figure 2 .like Figure 2 As shown, the data copy adjustment method includes the following steps: It should be noted that S210-S220, S230-S24, S250-S260 in the embodiment of the present application do not have a strong correlation with each other, and there is no dependency between the steps. Each stage can be based on Figure 1 The corresponding embodiments are implemented on the basis of the schemes of the above embodiments.

[0040] S210. For each storage device, determine an average value of performance data detected by the storage device in a unit time period as average performance data.

[0041] For example, for each storage device, the average value of the performance data detected by the storage device in a unit time period can be determined as the average performance data. Assuming that the unit time is 15 seconds, 15 seconds is a monitoring window, and the average value of the performance data of the storage device detected in the calculation window is calculated, which is recorded as Dki , where k represents the kth storage device and i represents the first monitoring window.

[0042] S220: Determine relative performance data of the target storage device with respect to all storage devices based on the average performance data of the target storage device and the average performance data of all storage devices.

[0043] The relative performance data of the target storage device relative to all storage devices can be determined based on the average performance data of the target storage device and the average performance data of all storage devices. For example, the relative performance data can be determined by performing a ratio, a difference, or other calculation on the characteristic form of the average performance data of the target storage device and the average performance data of all storage devices.

[0044] In the embodiment of the present application, determining the relative performance data of the target storage device relative to all storage devices according to the average performance data of the target storage device and the average performance data of all storage devices includes:

[0045] Determining a median of the average performance data of each of the storage devices;

[0046] The ratio of the average performance data of the target storage device to the median is used as the relative performance data of the target storage device relative to all storage devices.

[0047] Exemplarily, the median of the average performance data of each storage device can be determined, and the ratio of the average performance data of the target storage device to the median can be used as the relative performance data of the target storage device and all storage devices. For example, RDki = Dki / median(D1i, D2i, ..., Dki), where RDki is the relative performance data of the target storage device, Dki is the average performance data of the target storage device, and median(D1i, D2i, ..., Dki) is the median of the average performance data of all storage devices. The above scheme can more objectively determine the relative performance data of the target storage device relative to all storage devices, and accurately reflect the performance of the target storage device in the current business scenario.

[0048] Exemplarily, the average value of the average performance data of all storage devices may also be determined, and the ratio of the average performance data of the target storage device to the average value of the average performance data of all storage devices may be used as the relative performance data of the target storage device relative to all storage devices.

[0049] S230. For at least two unit time periods included in the preset time period, respectively determine relative performance data of the target storage device relative to all storage devices within the unit time period, and obtain at least two relative performance data corresponding to the target storage device.

[0050] Exemplarily, the target storage device and all storage devices are continuously detected, with unit time as a monitoring window, at least two unit time periods included in a preset time period as at least two detection cycles, and at least two relative performance data corresponding to the target storage device in at least two monitoring windows are determined.

[0051] For example, assuming that the unit time is 15 seconds, that is, 15 seconds is a monitoring window. The preset time period is 5 minutes, and the preset time period includes 20 unit times and 20 monitoring windows. In each unit time, the relative performance data corresponding to a target storage device can be calculated according to the above process, and 20 relative performance data can be obtained in 5 minutes.

[0052] S240: Determine the descriptive statistics according to at least two relative performance data.

[0053] Exemplarily, at least two relative performance data may be operated to determine the descriptive statistics. The operation method is not limited and may be set according to actual conditions.

[0054] In an embodiment of the present application, the descriptive statistics are determined according to at least two relative performance data, including:

[0055] At least one of a mean, a variance and a standard deviation of at least two relative performance data is determined as the descriptive statistic.

[0056] Exemplarily, assuming that the unit time within the preset time period includes the i-th unit time, the i+1-th unit time, the i+2-th unit time, ..., the j-th unit time, then the relative performance data of the k-th target storage device can be obtained including RDKi, RDKi+1, RDKi+2, ..., RDKj. Assuming that the mean is used as the calculation method of the descriptive statistics, the descriptive statistics of the relative performance data are Ek,i,j=mean(RDKi, RDKi+1, RDKi+2, ..., RDKj). mean represents the average value. The above scheme can accurately reflect the relatively stable performance status of the target storage device within the preset time period through descriptive statistics, and avoid accidental performance changes that affect the accuracy of the performance evaluation of the target storage device.

[0057] S250: Determine whether the descriptive statistics corresponding to the target storage device exceeds a normal statistics range threshold.

[0058] Exemplarily, a normal statistic range threshold may be preset to reflect the critical value of the normal and abnormal descriptive statistics of the target storage device. When the preset time period is fixed, the performance of the target storage device may be determined based on the specific value of the descriptive statistics of the target storage device within the preset time period, that is, whether the descriptive statistics corresponding to the target storage device exceeds the normal statistic range threshold.

[0059] S260: If it exceeds, determine that the performance of the target storage device is abnormal.

[0060] Exemplarily, if the descriptive statistics corresponding to the target storage device exceeds the normal statistics range threshold, the performance of the target storage device is determined to be abnormal. For example, assuming that the relative performance data is a descriptive statistic of the relative delay, for example, 2 microseconds, and the normal statistics range threshold corresponding to the delay is 0.5 microseconds. Since the relative delay is negatively correlated with the performance of the target storage device, when the descriptive statistics of the relative delay exceeds the normal statistics range threshold corresponding to the delay along the increasing trend, it is considered that the target storage device is abnormal. 2 microseconds is greater than 0.5 microseconds, and the descriptive statistics of the relative delay exceeds the normal statistics range threshold corresponding to the delay, it is determined that the performance of the target storage device is abnormal. Assuming that the relative performance data is a descriptive statistic of the relative bandwidth, for example, 6Mbps, and the normal statistics range threshold corresponding to the bandwidth is 10Mbps. Since the relative bandwidth is positively correlated with the performance of the target storage device, when the descriptive statistics of the relative bandwidth exceeds the normal statistics range threshold corresponding to the bandwidth along the decreasing trend, it is considered that the target storage device is abnormal. If 6Mbps is less than 10Mbps and the descriptive statistics of the relative delay exceed the normal statistical range threshold corresponding to the delay, it is determined that the performance of the target storage device is abnormal.

[0061] In the embodiment of the present application, determining that the performance of the target storage device is abnormal includes:

[0062] In the case where the preset time period is fixed, for different target storage devices whose descriptive statistics all exceed the preset statistical threshold, if the performance data includes at least one of processing latency, data write volume, storage device temperature, and number of data write errors, it is determined that the performance abnormality degree of the target storage device is positively correlated with the descriptive statistics;

[0063] If the performance data includes bandwidth, it is determined that the degree of performance abnormality of the target storage device is negatively correlated with the descriptive statistics.

[0064] Exemplarily, if the performance data includes at least one of processing delay, data write volume, storage device temperature, and number of data write errors, the larger the performance data, the worse the performance of the target storage device, and the higher the degree of performance abnormality. Therefore, it can be determined that the degree of performance abnormality of the target storage device is positively correlated with the descriptive statistics. If the performance data includes bandwidth, the larger the performance data, the better the performance of the target storage device, and the lower the degree of performance abnormality. Therefore, it can be determined that the degree of performance abnormality of the target storage device is negatively correlated with the descriptive statistics. The above scheme can intuitively reflect the degree of abnormality of the target storage device according to the descriptive statistics of the relative performance data when it is determined that the target storage device has an abnormality, so as to adaptively determine the adjustment method of the data copy according to the degree of abnormality.

[0065] In the embodiment of the present application, determining that the performance of the target storage device is abnormal includes:

[0066] When the preset time period is not fixed, if the descriptive statistic corresponding to the target storage device exceeds the normal statistic range threshold, the performance abnormality of the target storage device is positively correlated with the duration that the descriptive statistic exceeds the normal statistic range threshold.

[0067] Exemplarily, if the preset time period is not fixed, the duration for which the descriptive statistics corresponding to the target storage device exceeds the normal statistics range threshold may be counted. The longer the duration, the greater the degree of performance abnormality of the target storage device.

[0068] The scheme in the embodiment of the present application determines, for each storage device, the average value of the performance data detected by the storage device within a unit time period as the average performance data; determines the relative performance data of the target storage device relative to all storage devices based on the average performance data of the target storage device and the average performance data of all storage devices. Determine the descriptive statistics of the relative performance data of the target storage device detected within a preset time period, and determine the performance evaluation result of the target storage device based on the descriptive statistics corresponding to the target storage device; determine the adjustment method for the data copy in the target storage device based on the performance evaluation result of the target storage device. The technical scheme of the embodiment of the present application can determine the average performance data of all storage devices, and determine the relative performance data of the target storage device relative to all storage devices based on the average performance data of all storage devices and the average performance data of the target storage device, so as to reflect the relative performance of the target storage device among all storage devices, and reflect the relative performance of the target storage device in the process of processing the current business, so as to select the target storage device with relatively poor performance to adjust the data copy.

[0069] Figure 3The process of the data copy adjustment method provided in the embodiment of the present application Figure 3 .like Figure 3 As shown, the data replica adjustment method includes the following steps:

[0070] S310. Determine, for a target storage device, relative performance data of the target storage device relative to all storage devices; wherein the performance data includes at least one of bandwidth, processing latency, data write volume, storage device temperature, and number of data write errors.

[0071] S320: Determine descriptive statistics of relative performance data of a target storage device detected within a preset time period, and determine a performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device.

[0072] S330: If the performance evaluation result of the target storage device is abnormal performance, determine whether there is currently any input / output operation on the target storage device.

[0073] Exemplarily, if the performance evaluation result of the target storage device is determined to be abnormal according to the solution of the above embodiment, the input and output operations of the target storage device can be adaptively detected, and the adjustment method of the data copy can be adaptively determined according to the input and output operations.

[0074] S340: Determine an adjustment method for the data copy in the target storage device according to the judgment result of the input and output operation.

[0075] Exemplarily, the adjustment method of the data copy in the target storage device may be determined according to the judgment result of the input and output operation.

[0076] In an embodiment of the present application, determining the adjustment method of the data copy in the target storage device according to the judgment result of the input and output operation includes:

[0077] If there is a write operation on the target storage device, synchronously perform the write operation on the target storage device and the new storage device;

[0078] If the write operation is completed, copying the data copy in the target storage device before the write operation is performed to the new storage device;

[0079] If there is a read operation on the target storage device, after the read operation is performed on the target storage device, the data copy in the target storage device is copied to the new storage device.

[0080] Exemplarily, if there is a write operation on the target storage device, a new storage device can be added, and the write operation can be performed synchronously on the target storage device and the new storage device to avoid affecting the normal execution of the write operation. If the write operation is completed, the data copy before the write operation is performed in the target storage device is copied to the new storage device. In the target storage device, the method for distinguishing the data copy before the write operation and the data copy generated by the write operation can be to distinguish according to the timestamp when the write operation starts, or to distinguish according to the version of the data copy. The above scheme can realize real-time backup of the write operation when there is a write operation on the target storage device, and back up the old data copy after the write operation, so as to realize the backup of the data copy in the target storage device, avoid the performance degradation of the target storage device affecting the normal access to the data copy, and will not affect the normal execution of the current write operation.

[0081] If there is a read operation in the target storage device, in order to avoid affecting the current read operation, the read operation can be performed in the target storage device first. After the read operation is completed, the data copy of the target storage device is copied to the new storage device to avoid affecting the read operation and realize the backup of the data copy in the target storage device.

[0082] In an embodiment of the present application, determining the adjustment method of the data copy in the target storage device according to the judgment result of the input and output operation includes:

[0083] If there is no input / output operation on the target storage device, the data copy in the target storage device is copied to the new storage device to form a target number plus one data copy synchronization; wherein the target number is the number of all storage devices except the new storage device.

[0084] Exemplarily, if there is no input / output operation in the target storage device, the data copy in the target storage device can be directly copied to the new storage device, forming the target number plus one data copy synchronization. The target number is the number of all existing storage devices. For example, if there are originally 3 storage devices, a new storage device is added, and the data copy in the target storage device is copied to the new storage device, forming 4 data copy synchronizations. The above scheme can synchronize the data copy of the target storage device in a timely manner, avoiding the problem of data copy access failure due to performance degradation of the target storage device. And it can achieve the synchronization of the target number plus one data copy, and when the target storage device does not actively fail, the target storage device will not be passively deactivated, maximizing the utilization of the target storage device.

[0085] In the embodiment of the present application, the process of copying the data copy to the new storage device includes:

[0086] If the priority of business processing is higher than the priority of data copy replication, then the storage system is checked for business processing operations;

[0087] If there is no business processing operation, copying the data copy to the new storage device;

[0088] If there is a business processing operation, wait for the business processing operation to end and copy the data copy to the new storage device.

[0089] For example, the priority of business processing and data copy replication can be determined in advance, that is, whether the real-time performance and success rate of business processing are more concerned or the stability and security of data copies are more concerned. If the priority of business processing is higher than the priority of data copy replication, the storage system can be checked whether there is a business processing operation before backing up the data copy. If there is no business processing operation, the data copy backup operation is performed. If there is a business processing operation, the business processing operation is waited for to end, and the data copy backup operation is performed to avoid affecting the business processing.

[0090] The scheme in the embodiment of the present application determines the relative performance data of the target storage device relative to all storage devices for the target storage device; wherein the performance data includes at least one of bandwidth, processing delay, data write volume, storage device temperature, and number of data write errors; determines the descriptive statistics of the relative performance data of the target storage device detected within a preset time period, and determines the performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device. If the performance evaluation result of the target storage device is abnormal performance, determines whether the target storage device currently has input and output operations; and determines the adjustment method of the data copy in the target storage device according to the judgment result of the input and output operations. The technical scheme of the embodiment of the present application adaptively detects the input and output operations of the target storage device, and adaptively determines the adjustment method of the data copy according to the input and output operations, so as to avoid backing up the data copy in the presence of input and output, which affects the normal execution of the input and output operations.

[0091] This application takes latency as an example of performance data and provides a specific implementation method, which is as follows:

[0092] 1. Identify sub-healthy storage devices during device aging

[0093] The evaluation method of sub-healthy storage devices is as follows:

[0094] (1) Calculate the relative I / O delay (RD) of each storage device

[0095] RDki=Dki / median(D1i, D2i,...,ki);

[0096] Where Dki represents the average latency of storage device k in the i-th monitoring window, and k represents the number of storage devices on the server. Median(D1i, D2i, …, Dki) represents the median of the average latency of all drives on the same storage server in the i-th monitoring window.

[0097] For example, during a 3-hour monitoring period, 15 seconds is a monitoring window, and there are 720 monitoring windows in total, i∈{1, 2, ..., 720}, and Dki is the average latency of device k during the i-th 15-second monitoring window. If there are 5 storage devices, then median(D1i, D2i, ..., D15i) is the median of the average latency of the 15 storage devices on the same node during the i-th 15-second monitoring window.

[0098] (2) Calculate storage device delay events

[0099] Ek,i,j=mean(RDKi,RDKi+1,…,RDKj);

[0100] Ek,i,j represents the average relative delay (RD) of storage device k between consecutive monitoring windows i to j (the monitoring window can be 5s, 10s or 15s, etc. The above example is 15 seconds, which can be set according to the actual environment).

[0101] For example, a server has 15 storage devices, and the average delays are 100us, 120us, 110us, 105us, 115us, 130us, 140us, 150us, 160us, 170us, 180us, 190us, 200us, 210us, and 300us. The median of the calculated delay is 150us.

[0102] Calculate the relative delay of the devices, for example, for device 15, RD15i=300 / 150=2, for device 1, RD1i=100 / 150≈0.67.

[0103] If the delay of device 15 is still high in the next monitoring window, calculate its average relative delay in 5 consecutive windows, assuming RD15,i=2; RD15,i+1=2.2; RD15,i+2=2.1; RD15,i+3=2.3; RD15,i+4=2.5; E15,i,i+4=(2+2.2+2.1+2.3+2.5) / 5=2.2.

[0104] (3) Calculate Ek,i,j within the time span and determine the degree of sub-health based on different thresholds.

[0105] In a time span, calculate the event information of Ek,i,j>TH.

[0106] For example, if the monitoring time window is 15s and the time span is 15 minutes, then the Ek,i,i+59 value of device k in 60 monitoring windows is calculated. The sub-health level of the device can be judged according to different time spans. If the time span is set to 3 hours, the value of 720 monitoring windows within 3 hours is calculated. If the value exceeds TH, it is judged as moderate sub-health. If the time span is set to 24 hours, that is, within 24 hours, the Ek,i,j of the device>TH, that is, compared with other devices, the delay of this device for a whole day is seriously higher than that of other devices, and it is judged as severe sub-health. The specific time span can be set according to the actual scenario.

[0107] 2. Adaptive copy adjustment

[0108] The specific adjustment methods are as follows:

[0109] Compared with healthy storage devices, subhealthy storage devices have different I / O access performance and different subhealth levels have different failure probabilities. When a storage device in a server node is identified as subhealthy:

[0110] When the storage device has I / O operations and is in a serious sub-health state:

[0111] When there is a write operation, find a healthy storage device that meets the disaster recovery rules and write new data to the healthy storage device. After the write operation is completed, synchronize the old data of the sub-healthy device to the healthy storage device to form four copies.

[0112] When there is a read operation, a healthy device that meets the disaster recovery rules is found. After the read operation is completed, the data of the sub-healthy device is copied to the healthy device to form four copies.

[0113] When the storage device has no I / O operations and is in a serious sub-health state

[0114] Find a healthy storage device that meets the disaster recovery rules. When the system is idle, copy the data of the sub-healthy storage device to the healthy storage device to form four copies.

[0115] Figure 4 This is a schematic diagram of the structure of the data replica adjustment device provided in the embodiment of the present application. Figure 4 As shown, the data replica adjustment device includes:

[0116] The relative performance data determination module 410 is used to determine, for a target storage device, relative performance data of the target storage device relative to all storage devices; wherein the performance data includes at least one of bandwidth, processing latency, data write volume, storage device temperature, and number of data write errors;

[0117] A descriptive statistics determination module 420, configured to determine descriptive statistics of relative performance data of a target storage device detected within a preset time period, and determine a performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device;

[0118] The adjustment mode determination module 430 is used to determine the adjustment mode of the data copy in the target storage device according to the performance evaluation result of the target storage device.

[0119] In some embodiments, determining relative performance data of the target storage device relative to all storage devices includes:

[0120] For each storage device, the relative performance data determination module 410 determines an average value of the performance data detected by the storage device in a unit time period as average performance data;

[0121] Relative performance data of the target storage device relative to all storage devices is determined based on the average performance data of the target storage device and the average performance data of all storage devices.

[0122] In some embodiments, the relative performance data determination module 410 determines the relative performance data of the target storage device relative to all storage devices according to the average performance data of the target storage device and the average performance data of all storage devices, including:

[0123] Determining a median of the average performance data of each of the storage devices;

[0124] The ratio of the average performance data of the target storage device to the median is used as the relative performance data of the target storage device relative to all storage devices.

[0125] In some embodiments, the descriptive statistics determination module 420 determines the descriptive statistics of the relative performance data of the target storage device detected within a preset time period, including:

[0126] For at least two unit time periods included in the preset time period, respectively determine the relative performance data of the target storage device relative to all storage devices in the unit time period, and obtain at least two relative performance data corresponding to the target storage device;

[0127] The descriptive statistics are determined based on at least two relative performance data.

[0128] In some embodiments, the descriptive statistics determination module 420 determines the descriptive statistics according to at least two relative performance data, including:

[0129] At least one of a mean, a variance and a standard deviation of at least two relative performance data is determined as the descriptive statistic.

[0130] In some embodiments, the adjustment mode determination module 430 determines the performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device, including:

[0131] When the preset time period is fixed, determining whether the descriptive statistics corresponding to the target storage device exceeds a normal statistics range threshold;

[0132] If it exceeds, it is determined that the performance of the target storage device is abnormal.

[0133] In some embodiments, the adjustment mode determination module 430 determines that the performance of the target storage device is abnormal, including:

[0134] For different target storage devices whose descriptive statistics all exceed a preset statistical threshold, if the performance data includes at least one of a processing delay, a data write amount, a storage device temperature, and a number of data write errors, it is determined that the performance abnormality degree of the target storage device is positively correlated with the descriptive statistics;

[0135] If the performance data includes bandwidth, it is determined that the degree of performance abnormality of the target storage device is negatively correlated with the descriptive statistics.

[0136] In some embodiments, the adjustment mode determination module 430 determines that the performance of the target storage device is abnormal, including:

[0137] If the descriptive statistic corresponding to the target storage device exceeds the normal statistic range threshold, the performance abnormality degree of the target storage device is positively correlated with the duration for which the descriptive statistic exceeds the normal statistic range threshold.

[0138] In some embodiments, the adjustment mode determination module 430 determines the adjustment mode of the data copy in the target storage device according to the performance evaluation result of the target storage device, including:

[0139] If the performance evaluation result of the target storage device is abnormal performance, determining whether there is currently an input / output operation on the target storage device;

[0140] According to the judgment result of the input and output operation, the adjustment method of the data copy in the target storage device is determined.

[0141] In some embodiments, the adjustment mode determination module 430 determines the adjustment mode of the data copy in the target storage device according to the judgment result of the input and output operation, including:

[0142] If there is a write operation on the target storage device, synchronously perform the write operation on the target storage device and the new storage device;

[0143] If the write operation is completed, copying the data copy in the target storage device before the write operation is performed to the new storage device;

[0144] If there is a read operation on the target storage device, after the read operation is performed on the target storage device, the data copy in the target storage device is copied to the new storage device.

[0145] In some embodiments, the adjustment mode determination module 430 determines the adjustment mode of the data copy in the target storage device according to the judgment result of the input and output operation, including:

[0146] If there is no input / output operation on the target storage device, the data copy in the target storage device is copied to the new storage device to form a target number plus one data copy synchronization; wherein the target number is the number of all storage devices except the new storage device.

[0147] In some embodiments, the process of copying the data copy to the new storage device by the adjustment mode determination module 430 includes:

[0148] If the priority of business processing is higher than the priority of data copy replication, then the storage system is checked for business processing operations;

[0149] If there is no business processing operation, copying the data copy to the new storage device;

[0150] If there is a business processing operation, wait for the business processing operation to end and copy the data copy to the new storage device.

[0151] The data copy adjustment device provided in the embodiment of the present application can be used to execute the technical solution of the data copy adjustment method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0152] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the relative performance data determination module 410 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the relative performance data determination module 410. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0153] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0154] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes the scheme in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (network processor, NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0155] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0156] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0157] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0158] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0159] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the data copy adjustment method in the above embodiment.

[0160] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the data copy adjustment method of the above embodiment.

[0161] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the data copy adjustment method in the above embodiment.

[0162] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0163] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data copy adjustment method, characterized in that: include: For a target storage device, determine relative performance data of the target storage device relative to all storage devices; wherein the performance data includes at least one of bandwidth, processing latency, data write volume, storage device temperature, and number of data write errors; Determine descriptive statistics of relative performance data of a target storage device detected within a preset time period, and determine a performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device; According to the performance evaluation result of the target storage device, an adjustment method for the data copy in the target storage device is determined.

2. The method according to claim 1, characterized in that: Determining relative performance data of the target storage device relative to all storage devices includes: For each storage device, determine an average value of performance data detected by the storage device in a unit time period as average performance data; Relative performance data of the target storage device relative to all storage devices is determined based on the average performance data of the target storage device and the average performance data of all storage devices.

3. The method according to claim 2, characterized in that Determining relative performance data of the target storage device relative to all storage devices according to the average performance data of the target storage device and the average performance data of all storage devices includes: Determining a median of the average performance data of each of the storage devices; The ratio of the average performance data of the target storage device to the median is used as the relative performance data of the target storage device relative to all storage devices.

4. The method according to claim 1, characterized in that: Determine descriptive statistics of the relative performance data of the target storage device detected during a preset time period, including: For at least two unit time periods included in the preset time period, respectively determine the relative performance data of the target storage device relative to all storage devices in the unit time period, and obtain at least two relative performance data corresponding to the target storage device; The descriptive statistics are determined based on at least two relative performance data.

5. The method according to claim 4, characterized in that Determining the descriptive statistics based on at least two relative performance data includes: At least one of a mean, a variance and a standard deviation of at least two relative performance data is determined as the descriptive statistic.

6. The method according to claim 1, characterized in that Determining a performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device includes: Determine whether the descriptive statistics corresponding to the target storage device exceed the normal statistics range threshold; If it exceeds, it is determined that the performance of the target storage device is abnormal.

7. The method according to claim 6, characterized in that Determining that the performance of the target storage device is abnormal includes: In the case where the preset time period is fixed, for different target storage devices whose descriptive statistics all exceed the preset statistical threshold, if the performance data includes at least one of processing latency, data write volume, storage device temperature, and number of data write errors, it is determined that the performance abnormality degree of the target storage device is positively correlated with the descriptive statistics; If the performance data includes bandwidth, it is determined that the degree of performance abnormality of the target storage device is negatively correlated with the descriptive statistics.

8. The method according to claim 6, characterized in that Determining that the performance of the target storage device is abnormal includes: When the preset time period is not fixed, if the descriptive statistic corresponding to the target storage device exceeds the normal statistic range threshold, the performance abnormality of the target storage device is positively correlated with the duration that the descriptive statistic exceeds the normal statistic range threshold.

9. The method according to claim 1, characterized in that: Determining, according to the performance evaluation result of the target storage device, an adjustment method for the data copy in the target storage device, including: If the performance evaluation result of the target storage device is abnormal performance, determining whether there is currently an input / output operation on the target storage device; According to the judgment result of the input and output operation, the adjustment method of the data copy in the target storage device is determined.

10. The method according to claim 9, characterized in that Determining, according to the judgment result of the input and output operation, a method for adjusting the data copy in the target storage device, including: If there is a write operation on the target storage device, synchronously perform the write operation on the target storage device and the new storage device; If the write operation is completed, copying the data copy in the target storage device before the write operation is performed to the new storage device; If there is a read operation on the target storage device, after the read operation is performed on the target storage device, the data copy in the target storage device is copied to the new storage device.

11. The method according to claim 9, characterized in that Determining, according to the judgment result of the input and output operation, a method for adjusting the data copy in the target storage device, including: If there is no input / output operation on the target storage device, the data copy in the target storage device is copied to the new storage device to form a target number plus one data copy synchronization; wherein the target number is the number of all storage devices except the new storage device.

12. The method according to claim 9 or 10, characterized in that: The process of copying the data copy to the new storage device includes: If the priority of business processing is higher than the priority of data copy replication, then the storage system is checked for business processing operations; If there is no business processing operation, copying the data copy to the new storage device; If there is a business processing operation, wait for the business processing operation to end and copy the data copy to the new storage device.

13. A data copy adjustment device, characterized in that: include: A relative performance data determination module, used to determine, for a target storage device, relative performance data of the target storage device relative to all storage devices; wherein the performance data includes at least one of bandwidth, processing latency, data write volume, storage device temperature, and number of data write errors; A descriptive statistics determination module, used to determine the descriptive statistics of the relative performance data of the target storage device detected within a preset time period, and determine the performance evaluation result of the target storage device according to the descriptive statistics corresponding to the target storage device; The adjustment mode determination module is used to determine the adjustment mode of the data copy in the target storage device according to the performance evaluation result of the target storage device.

14. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the data copy adjustment method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the data copy adjustment method according to any one of claims 1 to 12.

16. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the data copy adjustment method described in any one of claims 1 to 12.

Images