Data redundancy processing method

Abstract

The invention relates to the technical field of electric digital data processing, in particular to a data redundancy processing method. The method comprises the steps that if multi-copy redundancy is carried out on data to be written by a target I / O request, the number H of times that a target address is accessed in a latest first time period is traversed, and the average number PHi of times that the ith target address is accessed in the latest first time period is acquired; traversing the number D of times that the target disk is accessed in the latest first time period, and obtaining the average number PDi of times that the ith target disk is accessed in the latest first time period; and if the copy number adjustment judgment value F of the data to be written by the target I / O request is smaller than a lower limit value of a preset copy number adjustment judgment value interval, judging whether n is greater than a first preset copy number, and if yes, determining n-1 as a target copy number corresponding to the data to be written by the target I / O request. The storage access performance and the storage cost of the data can be considered.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a method for processing data redundancy. Background Art

[0002] The use of data redundancy schemes can improve data reliability. The data redundancy schemes commonly used in the prior art include multiple copies and checksums. Multiple copies refer to storing a piece of data repeatedly in multiple storage components, thereby avoiding data loss due to failure of a single storage component. Common copy redundancy numbers include two-copy redundancy, three-copy redundancy, and four-copy redundancy. More copies mean a greater number of failures can be tolerated and better storage access performance, but an increase in the number of copies will increase storage and maintenance costs. When performing multiple copy redundancy on data, how to balance the storage access performance and storage cost of the data is an urgent problem to be solved. Summary of the invention

[0003] The present invention aims to provide a method for processing data redundancy to take into account both data storage access performance and storage cost.

[0004] According to the present invention, a method for processing data redundancy is provided, comprising the following steps:

[0005] S100, if multiple copies of data to be written by the target I / O request are redundancy, then enter S200.

[0006] S200, obtaining the number of times the target address is accessed in the most recent first time period H, where H=(H1, H2, . . . , H i ,…,H n ), H i H is the number of times the i-th target address is accessed in the most recent first time period, i ranges from 1 to n, and n is the current number of copies of the data to be written by the target I / O request; the target address is the storage address corresponding to the data to be written by the target I / O request, and different target addresses correspond to different disks; H i =(H i,1 ,H i,2 ,…,H i,j ,…,H i,m ), H i,j is the number of times the i-th target address is accessed in the j-th sub-time period in the most recent first time period, the value range of j is 1 to m, and m is the number of sub-time periods included in the most recent first time period.

[0007] S300, traverse H, and obtain the average number of times PH that the i-th target address is accessed in the most recent first time period i , PH i =∑m j＝1 H i,j / m.

[0008] S400, obtaining the number of times D that the target disk is accessed in the most recent first time period, D=(D1, D2, . . . , D i ,…,D n ), D i is the number of times the i-th target disk is accessed in the most recent first time period, the i-th target disk being the disk corresponding to the i-th target address; D i =(D i,1 ,D i,2 ,…,D i,j ,…,D i,m ), D i,j is the number of times the i-th target disk is accessed in the j-th sub-time period in the most recent first time period.

[0009] S500, traverse D, and obtain the average number of times PD the i-th target disk is accessed in the most recent first time period i , P.D. i =∑ m j＝1 D i,j / m.

[0010] S600, according to PH i and PD i Obtain a copy quantity adjustment judgment value F of data to be written by the target I / O request.

[0011] S700, if F is less than the lower limit of the preset interval of judgment value for adjusting the number of replicas, determine whether n is greater than the first preset number of replicas, and if so, determine n-1 as the target number of replicas corresponding to the data to be written by the target I / O request; if F is greater than the upper limit of the preset interval of judgment value for adjusting the number of replicas, determine whether n is less than the second preset number of replicas, and if so, determine n+1 as the target number of replicas corresponding to the data to be written by the target I / O request; the lower limit of the preset interval of judgment value for adjusting the number of replicas is less than 0, and the upper limit of the preset interval of judgment value for adjusting the number of replicas is greater than 0.

[0012] Compared with the prior art, the present invention has at least the following beneficial effects:

[0013] In the present invention, if multiple copies of data to be written by the target I / O request are performed with redundancy, the average number of accesses to each target address corresponding to the data in the most recent first time period is obtained, and the average number of accesses can reflect the popularity of the target address corresponding to the data being accessed, and the number of target addresses corresponding to the data is the current number of copies of the data. Different target addresses corresponding to the data correspond to different disks. The present invention also obtains the average number of accesses to each disk corresponding to each target address corresponding to the data in the most recent first time period, and the average number of accesses can reflect the access pressure of the disk corresponding to the target address corresponding to the data; the present invention is based on the target address corresponding to the data The access heat and the access pressure of the disk corresponding to the target address determine the judgment value for adjusting the number of copies of the data, and determine whether to adjust the current number of copies of the data based on the judgment value for adjusting the number of copies, so that when the access heat of the target address corresponding to the data and the access pressure of the disk corresponding to the target address are both large, the number of copies corresponding to the data is increased, thereby improving the storage access performance corresponding to the data; when the access heat of the target address corresponding to the data and the access pressure of the disk corresponding to the target address are both small, the number of copies corresponding to the data is reduced, thereby reducing the storage cost corresponding to the data; thus, the present invention achieves the purpose of taking into account both the storage access performance and the storage cost of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 The present invention provides a flowchart of a method for processing data redundancy according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0017] According to the present invention, Figure 1 As shown, a method for processing data redundancy is provided, comprising the following steps:

[0018] S100, if multiple copies of data to be written by the target I / O request are redundancy, then enter S200.

[0019] As a specific implementation, the target I / O request is an upper-layer I / O request received by the database file system or an I / O request generated by the database file system itself.

[0020] As a specific implementation, the upper-layer I / O request received by the database file system is the I / O request generated when the user reads or writes. The I / O request generated by the database file system itself includes the I / O request for reading metadata triggered by user behavior or the I / O request generated by the scheduled task not triggered by user behavior, etc.

[0021] In this embodiment, before S100, it also includes a process of determining whether to perform multiple copy redundancy or checksum redundancy on the data to be written by the target I / O request. The process includes:

[0022] S010, if the target I / O request is a write request, the type of the target I / O request is determined according to the type of data to be written by the target I / O request; the type of data to be written by the target I / O request is a metadata type or a non-metadata type, and the type of the target I / O request is a performance I / O type or a computing I / O type.

[0023] In this embodiment, the I / O request is an input / output request, the input request corresponds to reading data, and the output request corresponds to writing data. When the I / O request corresponds to writing data, the I / O request is judged to be a write request; when the I / O request corresponds to reading data, the I / O request is judged to be a read request. As a specific implementation, the I / O request includes an identifier for characterizing whether the I / O request is a write request or a read request, and the I / O request can be obtained based on the identifier as to whether it is a write request or a read request.

[0024] In this embodiment, the type of data to be written by the target I / O request is known; if the I / O request is a write request, the I / O request also includes an identifier for characterizing the type of data to be written by the I / O request. As a specific implementation, if the identifier for characterizing the type of data to be written by the I / O request is 1, it indicates that the type of data to be written by the I / O request is a metadata type; if the identifier for characterizing the type of data to be written by the I / O request is 0, it indicates that the type of data to be written by the I / O request is a non-metadata type.

[0025] As a specific implementation, judging the type of the target I / O request according to the type of data to be written by the target I / O request includes:

[0026] S011, if the type of data to be written by the target I / O request is a metadata type, determine that the type of the target I / O request is a performance I / O type; if the type of data to be written by the target I / O request is a non-metadata type, enter S012.

[0027] S012: If the data to be written by the target I / O request is hot data, then determine that the type of the target I / O request is a performance I / O type; if the data to be written by the target I / O request is cold data, then determine that the type of the target I / O request is a computing I / O type.

[0028] In this embodiment, hot data in metadata type data and non-metadata type data are data that are frequently accessed, and cold data in non-metadata type data are data that are infrequently accessed.

[0029] In this embodiment, if the target I / O request is a read request, the type of the target I / O request is not determined, and subsequent steps are not performed, and any method for processing a read request in the prior art may be used. As a specific implementation, S012 also includes a process for determining whether the data to be written by the target I / O request is hot data or cold data, and the process includes:

[0030] S0121, obtaining the virtual address add1 of the data to be written in the target I / O request.

[0031] In this embodiment, if the target I / O request is a write request, the target I / O request also includes the address of the data to be written, which is a virtual address. The actual storage address (i.e., the physical address) of the data to be written in the target I / O request can be obtained based on the virtual address and the conversion relationship from the virtual address to the physical address.

[0032] S0122, obtaining the access frequency f1 of the physical address corresponding to add1 in the target historical time period.

[0033] In this embodiment, the target historical time period is the historical time period closest to the current moment and having a preset length.

[0034] In this embodiment, no matter whether data is written or read from the physical address corresponding to add1 within the target historical time period, it is regarded as the physical address corresponding to add1 being accessed; the frequency f1 of the physical address corresponding to add1 being accessed within the target historical time period is the sum of the frequency of the physical address corresponding to add1 being written to within the target historical time period and the frequency of the physical address corresponding to add1 being read from within the target historical time period.

[0035] S0123, if f1≥f0, determine that the data to be written by the target I / O request is hot data; otherwise, determine that the data to be written by the target I / O request is cold data.

[0036] In this embodiment, f0 is an empirical value.

[0037] S020, if the type of the target I / O request is a performance I / O type, multiple copies of the data to be written by the target I / O request are redundant; if the type of the target I / O request is a computing I / O type, verification and redundancy are performed on the data to be written by the target I / O request.

[0038] In this embodiment, if the type of the target I / O request is a performance I / O type, it means that the address to which the target I / O request intends to write data is frequently accessed, and the performance of storage access is more concerned, and the performance requirement for storage access is relatively high; if the type of the target I / O request is a computing I / O type, it means that the address to which the target I / O request intends to write data is not frequently accessed, and the performance of storage access is not concerned, and the performance requirement for storage access is relatively low.

[0039] In this embodiment, if multiple copies of data to be written by the target I / O request are redundant, N disks are selected from the disks of the target computer as the target disks of the target I / O request, and the data to be written in the target I / O request is written to each of the target disks respectively, so as to realize N copies of redundancy of the data to be written in the target I / O request in the target computer; N is the redundant number corresponding to the target I / O request; the target computer is the computer on which the database file system runs.

[0040] In this embodiment, the number of disks included in the target disk is N, N≥2, and the data to be written in the target I / O request is written to each disk in the target disk respectively, so that each disk in the target disk stores the data to be written in the target I / O request, and N copies of the data to be written in the target I / O request can be achieved. Redundancy of N copies of the data to be written in the target I / O request. Since the data redundancy scheme for multiple copies does not need to consume too many computing resources, therefore, in this embodiment, writing the data to be written in the target I / O request to the disk of the target computer does not consume a large amount of computing resources inside the target computer, and does not affect the speed of processing other computing tasks inside the computer.

[0041] In this embodiment, the process of obtaining N is S200-S700.

[0042] S200, obtaining the number of times the target address is accessed in the most recent first time period H, where H=(H1, H2, . . . , H i ,…,H n ), H iH is the number of times the i-th target address is accessed in the most recent first time period, i ranges from 1 to n, and n is the current number of copies of the data to be written by the target I / O request; the target address is the storage address corresponding to the data to be written by the target I / O request, and different target addresses correspond to different disks; H i =(H i,1 ,H i,2 ,…,H i,j ,…,H i,m ), H i,j is the number of times the i-th target address is accessed in the j-th sub-time period in the most recent first time period, the value range of j is 1 to m, and m is the number of sub-time periods included in the most recent first time period.

[0043] In this embodiment, the duration of the first time period and the duration of the sub-time period are both empirical values. As a specific implementation, the duration of the first time period is 1 minute, and the duration of the sub-time period is 1 second.

[0044] In this embodiment, n is known; the target I / O request includes a virtual address corresponding to the data to be written, and the storage address (which is a physical address) corresponding to the data to be written by the target I / O request can be obtained based on the virtual address and the conversion relationship from the virtual address to the physical address; if the number of copies corresponding to the data stored at the storage address is s, that is, the data stored at the storage address has backup in another s-1 disks, then n=s.

[0045] S300, traverse H, and obtain the average number of times PH that the i-th target address is accessed in the most recent first time period i , PH i =∑ m j＝1 H i,j / m.

[0046] S400, obtaining the number of times D that the target disk is accessed in the most recent first time period, D=(D1, D2, . . . , D i ,…,D n ), D i is the number of times the i-th target disk is accessed in the most recent first time period, the i-th target disk being the disk corresponding to the i-th target address; D i =(D i,1 ,D i,2 ,…,D i,j ,…,D i,m ), D i,j is the number of times the i-th target disk is accessed in the j-th sub-time period in the most recent first time period.

[0047] S500, traverse D, and obtain the average number of times PD the i-th target disk is accessed in the most recent first time period i , P.D. i =∑ m j＝1 D i,j / m.

[0048] S600, according to PH i and PD i Obtain a copy quantity adjustment judgment value F of data to be written by the target I / O request.

[0049] As a preferred specific implementation, F satisfies the following conditions: F = ∑ n i＝1 ((PH i / PD i -a / 1000)×1000+(PD i / TD i -b / 100)×100) / n, where a is the preset heat expansion threshold, b is the preset disk pressure threshold, TD i is the upper limit of the number of times the i-th target disk is accessed in a single sub-time period.

[0050] In this embodiment, a, b and TD i They are all empirical values. The larger the value of a is, the higher the access heat requirement for the target address corresponding to the data when the replica capacity is increased; the larger the value of b is, the higher the access pressure requirement for the disk corresponding to the target address when the replica capacity is increased; as a specific implementation method, a=1, b=80.

[0051] S700, if F is less than the lower limit of the preset interval of judgment value for adjusting the number of replicas, determine whether n is greater than the first preset number of replicas, and if so, determine n-1 as the target number of replicas corresponding to the data to be written by the target I / O request; if F is greater than the upper limit of the preset interval of judgment value for adjusting the number of replicas, determine whether n is less than the second preset number of replicas, and if so, determine n+1 as the target number of replicas corresponding to the data to be written by the target I / O request; the lower limit of the preset interval of judgment value for adjusting the number of replicas is less than 0, and the upper limit of the preset interval of judgment value for adjusting the number of replicas is greater than 0.

[0052] In this embodiment, the target number of copies corresponding to the data to be written by the target I / O request is the redundancy number corresponding to the target I / O request.

[0053] In this embodiment, when F is less than the lower limit of the preset replica quantity adjustment judgment value interval, if n is not greater than the first preset replica quantity, n is determined as the target replica quantity corresponding to the data to be written by the target I / O request.

[0054] In this embodiment, when F is greater than the upper limit of the preset replica number adjustment judgment value interval, if n is not less than the second preset replica number, n is determined as the target replica number corresponding to the data to be written by the target I / O request.

[0055] In this embodiment, if n-1 is determined as the target number of copies corresponding to the data to be written by the target I / O request, n-1 target disks are selected from the n target disks to store the data to be written by the target I / O request; if n+1 is determined as the target number of copies corresponding to the data to be written by the target I / O request, a disk of the target computer is added as the target disk, so that there are n+1 disks to store the data to be written by the target I / O request.

[0056] In this embodiment, if F belongs to the preset replica quantity adjustment judgment value interval, n is determined as the target replica quantity corresponding to the data to be written by the target I / O request.

[0057] In this embodiment, the lower limit value of the preset copy quantity adjustment judgment value interval, the upper limit value of the preset copy quantity adjustment judgment value interval, the first preset copy quantity and the second preset copy quantity are all empirical values. Optionally, the first preset copy quantity is 2 and the second preset copy quantity is 5.

[0058] In this embodiment, if multiple copies of data to be written by the target I / O request are performed, the average number of accesses to each target address corresponding to the data in the most recent first time period is obtained. The average number of accesses can reflect the popularity of the target address corresponding to the data being accessed, and the number of target addresses corresponding to the data is the current number of copies of the data. Different target addresses corresponding to the data correspond to different disks. This embodiment also obtains the average number of accesses to each disk corresponding to each target address corresponding to the data in the most recent first time period. The average number of accesses can reflect the access pressure of the disk corresponding to the target address corresponding to the data; this embodiment is based on the target address corresponding to the data. The access popularity of the target address and the access pressure of the disk corresponding to the target address determine a judgment value for adjusting the number of copies of the data, and determine whether to adjust the current number of copies of the data based on the judgment value for adjusting the number of copies, so that when the access popularity of the target address corresponding to the data and the access pressure of the disk corresponding to the target address are both large, the number of copies corresponding to the data is increased, thereby improving the storage access performance corresponding to the data; when the access popularity of the target address corresponding to the data and the access pressure of the disk corresponding to the target address are both small, the number of copies corresponding to the data is reduced, thereby reducing the storage cost corresponding to the data; thus, this embodiment achieves the purpose of taking into account both the storage access performance and the storage cost of the data.

[0059] In this embodiment, S100 further includes: if the data to be written by the target I / O request is verified and redundancy is performed, then the process proceeds to S800.

[0060] Since the data redundancy scheme of the checksum needs to consume a large amount of computing resources inside the target computer, and the computing resources inside the target computer are very tight, preferably, the scheme of this embodiment sends the target I / O request to the target external storage node through the Remote Direct Memory Access (RDMA) technology to offload the checksum calculation task originally performed inside the target computer to the external storage node. Therefore, the calculation of the checksum of this embodiment only consumes the computing resources of the external storage node, but not the computing resources inside the target computer; and inside the external storage node, since there is no business need to compete with the calculation of the checksum for computing resources, the efficiency of calculating the parity checksum can be improved, and more computing resources can be reserved for the business inside the target computer.

[0061] In this embodiment, the external storage nodes can be expanded as needed. As an optional specific implementation, the number of external storage nodes is greater than or equal to 2, and the process of acquiring the target external storage node is S800-S830.

[0062] S800, obtain the total computing power E of the external storage node, E=(E1, E2, …, E k ,…,E q ), E k is the total computing power of the kth external storage node. The value range of k is 1 to q, q is the number of external storage nodes, and q≥2.

[0063] As a preferred embodiment, E k The following conditions are met: k =(y k,1 ×c k ×u+y k,2 × k )×w k , where y k,1 is the main frequency of the CPU of the kth external storage node, y k,2 is the main frequency of the memory of the kth external storage node, c k is the number of CPU cores of the kth external storage node, z k is the ratio of the memory size of the kth external storage node to the unit memory size, u is the preset CPU weight, u>1, w k is the preset weight of the kth external storage node. Thus, the total computing power of each external storage node can be estimated more accurately and quickly.

[0064] In this embodiment, the unit memory size is 1 GB.

[0065] In this embodiment, w k and u are empirical values. As a specific implementation, u=10.

[0066] S810, obtain the computing power ratio R consumed by the external storage node, R = (R1, R2, ..., R k ,…,R q ), R k The percentage of computing power consumed by the kth external storage node.

[0067] As a preferred embodiment, R k Meet the following conditions: R k =(x k,1 ×u+x k,2 ) / (u+1), x k,1 is the CPU usage percentage of the kth external storage node, x k,2 is the memory usage percentage of the kth external storage node.

[0068] S820, traverse E and R, according to E k and R k Get the remaining computing power G of the kth external storage node k .

[0069] In this embodiment, G k Meet the following conditions: G k =E k ×(1-R k ).

[0070] S830, determining the external storage node with the largest remaining computing power as the target external storage node, and implementing verification and redundancy of the data to be written in the target I / O request in the target external storage node.

[0071] Based on S800-S830, the external storage node with the largest remaining computing power can be obtained more accurately and quickly, which is conducive to achieving load balancing of external storage nodes.

[0072] As a specific implementation, implementing the verification and redundancy of the data to be written in the target I / O request in the target external storage node includes:

[0073] S801, divide the data to be written in the target I / O request into Q slices; Q is a preset number of slices.

[0074] In this embodiment, Q is an empirical value, and optionally, Q=4.

[0075] S802, obtain the parity checksum of Q slices.

[0076] Those skilled in the art know that any method for calculating a parity checksum in the prior art falls within the protection scope of the present invention.

[0077] S803, use Q+M disks in the target external storage node to store Q shards and the parity sums of the Q shards; wherein, Q disks in the Q+M disks are used to store the Q shards, and different disks store different shards; M disks in the Q+M disks are used to store the parity sums of the Q shards, and the parity sums of the Q shards stored by different disks are the same, and M is the preset number of disks for storing parity sums.

[0078] In this embodiment, after the data to be written in the target I / O request is split, different shards in the obtained Q shards have different data corresponding to them, and the data corresponding to different shards are stored on different disks. Using Q disks can realize the storage of the data to be written in the target I / O request.

[0079] In this embodiment, M is an empirical value, and M=1 or M=2 is optional. If M=1, Q+1 disks are required to implement the checksum redundancy of the data to be written in the target I / O request in the target external storage node, where Q disks are used to store the data to be written in the target I / O request, and 1 disk is used to store the parity checksum of Q shards; if M=2, Q+2 disks are required to implement the checksum redundancy of the data to be written in the target I / O request in the target external storage node, where Q disks are used to store the data to be written in the target I / O request, and 2 disks are used to store the parity checksum of Q shards.

[0080] This embodiment first determines whether an upper-layer I / O request received by the database file system or a request generated by the database file system itself is a write type request. If so, the type of the request is determined according to the type of data to be written in the request; if the request is a performance I / O type, the data in the request is stored on the disk of the target computer and multiple copies are performed for redundancy; if the request is a computing I / O type, the request is sent to the target external storage node so that the target external storage node can verify and redundancy the data to be written in the request. This embodiment supports two data redundancy schemes in a file system. For I / O requests of performance I / O type (the storage address of the corresponding data to be written is accessed more frequently), a multi-copy data redundancy scheme is used to ensure higher storage access performance; for I / O requests of computing type (the storage address of the corresponding data to be written is accessed less frequently), a checksum data redundancy scheme is used to improve data storage utilization, and checksum redundancy is performed on the external storage node, which can offload the computing task from the computing node of the target computer to the external storage node, does not consume the resources of the computing node of the target computer, and reserves more computing resources for the computing node. At the same time, there is no task inside the external storage node to compete with the task of calculating the parity checksum, which can improve the efficiency of calculating the parity checksum.

[0081] Small-scale experiments have shown that this embodiment reduces the cost by more than 50% compared to a data redundancy solution that only uses multiple copies, improves the storage access performance of this embodiment by more than 70% compared to a data redundancy solution that only uses checksums, and reduces the computing resource consumption inside the target computer of this embodiment by about 80%.

[0082] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A method for processing data redundancy, characterized in that: The following steps are involved: S100, if multiple copies of data to be written by the target I / O request are redundancy, then enter S200; S200, obtaining the number of times the target address is accessed in the most recent first time period H, where H=(H1, H2, . . . , H i ,…,H n ), H i H is the number of times the i-th target address is accessed in the most recent first time period, i ranges from 1 to n, and n is the current number of copies of the data to be written by the target I / O request; the target address is the storage address corresponding to the data to be written by the target I / O request, and different target addresses correspond to different disks; H i =(H i,1 ,H i,2 ,…,H i,j ,…,H i,m ), H i,j is the number of times the i-th target address is accessed in the j-th sub-time period in the most recent first time period, where j ranges from 1 to m, and m is the number of sub-time periods included in the most recent first time period; S300, traverse H, and obtain the average number of times PH that the i-th target address is accessed in the most recent first time period i , PH i =∑ m j＝1 H i,j / m; S400, obtaining the number of times D that the target disk is accessed in the most recent first time period, D=(D1, D2, . . . , D i ,…,D n ), D i is the number of times the i-th target disk is accessed in the most recent first time period, the i-th target disk being the disk corresponding to the i-th target address; D i =(D i,1 ,D i,2 ,…,D i,j ,…,D i,m ), D i,j is the number of times the i-th target disk is accessed in the j-th sub-time period in the most recent first time period; S500, traverse D, and obtain the average number of times PD the i-th target disk is accessed in the most recent first time period i , P.D. i =∑ m j＝1 D i,j / m; S600, according to PH i and PD i Obtain a copy quantity adjustment judgment value F of the data to be written by the target I / O request; S700, if F is less than the lower limit of the preset copy number adjustment judgment value interval, determine whether n is greater than the first preset copy number, and if so, determine n-1 as the target copy number corresponding to the data to be written by the target I / O request; If F is greater than the upper limit of the preset replica number adjustment judgment value interval, determine whether n is less than the second preset replica number, and if so, determine n+1 as the target replica number corresponding to the data to be written by the target I / O request; The lower limit value of the preset replica quantity adjustment judgment value interval is less than 0, and the upper limit value of the preset replica quantity adjustment judgment value interval is greater than 0.

2. The method for processing data redundancy according to claim 1, characterized in that: F satisfies the following conditions: F = ∑ n i＝1 ((PH i / PD i -a / 1000)×1000+(PD i / TD i -b / 100)×100) / n, where a is the preset heat expansion threshold, b is the preset disk pressure threshold, TD i is the upper limit of the number of times the i-th target disk is accessed in a single sub-time period.

3. The method for processing data redundancy according to claim 1, characterized in that: S700 also includes: if F belongs to the preset replica quantity adjustment judgment value interval, n is determined as the target replica quantity corresponding to the data to be written by the target I / O request.

4. The method for processing data redundancy according to claim 1, characterized in that: S100 also includes: if the data to be written by the target I / O request is checked and redundancy is performed, then enter S800: S800, obtain the total computing power E of the external storage node, E=(E1, E2, …, E k ,…,E q ), E k is the total computing power of the kth external storage node, the value range of k is 1 to q, q is the number of external storage nodes, q ≥ 2; S810, obtain the computing power ratio R consumed by the external storage node, R = (R1, R2, ..., R k ,…,R q ), R k The percentage of computing power consumed by the kth external storage node; S820, traverse E and R, according to E k and R k Get the remaining computing power G of the kth external storage node k ; S830, determining the external storage node with the largest remaining computing power as the target external storage node, and implementing verification and redundancy of the data to be written in the target I / O request in the target external storage node.

5. The method for processing data redundancy according to claim 4, characterized in that: E k The following conditions are met: k =(y k,1 ×c k ×u+y k,2 × k )×w k , where y k,1 is the main frequency of the CPU of the kth external storage node, y k,2 is the main frequency of the memory of the kth external storage node, c k is the number of CPU cores of the kth external storage node, z k is the ratio of the memory size of the kth external storage node to the unit memory size, u is the preset CPU weight, u>1, w k is the preset weight of the kth external storage node.

6. The method for processing data redundancy according to claim 5, characterized in that: R k Meet the following conditions: R k =(x k,1 ×u+x k,2 ) / (u+1), x k,1 is the CPU usage percentage of the kth external storage node, x k,2 is the memory usage percentage of the kth external storage node.

7. The method for processing data redundancy according to claim 4, characterized in that: G k Meet the following conditions: G k =E k ×(1-R k ).

Images