Saving method of data and device thereof

[0041] In the present invention, by selecting a main array and a backup array from multiple arrays, and writing data to the main array and the backup array, only when the data written in the main array exceeds a preset threshold (for example, the main array is full) data), the primary and backup arrays will be reselected.

[0042] In the process of selecting the primary array and the backup array, the selection can be made according to the data writing times of each array and the number of times that each array is used as a backup array, so that the number of times each array is used as a primary array and the number of times that each array is used as a backup array are not much different, thus Make full use of each disk in the storage device, reduce the wear and tear of the disk in the disk cabinet, and increase the service life of the storage device. Moreover, it can ensure that only a small number of hard disks are running on each disk cabinet, and the combination of disk hibernation technology can achieve energy-saving effects.

[0043] Based on the above ideas, the present invention provides a data storage method, which is applied to a storage device including multiple arrays, such as figure 1 As shown, the method includes the following steps:

[0044] Step 101, select a primary array and a backup array from multiple arrays.

[0045] In the storage device, a disk cabinet (for example, 16 disks) is usually used as a carrier for monitoring data writing. In the present invention, each disk is made into a corresponding jbod array, and one of them is selected from each jbod array The jbod array is used as the backup array, and another jbod array is selected as the primary array. Among them, the jbod array is a "simple disk bundle", which is a widely used storage array. In the current storage field, the jbod array is a very important storage device. For the jbod array, when storing data, the data is stored from the first disk, and only when the storage space of the first disk is used up, the data will be stored in the following disks.

[0046] Of course, in practical applications, it is not limited to make each disk into a corresponding jbod array, as long as the data can be written to each disk in turn during the data storage process, the subsequent process will be directly taken as an example of the array illustrate.

[0047] In addition, in the present invention, in order to ensure the balanced use of each array, the number of data writes and the number of backup arrays of each array are also recorded, and when selecting the main array and backup array from multiple arrays, according to each array Select the primary array and the backup array based on the number of data write times and the number of times as the backup array.

[0048] Specifically, if no data has been written to each array, the number of data writes and the number of times each array is used as a backup array are both 0. At this time, when selecting the primary array and the backup array, you can Arbitrary choice of primary and backup arrays.

[0049] And when the number of times data is written to each array and the number of times it is used as a backup array is not 0, it means that data has been written in the corresponding array. In order to reduce the wear and tear of the disk and ensure the uniform use of each disk, then The main array and the backup array need to be selected according to the number of times data is written to each array and the number of times it is used as a backup array. For example, the array with the least number of data writes is selected as the main array, and the array that is used as the backup array the least number of times is selected as the backup array.

[0050] Step 102, when there is data to be stored in the storage device, write data into the primary array and the backup array respectively.

[0051] Specifically, in order to improve the reliability of data storage, in the present invention, by writing data into the main array and the backup array at the same time, when the disk corresponding to one of the arrays in the main array or the backup array fails, the other disk is still There is corresponding data, so as not to cause data loss and business interruption.

[0052] Step 103, when the data written in the primary array exceeds the preset threshold, save the data in the primary array, reselect the primary array and the backup array from the backup array and other arrays, and go to step 102.

[0053] The preset threshold can be selected arbitrarily according to needs. For example, for a jbod array, it is necessary to write data to other jbod arrays after a jbod array is full of data, then the preset threshold is the entire storage space of the jbod array at this time . Of course, in practical applications, other thresholds can also be selected. For example, 95% of the storage space of the array is selected as the preset threshold, which will not be described in detail in the present invention, and the preset threshold is taken as an example for the entire storage space of the array. Follow up instructions.

[0054] Specifically, since the storage space of each array is the same, when the main array is full of data, the corresponding backup array is also full of data. At this time, the data in the main array needs to be preserved, while the data in the backup array Data does not need to be retained. It can be understood that, after writing data into the primary array and the backup array, it is necessary to add 1 to the data writing times of the primary array, and add 1 to the data writing times of the backup array and the times of being used as the backup array respectively. Of course, the number of data writes and the number of backup arrays are not limited to adding 1. For example, after writing data to the main array and the backup array each time, the number of data writes to the main array and the backup array is increased by 2. Add 3 to the number of backup arrays used as backup arrays. For ease of description, in the present invention, the number of times of data writing and the number of times of being used as a backup array are both described by adding 1 as an example.

[0055] It should be noted that, in order to realize the process of retaining the data in the main array, but the data in the backup array does not need to be preserved, in the present invention, a flag bit can also be set, and by using this flag bit, it can be indicated that the data in the array needs to be preserved . That is, when the main array is full of data, set the flag bit for the main array and set the time to be reserved. In the subsequent process, the main array cannot be used for the main array or backup array again; and because the backup array is not set If the flag is set, the backup array can be used again as the main array or backup array in the subsequent process.

[0056] Of course, in practical applications, the data in the primary array can also be retained according to the selection rules of the primary array and the backup array, that is, each disk can only be used as the primary array once within the data retention time. For example, when the data needs to be retained for 15 days, when selecting the primary array and the backup array, it is necessary to ensure that each disk can only be used as the primary array once within 15 days.

[0057] It can be seen that since the main array has been filled with data, the main array cannot be used as an array for storing data again at this time, and according to the selection rules of the main array and the backup array, it can be guaranteed that the data in the main array will not be cleared ( destroyed) until the data on that primary array expires. As for the backup array and other arrays, since they can continue to write data, they can continue to serve as the main array and backup array to continue to store data. For the backup array, since the data has already been written, when the data is written again, it is only necessary to overwrite the original data.

[0058] In this step, when the main array is full of data, it is necessary to reselect the main array and the backup array from the backup array and other arrays, and go to step 102 .

[0059] Specifically, in the process of reselecting the primary array and the backup array from the backup array and other arrays, it is necessary to reselect the primary array and the backup array according to the number of times data is written to and used as the backup array by the backup array and other arrays. For example, select the array with the least number of data write times as the primary array, and select the array with the least number of times as the backup array as the backup array.

[0060] In addition, in order to ensure that the number of times each disk is used as a backup array is the same, a preset value can be selected according to the number of disks. When selecting a disk as a backup array, it is necessary to determine whether the number of times the disk is used as a backup array reaches the preset value. , if not, continue to select the disk as the backup array. Based on this situation, in the process of reselecting the primary array and the backup array from the backup array and other arrays, it is also necessary to determine whether the number of times the backup array has been used as a backup array exceeds the preset value; if so, according to the data of other arrays Write times to re-select the primary array, and re-select the backup array according to the number of times other arrays are used as backup arrays; otherwise, continue to select the backup array as the re-selected backup array, and re-select the primary array according to the data write times of other arrays.

[0061] It should be noted that by using the above-mentioned processing procedure, data can be written into the storage device, and in the present invention, in order to improve the reliability of data storage, the data can be written into the main array and the backup array at the same time , when the disk corresponding to one of the arrays in the primary array or the backup array fails, there is still corresponding data in the other disk, so that data loss and business interruption will not be caused.

[0062] Therefore, in the process of data writing, there are two situations to consider when a disk fails:

[0063] The first case:

[0064] When writing data to the backup array fails, you need to reselect the backup array from other arrays, continue to write data to the main array, and write the data to the reselected backup array;

[0065] Second case:

[0066] When writing data to the primary array fails, you need to use the backup array as the new primary array and reselect the backup array from other arrays to continue writing data to the new primary array and write data to the Reselected backup array.

[0067] In order to illustrate the technical solution provided by the present invention more clearly, the technical solution provided by the present invention will be described in detail below in conjunction with a specific application scenario. In this application scenario, it is assumed that each disk cabinet has 16 disks (the number of disks in the disk cabinet will not affect the specific implementation of the present invention), and 16 disks are made into 16 jbod arrays. For the convenience of description, each Each jbod array is numbered from 1 to 16, hereinafter referred to as array 1, array 2, and so on until array 16. In this application scenario, according to the business pressure that the disk can withstand (for example, each disk can have 30 cameras with 512K streams), the data recorded by the corresponding number of cameras (for example, 30 cameras) is completely distributed on one disk In, and through the cooperation of the monitoring scheme, it is possible to write down a jbod array when a jbod array is full. Based on the above situation, if figure 2 As shown, the data storage method further includes:

[0068] Step 201, create an array according to each disk. In this application scenario, 16 disks are formed into 16 arrays.

[0069] Step 202, select a primary array and a backup array in each array.

[0070] In the present invention, in order to distribute the wear and tear of the disks to each disk, the main array and the backup array can be made of 16 disks in turn. Therefore, before selecting the main array and the backup array, it is also necessary to record the data writing times of each array In the process of selecting the main array and the backup array, it is necessary to select the main array and the backup array according to the data writing times of each array and the number of times as the backup array.

[0071] Specifically, when initially writing data to each array, it is necessary to clear the counts of all disks as backup arrays, and set an array as the backup array by default. For example, first set array 16 as the backup array (of course, in practical applications, Other arrays can be selected as the backup array); and the number of times the disk is used as a backup array is recorded on the disk as 1, and the number of data write times is 0. In addition, it is also necessary to select an array as the main array, for example, first set array 1 as the main array (of course, other arrays can also be selected as the main array in practical applications).

[0072] It should be noted that if there are arrays in each array that have already written data, the corresponding data write times of the array and the number of backup arrays are not 0. At this time, the detailed process of selecting the primary array and the backup array will be Discussed in subsequent steps.

[0073] Step 203, when the data needs to be written to the storage device, form the primary array and the backup array into a temporary RAID1 array, and write the data to the primary array and the backup array at the same time. That is, when it is detected that there is data written into the array 1, the data needs to be written into the backup array (array 16) at the same time.

[0074] Step 204, when the data in the main array is full, the relationship of the temporary RAID1 array is released, the data in the main array is saved, and the process goes to step 202 to reselect the main array and the backup array from each array. Among them, since the storage space of each array is the same, when the main array is full of data, the corresponding backup array is also full of data. At this time, the data in the main array needs to be preserved, and the data in the backup array can Re-elected as primary or backup array.

[0075] In this application scenario, when the data on array 1 is full, the number of data writes in array 1 is 1, and the number of data writes in the backup array (array 16) is 1, which is 1 as the backup array. Since the main array has been filled with data, the main array cannot be used as an array for storing data again at this time, and for the backup array and other arrays, they can continue to be used as the main array and the backup array to continue storing data.

[0076] When going to step 202 and reselecting the primary array and the backup array from each array, it is necessary to reselect the primary array and the backup array according to the number of times data is written to the backup array and other arrays and the number of times they are used as backup arrays. For example, select the array with the least data writing times as the primary array (array 1 cannot be selected at this time), and select the array with the least number of times as the backup array as the backup array.

[0077]In practical applications, in order to distribute the loss of the disks to each disk, the backup array can be made of 16 disks in turn. Based on this premise, in the present invention, the number of times each array can be used as a backup array can be selected as disk The total number minus 1, for example, when array 1-array 15 is selected as the main array in turn, array 16 is always the backup array, that is, array 16 is used as the backup array 15 times; after that, array 15 is selected as the backup array, when array 1-array 14 and When the array 16 is selected as the main array in turn, the array 15 is always the backup array; by analogy, the backup array can be made by 16 disks in turn, and each disk is used as the backup array for the same number of times, so that the loss of the disk is evenly distributed to each disk. block disk.

[0078] Based on the above situation, a preset value can be selected according to the number of disks. When selecting a disk as a backup array, it is necessary to determine whether the number of times the disk is used as a backup array reaches the preset value. If not, continue to select the disk as the backup array. Backup array.

[0079] In this application scenario, after array 1 is full of data, the data will be migrated to the next array (for example, array 2, which is the re-selected main array. It is understandable that in the process of selecting the main array, it is necessary to select the data to write The array with the least number of times is stored in the primary array); and when it is detected that there is data traffic on the array 2, the data needs to be written to the backup array (still the array 16 at this time).

[0080] It should be noted that the data mentioned in this application scenario is time-sensitive continuous writing data, that is, when the aging time of the data is reached, it is determined that the data stored in the corresponding array is invalid, and it is determined that the array can continue to be selected for subsequent Primary array or backup array in data storage process. For example, if the aging time of data is 7 days, the data written in array 1 will become invalid after 7 days after the data writing time in array 1. At this time, array 1 can be used as the primary array or the backup array. Therefore, in the process of selecting the main array, the data write times of array 1 is 1, and if the data write times of other arrays are all less than 1, array 1 will not be selected as the main array, and if the data of other arrays When the write times are greater than 1, array 1 needs to be selected as the main array.

[0081] It should be noted that during the data writing process, the array is RAID1, that is, the data is written to the primary array and the backup array at the same time, so when a disk fails, it will not cause data loss and business interruption.

[0082] In this application scenario, in the process of data writing, if array 1 reports an error (that is, the main array fails), cancel the flag of backup array 16, replace array 1 with array 16 as the main array, and reselect an array as the backup Array (for example, array 15, now need to record the array 15 as backup array times is 1, data write times is 1), data is written to the main array (array 16) and backup array (array 15) at the same time. In the process of data writing, if the backup array 16 reports an error, the backup array is reselected, and the data continues to be written into the main array, and is written into the newly selected backup array.

[0083] In summary, it can be seen that by using the technical solution provided by the present invention, the loss of disks can be reduced, and the service life of disks and equipment can be increased; and only a small part of disks are running on each disk cabinet, and the combination of disk sleep technology can save energy. effect; even if there is a disk failure and data recovery is required, only the data of one disk needs to be restored, which will not affect the business and data on other disks; and it can increase the security of data writing, and will not be caused by hard disk failure As a result, data writing fails and services are interrupted.

[0084] For example, if the monitoring data needs to be kept for 15 days, a disk needs to be filled every day. The data of each day is completely written into a disk (primary array) that actually stores data and a disk (backup array) that actually stores data. There are only two running disks, and the rest of the disks are in a dormant state. On the one hand, it ensures the security of data writing and business continuity, and at the same time, it can achieve a good energy-saving effect.

[0085] Based on the same inventive concept as the above method, the present invention also proposes a data storage device applied to a storage device including multiple arrays, such as image 3 As shown, the device further includes:

[0086] A selection module 10, configured to select a main array and a backup array from the plurality of arrays; and when the data written in the main array exceeds a preset threshold, reselect the main array from the backup array and other arrays and backup arrays.

[0087] Specifically, in the process of selecting the main array and the backup array from the plurality of arrays, the selection module 10 is also used to select the main array and the backup array according to the data writing times of each array and the times of serving as backup arrays; And reselect the main array and the backup array according to the data write times of the backup array and other arrays and the number of times as backup arrays; and further used to determine whether the number of times of the backup array as a backup array exceeds a preset value; if so , then reselect the primary array according to the data write times of other arrays, and reselect the backup array according to the number of times other arrays are used as backup arrays; otherwise, continue to select the backup array as the reselected backup array, and reselect Number of writes to reselect the primary array.

[0088] The storage module 20 is connected with the selection module 10, and is used for writing data into the main array and the backup array when there is data to be stored in the storage device.

[0089] In addition, in the present invention, in order to ensure the balanced use of each array, it is also necessary to record the data writing times of each array and the number of backup arrays, and when selecting the main array and the backup array from multiple arrays, according to each array Select the primary array and the backup array based on the number of data write times and the number of times as the backup array. Based on this situation, the storage module 20 is further configured to record the number of times data is written in each array and the number of times it is used as a backup array.

[0090] The processing module 30 is connected with the storage module 20, and is configured to save the data in the main array when the data written in the main array exceeds a preset threshold.

[0091] After writing data into the main array and the backup array, the storage module 20 records the number of data writes in each array and the number of times it is used as a backup array, and the processing module 30 is also used to write the data of the main array Add 1 to the data writing times, and add 1 to the data writing times of the backup array and the backup array times.

[0092] It should be noted that, in order to improve the reliability of data storage, by writing data to the primary array and the backup array at the same time, when the disk corresponding to one of the primary array or the backup array fails, the other disk still has the corresponding disk. data, so as not to cause data loss and business interruption.

[0093] Therefore, in the process of data writing, when writing data to the backup array fails, the selection module 10 is also used to reselect the backup array from other arrays; the storage module 20 is also used to , continue to write data into the primary array, and write the data to the reselected backup array;

[0094] When writing data to the main array fails, the selection module 10 is also used to use the backup array as a new main array and reselect a backup array from other arrays; the storage module 20 is also used Then, continue to write data to the new primary array, and write data to the reselected backup array.

[0095] It should be noted that the data in the present invention includes time-sensitive continuous writing data. At this time, the processing module 30 is also used to determine that the data stored in the main array is invalid, and to determine that the data stored in the main array can continue to be selected. The main array mentioned above is the main array or the backup array in the subsequent data storage process.

[0096] Wherein, each module of the device of the present invention can be integrated into one body, or can be deployed separately. The above modules can be combined into one module, or can be further split into multiple sub-modules.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be realized by hardware, or by software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present invention can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.), including several The instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present invention.

[0098] Those skilled in the art can understand that the drawing is only a schematic diagram of a preferred embodiment, and the modules or processes in the drawing are not necessarily necessary for implementing the present invention.

[0099] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description in the embodiment, and can also be changed and located in one or more devices different from the device in the embodiment. The modules in the above embodiments can be combined into one module, and can also be further split into multiple sub-modules.

[0100] The above serial numbers of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0101] The above disclosures are only a few specific embodiments of the present invention, however, the present invention is not limited thereto, and any changes conceivable by those skilled in the art shall fall within the protection scope of the present invention.