Disk load balancing storage method, system, storage medium and computer

Abstract

The present invention provides a multi-disk load balancing storage method, system, storage medium and computer, the storage method includes: respectively obtaining the storage capacity of several disks used to store data, selecting the disk with the largest storage capacity as a reference disk based on the storage capacity of each disk; dividing each disk into several logical volumes; respectively defining variables for each disk, calculating the capacity value corresponding to each disk according to the variable definition of each disk, determining the write load of each disk according to the capacity value of each disk and sorting them according to the level of the write load; obtaining the data to be stored, traversing the write load of all disks, determining the disk with the lowest write load according to the level of the write load sorting and using it as the target disk for current data storage; storing the data to be stored in the target disk to complete the load balancing storage of the data. The present application provides a storage method, the load storage is uniform, and the storage cycle of each disk is consistent.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a multi-disk load balancing storage method, system, storage medium and computer. Background Art

[0002] In the IoT architecture for the next-generation information infrastructure, cloud-edge-end is a popular architecture. Cloud-edge-end provides a good solution to address the challenges of cloud computing and cloud storage, network pressure, improve user experience, and meet business needs. The edge storage and computing machine is one of the important devices in the cloud-edge-end architecture. The edge storage and computing machine gradually assumes more functions such as AI computing and data processing. For example, the edge storage and computing machine is responsible for accessing the real-time video, captured images, and alarm events of the edge device IPC (network camera) and NVR (network video recorder), and stores the data locally. The business layer can perform AI analysis on the historical videos stored locally to extract valuable data. The premise of doing a good job of AI analysis is to store the data well first. The edge storage and computing machine generally contains multiple disks, and the conventional configuration is more than 8 disks. The storage capacity of each disk is uncertain, which may be 8T, 12T or 16T.

[0003] At present, if the edge storage and computing machine needs to access multiple IPCs and perform video loop write storage and store multiple videos on multiple local hard disks, due to the non-uniform disk storage capacity, facing disks with different storage capacities, the existing storage method lacks targeted and reasonable division of video data accessed by each disk. Each disk cannot achieve load-balanced storage, resulting in difficulty in maintaining consistency in the storage cycles of multiple disks, affecting data storage efficiency. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a multi-disk load balancing storage method, system, storage medium and computer to solve the technical problems existing in the prior art.

[0005] The present invention provides a multi-disk load balancing storage method, comprising:

[0006] respectively obtaining storage capacities of several disks for storing data, and selecting a disk with the largest storage capacity as a reference disk based on the storage capacity of each disk;

[0007] Formatting each disk in turn according to a preset storage capacity, so as to divide each disk into storage units including a corresponding number of logical volumes, wherein the storage capacity of the logical volumes in different disks is the same;

[0008] Define variables for each disk respectively, calculate the capacity value corresponding to each disk according to the variable definition of each disk, determine the write load of each disk according to the capacity value of each disk, and sort the disks according to the write load;

[0009] Obtain the data to be stored, traverse the write loads of all disks, determine the disk with the lowest write load according to the write load sorting results, and use it as the target disk for current data storage;

[0010] The data to be stored is stored in the target disk to complete the load-balanced storage of data in multiple disks.

[0011] Preferably, the variable definition includes at least:

[0012] Maximum number of logical volumes: the maximum number of logical volumes is determined according to the storage capacity of the reference disk and the storage capacity of the logical volumes;

[0013] The number of disk logical volumes is determined in sequence according to the storage capacity of each disk and the storage capacity of the logical volumes;

[0014] Logical volume number: the logical volume number is edited in ascending order according to the number of logical volumes corresponding to each disk;

[0015] Write sequence number: The write sequence number gradually increases according to the number of times the logical volume corresponding to each disk is written.

[0016] Preferably, the step of calculating the capacity value corresponding to each disk according to the variable definition of each disk, determining the write load of each disk according to the capacity value of each disk, and sorting the disks according to the write load includes:

[0017] Sequentially obtain the current write serial numbers and corresponding numbers of disk logical volumes corresponding to a number of disks, and calculate the capacity value corresponding to each disk according to the current write serial number of the disk, the number of disk logical volumes and the maximum number of logical volumes;

[0018] The calculation expression of the capability value is:

[0019]

[0020] In the formula, Indicates The capacity value of each disk, For the The current write sequence number of the disk, Indicates the maximum number of logical volumes. For the The number of disk logical volumes per disk;

[0021] The capacity values ​​of the disks are sorted in order of size, and the write loads of the disks are sorted in order of high and low according to the sorting result of the capacity values, wherein a disk with a larger capacity value corresponds to a higher write load.

[0022] Preferably, after the data to be stored is stored in the target disk to complete the load-balanced storage of data in multiple disks, a blank disk without data stored is inserted, and the multi-disk load-balanced storage method further includes:

[0023] Acquiring the storage capacity of the blank disk and formatting it to divide the blank disk into storage units corresponding to a plurality of logical volumes;

[0024] Obtain a calibration disk with the lowest write load among other disks except the blank disk, and obtain capability information corresponding to the calibration disk, wherein the capability information at least includes a disk ID, a current write sequence number, a number of logical volumes, and a capability value of the disk;

[0025] updating the current write sequence number of the blank disk according to the capability information of the calibration disk and the number of logical volumes of the blank disk;

[0026] The capacity value of the blank disk is adjusted according to the current write sequence number of the blank disk, so that the write load of the blank disk corresponds to the write load of other disks except the blank disk.

[0027] Preferably, the expression for updating the current write sequence number of the blank disk is:

[0028]

[0029] In the formula, Indicates the current write sequence number for blank disk updates, Indicates the current write sequence number of the calibration disk, Indicates the number of logical volumes on a blank disk. Indicates the number of logical volumes of the calibration disk.

[0030] Preferably, the data to be stored is multi-channel video data. After the data to be stored is stored in the target disk to complete the load-balanced storage of the data in multiple disks, the multi-disk load-balanced storage method further includes:

[0031] defining adjustment window bandwidths of a plurality of disks so that a ratio of the adjustment window bandwidths of the disks corresponds to a ratio of the storage capacities of the disks, and determining a total adjustment window bandwidth according to the adjustment window bandwidths of the disks;

[0032] Acquire the bandwidth of each channel of video data in the multiple channels of video data and the total bandwidth of the multiple channels of video data, and determine the number of adjustment windows of each disk according to the total bandwidth of the multiple channels of video data and the total bandwidth of the adjustment windows;

[0033] Obtain the number of channels for accessing video data from each disk, and determine the current access bandwidth of each disk based on the bandwidth of each channel of video data;

[0034] The theoretical bandwidth of each disk is calculated, and the bandwidth of the disk is adjusted according to the theoretical bandwidth of each disk and the current access bandwidth of each disk, so that the data access storage cycle of each disk corresponds to each other.

[0035] Preferably, the step of calculating the theoretical bandwidth of each disk and adjusting the bandwidth of the disk according to the theoretical bandwidth of each disk and the current access bandwidth of each disk so that the data access storage period of each disk corresponds to each other includes:

[0036] Calculate the theoretical bandwidth of the disk based on the adjustment window bandwidth of each disk and the corresponding adjustment window quantity;

[0037] Calculate the difference between the current access bandwidth of each disk and the theoretical bandwidth corresponding to each disk, and arrange them in descending order;

[0038] A first cycle bandwidth adjustment is performed according to the difference sorting result, where the first cycle bandwidth adjustment includes:

[0039] Select the disk k with the largest difference and the disk j with the smallest difference, and sort the videos connected to the disk k in order of bandwidth from small to large;

[0040] According to the bandwidth sorting results of the plurality of videos connected to the disk k, a second cyclic bandwidth adjustment is performed on the disk k and the disk j. The second cyclic bandwidth adjustment includes:

[0041] Switch the videos with the smallest bandwidth on disk k to disk j for access and storage in sequence. After each switch, re-acquire the access bandwidth of disk k and disk j, and calculate the difference between the re-acquired access bandwidth of disk k and disk j and the corresponding theoretical bandwidth.

[0042] Until the difference between the access bandwidth of disk k and disk j and the corresponding theoretical bandwidth meets the preset requirement, the second cycle bandwidth adjustment of disk k and disk j is completed;

[0043] According to the sorting results of the difference calculation, the corresponding disks are selected from both ends of the difference sorting results in turn, and the above-mentioned second cyclic bandwidth adjustment is repeated until the difference between the access bandwidth of all disks and the corresponding theoretical bandwidth meets the preset requirements, and the first cyclic bandwidth adjustment is completed, so that the data access storage period of each disk corresponds.

[0044] The present invention also provides a multi-disk load balancing storage system, comprising:

[0045] A selection module is used to respectively obtain the storage capacity of several disks used to store data, and select a disk with the largest storage capacity as a reference disk based on the storage capacity of each disk;

[0046] A partitioning module, used to format each disk in turn according to a preset storage capacity, so as to divide each disk into storage units containing a corresponding number of logical volumes, wherein the storage capacity of the logical volumes in different disks is the same;

[0047] A definition module is used to define variables for each disk, calculate the capacity value corresponding to each disk according to the variable definition of each disk, determine the write load of each disk according to the capacity value of each disk, and sort the disks according to the write load;

[0048] The traversal module is used to obtain the data to be stored, traverse the write loads of all disks, and determine the disk with the lowest write load according to the sorting results of the write loads and use it as the target disk for current data storage;

[0049] The storage module is used to store the data to be stored in the target disk to complete the load-balanced storage of data in multiple disks.

[0050] The present invention also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the multi-disk load balancing storage method is implemented.

[0051] The present invention also proposes a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multi-disk load balancing storage method when executing the computer program.

[0052] Compared with the prior art, the present invention has the following beneficial effects: the multi-disk load balancing storage method provided by the present application first divides the logical volumes according to the size of the disk and the size of the logical volumes, and the data storage uses the logical volumes as the data writing units. When one logical volume is full, the next logical volume is applied for. When the next logical volume file is written, it needs to be re-applied for. The application process is a load balancing adjustment process, while ensuring distributed writing and load balancing; after all the logical volumes in the disk are written, the earliest written logical volume can be overwritten to achieve periodic cyclic storage of data; then, according to the calculated capacity value of each disk, the write load of the disk is determined according to the capacity value and sorted according to the write load, the data to be stored is obtained, the write load of all disks is traversed, and the disk with the lowest write load is determined according to the write load sorting and used as the target disk for current data storage and the data is stored in the target disk; when storing videos, a disk with the lowest write capacity value is selected from multiple local disks for writing, and the storage capacity of each disk is mapped to a write capacity value through an algorithm. Through the capacity value, disks with different storage capacities can be mapped to the same writing standard. When there are multiple disks with different storage capacities storing data at the same time, it can be ensured that different disks are written in a distributed manner according to the proportion of the disk storage capacity. This solves the problem of inconsistent storage cycles caused by the difficulty in achieving load balancing on disks with different storage capacities.

[0053] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of the multi-disk load balancing storage method in the first embodiment of the present invention;

[0055] Figure 2 FIG. 4 is a structural block diagram of a computer in a fourth embodiment of the present invention.

[0056] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0057] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0059] Embodiment 1

[0060] See also Figure 1 , which shows a multi-disk load balancing storage method in a first embodiment of the present invention, and the multi-disk load balancing storage method specifically includes steps S10 to S50:

[0061] S10, respectively obtaining the storage capacity of several disks for storing data, and selecting a disk with the largest storage capacity as a reference disk based on the storage capacity of each disk;

[0062] In order to better illustrate the technical solution of the present invention, illustratively, in this embodiment, monitoring within a certain area is taken as an example for explanation. The video data generated by the monitoring needs to be stored in real time. It can be understood that there may be multiple monitoring devices, that is, multiple channels of video may need to be stored in the edge storage and computing machine at the same time. The edge storage and computing machine generally includes multiple disks. The storage capacity of the disks is different, which may be 8T, 12T or 16T, etc. In order to ensure the timeliness of the data, the disks are stored in a circular manner. When a disk is full, the old data in the disk will be overwritten in turn; if the storage is performed randomly, the 8T disk may be full and the old data is overwritten, while the 16T disk still has a lot of free storage space left, resulting in different overall storage cycles of the multiple disks in the edge storage and computing machine, which is not convenient for subsequent analysis of the stored data.

[0063] First, the storage capacity of several disks used to store data in the edge storage and computing machine is obtained. For the convenience of explanation, in this embodiment, three disks with different storage capacities of 8T, 12T and 16T are used for explanation. Based on the storage capacity of each disk, the 16T disk with the largest storage capacity is selected as the reference disk, and the 8T and 12T disks are used as comparison disks.

[0064] S20, formatting each disk in sequence according to a preset storage capacity, so as to divide each disk into storage units including a corresponding number of logical volumes, wherein the storage capacity of the logical volumes in different disks is the same;

[0065] In the specific implementation, three disks with different storage capacities of 8T, 12T and 16T are formatted in turn to divide the disks into logical volumes containing several preset storage capacities. Optionally, in this embodiment, the storage capacity of the logical volumes in each disk is the same, which is 256MB. Then, the 8T disk can be divided into 32768 logical volumes, the 12T disk can be divided into 49152 logical volumes, and the 16T disk can be divided into 65536 logical volumes.

[0066] Furthermore, during the disk storage process, there may be a problem of long seek time causing single point failure of the disk. In order to improve the security of disk storage data and subsequent data reading and writing performance, in this embodiment, a dual controller can be used to solve the problem of single point failure of the disk in the system. The two controllers monitor each other's status in real time and implement fault switching when a failure occurs, thereby ensuring the continuity of work when the disk stores data.

[0067] In this embodiment, the data of each disk is stored in the logical volume contained therein, so the management of the disk by the dual controller can be equivalently understood as the management of the logical volume in the disk. The method for managing the logical volume by the dual controller includes:

[0068] Obtaining metadata of the logical volume and sending the metadata to two controllers for storage. Both controllers store all metadata, but a certain logical volume is only controlled by one controller at a time, that is, both controllers store metadata of the logical volume;

[0069] respectively define working modes of the dual controllers, the working modes at least including a startup state, a single load state, a dual load state and a failure state, wherein the single load state is a state in which both controllers are in normal working order;

[0070] In response to the system starting up, the controller enters the starting state, and enters the single load state after the normal preset time. The purpose of entering the single load state after the preset time is to prevent the other party from delaying the work due to the late start and to prevent repeated state switching;

[0071] After the current controller enters the single-load state, it monitors the state of the other controller in real time. If the other controller does not respond, it enters the decision layer to obtain control of the other controller and take over the logical volume and data of the other controller. At this time, the takeover controller is in a dual-load state, and the taken-over controller is in a failed state.

[0072] If the other controller does not respond, the steps of entering the decision layer to obtain the control right of the other controller include:

[0073] When the taken-over controller fails, the system fails to send messages to the taken-over controller. At this time, the system will make a reconnection request within a certain period of time;

[0074] In response to the taken-over controller being in a failed state, interrupting its external data service and setting all logical volumes under its control to a silent state, so that the taken-over controller vacates the channel network card occupied by it for use by the taking-over controller;

[0075] The controller taking over responds to the reconnection request of the system through the router, and sends its own physical address to the router for address refresh to match the physical address of the controller taking over with the IP address of the system, and then sends the data packet to the controller being taken over to obtain control of the controller being taken over.

[0076] S30, defining variables for each disk respectively, calculating the capability value corresponding to each disk according to the variable definition of each disk, determining the write load of each disk according to the capability value of each disk and sorting the disks according to the write load;

[0077] Optionally, in this example, the variable definition at least includes:

[0078] Maximum number of logical volumes: the maximum number of logical volumes is determined according to the storage capacity of the reference disk and the storage capacity of the logical volumes;

[0079] The number of disk logical volumes is determined in sequence according to the storage capacity of each disk and the storage capacity of the logical volumes;

[0080] Logical volume number: the logical volume number is edited in ascending order according to the number of logical volumes corresponding to each disk;

[0081] Write sequence number: the write sequence number gradually increases according to the number of times the logical volume corresponding to each disk is written;

[0082] Earliest write sequence number: The earliest write sequence number is the write sequence number of the initial logical volume. When a disk performs cyclic overwrite writing, the difference between the write sequence number and the earliest write sequence number is equal to the number of logical volumes corresponding to the disk.

[0083] Optionally, in this embodiment, the maximum number of logical volumes (Max Volume Num) is a fixed value of 65536; the number of disk logical volumes (Disk Volume Num), the number of disk logical volumes of an 8T disk is 32768, and the number of disk logical volumes of a 12T disk is 49152; the logical volume number (Volume Num), the logical volume number of an 8T disk is 1-32768, the logical volume number of a 12T disk is 1-49152, and the logical volume number of a 16T disk is 1-65536, each number corresponds to each logical volume file; the write sequence number (Write Sn), each disk has a corresponding write sequence number, the write sequence number is of int64 type, and increases continuously. Taking an 8T disk as an example, from the beginning of writing to full, the value of Write Sn is 1-32768. When the disk is full and continues to be overwritten, Write Sn will start from 32769 until 65536 is full, and so on. When the disk is continuously overwritten, Write Sn will keep growing; the earliest written sequence number (Earliest Sn) will be updated when the disk performs cyclic overwrite. The difference between Write Sn and Earliest Sn is exactly equal to the Disk Volume Num of the disk.

[0084] The steps of calculating the capacity value corresponding to each disk according to the variable definition of each disk, determining the write load of each disk according to the capacity value of each disk, and sorting the disks according to the write load include:

[0085] Sequentially obtain the current write serial numbers and corresponding numbers of disk logical volumes corresponding to a number of disks, and calculate the capacity value corresponding to each disk according to the current write serial number of the disk, the number of disk logical volumes and the maximum number of logical volumes;

[0086] The calculation expression of the capability value is:

[0087]

[0088] In the formula, Indicates The capacity value of each disk, For the The current write sequence number of the disk, Indicates the maximum number of logical volumes. For the The number of disk logical volumes per disk;

[0089] The capacity values ​​of the disks are sorted in order of size, and the write loads of the disks are sorted in order of high and low according to the sorting result of the capacity values, wherein a disk with a larger capacity value corresponds to a higher write load.

[0090] It is understandable that as Write Sn increases, the capacity value of the disk also increases, and when the disk storage capacity is different, the capacity value increases at different speeds. When a 16T disk fills up a disk logical volume, the capacity value increases in a trend of 1, 2, 3, 4, 5..., while when an 8T disk fills up a disk logical volume, the capacity value increases in a trend of 2, 4, 6, 8, 10...

[0091] S40, obtaining data to be stored, traversing the write loads of all disks, determining the disk with the lowest write load according to the write load sorting result and using it as the target disk for current data storage;

[0092] S50, storing the data to be stored in the target disk to complete the load-balanced storage of the data in multiple disks.

[0093] Optionally, the data to be stored is multi-channel video data. When storing data, the capacity value is positively correlated with the write load. Therefore, the disk with the lowest write load is used as the target disk for writing, that is, the disk with the smallest capacity value is used as the target disk for writing. As can be seen from the capacity value calculation formula, different disks have different capacity values. Among them, the capacity value of an 8T disk is 2 ， The capacity value of a 12T disk is 1.5 ， The capacity of a 16T disk is ，The capacity values ​​of a 16T disk for writing and storing are 1, 2, 3, 4, 5, ..., the capacity values ​​of a 12T disk for writing and storing are 1.5, 3, 4.5, 6, 7.5, ..., and the capacity values ​​of an 8T disk for writing and storing are 2, 4, 6, 8, 10, ..., therefore, in this example, the sequential cycles for storing data according to the capacity values ​​are: first sequential cycle: 1 (16T), 1.5 (12T), 2 (8T), 2 (16T), 3 (12T), 3 (16T), 4 (16T), 4 (8T), 4.5 (12T); second sequential cycle: 5 (16T), 6 (12T), 6 (8T), 6 (16T), 7 (16T), 7.5 (12T), 8 (8T), 8 (16T ), 9 (12T), and so on; it can be seen that when storing data, 9 logical volumes are used as a sequential cycle. In the first sequential cycle, logical volumes in 4 16T disks, logical volumes in 3 12T disks, and logical volumes in 2 8T disks are selected for storage; the order of selecting logical volumes in the second sequential cycle is similar to that in the first sequential cycle; and so on, that is, when using disks to store data, the ratio of logical volumes selected from 8T, 12T and 16T disks is 2:3:4; in this embodiment, by adding a write capacity value statistic to each disk, the larger the write capacity value, the greater the write load. When storing videos, the disk with the lowest write capacity value is selected from multiple local disks for writing, and disks with different storage capacities are mapped to a write capacity value through an algorithm. Through the capacity value, disks with different storage capacities can be mapped to the same write standard. When there are multiple disks with different storage capacities for simultaneous storage, it can be guaranteed that different disks are written in a distributed manner according to the disk storage capacity ratio; it solves the problem that disks with different storage capacities are difficult to achieve load balancing, resulting in inconsistent storage cycles.

[0094] Furthermore, in some optional embodiments, assuming that several disks in the edge storage and computing machine are storing data and are written evenly under normal circumstances, when the edge storage and computing machine suddenly needs to insert a blank disk that does not store data for data storage, it is understandable that since the current write serial number of the new disk is the initial value, compared with other disks that have already stored data, the current capacity value of the new disk is the smallest, that is, its write load is the lowest. As a result, all data writes in the subsequent period of time will be concentrated on this new disk. This storage method is unreasonable, and the data processing volume of the new disk increases dramatically, which is easy to cause damage.

[0095] Therefore, when the edge storage and computing machine needs to insert a blank disk that does not store data, the multi-disk load balancing storage method also includes:

[0096] Acquiring the storage capacity of the blank disk and formatting it to divide the blank disk into storage units corresponding to a plurality of logical volumes;

[0097] Obtain a calibration disk with the lowest write load among other disks except the blank disk, and obtain capability information corresponding to the calibration disk, wherein the capability information at least includes a disk ID, a current write sequence number, a number of logical volumes, and a capability value of the disk;

[0098] updating the current write sequence number of the blank disk according to the capability information of the calibration disk and the number of logical volumes of the blank disk;

[0099] The capacity value of the blank disk is adjusted according to the current write sequence number of the blank disk, so that the write load of the blank disk corresponds to the write load of other disks except the blank disk.

[0100] The expression for updating the current write sequence number of the blank disk is:

[0101]

[0102] In the formula, Indicates the current write sequence number for blank disk updates, Indicates the current write sequence number of the calibration disk, Indicates the number of logical volumes on a blank disk. Indicates the number of logical volumes of the calibration disk.

[0103] Optionally, in this embodiment, after a blank disk that does not store data is inserted into the edge storage and computing machine, the above-mentioned update operation is performed on the blank disk. Through the update operation on the blank disk, the write load of the blank disk is matched with other disks that have already written data to the same write standard for load scheduling, thereby improving the balanced storage characteristics of the disk.

[0104] Furthermore, since the bandwidths of different videos are not necessarily the same, the storage paths of multiple videos stored by load balancing storage may not necessarily ensure that the storage bandwidths of the disks are the same, that is, the storage cycles of the disks cannot be guaranteed to be the same; in some optional embodiments, the multi-disk load balancing storage method further includes:

[0105] defining adjustment window bandwidths of a plurality of disks so that a ratio of the adjustment window bandwidths of the disks corresponds to a ratio of the storage capacities of the disks, and determining a total adjustment window bandwidth according to the adjustment window bandwidths of the disks;

[0106] Acquire the bandwidth of each channel of video data in the multiple channels of video data and the total bandwidth of the multiple channels of video data, and determine the number of adjustment windows of each disk according to the total bandwidth of the multiple channels of video data and the total bandwidth of the adjustment windows;

[0107] Obtain the number of channels for accessing video data from each disk, and determine the current access bandwidth of each disk based on the bandwidth of each channel of video data;

[0108] The theoretical bandwidth of each disk is calculated, and the bandwidth of the disk is adjusted according to the theoretical bandwidth of each disk and the current access bandwidth of each disk, so that the data access storage cycle of each disk corresponds to each other.

[0109] The step of calculating the theoretical bandwidth of each disk and adjusting the bandwidth of the disk according to the theoretical bandwidth of each disk and the current access bandwidth of each disk so that the data access storage period of each disk corresponds to each other includes:

[0110] Calculate the theoretical bandwidth of the disk based on the adjustment window bandwidth of each disk and the corresponding adjustment window quantity;

[0111] Calculate the difference between the current access bandwidth of each disk and the theoretical bandwidth corresponding to each disk, and arrange them in descending order;

[0112] A first cycle bandwidth adjustment is performed according to the difference sorting result, where the first cycle bandwidth adjustment includes:

[0113] Select the disk k with the largest difference and the disk j with the smallest difference, and sort the videos connected to the disk k in order of bandwidth from small to large;

[0114] According to the bandwidth sorting results of the plurality of videos connected to the disk k, a second cyclic bandwidth adjustment is performed on the disk k and the disk j. The second cyclic bandwidth adjustment includes:

[0115] Switch the videos with the smallest bandwidth on disk k to disk j for access and storage in sequence. After each switch, re-acquire the access bandwidth of disk k and disk j, and calculate the difference between the re-acquired access bandwidth of disk k and disk j and the corresponding theoretical bandwidth.

[0116] Until the difference between the access bandwidth of disk k and disk j and the corresponding theoretical bandwidth meets the preset requirement, the second cycle bandwidth adjustment of disk k and disk j is completed;

[0117] According to the sorting results of the difference calculation, the corresponding disks are selected from both ends of the difference sorting results in turn, and the above-mentioned second cyclic bandwidth adjustment is repeated until the difference between the access bandwidth of all disks and the corresponding theoretical bandwidth meets the preset requirements, and the first cyclic bandwidth adjustment is completed, so that the data access storage period of each disk corresponds.

[0118] Optionally, when adjusting the bandwidth of the disk, the disk access bandwidth needs to be in a stable stage. The disk bandwidth statistics time can be 30s, that is, each video is accessed to the disk for 30s before the bandwidth access between the disk and the video is considered to be in a stable state; define the adjustment window bandwidth of 8T, 12T, and 16T to be 8Mb, 12Mb, and 16Mb respectively. When accessing, the video is connected to the disk window accordingly. The total bandwidth of the adjustment window is 36Mb. Assuming that the total bandwidth of the accessed video is 360 Mb, each disk has 10 adjustment windows, and the theoretical bandwidths of the disks are 80Mb, 120Mb, and 160Mb respectively; since the bandwidth of each video channel may be different, the access of 360Mb video bandwidth to 8T, 12T, and 16T disks is not necessarily in the ratio of 2:3:4. The access bandwidth of some disks may be greater than their theoretical bandwidth, and the access bandwidth of other disks may be less than their theoretical bandwidth, that is, the difference between the current access bandwidth of each disk and the theoretical bandwidth corresponding to each disk may be positive or negative; therefore, after the video access is stable, the number of video channels and bandwidths accessed by each disk are further dynamically adjusted, so that 8T, 12T, and 16T disks access video data in the ratio of 2:3:4, and the data access cycle corresponding to each disk is the same.

[0119] In summary, the multi-disk load balancing storage method provided by the present application first divides the logical volumes according to the size of the disk and the size of the logical volumes. The data storage uses the logical volume as the data writing unit. When one logical volume is full, the next logical volume is applied for. When the next logical volume file is written, it needs to be reapplied. The application process is a load balancing adjustment process, while ensuring distributed writing and load balancing; after all the logical volumes in the disk are written, the earliest written logical volume can be overwritten to achieve periodic storage of data; then, according to the calculated capacity value of each disk, the write load of the disk is determined according to the capacity value and sorted according to the write load, the data to be stored is obtained, the write load of all disks is traversed, and the disk with the lowest write load is determined according to the write load sorting and used as the target disk for current data storage and the data is stored in the target disk; when storing the video, a disk with the lowest write capacity value is selected from multiple local disks for writing, and the storage capacity of each disk will be mapped to a write capacity value through an algorithm. Through the capacity value, disks with different storage capacities can be mapped to the same writing standard. When there are multiple disks with different storage capacities storing data at the same time, it can be ensured that different disks are written in a distributed manner according to the proportion of the disk storage capacity. This solves the problem of inconsistent storage cycles caused by the difficulty in achieving load balancing on disks with different storage capacities.

[0120] Embodiment 2

[0121] This embodiment provides a multi-disk load balancing storage system, including:

[0122] A selection module is used to respectively obtain the storage capacity of several disks used to store data, and select a disk with the largest storage capacity as a reference disk based on the storage capacity of each disk;

[0123] A partitioning module, used to format each disk in turn according to a preset storage capacity, so as to divide each disk into storage units containing a corresponding number of logical volumes, wherein the storage capacity of the logical volumes in different disks is the same;

[0124] A definition module is used to define variables for each disk, calculate the capacity value corresponding to each disk according to the variable definition of each disk, determine the write load of each disk according to the capacity value of each disk, and sort the disks according to the write load;

[0125] The traversal module is used to obtain the data to be stored, traverse the write loads of all disks, and determine the disk with the lowest write load according to the sorting results of the write loads and use it as the target disk for current data storage;

[0126] The storage module is used to store the data to be stored in the target disk to complete the load-balanced storage of data in multiple disks.

[0127] Wherein, the variable definition at least includes:

[0128] Maximum number of logical volumes: the maximum number of logical volumes is determined according to the storage capacity of the reference disk and the storage capacity of the logical volumes;

[0129] The number of disk logical volumes is determined in sequence according to the storage capacity of each disk and the storage capacity of the logical volumes;

[0130] Logical volume number: the logical volume number is edited in ascending order according to the number of logical volumes corresponding to each disk;

[0131] Write sequence number: The write sequence number gradually increases according to the number of times the logical volume corresponding to each disk is written.

[0132] The steps of calculating the capacity value corresponding to each disk according to the variable definition of each disk, determining the write load of each disk according to the capacity value of each disk, and sorting the disks according to the write load include:

[0133] Sequentially obtain the current write serial numbers and corresponding numbers of disk logical volumes corresponding to a number of disks, and calculate the capacity value corresponding to each disk according to the current write serial number of the disk, the number of disk logical volumes and the maximum number of logical volumes;

[0134] The calculation expression of the capability value is:

[0135]

[0136] In the formula, Indicates The capacity value of each disk, For the The current write sequence number of the disk, Indicates the maximum number of logical volumes. For the The number of disk logical volumes per disk;

[0137] The capacity values ​​of the disks are sorted in order of size, and the write loads of the disks are sorted in order of high and low according to the sorting result of the capacity values, wherein a disk with a larger capacity value corresponds to a higher write load.

[0138] An insert module, the insert module being used for:

[0139] Acquiring the storage capacity of the blank disk and formatting it to divide the blank disk into storage units corresponding to a plurality of logical volumes;

[0140] Obtain a calibration disk with the lowest write load among other disks except the blank disk, and obtain capability information corresponding to the calibration disk, wherein the capability information at least includes a disk ID, a current write sequence number, a number of logical volumes, and a capability value of the disk;

[0141] updating the current write sequence number of the blank disk according to the capability information of the calibration disk and the number of logical volumes of the blank disk;

[0142] The capacity value of the blank disk is adjusted according to the current write sequence number of the blank disk, so that the write load of the blank disk corresponds to the write load of other disks except the blank disk.

[0143] The expression for updating the current write sequence number of the blank disk is:

[0144]

[0145] In the formula, Indicates the current write sequence number for blank disk updates, Indicates the current write sequence number of the calibration disk, Indicates the number of logical volumes on a blank disk. Indicates the number of logical volumes of the calibration disk.

[0146] An adjustment module, wherein the adjustment module is used to:

[0147] defining adjustment window bandwidths of a plurality of disks so that a ratio of the adjustment window bandwidths of the disks corresponds to a ratio of the storage capacities of the disks, and determining a total adjustment window bandwidth according to the adjustment window bandwidths of the disks;

[0148] Acquire the bandwidth of each channel of video data in the multiple channels of video data and the total bandwidth of the multiple channels of video data, and determine the number of adjustment windows of each disk according to the total bandwidth of the multiple channels of video data and the total bandwidth of the adjustment windows;

[0149] Obtain the number of channels for accessing video data from each disk, and determine the current access bandwidth of each disk based on the bandwidth of each channel of video data;

[0150] The theoretical bandwidth of each disk is calculated, and the bandwidth of the disk is adjusted according to the theoretical bandwidth of each disk and the current access bandwidth of each disk, so that the data access storage cycle of each disk corresponds to each other.

[0151] The step of calculating the theoretical bandwidth of each disk and adjusting the bandwidth of the disk according to the theoretical bandwidth of each disk and the current access bandwidth of each disk so that the data access storage period of each disk corresponds to each other includes:

[0152] Calculate the theoretical bandwidth of the disk based on the adjustment window bandwidth of each disk and the corresponding adjustment window quantity;

[0153] Calculate the difference between the current access bandwidth of each disk and the theoretical bandwidth corresponding to each disk, and arrange them in descending order;

[0154] A first cycle bandwidth adjustment is performed according to the difference sorting result, where the first cycle bandwidth adjustment includes:

[0155] Select the disk k with the largest difference and the disk j with the smallest difference, and sort the videos connected to the disk k in order of bandwidth from small to large;

[0156] According to the bandwidth sorting results of the plurality of videos connected to the disk k, a second cyclic bandwidth adjustment is performed on the disk k and the disk j. The second cyclic bandwidth adjustment includes:

[0157] Switch the videos with the smallest bandwidth on disk k to disk j for access and storage in sequence. After each switch, re-acquire the access bandwidth of disk k and disk j, and calculate the difference between the re-acquired access bandwidth of disk k and disk j and the corresponding theoretical bandwidth.

[0158] Until the difference between the access bandwidth of disk k and disk j and the corresponding theoretical bandwidth meets the preset requirement, the second cycle bandwidth adjustment of disk k and disk j is completed;

[0159] According to the sorting results of the difference calculation, the corresponding disks are selected from both ends of the difference sorting results in turn, and the above-mentioned second cyclic bandwidth adjustment is repeated until the difference between the access bandwidth of all disks and the corresponding theoretical bandwidth meets the preset requirements, and the first cyclic bandwidth adjustment is completed, so that the data access storage period of each disk corresponds.

[0160] It should be noted that the multi-disk load balancing storage system part in the embodiment of the present disclosure corresponds to the method part of the multi-disk load balancing storage method in the embodiment of the present disclosure, and their specific implementation details are also the same, which will not be repeated here.

[0161] Embodiment 3

[0162] A third embodiment of the present invention provides a storage medium on which a computer program is stored. When the program is executed by a processor, the multi-disk load balancing storage method as described above is implemented.

[0163] Embodiment 4

[0164] The present invention also provides a computer, see Figure 2 , shown is a computer in the fourth embodiment of the present invention, including a memory 10, a processor 20, and a computer program 30 stored in the memory 10 and executable on the processor 20. When the processor 20 executes the computer program 30, the above-mentioned multi-disk load balancing storage method is implemented.

[0165] The memory 10 includes at least one type of storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10 may be an internal storage unit of a computer, such as a hard disk of the computer. In other embodiments, the memory 10 may also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Further, the memory 10 may also include both an internal storage unit of the computer and an external storage device. The memory 10 may be used not only to store application software and various types of data installed in the computer, but also to temporarily store data that has been output or is to be output.

[0166] Among them, in some embodiments, the processor 20 can be an electronic control unit (Electronic Control Unit, abbreviated as ECU, also known as a vehicle computer), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run the program code stored in the memory 10 or process data, such as executing access restriction programs, etc.

[0167] It should be pointed out that Figure 2 The structure shown does not constitute a limitation on the computer. In other embodiments, the computer may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0168] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0169] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0170] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0171] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A multi-disk load balancing storage method, characterized in that: include; respectively obtaining storage capacities of several disks for storing data, and selecting a disk with the largest storage capacity as a reference disk based on the storage capacity of each disk; Formatting each disk in turn according to a preset storage capacity, so as to divide each disk into storage units including a corresponding number of logical volumes, wherein the storage capacity of the logical volumes in different disks is the same; Define variables for each disk respectively, calculate the capacity value corresponding to each disk according to the variable definition of each disk, determine the write load of each disk according to the capacity value of each disk, and sort the disks according to the write load; Obtain the data to be stored, traverse the write loads of all disks, determine the disk with the lowest write load according to the write load sorting results, and use it as the target disk for current data storage; Storing the data to be stored in the target disk to complete the load-balanced storage of the data in multiple disks; After the data to be stored is stored in the target disk to complete the load-balanced storage of the data in the multiple disks, a blank disk without data stored is inserted. The multiple disk load-balanced storage method further includes: Acquiring the storage capacity of the blank disk and formatting it to divide the blank disk into storage units corresponding to a plurality of logical volumes; Obtain a calibration disk with the lowest write load among other disks except the blank disk, and obtain capability information corresponding to the calibration disk, wherein the capability information at least includes a disk ID, a current write sequence number, a number of logical volumes, and a capability value of the disk; updating the current write sequence number of the blank disk according to the capability information of the calibration disk and the number of logical volumes of the blank disk; The capacity value of the blank disk is adjusted according to the current write sequence number of the blank disk, so that the write load of the blank disk corresponds to the write load of other disks except the blank disk.

2. The multi-disk load balancing storage method according to claim 1, characterized in that: The variable definition includes at least: Maximum number of logical volumes: the maximum number of logical volumes is determined according to the storage capacity of the reference disk and the storage capacity of the logical volumes; The number of disk logical volumes is determined in sequence according to the storage capacity of each disk and the storage capacity of the logical volumes; Logical volume number: the logical volume number is determined in ascending order according to the number of logical volumes corresponding to each disk; Write sequence number: The write sequence number gradually increases according to the number of times the logical volume corresponding to each disk is written.

3. The multi-disk load balancing storage method according to claim 2, characterized in that: The steps of calculating the capacity value corresponding to each disk according to the variable definition of each disk, determining the write load of each disk according to the capacity value of each disk, and sorting the disks according to the write load include: Sequentially obtain the current write serial numbers and corresponding numbers of disk logical volumes corresponding to a number of disks, and calculate the capacity value corresponding to each disk according to the current write serial number of the disk, the number of disk logical volumes and the maximum number of logical volumes; The calculation expression of the capability value is: In the formula, Indicates The capacity value of each disk, For the The current write sequence number of the disk, Indicates the maximum number of logical volumes. For the The number of disk logical volumes per disk; The capacity values ​​of the disks are sorted in order of size, and the write loads of the disks are sorted in order of high and low according to the sorting result of the capacity values, wherein a disk with a larger capacity value corresponds to a higher write load.

4. The multi-disk load balancing storage method according to claim 1, characterized in that: The expression for updating the current write sequence number of the blank disk is: In the formula, Indicates the current write sequence number for blank disk updates, Indicates the current write sequence number of the calibration disk, Indicates the number of logical volumes on a blank disk. Indicates the number of logical volumes of the calibration disk.

5. The multi-disk load balancing storage method according to claim 1, characterized in that: The data to be stored is multi-channel video data. After the data to be stored is stored in the target disk to complete the load-balanced storage of the data in multiple disks, the multi-disk load-balanced storage method further includes: defining adjustment window bandwidths of a plurality of disks so that a ratio of the adjustment window bandwidths of the disks corresponds to a ratio of the storage capacities of the disks, and determining a total adjustment window bandwidth according to the adjustment window bandwidths of the disks; Acquire the bandwidth of each channel of video data in the multiple channels of video data and the total bandwidth of the multiple channels of video data, and determine the number of adjustment windows of each disk according to the total bandwidth of the multiple channels of video data and the total bandwidth of the adjustment windows; Obtain the number of channels for accessing video data from each disk, and determine the current access bandwidth of each disk based on the bandwidth of each channel of video data; The theoretical bandwidth of each disk is calculated, and the bandwidth of the disk is adjusted according to the theoretical bandwidth of each disk and the current access bandwidth of each disk, so that the data access storage cycle of each disk corresponds to each other.

6. The multi-disk load balancing storage method according to claim 5, characterized in that: The step of calculating the theoretical bandwidth of each disk and adjusting the bandwidth of the disk according to the theoretical bandwidth of each disk and the current access bandwidth of each disk so that the data access storage period of each disk corresponds to each other includes: Calculate the theoretical bandwidth of the disk based on the adjustment window bandwidth of each disk and the corresponding adjustment window quantity; Calculate the difference between the current access bandwidth of each disk and the theoretical bandwidth corresponding to each disk, and arrange them in descending order; A first cycle bandwidth adjustment is performed according to the difference sorting result, where the first cycle bandwidth adjustment includes: Select the disk k with the largest difference and the disk j with the smallest difference, and sort the videos connected to the disk k in order of bandwidth from small to large; According to the bandwidth sorting results of the plurality of videos connected to the disk k, a second cyclic bandwidth adjustment is performed on the disk k and the disk j. The second cyclic bandwidth adjustment includes: Switch the videos with the smallest bandwidth on disk k to disk j for access and storage in sequence. After each switch, re-acquire the access bandwidth of disk k and disk j, and calculate the difference between the re-acquired access bandwidth of disk k and disk j and the corresponding theoretical bandwidth. Until the difference between the access bandwidth of disk k and disk j and the corresponding theoretical bandwidth meets the preset requirement, the second cycle bandwidth adjustment of disk k and disk j is completed; According to the sorting results of the difference calculation, the corresponding disks are selected from both ends of the difference sorting results in turn, and the above-mentioned second cyclic bandwidth adjustment is repeated until the difference between the access bandwidth of all disks and the corresponding theoretical bandwidth meets the preset requirements, and the first cyclic bandwidth adjustment is completed, so that the data access storage period of each disk corresponds.

7. A multi-disk load balancing storage system, characterized in that: include: A selection module is used to respectively obtain the storage capacity of several disks used to store data, and select a disk with the largest storage capacity as a reference disk based on the storage capacity of each disk; A partitioning module, used to format each disk in turn according to a preset storage capacity, so as to divide each disk into storage units containing a corresponding number of logical volumes, wherein the storage capacity of the logical volumes in different disks is the same; A definition module is used to define variables for each disk, calculate the capacity value corresponding to each disk according to the variable definition of each disk, determine the write load of each disk according to the capacity value of each disk, and sort the disks according to the write load; The traversal module is used to obtain the data to be stored, traverse the write loads of all disks, and determine the disk with the lowest write load according to the sorting results of the write loads and use it as the target disk for current data storage; A storage module, used for storing the data to be stored in the target disk, so as to complete the load-balanced storage of data in multiple disks; An inserting module, used for inserting a blank disk without storing data, obtaining the storage capacity of the blank disk and formatting the blank disk so as to divide the blank disk into storage units corresponding to a plurality of logical volumes; Obtain a calibration disk with the lowest write load among other disks except the blank disk, and obtain capability information corresponding to the calibration disk, wherein the capability information at least includes a disk ID, a current write sequence number, a number of logical volumes, and a capability value of the disk; updating the current write sequence number of the blank disk according to the capability information of the calibration disk and the number of logical volumes of the blank disk; The capacity value of the blank disk is adjusted according to the current write sequence number of the blank disk, so that the write load of the blank disk corresponds to the write load of other disks except the blank disk.

8. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multi-disk load balancing storage method as described in any one of claims 1 to 6 is implemented.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the multi-disk load balancing storage method as described in any one of claims 1 to 6 is implemented.

Images