A cluster network storage system and storage method thereof

Abstract

The present invention relates to the technical field of storage systems, and discloses a cluster network storage system and a storage method thereof, wherein the storage system comprises a storage management module, the storage management module is provided with a plurality of network ports; the storage management module also provides a plurality of storage interfaces; the storage management module comprises a storage drive unit, a storage unit and a management unit; the method comprises: when the management unit detects that a newly added NAS device is connected to the network port of the storage management module, the IP address is allocated to the newly added NAS device; when the storage drive unit identifies the newly added IP address, the configuration operation is performed on the newly added NAS device to expand the newly added NAS device to the storage pool; the storage unit detects the read and write request sent by the user end to the storage management module through the storage interface to perform read and write operations on the storage pool. The present invention is conducive to solving the problem that the links in the existing large-scale network storage process are complex and easy to trigger delay errors.

Description

Technical Field

[0001] The present invention relates to the technical field of storage systems, and in particular to a cluster network storage system and a storage method thereof. Background Art

[0002] In many cases of large-scale network storage on the market, distributed storage solutions are used. However, distributed storage has several defects:

[0003] First, the distributed storage link is complex. There are at least three functional nodes in the storage link: monitoring node, security node and storage node. Each node is a separate device. The number of monitoring nodes and security nodes is at least 2, and the number of storage nodes is at least 3. In addition, during the storage process, different types of nodes need to communicate with each other, and the same type of nodes also need to communicate with each other, which makes the communication of the link in the storage process very complicated, resulting in common network delays. In addition, the network delay of communication between each two nodes will be superimposed, resulting in more serious network delays in distributed storage. When distributed storage processes a file into multiple copies and stores them in different storage units, it is necessary to detect data delays. When the data delay reaches more than ten milliseconds, it is easy to trigger an error, resulting in frequent errors in the distributed storage process.

[0004] Therefore, a more reliable network storage system needs to be provided. Summary of the invention

[0005] The main purpose of the present invention is to provide a cluster network storage system and a storage method thereof, aiming to solve the problem that the links in the existing large-scale network storage process are complex and prone to delay and error reporting.

[0006] To achieve the above object, the present invention provides a storage method for a cluster network storage system, wherein the storage system includes a storage management module, the storage management module is provided with a plurality of network ports; the storage management module also provides a plurality of storage interfaces; the storage management module includes a storage drive unit, a storage unit and a management unit; the method includes the following steps:

[0007] When the management unit detects that a newly added NAS device is connected to the network port of the storage management module, the management unit allocates an IP address to the newly added NAS device;

[0008] When the storage driver unit identifies the newly added IP address, it performs a configuration operation on the newly added NAS device to expand the newly added NAS device to the storage pool;

[0009] The storage unit detects a read / write request sent by the user end to the storage management module through the storage interface, and performs a read / write operation on the storage pool according to the read / write request.

[0010] Preferably, when the storage driver unit identifies the newly added IP address, the step of performing a configuration operation on the newly added NAS device to expand the newly added NAS device to the storage pool includes:

[0011] When the storage drive unit identifies the newly added IP address, it allocates an ID number to the newly added NAS device;

[0012] The storage drive unit obtains storage space information of the newly added NAS device, and determines whether the newly added NAS device supports the RAID function;

[0013] If the newly added NAS device does not support the RAID function, the storage drive unit performs RAID function configuration on the newly added NAS device so that the newly added NAS device supports the RAID function;

[0014] If the newly added NAS device supports RAID function, the storage drive unit performs storage configuration and RAID function partition configuration on the newly added NAS device, so that the newly added NAS device is divided into different storage blocks, each of which supports different RAID functions;

[0015] The storage driver unit expands the newly added NAS device into the storage pool according to the configuration information of the newly added NAS device, and updates the configuration information of the storage pool.

[0016] Preferably, the storage management module includes a local storage module, and the local storage module is in communication with the storage pool; the storage unit detects a read / write request sent by the user end to the storage management module through the storage interface, and performs a read / write operation on the storage pool according to the read / write request, further comprising:

[0017] The storage unit establishes a first storage link and a second storage link;

[0018] The storage unit acquires the data to be stored sent by the user end to the storage management module through the first storage link, and stores the data to be stored in the local storage module;

[0019] The storage unit reads the data to be stored stored in the local storage module to the storage pool through the second storage link, so as to write the data into the storage pool.

[0020] Preferably, the step of the storage unit acquiring the data to be stored sent by the user end to the storage management module through the first storage link, and storing the data to be stored in the local storage module includes:

[0021] The storage unit acquires the storage time of the data stored in the local storage module;

[0022] The storage unit acquires the data to be stored sent by the user end to the storage management module through the first storage link;

[0023] The storage unit obtains the storage space required for the data to be stored, obtains the remaining storage space of the local storage module, and determines whether the remaining storage space reaches the required storage space;

[0024] If so, the storage unit stores the data to be stored in the local storage module;

[0025] If not, the storage unit erases the data with an earlier storage time in the local storage module according to the storage time of the data stored in the local storage module, and then stores the data to be stored in the local storage module.

[0026] Preferably, the step of the storage unit reading the data to be stored stored in the local storage module to the storage pool through the second storage link to write the data into the storage pool comprises:

[0027] The storage unit obtains the data type corresponding to the data to be stored sent by the user terminal;

[0028] When the data type is security mode data, the storage unit stores the data to be stored in a security storage block in the NAS device in a serial manner through the second storage link;

[0029] When the data type is performance mode data, the storage unit segments the data to be stored through the second storage link according to the number of storage nodes corresponding to the performance storage block to obtain a plurality of sub-data, so as to store each of the sub-data in the performance storage block in the NAS device in a striped storage mode, wherein the security storage block is in a data backup mode and the performance storage block is in a striped storage mode.

[0030] Preferably, the method further comprises:

[0031] The management unit detects the transmission rate of the data to be stored sent by the user terminal to the storage management module, and detects the status of each of the NAS devices in the storage pool;

[0032] When the transmission rate reaches a preset value, the management unit expands at least one of the NAS devices into a temporary expansion module of the local storage module according to the status of each of the NAS devices.

[0033] Preferably, the step of the storage unit detecting a read / write request sent by the user end to the storage management module through the storage interface, and performing a read / write operation on the storage pool according to the read / write request, further includes:

[0034] The storage unit acquires the data to be stored sent by the user terminal to the storage management module through the storage interface;

[0035] The storage unit stores the attribute information of the data to be stored in the storage management module;

[0036] The storage unit sends the data information of the data to be stored to the storage pool for storage.

[0037] Preferably, the cluster network storage system further comprises a subordinate storage management module; the subordinate storage management module is communicatively connected to the storage management module, and the subordinate storage management module is communicatively connected to each NAS device; the method further comprises:

[0038] The management unit synchronizes the interaction information between the user terminal and the storage management module, and the interaction information between the storage management module and the storage pool to the slave storage management module respectively;

[0039] When it is detected that the state of the storage management module is abnormal, the slave storage management module is used to replace the storage management module.

[0040] Preferably, the data type of the data to be stored is determined, and the corresponding storage block is determined according to the following method:

[0041] Acquire data characteristics of a data source to which the data to be stored belongs, wherein the data characteristics include a read frequency, a write frequency, a data importance level, and a data volume;

[0042] Calculating characteristic values ​​of the data to be stored according to the data characteristics of the data to be stored;

[0043] Determine the data type of the data to be stored according to the characteristic value interval corresponding to the characteristic value of the data to be stored, and determine the RAID mode corresponding to the data to be stored according to the data type of the data to be stored;

[0044] Determine the corresponding storage block according to the RAID mode corresponding to the data to be stored;

[0045] ;

[0046] Among them, C is the characteristic value of the data to be stored, , n is the number of data features; is the weight value of the i-th data feature of the data to be stored, , is the score of the i-th data feature of the data to be stored;

[0047] Determined based on the actual storage situation of the data of the same data source stored in the storage pool, including:

[0048] Determine a corresponding read frequency score according to the actual read frequency of the storage data of the same data source in the storage pool;

[0049] Determine a corresponding write frequency score according to the actual write frequency of the storage data of the same data source in the storage pool;

[0050] The importance level of data is determined according to the importance level of the data source to which the data to be stored belongs;

[0051] The data size of the data source to which the data to be stored belongs is the total data size of the storage data of the same data source in the storage pool.

[0052] To achieve the above object, the present invention also proposes a cluster network storage system for executing the method; the storage system includes a storage management module, the storage management module is provided with a plurality of network ports; the storage management module also provides a plurality of storage interfaces; the storage management module includes a storage drive unit, a storage unit, and a management unit;

[0053] The management unit is used to allocate an IP address to the newly added NAS device when detecting that a newly added NAS device is connected to the network port of the storage management module;

[0054] The storage driver unit is used to perform a configuration operation on the newly added NAS device when the newly added IP address is identified, so as to expand the newly added NAS device to the storage pool;

[0055] The storage unit is used to detect the read and write request sent by the user end to the storage management module through the storage interface, and perform read and write operations on the storage pool according to the read and write request.

[0056] In the technical solution of the present invention, the storage system is provided with a storage management module, and each network port is provided through the storage management module so that each NAS device can access the network port, and each NAS device is configured as a storage pool. When a new NAS device is added, it can be flexibly expanded to the storage pool at any time; and the user end can access the storage system through the storage interface provided by the storage management module, and implement read and write operations on the storage pool. Therefore, the storage system of the present invention has a simple structure, and the storage process is the interaction between the storage management module and the storage pool. The link communication in the storage process is simple, which overcomes the serious network delay in the distributed storage in the prior art, and there will be no error caused by network delay. Therefore, the technical solution of the present invention is conducive to overcoming the technical problem in the prior art that the link is complex and easy to trigger delay errors in the large network storage process. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a module connection block diagram of the present invention;

[0058] Figure 2 FIG. 1 is a flow chart of the first embodiment of the present invention.

[0059] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0061] In the following description, suffixes such as "unit", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "unit", "component" or "unit" can be used in a mixed manner.

[0062] See also Figure 1 to Figure 2 In a first embodiment of the present invention, a storage method of a cluster network storage system is provided, wherein the storage system comprises a storage management module, the storage management module is provided with a plurality of network ports; the storage management module also provides a plurality of storage interfaces; the storage management module comprises a storage drive unit, a storage unit, and a management unit; the method comprises the following steps:

[0063] Step S10, when the management unit detects that a newly added NAS device is connected to the network port of the storage management module, the management unit allocates an IP address to the newly added NAS device;

[0064] Step S20, when the storage driver unit identifies the newly added IP address, it performs a configuration operation on the newly added NAS device to expand the newly added NAS device to the storage pool;

[0065] Step S30: the storage unit detects a read / write request sent by the client to the storage management module through the storage interface, and performs a read / write operation on the storage pool according to the read / write request.

[0066] In the technical solution of the present invention, the storage system is provided with a storage management module, and each network port is provided through the storage management module so that each NAS device can access the network port, and each NAS device is configured as a storage pool. When a new NAS device is added, it can be flexibly expanded to the storage pool at any time; and the user end can access the storage system through the storage interface provided by the storage management module, and implement read and write operations on the storage pool. Therefore, the storage system of the present invention has a simple structure, and the storage process is the interaction between the storage management module and the storage pool. The link communication in the storage process is simple, which overcomes the serious network delay in the distributed storage in the prior art, and there will be no error caused by network delay. Therefore, the technical solution of the present invention is conducive to overcoming the technical problem in the prior art that the link is complex and easy to trigger delay errors in the large network storage process.

[0067] NAS (Network Attached Storage) is a special file storage device. In the present invention, each NAS device serves as a storage node.

[0068] The storage system of the present invention aggregates the capacity of each NAS device into a storage pool and provides it to users for access; when a new NAS device is connected to the network port of the storage management module, it supports dynamic update of the storage pool capacity; the file management of the storage system is managed by MySQL (My Structured Query Language, a widely used open source relational database management system) database.

[0069] The storage management module includes a storage driver unit, a storage unit and a management unit; wherein the storage driver unit is mainly used to merge various NAS devices into a storage pool and is responsible for accessing the storage pool; the storage unit is responsible for the specific read and write operation process; the management unit is used to manage the added NAS devices, manage the subordinate storage management modules, and manage the issuance of instructions.

[0070] The storage system in the present invention is implemented based on a Web server. Its main functions include providing an interactive experience for users through a self-developed Web front-end interface, real-time monitoring of the status of the storage system, automatic configuration of downstream NAS devices, and automatic mounting of user-side Samba network file drivers.

[0071] Specifically, when the storage system is mounted with multiple NAS devices, the storage management module displays the remaining capacity of the storage pool according to the sum of the remaining capacity of each NAS device, and the storage management module displays the total capacity of the storage pool according to the sum of the total capacity of each NAS device.

[0072] The present invention supports adding new NAS devices online, and the storage capacity of the storage pool can be expanded without downtime.

[0073] Based on the first embodiment of the storage method of the cluster network storage system of the present invention, in the second embodiment of the storage method of the cluster network storage system of the present invention, the step S20 includes:

[0074] Step S21, when the storage drive unit identifies the newly added IP address, it assigns an ID number to the newly added NAS device;

[0075] Step S22, the storage drive unit obtains storage space information of the newly added NAS device, and determines whether the newly added NAS device supports RAID function;

[0076] If the newly added NAS device does not support the RAID function, executing step S23: the storage drive unit performs RAID function configuration on the newly added NAS device so that the newly added NAS device supports the RAID function;

[0077] If the newly added NAS device supports the RAID function, executing step S24: the storage drive unit performs storage configuration and RAID function partition configuration on the newly added NAS device, so that the newly added NAS device is divided into different storage blocks, each of which supports different RAID functions;

[0078] Step S25: the storage driver unit expands the newly added NAS device into the storage pool according to the configuration information of the newly added NAS device, and updates the configuration information of the storage pool.

[0079] Specifically, each NAS device connected to the storage management module is assigned an IP address, and an ID number is assigned to each NAS device connected to the storage management module according to the IP address to distinguish each NAS device.

[0080] The storage drive unit obtains storage space information of the newly added NAS device, where the storage space information includes the total capacity and remaining capacity of the NAS device.

[0081] Specifically, the storage driver unit configuring the RAID function of the newly added NAS device means that the storage driver unit sends a patch package of the RAID function to the newly added NAS device.

[0082] The storage driver unit is used to integrate various NAS devices. The storage driver unit generates block device nodes for NAS devices and integrates multiple NAS devices into a storage pool. The block device nodes and storage pool are shared with the user end through network file systems such as Samba (a free software that implements the SMB protocol on Linux and UNIX systems). When the user end adds a storage node through the management unit, the storage driver unit updates the storage pool information in the storage unit to implement the dynamic adjustment function of the storage pool. The storage pool information includes: the various NAS device nodes included in the storage pool, the storage pool capacity, the health status of each NAS device, etc.

[0083] After the management unit detects a newly added NAS device, it automatically configures the storage for the newly added NAS device according to the storage mechanism selected by the user. The storage configuration includes: the mode of the NAS device disk array, the type of file system, the configuration of the network file system, and the properties and index number of the NAS device node.

[0084] Specifically, in the present invention, the newly added NAS device is configured with RAID function partitions so that different storage blocks of the newly added NAS device support different RAID functions, for example, the first partition supports RAID 0 function, and the second partition supports RAID 5 function.

[0085] Based on the second embodiment of the storage method of the cluster network storage system of the present invention, in the third embodiment of the storage method of the cluster network storage system of the present invention, the storage management module includes a local storage module, and the local storage module is communicatively connected with the storage pool; the step S30 includes:

[0086] Step S31, the storage unit establishes a first storage link and a second storage link;

[0087] Step S32, the storage unit obtains the data to be stored sent by the user end to the storage management module through the first storage link, and stores the data to be stored in the local storage module;

[0088] Step S33: the storage unit reads the data to be stored stored in the local storage module to the storage pool through the second storage link, so as to write the data into the storage pool.

[0089] In the prior art, there is only one link in the storage process, which needs to serve the process of the user end writing to the system and the process of the system reading from the hard disk. As a result, data cannot be read when writing, and cannot be written when reading, resulting in that writing and reading cannot be performed at the same time, resulting in low read and write efficiency.

[0090] In the present invention, the storage management module can be externally connected to a local storage module for temporarily storing the data to be stored. The storage space of the local storage module can be several hundred megabytes. The storage operation mode of the local storage module is as follows:

[0091] The first storage link, the user end writes data to the local storage module of the storage management module;

[0092] The second storage link reads data in the local storage module to the NAS device in the storage pool and writes it to the NAS device.

[0093] The two links are in parallel, one link is responsible for writing data to the local storage module, and the other link is responsible for writing data from the local storage module to the storage pool. The two links do not interfere with each other and run independently. Writing data to the local storage module and writing data to the storage pool can be done in parallel, with high read and write efficiency.

[0094] Based on the third embodiment of the storage method of the cluster network storage system of the present invention, in the fourth embodiment of the storage method of the cluster network storage system of the present invention, the step S32 includes:

[0095] Step S321, the storage unit obtains the storage time of the data stored in the local storage module;

[0096] Step S322, the storage unit obtains the data to be stored sent by the user end to the storage management module through the first storage link;

[0097] Step S323, the storage unit obtains the storage space required for the data to be stored, obtains the remaining storage space of the local storage module, and determines whether the remaining storage space reaches the required storage space;

[0098] If yes, executing step S324: the storage unit stores the data to be stored in the local storage module;

[0099] If not, execute step S325: the storage unit erases the data with earlier storage time in the local storage module according to the storage time of the data stored in the local storage module, and then stores the data to be stored in the local storage module.

[0100] Specifically, if the storage space of the local storage module is small, if the storage space of the local storage module is insufficient, the earliest written data will be erased to expand the available storage space of the local storage module to meet the current data storage requirements, so that the local storage module can continue to meet the storage requirements through a smaller storage space.

[0101] Based on the fourth embodiment of the storage method of the cluster network storage system of the present invention, in the fifth embodiment of the storage method of the cluster network storage system of the present invention, the step S33 includes:

[0102] Step S331, the storage unit obtains the data type corresponding to the data to be stored sent by the user terminal;

[0103] When the data type is security mode data, step S332 is executed: the storage unit stores the data to be stored in a security storage block in the NAS device in a serial manner through the second storage link;

[0104] When the data type is performance mode data, execute step S333: the storage unit segments the data to be stored through the second storage link according to the number of storage nodes corresponding to the performance storage block, and obtains a plurality of sub-data, so as to store each of the sub-data in a striped storage mode in the performance storage block in the NAS device, wherein the security storage block is in data backup mode and the performance storage block is in striped storage mode.

[0105] Specifically, whether the data type of the data to be stored is security mode data or performance mode data can be determined by a user's selection, or can be automatically determined by the system based on the source of the data to be stored.

[0106] Specifically, the storage mode corresponding to the security mode data is the security mode, the corresponding storage block is the security storage block, and the corresponding RAID mode is the RAID 0 storage mode; the storage mode corresponding to the performance mode data is the performance mode, the corresponding storage block is the performance storage block, and the corresponding RAID mode is the RAID 5 storage mode.

[0107] In the security mode, the disk array of the storage node of each NAS device is in data backup mode. When the storage drive unit receives the file sent by the user, the storage unit sends it to the storage node of the NAS device in a serial manner to ensure the security of the data.

[0108] In performance mode, the storage node disk arrays of each NAS device are in striped storage mode. The storage unit will store the files in segments in each storage node according to the number of storage nodes in the current storage pool. Both reading and writing can be processed in parallel to improve the storage and access efficiency of files.

[0109] Specific:

[0110] Determine the data type of the data to be stored and the corresponding storage block according to the following method:

[0111] The data characteristics of the data source to which the data to be stored belongs are obtained, wherein the data characteristics include a reading frequency, a writing frequency, a data importance level, and a data volume; of course, the types of data characteristics may not be limited thereto.

[0112] Calculating characteristic values ​​of the data to be stored according to the data characteristics of the data to be stored;

[0113] Determine the data type of the data to be stored according to the characteristic value interval corresponding to the characteristic value of the data to be stored, and determine the RAID mode corresponding to the data to be stored according to the data type of the data to be stored;

[0114] Determine the corresponding storage block according to the RAID mode corresponding to the data to be stored;

[0115] ;

[0116] Among them, C is the characteristic value of the data to be stored, , n is the number of data features; is the weight value of the i-th data feature of the data to be stored, , is the score of the i-th data feature of the data to be stored;

[0117] Determined based on the actual storage situation of the data of the same data source stored in the storage pool, including:

[0118] Determine a corresponding read frequency score according to the actual read frequency of the storage data of the same data source in the storage pool, wherein different read frequency times correspond to different read frequency intervals, and each read frequency interval corresponds to a read frequency score;

[0119] Determine a corresponding write frequency score according to the actual write frequency of the storage data of the same data source in the storage pool, wherein different write frequency times correspond to different write frequency intervals, and each write frequency interval corresponds to a write frequency score;

[0120] The data importance level is determined according to the importance level corresponding to the data source to which the data to be stored belongs; specifically, the data source includes the source client and the specific application on the source client, and the importance level is determined according to the importance of the source client and the importance of the specific application of the source.

[0121] The data volume of the data source to which the data to be stored belongs is the total amount of data stored in the storage pool of the same data source in the storage pool. In the storage system, the data volume of the same data source can be counted, and the data volume of the data source to which the data to be stored belongs can be determined based on the statistical results, and the corresponding feature score can be determined based on the data volume.

[0122] Determine the data type of the data to be stored based on the eigenvalue interval corresponding to the eigenvalue of the data to be stored. The specific steps are as follows:

[0123] Define the data type corresponding to each characteristic value interval (for example, security mode data, performance mode data);

[0124] According to the characteristic value corresponding to the data to be stored, the characteristic value interval described by the characteristic value can be determined, so that the corresponding data type can be determined.

[0125] After the data type of the data to be stored is determined, the RAID mode of the data to be stored can be determined.

[0126] Furthermore, in the present invention, each data type corresponds to a storage block, and the storage space size of each storage block can be matched according to the actual data storage situation in the storage pool, specifically:

[0127] When each newly added NAS device is expanded to the storage pool, the storage block division of the newly added NAS device is completed according to the configuration ratio of the preset functional partitions, so that each storage block supports different RAID functions;

[0128] Count the remaining storage space of each storage block, and count the used storage space of each storage block;

[0129] The ratio of the used storage space of each used storage block is calculated to obtain a first ratio, and the ratio of the remaining storage space of each storage block is calculated to obtain a second ratio.

[0130] According to the first ratio and the second ratio, a storage block with insufficient remaining storage space is determined as a block to be supplemented; part of the storage space in the storage block with the most remaining storage space is adjusted to the storage block type corresponding to the block to be supplemented, or when the newly added NAS device is dividing the storage blocks, more storage space is allocated to the block to be supplemented.

[0131] Based on the first to fifth embodiments of the storage method of the cluster network storage system of the present invention, in a sixth embodiment of the storage method of the cluster network storage system of the present invention, the method further includes:

[0132] Step S40, the management unit detects the transmission rate of the data to be stored sent by the user terminal to the storage management module, and detects the status of each of the NAS devices in the storage pool;

[0133] Step S50: When the transmission rate reaches a preset value, the management unit expands at least one of the NAS devices into a temporary expansion module of the local storage module according to the status of each of the NAS devices.

[0134] Specifically, the state of the NAS device includes an idle state or a busy state.

[0135] When the transmission rate is high, it means that the amount of data that the local storage module needs to store in a short time is large, which may exceed the short-term storage capacity of the local storage module. At this time, by detecting the transmission rate of the data to be stored sent by the user terminal to the storage management module, the idle NAS device is expanded in advance as a temporary expansion module of the local storage module to increase the temporary storage capacity, thereby avoiding data loss due to insufficient storage space of the local storage module.

[0136] Based on the first to fifth embodiments of the storage method of the cluster network storage system of the present invention, in the seventh embodiment of the storage method of the cluster network storage system of the present invention, the step S30 further includes:

[0137] Step S34, the storage unit acquires the data to be stored sent by the user terminal to the storage management module through the storage interface;

[0138] Step S35, the storage unit stores the attribute information of the data to be stored in the storage management module;

[0139] Step S36: the storage unit sends the data information of the data to be stored to the storage pool for storage.

[0140] The storage management unit uses MySQL to manage the file name, file type, file size, file permission and owner, creation time, modification time, access time, file storage location (storage location on the storage drive unit), hard link and symbolic link data block addresses (multiple in performance mode) of the data to be stored, and adds, deletes, modifies and queries the file information through the database;

[0141] The storage management unit mainly includes the transmission management of the data to be stored, the segmentation, synthesis, error detection and correction, and buffer management of the files. If the user sends a file to be stored in the performance mode, the storage unit will package and verify the file after receiving it, generate a verification code, and send the file in segments according to the currently allocated NAS device as the storage node.

[0142] If the size of the file received by the storage unit does not meet the segment size, it waits in the storage unit to determine whether the waiting time has timed out; if it times out and the user stops sending, it is written to the NAS device.

[0143] Specifically, the attribute information of the data to be stored is stored in the storage management module, and the data information of the data to be stored is sent to the storage pool for storage. When reading data from the storage pool, only the read data fragment can be restored to the read data and sent to the user end together with the attribute information stored in the storage management module, thereby shortening the data reading time.

[0144] Based on the first to fifth embodiments of the storage method of the cluster network storage system of the present invention, in an eighth embodiment of the storage method of the cluster network storage system of the present invention, the cluster network storage system further comprises a subordinate storage management module; the subordinate storage management module is communicatively connected to the storage management module, and the subordinate storage management module is communicatively connected to each NAS device; the method further comprises:

[0145] Step S60, the management unit synchronizes the interaction information between the user terminal and the storage management module, and the interaction information between the storage management module and the storage pool to the slave storage management module respectively;

[0146] Step S70: When it is detected that the storage management module is in an abnormal state, the slave storage management module is used to replace the storage management module.

[0147] The system management in the present invention supports dual-path dual active and is implemented using a master-slave mechanism. The slave storage management module serves as a backup machine to receive all data from the user end but does not respond. At the same time, it backs up the database of the storage management module serving as the host. The backup machine and the host are monitored through real-time heartbeat data. When a host failure occurs, the backup machine takes over the user end instructions to achieve high availability of the system.

[0148] The scalability of NAS devices is usually limited by physical hardware conditions, such as the number of hard disk slots and the upper capacity limit; at the same time, NAS devices are usually based on a single device. Once the device fails, it may cause data inaccessibility or data loss risks.

[0149] The storage system in the present invention is a storage system based on the Feiteng platform. The storage management module acts as a controller to manage downstream storage devices (various NAS devices). The storage management module developed by the company provides a storage interface to the upstream. This storage system implements the following functions:

[0150] First, realize the storage media integration between NAS devices to form at least one storage pool as needed;

[0151] Second, through the performance storage mode, the data to be stored is stored in segments, and the segmented data to be stored is stored in each allocated storage block through parity check, and a check code for the data to be stored is generated. When a storage block is damaged, the data segment stored in the damaged storage block can be restored through the check code, thereby improving data reconstruction efficiency;

[0152] Third, there is no minimum limit on NAS devices, which reduces the cost of use;

[0153] Fourth, after networking, each NAS device can automatically join the storage system, reducing management difficulty;

[0154] Fifth, the storage management unit supports dual-path dual-active to achieve high availability.

[0155] The present invention realizes a network storage system based on the Feiteng platform, which effectively solves some shortcomings existing in NAS devices and distributed storage systems, reduces the use cost, and provides a convenient management system that users can choose according to different use scenarios.

[0156] To achieve the above object, the present invention also proposes a cluster network storage system for executing the method; the storage system includes a storage management module, the storage management module is provided with a plurality of network ports; the storage management module also provides a plurality of storage interfaces; the storage management module includes a storage drive unit, a storage unit, and a management unit;

[0157] The management unit is used to allocate an IP address to the newly added NAS device when detecting that a newly added NAS device is connected to the network port of the storage management module;

[0158] The storage driver unit is used to perform a configuration operation on the newly added NAS device when the newly added IP address is identified, so as to expand the newly added NAS device to the storage pool;

[0159] The storage unit is used to detect the read and write request sent by the user end to the storage management module through the storage interface, and perform read and write operations on the storage pool according to the read and write request.

[0160] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device to enter the method described in each embodiment of the present invention.

[0161] In the description of this specification, the description with reference to the terms "an embodiment", "another embodiment", "other embodiments", or "first embodiment to Xth embodiment" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, method steps or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0162] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0163] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0164] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A storage method of a cluster network storage system, characterized in that: The storage system includes a storage management module, the storage management module is provided with a plurality of network ports; the storage management module also provides a plurality of storage interfaces; the storage management module includes a storage drive unit, a storage unit and a management unit; the storage management module includes a local storage module, and the local storage module is communicatively connected with a storage pool; the method includes the following steps: When the management unit detects that a newly added NAS device is connected to the network port of the storage management module, the management unit allocates an IP address to the newly added NAS device; When the storage driver unit identifies the newly added IP address, it performs a configuration operation on the newly added NAS device to expand the newly added NAS device to the storage pool; The storage unit detects a read / write request sent by the user end to the storage management module through the storage interface, and performs a read / write operation on the storage pool according to the read / write request, including: the storage unit establishes a first storage link and a second storage link; the storage unit reads the data to be stored stored in the local storage module to the storage pool through the second storage link, so as to write the data into the storage pool; The step of the storage unit reading the data to be stored stored in the local storage module to the storage pool through the second storage link to write the data into the storage pool includes: The storage unit obtains the data type corresponding to the data to be stored sent by the user terminal; When the data type is security mode data, the storage unit stores the data to be stored in a security storage block in the NAS device in a serial manner through the second storage link; When the data type is performance mode data, the storage unit segments the data to be stored through the second storage link according to the number of storage nodes corresponding to the performance storage block to obtain a plurality of sub-data, so as to store each of the sub-data in the performance storage block in the NAS device in a striped storage mode, wherein the security storage block is in a data backup mode and the performance storage block is in a striped storage mode; Determine the data type of the data to be stored and the corresponding storage block according to the following method: Acquire data characteristics of a data source to which the data to be stored belongs, wherein the data characteristics include a read frequency, a write frequency, a data importance level, and a data volume; Calculating characteristic values ​​of the data to be stored according to the data characteristics of the data to be stored; Determine the data type of the data to be stored according to the characteristic value interval corresponding to the characteristic value of the data to be stored, and determine the RAID mode corresponding to the data to be stored according to the data type of the data to be stored; Determine the corresponding storage block according to the RAID mode corresponding to the data to be stored; ； Among them, C is the characteristic value of the data to be stored, , n is the number of data features; is the weight value of the i-th data feature of the data to be stored, , is the score of the i-th data feature of the data to be stored; Determined based on the actual storage situation of the data of the same data source stored in the storage pool, including: Determine a corresponding read frequency score according to the actual read frequency of the storage data of the same data source in the storage pool; Determine a corresponding write frequency score according to the actual write frequency of the storage data of the same data source in the storage pool; The importance level of data is determined according to the importance level of the data source to which the data to be stored belongs; The data size of the data source to which the data to be stored belongs is the total data size of the storage data of the same data source in the storage pool.

2. The storage method of the cluster network storage system according to claim 1, characterized in that: The step of the storage driver unit performing a configuration operation on the newly added NAS device to expand the newly added NAS device to a storage pool when identifying the newly added IP address includes: When the storage drive unit identifies the newly added IP address, it allocates an ID number to the newly added NAS device; The storage drive unit obtains storage space information of the newly added NAS device, and determines whether the newly added NAS device supports the RAID function; If the newly added NAS device does not support the RAID function, the storage drive unit performs RAID function configuration on the newly added NAS device so that the newly added NAS device supports the RAID function; If the newly added NAS device supports RAID function, the storage drive unit performs storage configuration and RAID function partition configuration on the newly added NAS device, so that the newly added NAS device is divided into different storage blocks, each of which supports different RAID functions; The storage driver unit expands the newly added NAS device into the storage pool according to the configuration information of the newly added NAS device, and updates the configuration information of the storage pool.

3. The storage method of the cluster network storage system according to claim 2, characterized in that: The step of the storage unit detecting a read / write request sent by the user end to the storage management module through the storage interface, and performing a read / write operation on the storage pool according to the read / write request, further includes: The storage unit obtains the data to be stored sent by the user end to the storage management module through the first storage link, and stores the data to be stored in the local storage module.

4. The storage method of the cluster network storage system according to claim 3, characterized in that: The step of the storage unit acquiring the data to be stored sent by the user end to the storage management module through the first storage link, and storing the data to be stored in the local storage module includes: The storage unit acquires the storage time of the data stored in the local storage module; The storage unit acquires the data to be stored sent by the user end to the storage management module through the first storage link; The storage unit obtains the storage space required for the data to be stored, obtains the remaining storage space of the local storage module, and determines whether the remaining storage space reaches the required storage space; If so, the storage unit stores the data to be stored in the local storage module; If not, the storage unit erases the data with an earlier storage time in the local storage module according to the storage time of the data stored in the local storage module, and then stores the data to be stored in the local storage module.

5. The storage method of the cluster network storage system according to any one of claims 3 to 4, characterized in that: The method further comprises: The management unit detects the transmission rate of the data to be stored sent by the user terminal to the storage management module, and detects the status of each of the NAS devices in the storage pool; When the transmission rate reaches a preset value, the management unit expands at least one of the NAS devices into a temporary expansion module of the local storage module according to the status of each of the NAS devices.

6. The storage method of the cluster network storage system according to any one of claims 1 to 4, characterized in that: The step of the storage unit detecting a read / write request sent by the user end to the storage management module through the storage interface, and performing a read / write operation on the storage pool according to the read / write request, further includes: The storage unit acquires the data to be stored sent by the user end to the storage management module through the storage interface; The storage unit stores the attribute information of the data to be stored in the storage management module; The storage unit sends the data information of the data to be stored to the storage pool for storage.

7. The storage method of the cluster network storage system according to any one of claims 1 to 4, characterized in that: The cluster network storage system also includes a slave storage management module; The slave storage management module is communicatively connected to the storage management module, and the slave storage management module is communicatively connected to each NAS device; The method further comprises: The management unit synchronizes the interaction information between the user terminal and the storage management module, and the interaction information between the storage management module and the storage pool to the slave storage management module respectively; When it is detected that the state of the storage management module is abnormal, the slave storage management module is used to replace the storage management module.

8. A cluster network storage system, characterized in that: Used to execute the method according to any one of claims 1 to 7; the storage system includes a storage management module, the storage management module is provided with a plurality of network ports; the storage management module also provides a plurality of storage interfaces; the storage management module includes a storage drive unit, a storage unit, and a management unit; The management unit is used to allocate an IP address to the newly added NAS device when detecting that a newly added NAS device is connected to the network port of the storage management module; The storage driver unit is used to perform a configuration operation on the newly added NAS device when the newly added IP address is identified, so as to expand the newly added NAS device to the storage pool; The storage unit is used to detect the read and write request sent by the user end to the storage management module through the storage interface, and perform read and write operations on the storage pool according to the read and write request.

Images