Network data storage system and method

Abstract

The invention discloses a network data storage system and method, and belongs to the field of data storage, the network data storage system comprises a distributed storage node module, a dynamic load balancing module, an intelligent redundancy management module, a data processing module, a backup recovery module and an extension optimization module; the distributed storage node module consists of a plurality of nodes with independent storage capability; according to the invention, a distributed architecture is adopted, technologies of data compression, encryption, backup and the like are combined, and data are distributed and stored, dynamic load balancing, intelligent redundancy and fault tolerance, hybrid compression and encryption, efficient backup and intelligent recovery, flexible expansion, performance optimization and the like are combined; the problems of storage bottleneck, system reliability, data security and the like of a traditional centralized data storage scheme during large-scale data processing are solved. The storage capability, the data access speed, the data security and the fault-tolerant capability of the system are improved, and the method is suitable for scenes such as a cloud storage platform and an enterprise data center.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a network data storage system and method. Background Technology

[0002] With the rapid development of the Internet and the Internet of Things, the amount of data has exploded. Traditional centralized data storage solutions have many drawbacks when dealing with large-scale data.

[0003] Regarding storage bottlenecks, centralized storage has limited scalability. When data volume increases dramatically, it struggles to meet storage demands, leading to a significant rise in storage costs. For example, during a promotional period, a large e-commerce platform experienced a surge in user and transaction data. The centralized storage system, unable to quickly expand its capacity, experienced data storage delays, impacting the user experience.

[0004] In terms of system reliability, if a single storage center fails—such as due to hardware damage, software vulnerabilities, or attacks—a large amount of data will be at risk of being lost, severely impacting business continuity. For example, a company's data center experienced a server hardware failure that rendered core business data inaccessible, causing business interruption for several hours and resulting in significant economic losses.

[0005] In terms of data security, centralized storage is vulnerable to hacker attacks, and data breaches can cause incalculable losses to businesses and users. Reports indicate that a cloud storage platform, due to inadequate security measures, suffered a massive leak of user personal information, triggering a serious crisis of trust.

[0006] Therefore, an efficient, reliable, and flexible network data storage system and method are needed to solve the problems mentioned above. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a network data storage system and method to solve problems such as storage bottlenecks, low system reliability, and poor data security in traditional centralized data storage solutions, thereby improving the system's storage capacity, data access speed, data security, and fault tolerance.

[0008] Technical solution: To solve the above-mentioned technical problems, according to one aspect of the present invention, more specifically, a network data storage system includes a distributed storage node module, a dynamic load balancing module, an intelligent redundancy management module, a data processing module, a backup and recovery module, and an expansion and optimization module.

[0009] Distributed storage node module: It consists of multiple nodes with independent storage capabilities. The nodes are interconnected through a network to form a storage cluster for distributed data storage.

[0010] Dynamic load balancing module: Connected to the distributed storage node module, it monitors the load of each node and automatically adjusts data storage and access requests based on the node load to achieve dynamic load balancing and avoid single points of failure and performance bottlenecks.

[0011] Intelligent Redundancy Management Module: Connected to the distributed storage node module and the dynamic load balancing module, it intelligently selects the redundancy level based on data redundancy algorithms and adaptive redundancy ratio technology, according to data importance and access frequency. It maintains higher redundancy copies for frequently accessed data, thereby improving fault tolerance.

[0012] Data processing module: includes a compression unit and an encryption unit. The compression unit uses a combination of various compression algorithms and automatically selects the appropriate compression method based on the data type; the encryption unit uses high-strength encryption technology to encrypt data during transmission and storage.

[0013] Backup and recovery module: It adopts a combination of full and incremental backup, performs data backup through intelligent backup strategies, and performs rapid recovery by selecting the optimal recovery path and strategy when data is lost through an intelligent recovery engine.

[0014] Expansion and optimization module: Supports on-demand expansion of storage capacity, and adopts intelligent cache management and adaptive storage scheduling technology to optimize storage performance.

[0015] According to another aspect of the present invention, and more specifically a network data storage method, the method comprises the following steps:

[0016] S1. Distributed Data Storage and Dynamic Load Balancing. A distributed storage architecture is adopted, distributing data across multiple nodes in the distributed storage node module; the dynamic load balancing module monitors node load in real time and automatically adjusts data storage and access requests.

[0017] S2. Intelligent Redundancy Management. The intelligent redundancy management module intelligently selects the redundancy level based on data importance and access frequency, using advanced data redundancy algorithms and adaptive redundancy ratio technology, maintaining higher redundancy copies for frequently accessed data.

[0018] S3. Data Compression and Encryption. The compression unit of the data processing module automatically selects the appropriate compression method based on the data type; the encryption unit of the data processing module encrypts the compressed data during transmission and storage.

[0019] S4. Data Backup and Recovery. The backup and recovery module uses a combination of full and incremental backups, employing intelligent backup strategies. When data is lost, the module's intelligent recovery engine selects the optimal recovery path and strategy for rapid recovery based on the health status of the backup data.

[0020] S5. System Expansion and Performance Optimization. The expansion and optimization module dynamically adds storage nodes according to demand and automatically balances data load; the expansion and optimization module adopts intelligent cache management and adaptive storage scheduling technology to optimize storage performance.

[0021] The beneficial effects of the network data storage system and method of the present invention are as follows:

[0022] (1) This invention avoids single point of failure and performance bottleneck through distributed storage architecture and dynamic load balancing technology, thereby improving the reliability and stability of the system.

[0023] By employing intelligent redundancy and fault tolerance technologies, the redundancy level is intelligently adjusted based on data importance and access frequency, thereby reducing storage costs while ensuring data security.

[0024] The application of hybrid compression and encryption technologies minimizes storage footprint while ensuring data security during transmission and storage, preventing data leaks and unauthorized access.

[0025] The efficient backup and intelligent recovery mechanism ensures rapid recovery in the event of data loss, minimizing the losses caused by data loss.

[0026] The system's ability to flexibly expand and optimize storage performance enables it to meet storage needs of varying scales and ensure rapid response for data access. Attached Figure Description

[0027] The present invention will now be described in further detail with reference to the accompanying drawings and specific implementation methods.

[0028] Figure 1 This is a schematic diagram of the network data storage system of the present invention;

[0029] Figure 2 This is a flowchart illustrating the network data storage method of the present invention. Detailed Implementation

[0030] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0031] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] Example 1

[0033] Reference Figures 1-2 Application of a network data storage system and method in a cloud storage platform:

[0034] When a network data storage system is applied to a cloud storage platform, the distributed storage node module consists of multiple cloud server nodes, which are interconnected through a network to form a high-efficiency storage cluster.

[0035] When a user uploads data, the system first performs distributed data storage and dynamic load balancing. The data is distributed across different cloud server nodes, and the dynamic load balancing module monitors the load of each node in real time. If a node is overloaded, it will automatically distribute new data storage requests to nodes with lower loads, thus avoiding data storage efficiency being affected by the overload of a single node.

[0036] The intelligent redundancy management module manages redundancy based on the importance and access frequency of user data. For important data that users frequently access, such as personal photo albums and important documents, the system adds redundant copies and stores them on different cloud server nodes, ensuring normal data access even if some nodes fail. For less frequently accessed, cold data, such as old email archives, fewer redundant copies are maintained to reduce storage costs.

[0037] The data processing module processes the uploaded data. The compression unit compresses data according to its data type. For text data, it uses a lossless compression algorithm, while for images, videos, and other data, it selects an appropriate compression method based on their format and quality requirements to reduce storage usage. The encryption unit uses the AES encryption algorithm to encrypt the data, ensuring its security during transmission to and storage on the cloud server and preventing unauthorized access.

[0038] The backup and recovery module backs up data according to an intelligent backup strategy. For user data, a full backup is performed every morning at midnight, and incremental backups are performed during the day based on data changes. When a user accidentally deletes data or data is lost due to node failure, the intelligent recovery engine quickly finds the latest and healthiest backup data, selects the optimal recovery path, and quickly restores the data to the user.

[0039] As the number of cloud storage users and the volume of data increase, the expansion and optimization module can dynamically add cloud server nodes according to demand. After a new node is added, the system will automatically redistribute existing data among the nodes to achieve load balancing. At the same time, intelligent caching management technology caches frequently accessed data on high-speed storage media, improving data access speed and enhancing user experience.

[0040] Example 2

[0041] Reference Figures 1-2 Application of a network data storage system and method in enterprise data centers:

[0042] In enterprise data centers, the network data storage system of this invention also delivers excellent performance. The distributed storage node module consists of multiple internal enterprise servers, forming an internal storage cluster.

[0043] Enterprise core business data, such as financial data and customer information, are distributed across different server nodes through a data distribution and dynamic load balancing mechanism. The dynamic load balancing module ensures that the load of each server node is balanced, guaranteeing efficient access to data by the business system.

[0044] The intelligent redundancy management module sets a high redundancy level for core business data to ensure that data is not lost and business operations continue even if a server node fails. For non-core data with low access frequency, the redundancy level is appropriately reduced to save storage resources.

[0045] The data processing module compresses and encrypts enterprise data. For large amounts of text-based business documents, efficient lossless compression algorithms are used to reduce storage space; for data such as video conference recordings, appropriate compression methods are selected. Encryption ensures the security of sensitive enterprise data during internal transmission and storage, preventing data leakage.

[0046] The backup and recovery module provides reliable backup and recovery protection for enterprise data. Full backups are performed weekly, while incremental backups are performed in real time. In the event of data loss due to unforeseen circumstances, data can be quickly recovered from backups, minimizing the impact of data loss on business operations.

[0047] When a business expands and data volume increases dramatically, the expansion and optimization module can easily add new server nodes to expand storage capacity. The system automatically balances data load, combining intelligent cache management and adaptive storage scheduling technology to ensure fast data access for business systems and meet the needs of business development.

[0048] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A network data storage system, characterized in that, It includes a distributed storage node module, a dynamic load balancing module, an intelligent redundancy management module, a data processing module, a backup and recovery module, and an expansion and optimization module; The distributed storage node module consists of multiple nodes with independent storage capabilities, which are interconnected through a network to form a storage cluster. The dynamic load balancing module is connected to the distributed storage node module and is used to monitor the load of each node and automatically adjust data storage and access requests according to the node load. The intelligent redundancy management module is connected to the distributed storage node module and the dynamic load balancing module. Based on the data redundancy algorithm and adaptive redundancy ratio technology, it intelligently selects the redundancy level according to the importance of the data and the access frequency. The data processing module includes a compression unit and an encryption unit. The compression unit uses a combination of multiple compression algorithms and automatically selects the appropriate compression method according to the data type. The encryption unit is used to encrypt data during transmission and storage. The backup and recovery module uses a combination of full and incremental backups, employs intelligent backup strategies to back up data, and performs rapid recovery in case of data loss. The expansion and optimization module is used to support on-demand expansion of storage capacity and to optimize storage performance using intelligent cache management and adaptive storage scheduling technology.

2. The network data storage system according to claim 1, characterized in that, The encryption unit uses the AES encryption algorithm.

3. A network data storage system according to claim 2, characterized in that, The intelligent recovery engine of the backup and recovery module can select the optimal recovery path and strategy based on the health status of the backup data.

4. A network data storage system according to claim 3, characterized in that, The extended optimization module also includes a performance monitoring unit, which collects indicators such as system response time, data transmission rate, and node resource utilization in real time, and dynamically adjusts the intelligent caching strategy and storage scheduling parameters based on the indicator analysis results to continuously optimize system performance.

5. A network data storage method, using the network data storage system of claim 4, characterized in that, Includes the following steps: S1. Distributed Data Storage and Dynamic Load Balancing: A distributed storage architecture is adopted to distribute data across multiple nodes; Real-time monitoring of node load status and automatic adjustment of data storage and access requests; S2. Intelligent Redundancy Management: Based on data importance and access frequency, and using data redundancy algorithms and adaptive redundancy ratio technology, the redundancy level is intelligently selected to maintain higher redundancy copies for frequently accessed data. S3. Data Compression and Encryption: Automatically selects the appropriate compression method based on the data type; encrypts the compressed data during transmission and storage. S4. Data Backup and Recovery: Employs a combination of full and incremental backups, using intelligent backup strategies for data backup; When data is lost, the optimal recovery path and strategy are selected for rapid recovery based on the health status of the backup data. S5. System Expansion and Performance Optimization: Dynamically add storage nodes according to demand and automatically balance data load; adopt intelligent cache management and adaptive storage scheduling technology to optimize storage performance.

6. A network data storage method according to claim 5, characterized in that, In the S3 data compression and encryption step, an efficient lossless compression algorithm is used for text data, and appropriate lossy or lossless compression methods are selected for image and video data according to their characteristics.

7. A network data storage method according to claim 5, characterized in that, In the S4 data backup and recovery steps, full backups periodically perform complete backups of all data, while incremental backups only back up the data that has changed since the last backup.

8. A network data storage method according to claim 7, characterized in that, The intelligent cache management technology analyzes the frequency and timeliness of data access, prioritizing the caching of frequently accessed and recently active data to high-speed storage media, while periodically cleaning up infrequently accessed cached data to free up space.

Images