Internet-based data cloud storage system

Abstract

The invention discloses a data cloud storage system based on the Internet, and relates to the technical field of data cloud storage, the data cloud storage system comprises a management center, the management center is in communication connection with a data acquisition module, a data analysis module, a data processing module, a user confirmation module and a data storage module; the data acquisition module is used for acquiring data information required to be stored by a user and storage information of a cloud space; the data analysis module is used for analyzing whether the residual storage capacity of the cloud space can meet user requirements or not; the data processing module is used for providing a cloud space storage capacity change scheme for a user and generating corresponding charging information; the user confirmation module is used for confirming a cloud space storage capacity change scheme by a user and paying fees; and the data storage module is used for changing the cloud space storage capacity according to the cloud space storage capacity change scheme confirmed by the user, and storing data information required to be stored by the user.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and specifically to an Internet-based data cloud storage system. Background Art

[0002] With the development of Internet technology, data storage technology has been improved. While being more and more widely used in daily life, there are many potential problems. For example, users have a greater demand for the storage space capacity of data; there is excessive waste of cloud space storage capacity; data loss may occur during the data storage process;

[0003] In the prior art, when users store data in cloud space, the storage capacity does not meet the user's needs, resulting in excessive waste of cloud space storage capacity. When data is uploaded to cloud space for storage, there is a risk of data loss. How to improve the rationality of cloud space storage capacity, the integrity and efficiency of storing data in data cloud storage is a problem we need to solve. For this reason, an Internet-based data cloud storage system is provided herein. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide an Internet-based data cloud storage system, including a management center, which is communicatively connected to a data acquisition module, a data analysis module, a data processing module, a user confirmation module, and a data storage module;

[0005] The data acquisition module is used to acquire the data information required by the user for storage and the storage information of the cloud space;

[0006] The data analysis module is used to analyze whether the remaining storage capacity of the cloud space can meet the user's needs;

[0007] The data processing module is used to provide the user with a cloud space storage capacity change plan and generate corresponding billing information;

[0008] The user confirmation module is used for the user to confirm the cloud space storage capacity change plan and make a payment;

[0009] The data storage module is used to change the cloud space storage capacity according to the cloud space storage capacity change plan confirmed by the user, and store the data information required by the user for storage.

[0010] Further, the process of the data acquisition module acquiring the data information required by the user for storage and the storage information of the cloud space includes:

[0011] Set a time period t;

[0012] Periodically obtain the data information required to be stored by the user according to the set time period t, package the obtained data information to generate a data packet, and record the data size corresponding to the data packet as c t , and read the data types of the data information, obtain the historical storage records of different data types of the user, obtain the storage frequencies of different data types according to the historical storage records of different data types, and record the data information as a frequently used data type or an infrequently used data type according to the storage frequencies of each data type;

[0013] Mark the data size of the frequently used data type corresponding to the data information as a t and mark the data size of the infrequently used data type corresponding to the data information as b t ;

[0014] Obtain the remaining storage capacity Q3 of the cloud space.

[0015] Furthermore, the process by which the data analysis module analyzes whether the remaining storage capacity of the cloud space can meet the user's needs includes:

[0016] Perform data duplicate checking on the data information required to be stored by the user, and make a judgment according to the data duplicate checking result;

[0017] When there is no duplicate data information, compare the data size of the data required to be stored by the user with the remaining storage capacity of the cloud space;

[0018] When c t > Q3, it is necessary to expand the cloud space storage capacity, generate a non-duplicate data expansion request, and send it to the data processing module;

[0019] When c t = Q3, send a user cloud space storage capacity warning message to the user confirmation module;

[0020] When c t < Q3, there is no need to change the cloud space storage capacity;

[0021] If there is duplicate data information, when c t < Q3, there is no need to change the cloud space storage capacity;

[0022] When c t = Q3, send a user cloud space storage capacity warning message to the user confirmation module;

[0023] When c t > Q3, obtain the data size of the non-duplicate data information, mark the data size as d t , and compare the data size of the non-duplicate data information with the remaining storage capacity of the cloud space;

[0024] When dt When it is > Q3, it is necessary to expand the cloud space storage capacity, generate a duplicate data expansion request, and send it to the data processing module;

[0025] When d t = Q3, a warning message that the user's cloud space storage capacity is about to be insufficient is sent to the user confirmation module;

[0026] When d t < Q3, there is no need to change the cloud space storage capacity.

[0027] Furthermore, the process in which the data processing module provides the user with a cloud space storage capacity change plan and generates corresponding billing information includes:

[0028] Set up hot storage data units and cold storage data units;

[0029] When receiving a non-duplicate data expansion request, according to the data size a t of the commonly used data types that the user needs to store and the data size b t of the infrequently used data types, generate the expanded storage capacity required for the hot storage data unit of the cloud space and the expanded storage capacity required for the cold storage data unit, and generate the required additional fees;

[0030] When receiving a duplicate data expansion request, obtain the data size of the duplicate data of the commonly used data types and the duplicate data size of the infrequently used data types in the duplicate data information. According to a t 、b t and the data size of the duplicate data of the commonly used data types and the duplicate data size of the infrequently used data types in the duplicate data, obtain the data size of the non-duplicate data of the commonly used data types and the data size of the non-duplicate data of the infrequently used data types that the user needs to store. According to the data size of the non-duplicate data of the commonly used data types and the data size of the non-duplicate data of the infrequently used data types that the user needs to store, generate the expanded storage capacity required for the hot storage data unit of the cloud space and the expanded storage capacity required for the cold storage data unit, and generate the required additional fees.

[0031] Furthermore, the process in which the user confirmation module allows the user to confirm the cloud space storage capacity change plan and make a payment includes:

[0032] Set up a service window;

[0033] When the management center sends the cloud space storage capacity change plan to the user confirmation module, the user confirms the plan at the service window. If the user confirms the storage capacity change and makes a payment, the user confirmation module feedbacks the confirmed cloud space storage capacity change plan to the data storage module.

[0034] Further, the process in which the data storage module changes the cloud space storage capacity according to the cloud space storage capacity change plan confirmed by the user and stores the data information required by the user includes:

[0035] According to the cloud space storage capacity change plan confirmed by the user, expand the storage capacity of the hot storage data unit and the cold storage data unit;

[0036] Set up a virtual platform;

[0037] The user uploads the data information required to be stored into the virtual platform. If there are duplicate data in the data information required to be stored by the user, the data information required to be stored uploaded by the user is preprocessed in the virtual platform, and the data information of common data types and the data information of uncommon data types are de-duplicated through a hash function;

[0038] Store the de-duplicated data information of common data types and the data information of uncommon data types into the expanded hot storage data unit and cold storage data unit;

[0039] If there is no duplicate data, store the data information of common data types and the data information of uncommon data types into the expanded hot storage data unit and cold storage data unit.

[0040] Further, the process of preprocessing the data information required to be stored uploaded by the user in the virtual platform includes:

[0041] According to the hash function, screen out the duplicate data information of the data information of common data types and the data information of uncommon data types, and at the same time compare the data information of the previous time period with the data information of the current time period for duplicate data information, and de-duplicate the data information of common data types and the data information of uncommon data types;

[0042] Compared with the prior art, the beneficial effects of the present invention are: using a virtual platform to classify and de-duplicate the data required to be stored by the user, setting up a cold storage data unit and a hot storage data unit, classifying and storing the data, improving the integrity and efficiency of uploading data to the cloud space storage, expanding the storage capacity of the storage unit according to the user's needs, and enabling the user to reasonably utilize the cloud space storage capacity. Description of the Drawings

[0043] Figure 1 It is a schematic diagram of a data cloud storage system based on the Internet according to an embodiment of the present application. Detailed Embodiments

[0044] Such as Figure 1As shown in the figure, an Internet-based data cloud storage system includes a management center, which is communicatively connected to a data acquisition module, a data analysis module, a data processing module, a user confirmation module, and a data storage module;

[0045] The data acquisition module is used to acquire the data information required by the user for storage and the storage information of the cloud space;

[0046] It should be further noted that in the specific implementation process, the cloud space is a storage space used to provide storage information for users;

[0047] The data analysis module is used to analyze whether the remaining storage capacity of the cloud space can meet the user's needs;

[0048] The data processing module is used to provide a cloud space storage capacity change plan for the user and generate corresponding billing information;

[0049] The user confirmation module is used for the user to confirm the cloud space storage capacity change plan and make payments;

[0050] The data storage module is used to change the cloud space storage capacity according to the cloud space storage capacity change plan confirmed by the user and store the data information required by the user.

[0051] The data acquisition module is used to acquire the data information required by the user for storage and the storage information of the cloud space. The process includes:

[0052] Set a time period t;

[0053] According to the set time period t, periodically obtain the data information required by the user for storage, pack the obtained data information to generate a data packet, and record the data size corresponding to the data packet as c t and read the data type of the data information. The data types include pictures, videos, documents, music, and others. Obtain the historical storage records of different data types of the user, obtain the storage frequencies of different data types according to the historical storage records of different data types, and record the data information as common data types or uncommon data types according to the storage frequencies of each data type;

[0054] Mark the data size of the common data type as a t and mark the data size of the uncommon data type as b t where a t represents the data size of the common data type in the data packet of the t-th time period, and b t represents the data size of the uncommon data type in the data packet of the t-th time period, and t = 1, 2,..., n, where n is an integer.

[0055] It should be further noted that in the specific implementation process, the data storage frequency threshold is set to v, where v > 0. If the storage frequency of user data within the time period t is greater than v, it is called frequently used data. If the storage frequency of user data within the time period t is less than v, it is called infrequently used data;

[0056] It should be further noted that in the specific implementation process, when obtaining the historical storage records of different data types of users, if there is no historical storage record, the storage frequency of the user data is set to 0;

[0057] Perform data duplicate checking on the obtained frequently used data and infrequently used data respectively. According to the data duplicate checking results, upload the data to be stored to the virtual platform for data deduplication processing, and then determine whether the storage space of the cloud space meets the requirements;

[0058] Obtain the storage space allocated to the user by the cloud space, and obtain the remaining storage capacity Q3 of the cloud space;

[0059] The data analysis module is used to analyze whether the remaining storage capacity of the cloud space can meet the user's needs. The specific process includes:

[0060] The process of performing data duplicate checking on the obtained frequently used data and infrequently used data respectively includes: comparing the frequently used data and infrequently used information in the data packet of the (t - 1)th time period with the frequently used data and infrequently used data in the data packet of the tth time period. If there are duplicate data information, obtain the frequently used data information and infrequently used data information in the data packet of the (t - 1)th time period, the duplicate data information of the frequently used data and the duplicate data information of the infrequently used data in the data packet of the tth time period, and obtain the data size of the duplicate data information of the frequently used data, mark the data size as m, the data size of the duplicate data information of the non - duplicate data, mark the data size as h, generate a query pointer according to the duplicate data information in the data packet, record the data size of the query pointer as L, and then associate the generated query pointer with the corresponding duplicate data in the previous period to avoid the loss of duplicate data information storage. The memory space of the query pointer is small. If there is no duplicate information, directly upload the frequently used data and infrequently used data to the virtual platform;

[0061] If there is no duplicate data information between the data information in the data packet of the (t - 1)th time period and the data information in the data packet of the tth time period, compare the data size of the data that the user needs to store with the remaining storage capacity Q3 of the cloud space, and judge whether the storage space allocated to the user by the cloud space is sufficient according to the comparison result;

[0062] When c t > Q3, the remaining storage capacity of the cloud space cannot meet the user's needs, and it is necessary to expand the storage capacity of the cloud space, generate a non - duplicate data expansion request, and send it to the data processing module;

[0063] When c t = Q3, it is estimated that the remaining storage capacity of the cloud space is full, and a warning message about the user's cloud space storage capacity is sent to the user confirmation module;

[0064] When c t < Q3, the remaining storage capacity of the cloud space meets the user's needs, and there is no need to change the cloud space storage capacity;

[0065] If there are duplicate data information in the data information of the data packet in the (t - 1)-th time period and the data information of the data packet in the t-th time period, when c t < Q3, there is no need to change the cloud space storage capacity;

[0066] When c t = Q3, a warning message about the user's cloud space storage capacity is sent to the user confirmation module;

[0067] When c t > Q3, obtain the non-duplicate data information in the data information of the data packet in the t-th time period and the data information of the data packet in the (t - 1)-th time period, and obtain the data size of the non-duplicate data information, and mark the data size as d t , where d t = k t - m - h, compare the data size d t of the non-duplicate data information with the remaining storage capacity Q3 of the cloud space, and judge whether the storage space allocated to the user by the cloud space is sufficient according to the comparison result;

[0068] When d t > Q3, the remaining storage capacity of the cloud space cannot meet the user's needs, and it is necessary to expand the cloud space storage capacity, generate a duplicate data expansion request, and send it to the data processing module;

[0069] When d t = Q3, it is estimated that the remaining storage capacity of the cloud space is full, and a warning message that the user's cloud space storage capacity will soon be insufficient is sent to the user confirmation module;

[0070] When d t < Q3, the remaining storage capacity of the cloud space meets the user's needs, and there is no need to change the cloud space storage capacity;

[0071] The data processing module is used to provide a cloud space storage capacity change plan for the user and generate corresponding billing information. The specific process includes:

[0072] Set up a hot storage data unit and a cold storage data unit;

[0073] It should be further noted that in the specific implementation process, the hot storage data unit has a small storage space, but fast data storage and data extraction speeds, and is suitable for storing frequently used data. The cold storage data unit has a large storage space, but slow data storage and data extraction speeds, and is suitable for storing infrequently used data;

[0074] Set the storage capacity to be expanded for the hot storage data unit in the cloud space as G1, and the storage capacity to be expanded for the cold storage data unit as G2. Denote the expansion charging standard for the hot storage data unit as q*s, the expansion charging standard for the cold storage data unit as p*s, and the additional cost required as Y, where s is the capacity of one storage unit, q is the cost required to expand one hot storage data unit, and p is the cost required to expand one cold storage data unit;

[0075] When receiving a non-duplicate data expansion request, based on the data sizes of the frequently used data types and infrequently used data types that the user needs to store, obtain the storage capacity G1 that needs to be expanded for the hot storage data unit in the cloud space and the storage capacity G2 that needs to be expanded for the cold storage data unit, where G1 = a t , G2 = b t , then the storage capacity of the hot storage data unit in the cloud space that needs to be expanded is a t , the storage capacity of the cold storage data unit in the cloud space that needs to be expanded is b t , and the additional cost Y = a t / s * q + b t / s * p;

[0076] When receiving a duplicate data expansion request, based on the data sizes of the frequently used data types and infrequently used data types that the user needs to store, obtain the storage capacity G1 that needs to be expanded for the hot storage data unit in the cloud space and the storage capacity G2 that needs to be expanded for the cold storage data unit, where G1 = a t +L - m, G2 = b t +L - h, then the storage capacity of the hot storage data unit in the cloud space that needs to be expanded is a t +L - m, the storage capacity of the cold storage data unit in the cloud space that needs to be expanded is b t +L - h, and the additional cost Y = (a t +L - m) / s * q + (b t +L - h) / s * p;

[0077] The user confirmation module is used for the user to confirm the cloud space storage capacity change plan. The specific process includes:

[0078] Set up a service window;

[0079] It should be further noted that in the specific implementation process, the service window is a two-way interaction platform that provides users with cloud storage services, facilitating users' management of the remaining storage capacity of the cloud space, rational application of the storage capacity, efficient and rapid storage of data information, and saving storage costs;

[0080] When the management center sends the cloud space storage capacity change plan and corresponding billing information to the user confirmation module, the user confirms the plan at the service window. After the user confirms the storage capacity change and pays the fee, the selected plan by the user is sent to the management center, and the management center sends the confirmed cloud space storage capacity change plan by the user to the data storage module.

[0081] The data storage module is used to change the cloud space storage capacity according to the confirmed cloud space storage capacity change plan by the user and store the data information required by the user. The specific process includes:

[0082] According to the user confirmation information, in accordance with the user cloud space storage capacity change plan provided by the data processing module, expand the storage capacity of the hot storage data unit and the cold storage data unit;

[0083] Set up a virtual platform;

[0084] The user uploads the data information required to be stored into the virtual platform, and preprocesses the data information required to be stored uploaded by the user within the virtual platform. The specific process includes:

[0085] Upload the data information required to be stored by the user to the virtual platform, detect the duplicate data information in the current time period and the previous time period through a hash function, screen out the duplicate data information of the data information of the common data type and the data information of the uncommon data type according to the detection result, and perform deduplication processing on the data information of the common data type and the data information of the uncommon data type;

[0086] If there is duplicate data in the data information required to be stored by the user, preprocess the data information required to be stored uploaded by the user within the virtual platform, and perform deduplication processing on the data information of the common data type and the data information of the uncommon data type through a hash function;

[0087] According to the analysis result of the data analysis module, store the deduplicated data information of the common data type and the data information of the uncommon data type into the expanded hot storage data unit and cold storage data unit;

[0088] If there is no duplicate data, according to the analysis result of the data analysis module, store the data information of the common data type and the data information of the uncommon data type and the associated query pointers into the expanded hot storage data unit and cold storage data unit.

[0089] It should be further noted that during the specific implementation process, the cloud space is automatically released when the user finishes storing.

[0090] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A data cloud storage system based on the Internet, including a management center, characterized in that: The management center is communicatively connected with a data acquisition module, a data analysis module, a data processing module, a user confirmation module and a data storage module; The data collection module is used to collect data information that users need to store and storage information of the cloud space; The data analysis module is used to analyze whether the remaining storage capacity of the cloud space can meet user needs; The data processing module is used to provide users with cloud space storage capacity change solutions and generate corresponding billing information; The user confirmation module is used for the user to confirm the cloud space storage capacity change plan and make payment; The data storage module is used to change the cloud space storage capacity according to the cloud space storage capacity change plan confirmed by the user, and store the data information required to be stored by the user.

2. The Internet-based data cloud storage system according to claim 1, characterized in that: The process of the data collection module collecting the data information required to be stored by the user and the storage information of the cloud space includes: Set the time period t; According to the set time period t, the data information that the user needs to store is periodically obtained, the obtained data information is packaged to generate a data packet, and the data size corresponding to the data packet is recorded as c t , and read the data type of the data information, obtain the historical storage records of different data types of the user, obtain the storage frequency of different data types according to the historical storage records of different data types, and record the data information as a common data type or an uncommon data type according to the storage frequency of each data type; The data size of the common data type corresponding to the data information is marked as a t The data size of the uncommon data type corresponding to the data information is marked as b t ; Get the remaining storage capacity of the cloud space Q3.

3. The Internet-based data cloud storage system according to claim 2, characterized in that: The process of the data analysis module analyzing whether the remaining storage capacity of the cloud space can meet the user's needs includes: Check the data information that the user needs to store for duplication, and make judgments based on the results of the duplication check; If there is no duplicate data information, the size of the data that the user needs to store is compared with the remaining storage capacity of the cloud space; When c t >Q3, it is necessary to expand the cloud space storage capacity, generate a non-duplicate data expansion request, and send it to the data processing module; When c t =Q3, a warning message about the user's cloud space storage capacity is sent to the user confirmation module; When c t <When Q3, there is no need to change the cloud storage capacity; When there is duplicate data information, when c t <Q3, there is no need to change the cloud storage capacity; When c t =Q3, a warning message about the user's cloud space storage capacity is sent to the user confirmation module; When c t >Q3, obtain the data size of non-repeated data information and mark the data size as d t , compare the data size of non-duplicate data information with the remaining storage capacity of the cloud space; When t >Q3, it is necessary to expand the cloud space storage capacity, generate duplicate data expansion requests, and send them to the data processing module; When t =Q3, a warning message is sent to the user confirmation module that the user's cloud space storage capacity is about to be insufficient; When d t <When Q3, there is no need to change the cloud storage capacity.

4. The Internet-based data cloud storage system according to claim 3, characterized in that: The process of the data processing module providing a cloud space storage capacity change plan for the user and generating corresponding billing information includes: Setting up hot storage data units and cold storage data units; When a request for non-duplicate data expansion is received, the data size of the commonly used data types that the user needs to store is a t and the data size b of uncommon data types t , generating the expanded storage capacity required for the hot storage data unit and the cold storage data unit of the cloud space, and generating the required additional costs; When a duplicate data expansion request is received, the data size of the duplicate data of the common data type and the duplicate data size of the uncommon data type in the duplicate data information are obtained, and the duplicate data size of the uncommon data type is obtained according to a t , b t and the data size of duplicate data of commonly used data types and the data size of duplicate data of uncommonly used data types in the duplicate data, obtain the data size of non-duplicate data of commonly used data types and the data size of non-duplicate data of uncommonly used data types that the user needs to store, generate the expanded storage capacity required for the hot storage data unit and the expanded storage capacity required for the cold storage data unit of the cloud space according to the data size of non-duplicate data of commonly used data types and the data size of non-duplicate data of uncommonly used data types that the user needs to store, and generate the required increased fees.

5. The Internet-based data cloud storage system according to claim 4, characterized in that: The process in which the user confirms the cloud space storage capacity change plan and pays the fee includes: Set up service window; When the management center sends the cloud space storage capacity change plan to the user confirmation module, the user confirms the plan in the service window. If the user confirms the storage capacity change and pays the fee, the cloud space storage capacity change plan confirmed by the user will be fed back to the data storage module.

6. The Internet-based data cloud storage system according to claim 5, characterized in that: The process in which the data storage module changes the cloud space storage capacity according to the cloud space storage capacity change plan confirmed by the user and stores the data information required to be stored by the user includes: Expand the storage capacity of hot storage data units and cold storage data units according to the cloud space storage capacity change plan confirmed by the user; Setting up a virtual platform; The user uploads the data information to be stored to the virtual platform. If there is duplicate data in the data information to be stored by the user, the data information uploaded by the user to be stored is pre-processed in the virtual platform, and the data information of the commonly used data types and the data information of the uncommon data types are deduplicated through the hash function; The data information of the commonly used data types and the data information of the uncommon data types after deduplication processing are stored in the expanded hot storage data unit and the cold storage data unit; If there is no duplicate data, the data information of the commonly used data types and the data information of the uncommon data types are stored in the expanded hot storage data units and cold storage data units.

7. The Internet-based data cloud storage system according to claim 6, characterized in that: The process of preprocessing the data information uploaded by the user and required to be stored in the virtual platform includes: The duplicate data information of the commonly used data types and the data information of the uncommonly used data types are filtered out according to the hash function. At the same time, the duplicate data information of the previous time period is compared with the data information of the current time period, and the duplicate data information of the commonly used data types and the data information of the uncommonly used data types are deduplicated.

Images