Enterprise data storage management system based on cloud storage

Abstract

The invention discloses an enterprise data storage management system based on cloud storage, and belongs to the technical field of data storage, and the system comprises a memory division module which is used for dividing a storage space in a cloud server into a plurality of sub-memories according to different storage contents; the distribution module is used for inputting the data input into the cloud server into the corresponding sub-memories for storage according to the deviation coefficient condition of the data; and the cleaning module is used for irregularly cleaning the stored data in the sub-memory. And the space adjusting module is used for dynamically adjusting the storage space according to the storage condition of each sub-storage. According to the invention, the storage space in each sub-memory can be automatically and dynamically adjusted through the space adjustment module according to the storage condition in each sub-memory, so that the storage space in the cloud server can be more reasonably used, the waste of storage resources is reduced, and the storage cost is saved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data storage, and in particular relates to an enterprise data storage management system based on cloud storage. Background Art

[0002] With the rapid growth of enterprise data, traditional local storage methods can no longer meet the needs of rapid expansion and high reliability. Therefore, cloud storage technology came into being. Cloud storage technology is a data storage technology developed on the basis of cloud computing. It integrates various storage resources in the network through the network and provides them to users in the form of storage services. It integrates distributed storage, virtualization, network transmission and other technologies, stores data on remote server clusters through the network, and realizes cross-regional and cross-device access and sharing of data.

[0003] When using cloud storage technology to store enterprise data in the corresponding cloud server, the cloud server is generally planned and classified into multiple sub-storage devices first, and then the enterprise data is stored in the corresponding sub-storage devices. However, due to the large variety of enterprise data, data storage errors are prone to occur during storage, which is inconvenient for subsequent retrieval. In addition, due to the growth of stored data, a large amount of storage space is required, and the storage cost may exceed expectations. How to more reasonably allocate and utilize the space of each sub-storage device to reduce the output of storage costs is becoming increasingly important. Summary of the invention

[0004] The purpose of the present invention is to provide an enterprise data storage management system based on cloud storage to solve the problems faced in the above-mentioned background technology.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] An enterprise data storage management system based on cloud storage, the management system comprising:

[0007] A storage partitioning module, which is used to divide the storage space in the cloud server into multiple sub-storages according to different storage contents, and each sub-storage is provided with a matching feature set trained by a neural network model, and the matching feature set contains multiple matching features related to the storage content of the sub-storage;

[0008] An allocation module, the allocation module is used to input the data input into the cloud server into the corresponding sub-storage for storage according to the deviation coefficient;

[0009] A cleaning module is used to clean up the storage data in the sub-storage device irregularly.

[0010] The space adjustment module is used to dynamically adjust the storage space according to the storage conditions of each sub-storage.

[0011] Furthermore, the allocation module works in the following way:

[0012] The allocation is divided into the first and second stages:

[0013] Phase 1: Extract multiple matching features of the data input into the cloud server to form an initial matching feature set, and compare the initial matching feature set with the matching feature sets in each sub-storage to select the sub-storage containing all the initial matching feature sets;

[0014] The second stage: obtain multiple matching features of the data input into the cloud service, and obtain the proportion value ρ of each matching feature based on the number of times the same matching feature appears i , the proportion of each matching feature ρ i The standard proportion value ρ of each matching feature in each sub-storage thi Compare, by formula Get the deviation value R of each matching feature i ;

[0015] Then through the formula F = R1 + ... + R i +……+R n Obtain the deviation coefficient F of each sub-storage;

[0016] Input the data into the cloud service into the sub-storage with the smallest deviation coefficient;

[0017] Among them, n is the number of matching feature items obtained and input into the cloud service data, R1 is the deviation value of the first matching feature, and i∈(1,n).

[0018] Furthermore, the working method of the cleaning module is:

[0019] Obtaining the time when the storage data in each sub-storage is entered into the sub-storage, and when the storage time exceeds the preset storage time, clearing the storage time;

[0020] The data cleaning method is to remove the expired storage data from the sub-storage and convert the storage data to cold storage mode.

[0021] Furthermore, the working method of the space adjustment module is:

[0022] Set multiple adjustment time points, and obtain the storage space size EM in the sub-storage at the current adjustment time point;

[0023] By formula Obtaining the estimated storage value EP;

[0024] When EP>EM*90%, the storage space of the sub-storage is adjusted;

[0025] Among them, R a is the first estimated value, R s is the second estimated value, τ1 and τ2 are preset coefficients.

[0026] Furthermore, the first estimated value R a The acquisition method is:

[0027] Get the current adjustment time point t from the historical storage data b To the next adjustment time point t c The storage data changes with time curve R ax (t), and the last adjustment time point t a To the current adjustment time point t b The storage data changes with time curve R ay (t);

[0028] By formula

[0029]

[0030] Get the first estimate R a ;

[0031] Among them, WR ax (t) is the preset current adjustment time point t b To the next adjustment time point t c Standard storage data change curve over time, WR ay (t) is the preset last adjustment time point t a To the current adjustment time point t b The standard storage data changes over time.

[0032] Furthermore, the second estimated value R s The acquisition method is:

[0033] Get the last adjustment time point t a To the current adjustment time point t b The business generation volume Q of the internal enterprise, the frequency rate of the enterprise's daily business activities h, and the change volume P of the enterprise's internal basic data;

[0034] So through the formula

[0035]

[0036] The second estimated value R s ;

[0037] Among them, α1 and α2 are preset proportional coefficients, Q0 is the preset last adjustment time point t a To the current adjustment time point t b The standard business generation volume of the enterprise, h(t) is the curve of the frequency rate of daily business activities of the enterprise over time, h(t b ) is t b The frequency of daily business activities of an enterprise over time.

[0038] Furthermore, the method in which the space adjustment module adjusts the storage space of the sub-storage is:

[0039] Select the number m of sub-storages with EP>EM*90%, increase their storage space, and use the formula S=EP-EM*90% to obtain the storage deviation value S of each sub-storage, and sort the sub-storages S1, S2, ... S from large to small according to the storage deviation value S. m , and increase them in sequence according to the sorting order;

[0040] At the same time, select the number of sub-storages k with EP≤EM*90%, reduce their storage space, and obtain the storage free value D of each sub-storage through the formula D=EM*90%-EP, and sort the sub-storages D1, D2...D from large to small according to the storage free value D. k , and then reduce them in order according to the sorting order.

[0041] Beneficial effects of the present invention:

[0042] The present invention can divide the storage space in the cloud server into multiple sub-storages in advance according to the different content of the enterprise's stored data, and obtain the deviation coefficient of each sub-storage according to the matching characteristics of the data input into the cloud server and the matching characteristics in each sub-storage through the allocation module, so as to input the data into the corresponding sub-storage for storage according to the deviation coefficient, so that the data can be input into the corresponding storage space more accurately, thereby reducing storage errors and facilitating subsequent query and management;

[0043] The present invention can automatically and dynamically adjust the storage space in each sub-storage according to the storage conditions in each sub-storage through the space adjustment module, so that the storage space in the cloud server can be used more reasonably, the waste of storage resources can be reduced, and storage costs can be saved.

[0044] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0046] Figure 1 It is a system block diagram of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In one embodiment, a cloud storage-based enterprise data storage management system is disclosed, such as Figure 1 As shown, the management system includes:

[0049] A storage partitioning module is used to divide the storage space in the cloud server into multiple sub-storages according to different storage contents, and each sub-storage is provided with a matching feature set trained by a neural network model, and the matching feature set contains multiple matching features related to the storage content of the sub-storage;

[0050] The allocation module is used to input the data input into the cloud server into the corresponding sub-storage for storage according to the deviation coefficient;

[0051] The cleaning module is used to clean up the storage data in the sub-storage device irregularly.

[0052] The space adjustment module is used to dynamically adjust the storage space according to the storage conditions of each sub-storage.

[0053] Through the above technical scheme, the present application can divide the storage space in the cloud server into multiple sub-storages in advance according to the different data contents stored in the enterprise, and compare the matching features of the data input into the cloud server with the matching features in each sub-storage one by one through the allocation module to obtain the deviation coefficient, so as to input the data into the corresponding sub-storage for storage according to the deviation coefficient. In this way, the data can be input into the corresponding storage space more accurately, storage errors can be reduced, and subsequent queries and management can be facilitated. At the same time, the present application can also automatically and dynamically adjust the storage space in each sub-storage according to the storage situation in each sub-storage through the space adjustment module, so as to more reasonably use the storage space in the cloud server, reduce the waste of storage resources, and save storage costs.

[0054] As an implementation mode of the present invention, the method in which the allocation module works is as follows:

[0055] The allocation is divided into the first and second stages:

[0056] Phase 1: Extract multiple matching features of the data input into the cloud server to form an initial matching feature set, and compare the initial matching feature set with the matching feature sets in each sub-storage to select the sub-storage containing all the initial matching feature sets;

[0057] The second stage: obtain multiple matching features of the data input into the cloud service, and obtain the proportion value ρ of each matching feature based on the number of times the same matching feature appears i , the proportion of each matching feature ρ i The standard proportion value ρ of each matching feature in each sub-storage thi Compare, by formula Get the deviation value R of each matching feature i ;

[0058] Then through the formula F = R1 + ... + R i +……+R n Obtain the deviation coefficient F of each sub-storage;

[0059] Input the data into the cloud service into the sub-storage with the smallest deviation coefficient;

[0060] Among them, n is the number of matching feature items obtained and input into the cloud service data, R1 is the deviation value of the first matching feature, and i∈(1,n).

[0061] Through the above technical solution, this embodiment provides a specific method for the allocation module to allocate and store the data input into the cloud server. The entire allocation is divided into a first stage and a second stage; the first stage: extract multiple matching features of the data input into the cloud server to form an initial matching feature set, and compare the initial matching feature set with the matching feature sets in each sub-storage, and filter out the sub-storage containing all the initial matching feature sets, so that the sub-storage related to the data to be stored can be found to narrow the storage selection range; the second stage: obtain multiple matching features of the data input into the cloud service, and according to the number of times the same matching feature appears, the proportion value ρ of each matching feature is obtained i , the proportion of each matching feature ρ i The standard proportion value ρ of each matching feature in each sub-storage thi Compare, by formula Get the deviation value R of each matching feature i ; Then through the formula F = R1 + ... + R i +……+R n Obtain the deviation coefficient F of each sub-storage, and input the data into the cloud service into the sub-storage with the smallest deviation coefficient; after narrowing the storage selection range, in order to more accurately determine the storage location, further precise division can be performed, so first obtain multiple matching features of the data input into the cloud service, and according to the number of times the same matching feature appears, the proportion value ρ of each matching feature is obtained i , the proportion of each matching feature ρ i The standard proportion value ρ of each matching feature in each sub-storage thi Compare, by formula Get the deviation value R of each matching feature i It can be seen that the smaller the deviation value is, the greater the possibility that the input data is stored in the sub-storage is, and then through the formula F = R1 + ... + R i +……+R n By synthesizing the deviation coefficient F of each sub-storage, it can be seen that the smaller the overall deviation coefficient, the higher the possibility that the input data is stored in the sub-storage, so the data input into the cloud service is input into the sub-storage with the smallest deviation coefficient. In this way, the location where the input data needs to be stored can be accurately determined based on the deviation coefficient between the input data and each sub-storage, thereby reducing the occurrence of storage errors.

[0062] It should be noted that the standard ratio of each matching feature in each sub-storage is thi It can be independently formulated according to the storage content in each sub-storage device based on experience, and will not be described in detail here.

[0063] As an embodiment of the present invention, the working method of the cleaning module is:

[0064] Obtaining the time when the storage data in each sub-storage is entered into the sub-storage, and when the storage time exceeds the preset storage time, clearing the storage time;

[0065] The data cleaning method is to remove the expired storage data from the sub-storage and convert the storage data to cold storage mode.

[0066] Through the above technical scheme, this embodiment provides a specific method for the cleaning module to work. In order to better utilize the storage space and reduce waste, it is necessary to clean up the data in the storage space from time to time. Specifically, the time when the storage data in each sub-storage is entered into the sub-storage is first obtained, and a storage duration is set. When its storage time exceeds the preset storage duration, the storage time is cleaned up, which means that the storage data has expired. In order to reduce its occupancy of storage space, the expired storage data is removed from the sub-storage and converted to a cold storage mode to ensure full utilization of the storage space.

[0067] As an implementation mode of the present invention, the working method of the space adjustment module is: setting multiple adjustment time points, obtaining the storage space size EM in the sub-storage at the current adjustment time point;

[0068] By formula Obtaining the estimated storage value EP;

[0069] When EP>EM*90%, the storage space of the sub-storage is adjusted;

[0070] Among them, R a is the first estimated value, R s is the second estimated value, τ1 and τ2 are preset coefficients;

[0071] The first estimate R a The acquisition method is: obtain the current adjustment time point t from the historical storage data b To the next adjustment time point t c The storage data changes with time curve R ax (t), and the last adjustment time point t a To the current adjustment time point t b The storage data changes with time curve R ay (t);

[0072] By formula

[0073]

[0074] Get the first estimate Ra ;

[0075] Among them, WR ax (t) is the preset current adjustment time point t b To the next adjustment time point t c Standard storage data change curve over time, WR ay (t) is the preset last adjustment time point t a To the current adjustment time point t b The time-varying curve of standard storage data;

[0076] The second estimated value R s The acquisition method is: Get the last adjustment time point t a To the current adjustment time point t b The business generation volume Q of the internal enterprise, the frequency rate of the enterprise's daily business activities h, and the change volume P of the enterprise's internal basic data;

[0077] So through the formula

[0078]

[0079] The second estimated value R s ;

[0080] Among them, α1 and α2 are preset proportional coefficients, Q0 is the preset last adjustment time point t a To the current adjustment time point t b The standard business generation volume of the enterprise, h(t) is the curve of the frequency rate of daily business activities of the enterprise over time, h(t b ) is t b The frequency of daily business activities of the enterprise over time;

[0081] The method for adjusting the storage space of the sub-storage by the space adjustment module is as follows: select the number m of sub-storages with EP>EM*90%, increase their storage space, obtain the storage deviation value S of each sub-storage through the formula S=EP-EM*90%, and sort the sub-storages from large to small according to the storage deviation value S as S1, S2...S m , and increase them in sequence according to the sorting order;

[0082] At the same time, select the number of sub-storages k with EP≤EM*90%, reduce their storage space, and obtain the storage free value D of each sub-storage through the formula D=EM*90%-EP, and sort the sub-storages D1, D2...D from large to small according to the storage free value D. k , and then reduce them in order according to the sorting order.

[0083] Through the above technical solution, this embodiment provides a method for dynamically adjusting the storage space by the space adjustment module, firstly setting multiple adjustment time points, obtaining the storage space size EM in the sub-storage at the current adjustment time point; then obtaining the current adjustment time point t from the historical storage data b To the next adjustment time point t c The storage data changes with time curve R ax (t), and the last adjustment time point t a To the current adjustment time point t b The storage data changes with time curve R ay (t), through the formula

[0084]

[0085] Get the first estimate R a ; From the formula Represented as the same time period (t c -t b ) The difference between the change of the stored data in the standard storage data and the change of the standard storage data. The larger the value, the larger the storage data, and the larger the storage space required. The same formula Indicates the same time period (t b -t a ) and the difference between the storage data changes in the standard storage data. The larger the difference, the larger the storage space required. Therefore, a comprehensive analysis is performed on the storage data of the previous period at the same time point in the historical storage data and the storage data of the next period to obtain the first estimated value. In this way, the required storage space can be better predicted based on the historical storage situation. Then, the last adjustment time point t is obtained. a To the current adjustment time point t b The business generation volume Q of the enterprise, the frequency rate of the enterprise's daily business activities h and the change volume P of the enterprise's internal basic data are used to calculate the value of the enterprise's internal business volume Q, the frequency rate of the enterprise's daily business activities h and the change volume P of the enterprise's internal basic data.

[0086]

[0087] The second estimated value R s ;formula Indicates that at t a ~t b The difference between the frequency rate of daily activities of enterprises in the second half of the time and the frequency rate of daily activities of enterprises in the first half of the time, the larger the value, the more storage space is needed; so combined with t a ~t bThe second estimated value is obtained by comprehensively analyzing the business generation volume Q of the enterprise during the period, the frequency rate of the enterprise's daily business activities h, and the change volume of the enterprise's internal basic data. It can be seen that during this period, the more the enterprise's business generation volume Q, the greater the change volume P of the enterprise's internal basic data, and the higher the frequency rate of the enterprise's daily business activities, the larger the storage space required; therefore, the formula is finally used Obtain a storage estimate EP, and compare the obtained storage estimate with the storage space size EM in the sub-storage at the current adjustment time point: when EP>EM*90%, it means that the storage space is insufficient, otherwise it means that the storage space is sufficient, then the storage space of the sub-storage is adjusted accordingly, and the specific adjustment is: select the number m of sub-storages with EP>EM*90%, increase their storage space, obtain the storage deviation value S of each sub-storage through the formula S=EP-EM*90%, and sort the sub-storages from large to small according to the storage deviation value S as S1, S2...S m , and increase in sequence according to the sorting order, and the higher the ranking, the more storage space is needed, and the more storage space the sub-storage increases; at the same time, select the number of sub-storages k with EP≤EM*90%, reduce their storage space, and obtain the storage free value D of each sub-storage through the formula D=EM*90%-EP, and sort the sub-storages D1, D2...D from large to small according to the storage free value D k , and it is reduced in sequence according to the sorting order. The higher its ranking, the more sufficient its storage space is, and the more storage space the sub-storage reduces. In this way, the storage space in each sub-storage can be automatically adjusted dynamically according to the storage situation in each sub-storage, so that the storage space in the cloud server can be used more reasonably, the waste of storage resources can be reduced, and storage costs can be saved. It should be noted that the determination of the adjustment time point can be artificially formulated, and the preset coefficients τ1 and τ2, the preset proportional coefficients α1 and α2 can be determined based on empirical data, and the standard business generation volume Q0 of the enterprise between each adjustment time point and the standard storage data change curve WR over time ax (t), WR ay (t) etc. can be formulated based on the relevant historical data of the enterprise and will not be elaborated in detail here.

[0088] The above contents are merely examples and explanations of the concept of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. An enterprise data storage management system based on cloud storage, characterized in that: The management system comprises: A storage partitioning module, which is used to divide the storage space in the cloud server into multiple sub-storages according to different storage contents, and each sub-storage is provided with a matching feature set trained by a neural network model, and the matching feature set contains multiple matching features related to the storage content of the sub-storage; An allocation module, the allocation module is used to input the data input into the cloud server into the corresponding sub-storage for storage according to the deviation coefficient; A cleaning module is used to clean up the storage data in the sub-storage device irregularly. The space adjustment module is used to dynamically adjust the storage space according to the storage conditions of each sub-storage.

2. The cloud storage-based enterprise data storage management system according to claim 1, characterized in that: The method in which the allocation module works is: The allocation is divided into the first and second stages: Phase 1: Extract multiple matching features of the data input into the cloud server to form an initial matching feature set, and compare the initial matching feature set with the matching feature sets in each sub-storage to select the sub-storage containing all the initial matching feature sets; The second stage: obtain multiple matching features of the data input into the cloud service, and obtain the proportion value ρ of each matching feature based on the number of times the same matching feature appears i , the proportion of each matching feature ρ i The standard proportion value ρ of each matching feature in each sub-storage thi Compare, by formula Get the deviation value R of each matching feature i ; Then through the formula F = R1 + ... + R i +……+R n Obtain the deviation coefficient F of each sub-storage; Input the data into the cloud service into the sub-storage with the smallest deviation coefficient; Among them, n is the number of matching feature items obtained and input into the cloud service data, R1 is the deviation value of the first matching feature, and i∈(1,n).

3. The cloud storage-based enterprise data storage management system according to claim 1, characterized in that: The working method of the cleaning module is: Obtaining the time when the storage data in each sub-storage is entered into the sub-storage, and when the storage time exceeds the preset storage time, clearing the storage time; The data cleaning method is to remove the expired storage data from the sub-storage and convert the storage data to cold storage mode.

4. The cloud storage-based enterprise data storage management system according to claim 1, characterized in that: The working method of the space adjustment module is: Set multiple adjustment time points, and obtain the storage space size EM in the sub-storage at the current adjustment time point; By formula Obtaining the estimated storage value EP; When EP>EM*90%, the storage space of the sub-storage is adjusted; Among them, R a is the first estimated value, R s is the second estimated value, τ1 and τ2 are preset coefficients.

5. The cloud storage-based enterprise data storage management system according to claim 4, characterized in that: The first estimated value R a The acquisition method is: Get the current adjustment time point t from the historical storage data b To the next adjustment time point t c The storage data changes with time curve R ax (t), and the last adjustment time point t a To the current adjustment time point t b The storage data changes with time curve R ay (t); By formula Get the first estimate R a ; Among them, WR ax (t) is the preset current adjustment time point t b To the next adjustment time point t c Standard storage data change curve over time, WR ay (t) is the preset last adjustment time point t a To the current adjustment time point t b The standard storage data changes over time.

6. The cloud storage-based enterprise data storage management system according to claim 4, characterized in that: The second estimated value R s The acquisition method is: Get the last adjustment time point t a To the current adjustment time point t b The business generation volume Q of the internal enterprise, the frequency rate of the enterprise's daily business activities h, and the change volume P of the enterprise's internal basic data; So through the formula The second estimated value R s ; Among them, α1 and α2 are preset proportional coefficients, Q0 is the preset last adjustment time point t a To the current adjustment time point t b The standard business generation volume of the enterprise, h(t) is the curve of the frequency rate of daily business activities of the enterprise over time, h(t b ) is t b The frequency of daily business activities of an enterprise over time.

7. The cloud storage-based enterprise data storage management system according to claim 6, characterized in that: The method by which the space adjustment module adjusts the storage space of the sub-storage is: Select the number m of sub-storages with EP>EM*90%, increase their storage space, and use the formula S=EP-EM*90% to obtain the storage deviation value S of each sub-storage, and sort the sub-storages S1, S2, ... S from large to small according to the storage deviation value S. m , and increase them in sequence according to the sorting order; At the same time, select the number of sub-storages k with EP≤EM*90%, reduce their storage space, and obtain the storage free value D of each sub-storage through the formula D=EM*90%-EP, and sort the sub-storages D1, D2...D from large to small according to the storage free value D. k , and then reduce them in order according to the sorting order.

Images