A data storage management system based on a cloud server

Abstract

The application discloses a data storage management system based on a cloud server and particularly relates to the technical field of data storage management, and solves the technical problems that it is difficult to extract unstructured and semi-structured data features to combine structured data features to construct a data type classification model for real-time classification, that there is a lack of value evaluation index values obtained by comprehensively using data frequency, data key ratio and data sensitivity, and that there is a lack of hierarchical storage of various data types according to the value evaluation index values. The system comprises the following modules: a data acquisition module used for deploying various data acquisition units to acquire data; a data classification module used for constructing a classification model; a data processing and analysis module used for analyzing and calculating data frequency, data key ratio and data sensitivity to obtain value evaluation index values; and a data storage and backup module used for storing and distributing data according to the value evaluation index values.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage management, and particularly relates to a data storage management system based on a cloud server. BACKGROUND

[0002] With the advent of the big data era, the types of data generated and processed by enterprises are increasingly complex and diverse. Traditional data storage and management solutions are relatively mature in dealing with structured data, but for unstructured data such as text files, images, audio, and semi-structured data such as XML and JSON format data, the management of multiple data types faces many challenges. These data are often stored in different formats and are difficult to store, retrieve and analyze efficiently, resulting in low data utilization and increased operating costs for enterprises. Although existing data storage management systems based on cloud servers have achieved data storage management to some extent, they still have some problems, such as difficulty in extracting multiple features contained in unstructured and semi-structured data in combination with structured data features to build a data type classification model for real-time classification of data types, lack of value evaluation index values obtained by comprehensively considering data usage frequency, data key ratio and data sensitivity, and lack of hierarchical storage of multiple data types according to the value evaluation index values, resulting in low data storage efficiency, limited processing capacity for different types of data, and lack of intelligent storage management strategies. SUMMARY

[0003] The present application aims to at least solve one of the technical problems existing in the prior art; for this purpose, the present application provides a data storage management system based on a cloud server, which is used to solve the following technical problems:

[0004] Although existing data storage management systems based on cloud servers have achieved data storage management to some extent, they still have some problems, such as difficulty in extracting multiple features contained in unstructured and semi-structured data in combination with structured data features to build a data type classification model for real-time classification of data types, lack of value evaluation index values obtained by comprehensively considering data usage frequency, data key ratio and data sensitivity, and lack of hierarchical storage of multiple data types according to the value evaluation index values, resulting in low data storage efficiency, limited processing capacity for different types of data, and lack of intelligent storage management strategies.

[0005] To solve the above problems, the present application provides a data storage management system based on a cloud server, which comprises the following modules:

[0006] Data acquisition module: deploying structured data acquisition unit, unstructured data acquisition unit and semi-structured data acquisition unit to acquire data;

[0007] Data classification module: build a classification model to classify the data types in the three collection units;

[0008] Data processing and analysis module: after analyzing and calculating the data usage frequency, data key ratio and data sensitivity of the three data types respectively, the values of the value evaluation index of the corresponding data types are obtained by adding them up;

[0009] Data storage and backup module: send the processed data to the cloud storage resource layer for storage, allocate data according to the value evaluation index value, and control and backup data regularly according to the preset access control backup strategy.

[0010] Preferably, the deployment of structured data collection unit, unstructured data collection unit and semi-structured data collection unit collects data, including the following steps:

[0011] The data collection module is composed of three data type collection units. The structured data collection unit is connected to the existing business database of the enterprise, and the structured data generated by the business operation is captured in real time through the database interface.

[0012] The unstructured data collection unit deploys a file crawler program to traverse and collect documents, pictures and audio resources in the enterprise internal file server and cloud storage.

[0013] The semi-structured data collection unit extracts key data elements from configuration information and log files stored in a specified format using corresponding parsers, and includes semi-structured data in the unified collection system. The specified format refers to XML and JSON formats.

[0014] Preferably, the classification model classifies the data in the three collection units, including the following steps:

[0015] Clean and format the structured data;

[0016] Convert the text into a numerical vector using a pre-trained Word2Vec model; extract multi-scale features from images through a convolutional neural network, and perform fusion and normalization operations; extract time domain, frequency domain and time-frequency domain features from audio, and then reduce dimensionality and fuse through principal component analysis;

[0017] Perform flattening processing on the JSON in the obtained semi-structured data; convert XML data into table form;

[0018] The processed data is labeled using a semi-automatic labeling tool, and features are extracted from the labeled data, including fields of structured data, numerical representations of unstructured data, and key fields of semi-structured data. A data type classification model is built using a neural network-based framework, and the extracted features are input into the model for training. The trained model is deployed to a server for real-time classification of data.

[0019] Preferably, the data usage frequency comprises the following steps:

[0020] The value evaluation indicators of structured data, unstructured data, and semi-structured data are calculated respectively, wherein the value evaluation indicators include data usage frequency, data key ratio, and data sensitivity;

[0021] In the time range of working days in the past month, the number of times and the time of calling, viewing, processing, and using other ways of different types of data sets are counted respectively, and the total number of times and the total time of use are obtained by adding them up. The data usage frequency is obtained by combining the total number of times of data usage divided by the total number of days and the total time of data usage divided by the total time.

[0022] Data usage frequency calculation formula:

[0023]

[0024] Wherein, represents the data usage frequency, represents the number of calls, represents the number of views, represents the number of processing, represents the number of other ways of use, represents the total number of working days in the past month, represents the time of calling, represents the time of viewing, represents the time of processing, represents the time of other ways of use, represents the total time of working days in the past month.

[0025] Preferably, the data key ratio comprises the following steps:

[0026] Collect relevant business objectives from the enterprise's strategic planning documents, annual reports and business department plans, classify and organize the collected objectives, classify the collected objectives according to time dimension, including short-term goals, medium-term goals and long-term goals, decompose each business objective into a plurality of measurable sub-goals, and determine the key success factors of each sub-goal; Build a hierarchical model, the highest layer is the overall goal of the enterprise, the middle layer is the sub-goal of each business field, and the bottom layer is the key success factor of each sub-goal. Label the targets and corresponding time attributes of different layers, train the hierarchical model with the labeled data for the training set, and deploy the trained model to the server;

[0027] Determine the dependency relationship between each level of objectives, analyze the target dependency relationship, and find the target sequence that plays a decisive role in the overall target as the key target path by combining the key path method;

[0028] Map the collected data to each target and key success factor in the target hierarchy, determine whether the corresponding key success factor is located on the key target path, and count the number of nodes located on the key target path and the total number of nodes of the key target path. Divide the data key ratio by the total number of nodes of the key target path to obtain the data key ratio;

[0029] Data key ratio calculation formula:

[0030]

[0031] Wherein, Data key ratio, The number of nodes located on the key target path, The total number of nodes of the key target path.

[0032] Preferably, the data sensitivity degree comprises the following steps:

[0033] Divide the data information type into personal identity information, financial information, health information and business secrets, define the sensitivity base value of different data information types, assign the data volume factor according to the proportion value of the data volume of different data information types in the total data volume according to the specified rules, assign the risk degree factor according to the severity of the leakage and theft of different data information types, and assign the ease of use factor according to the ease of access and use of different data information types. After normalizing the sensitivity base value of different data information types, the data volume factor, the risk degree factor and the ease of use factor are obtained simultaneously to obtain the data sensitivity degree of the corresponding data information type;

[0034] Data sensitivity degree calculation formula:

[0035]

[0036] wherein, represents the data sensitivity, represents the sensitivity base value, represents the sensitivity base minimum value, represents the sensitivity base maximum value, represents the data volume factor, represents the risk degree factor, represents the ease of use factor, , and represents the corresponding weight coefficient.

[0037] Preferably, the value evaluation index value comprises the following steps:

[0038] The data usage frequency, the data key ratio and the data sensitivity are added to obtain the value evaluation index value, and the value evaluation index value calculation formula is:

[0039]

[0040] wherein, represents the value evaluation index value, represents the data usage frequency, represents the data key ratio, represents the data sensitivity;

[0041] The value evaluation index values of different data types are collected, and the ascending order function in the sorting of the electronic spreadsheet software is used to sort them respectively, and according to the distribution of the sorted data, the division standards of three levels are determined, and the 33% and 67% quantiles are taken as the division points;

[0042] According to the determined division mark, the value evaluation index values of different data types are divided into levels:

[0043] The value evaluation index value between 0%-33% quantile is taken as the first-level data;

[0044] The value evaluation index value between 34%-67% quantile is taken as the second-level data;

[0045] The value evaluation index value between 68%-100% quantile is taken as the third-level data.

[0046] Preferably, the processed data is sent to the cloud storage resource layer for storage, the data is distributed according to the value evaluation index value, and the data is controlled and backed up regularly according to the preset access control backup strategy, comprising the following steps:

[0047] For structured data corresponding to the first level data using high-performance database service, the second level data using standard database service, the third level data using low-cost database service; for unstructured data corresponding to the first level data stored to the high-performance layer of object storage, the second level data stored to the standard layer of object storage, the third level data stored to the low-cost layer of object storage; for semi-structured data corresponding to the first level data using high-performance NoSQL database, the second level data using standard NoSQL database, the third level data using low-cost NoSQL database;

[0048] At the same time, for all data types corresponding to the first level data, strict access control and high-frequency backup are configured, the second level data is set to medium intensity access control and regular backup, and the third level data is set to basic access control and low-frequency backup.

[0049] The beneficial effects of the present application are:

[0050] The present application extracts and analyzes the text, image and audio data in the unstructured data respectively to obtain the numerical representation of the text, image and audio, extracts and analyzes the JOSN and XML format data in the semi-structured data to obtain the key numerical representation, and combines the features in the structured data to construct a data type classification model based on neural network to classify the data in real time, realizes accurate feature extraction for different data types, learns the association and difference between different features, and thus more accurately classifies the data types.

[0051] The present application obtains the data usage frequency by taking the average value of the average number of daily usage and the average value of the data usage time, constructs a hierarchical model, the highest layer is the overall goal of the enterprise, the middle layer is the sub-goal of each business field, and the bottom layer is the key success factor of each sub-goal, deploys it to the server, finds out the target sequence which plays a decisive role in the overall goal as the key target path in combination with the target dependency relationship and the key guide method, obtains the data key proportion according to the proportion value occupied by the target on the key directory path, obtains the data sensitivity degree through normalization and multiple factor combination of dividing data information type and formulating rule allocation sensitivity basic value, adds the data usage frequency, the data key proportion and the data sensitivity degree to obtain the value evaluation index value, divides the grades of different data types according to the value evaluation index value and stores them in the corresponding database, realizes multi-dimensional calculation of the value evaluation index value, the data usage frequency reflects the application frequency of the data in daily business; the data key proportion reflects the importance of the data in the key target path; the data sensitivity degree focuses on the security and sensitivity of the data; through this multi-dimensional comprehensive evaluation, the value of the data can be more comprehensively and objectively quantified, and the one-sidedness caused by single-dimensional evaluation can be avoided. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The module flowchart of the present application is shown. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0054] Please refer to Figure 1 The present application is a cloud server data storage management system, which comprises the following modules:

[0055] The data collection module: deploy structured data collection unit, unstructured data collection unit and semi-structured data collection unit to collect data;

[0056] The data classification module: build a classification model to classify the data types in the three collection units;

[0057] The data processing and analysis module: after analyzing and calculating the data usage frequency, data key ratio and data sensitivity of the three data types respectively, add them to obtain the value evaluation index value of the corresponding data type;

[0058] The data storage and backup module: send the processed data to the cloud storage resource layer for storage, allocate the data according to the value evaluation index value, and control and backup the data regularly according to the preset access control backup strategy.

[0059] Specifically, deploy structured data collection unit, unstructured data collection unit and semi-structured data collection unit to collect data; extract features from the fields in structured data, text values, image values and audio values in unstructured data, and key fields in semi-structured data after analysis, input the extracted features into the model to build a data type classification model; get the data usage frequency by dividing the total number of data usage times by the total number of days and dividing the data usage time by the total time, judge whether the corresponding key success factor is located on the key target path, and count the number of nodes of the factor located on the key target path and the total number of nodes of the key target path, then divide them to get the data key ratio, normalize the sensitivity base value of different data information types, and then get the data sensitivity of the corresponding data information type by combining the data volume factor, risk factor and ease of use factor, add the data usage frequency, data key ratio and data sensitivity to get the value evaluation index value; store the different data types after grading according to the value evaluation index value, including first-level data, second-level data and third-level data.

[0060] In one embodiment of the present application, the deployment of the structured data acquisition unit, the unstructured data acquisition unit and the semi-structured data acquisition unit acquires data, including the following steps:

[0061] The data acquisition module is constructed by three types of data acquisition units, the structured data acquisition unit is connected to the existing business database of the enterprise, and the structured data generated by the business operation is captured in real time through the database interface;

[0062] The unstructured data acquisition unit deploys a file crawler program, and traverses and collects documents, pictures and audio resources in the enterprise internal file server and cloud storage;

[0063] The semi-structured data acquisition unit extracts key data elements by using corresponding parsers for configuration information and log files stored in a specified format, and includes semi-structured data in the unified acquisition system, wherein the specified format refers to XML and JSON formats.

[0064] Specifically, the connection information of the existing business database of the enterprise is determined, the corresponding database connection driver is used, the connection with the business database is established in the data acquisition program, the database tables, fields and data change events involved in the structured data generated by the business operation that needs to be captured are determined according to the business requirements, and the captured structured data is transmitted to the temporary storage area of the data acquisition module; the documents, pictures and audio in the enterprise internal file server and cloud storage are determined, a suitable file crawler framework is selected, the crawler program is written according to the resource storage path and directory structure, the file crawler program is started, the file resources in the enterprise internal file server and cloud storage are traversed from the specified starting path, the file type is judged according to the extension name, and the extracted file content or basic information is organized according to the set data format; according to the technical architecture and business system deployment of the enterprise, the server location, directory path and file naming rules of the stored configuration information and log files are determined, the suitable XML parser is selected for the XML format configuration information and log files, the corresponding JSON parsing library is used for the JSON format files, the parsing rules and scripts are written according to the structure characteristics of the configuration files and log files, the parsing program is started, the positioned XML and JSON format files are parsed according to the parsing rules, the key data elements are extracted from the parsing results, and the extracted key data elements are organized according to the pre-designed data structure and format, so as to be included in the unified acquisition system.

[0065] In one embodiment of the present application, the construction of the classification model classifies the data in the three acquisition units, including the following steps:

[0066] Cleaning and formatting operations are performed on structured data;

[0067] Text is converted into numerical vectors using a pre-trained Word2Vec model; multi-scale features are extracted from images using a convolutional neural network, and fusion and normalization operations are performed; time-domain, frequency-domain, and time-frequency domain features are extracted from audio, and principal component analysis is used for dimensionality reduction and fusion;

[0068] JSON in the obtained semi-structured data is flattened; XML data is converted into table form;

[0069] A semi-automatic labeling tool is used to label the processed data, and features are extracted from the labeled data, including the fields of structured data, numerical representations of unstructured data, and key fields of semi-structured data. A data type classification model is built using a neural network-based framework, the extracted features are input into the model for training, and the trained model is deployed to a server for real-time data classification.

[0070] Specifically, the collected text data is preprocessed and segmented, a pre-trained Word2Vec model is loaded, the average value of all word vectors is calculated to obtain the numerical representation vector of the text, which is used as the feature vector of the text data for subsequent processing; the collected image data is uniformly adjusted to a fixed size, such as 224x224 pixels, and normalized, mapping the pixel value range to the interval [0, 1] or [-1, 1], a pre-trained convolutional neural network is selected as the backbone network, the resized and normalized image is input into the backbone network, feature maps are extracted from different layers of the backbone network, deep features are upsampled, then the upsampled deep features are added to the corresponding shallow features to obtain the fused feature maps, and multiple scale feature maps are output, all scale feature maps are spliced together, weighted sum is performed on feature maps of different scales, the fused features are normalized to make the numerical range consistent, and the numerical representation vector of the image is generated; an audio processing library is used to load an audio file, time domain features, frequency domain features and time-frequency domain features of the audio are extracted, the extracted features of various types are normalized to eliminate dimensional differences, the processed multiple features are spliced to form a complete audio feature vector, weights are set according to the importance of different features, weighted sum is performed on the spliced feature vector, then principal component analysis is used for further dimension reduction and feature fusion to obtain the final audio numerical representation vector; the extracted JSON data key-value pairs are cleaned, nested fields are flattened, for example, for JSON configuration information containing multiple levels of nesting, the nested structure is expanded into a single layer of key-value pair form, facilitating subsequent processing and analysis, after the XML data content is extracted using an XPath tool, it is converted into a table form according to the established rules, such as converting XML format log data into a two-dimensional table form, each row representing a log record, and each column representing different data fields; a detailed annotation guide is developed to clearly define the annotation content and rules of various data. For example, for structured data, the data type and specific meaning of each field are annotated; for text values, image values and audio values in unstructured data, the corresponding numerical feature vectors are annotated; for key fields parsed in semi-structured data, the semantics and role in the original data are annotated, a suitable neural network framework is selected to build a data type classification model, the extracted structured data features, unstructured data features and semi-structured data features are used as the input of the model, and the classification result of the data is output by the model, after the classification model is trained, a suitable cloud computing server is selected for model deployment.

[0071] In one embodiment of the present application, the data usage frequency comprises the following steps:

[0072] The value evaluation indexes of the structured data, the unstructured data and the semi-structured data are calculated respectively, wherein the value evaluation indexes include data usage frequency, data key ratio and data sensitivity;

[0073] In the time range of working days in the past month, the number of times and the time of calling, viewing, processing and using other ways of different types of data sets are counted respectively, and the total number of times and the total time of use are obtained by adding respectively, and the data usage frequency is obtained by combining the total number of data usage times divided by the total number of days and the data usage time divided by the total time;

[0074] Data usage frequency calculation formula:

[0075]

[0076] Among them, represents the data usage frequency, represents the number of times of calling, represents the number of times of viewing, represents the number of times of processing, represents the number of times of using other ways, represents the total number of days of working days in the past month, represents the time of calling, represents the time of viewing, represents the time of processing, represents the time of using other ways, represents the total time of working days in the past month.

[0077] Specifically, it is determined that the structured data set, the unstructured data set and the semi-structured data set are involved, the specific time point and the time length of each data set called, viewed, processed and used in other ways in the time range of domestic working days in the past one month are recorded in detail through system logs, built-in log functions of application programs or manual records, and it is ensured that the recorded information is accurate and complete, including date, time, data set type, use mode, use time length and the like. The collected record data can be recorded in the form of a table, and different types of data sets are counted respectively. For each data set type, the total number of times and the total time of calling, viewing, processing and using in other ways in the working days in the past one month are calculated, the total number of days of working days in the past one month is determined, and the total time length of all data sets used in the working days in the past one month is calculated. The total time length of each data set type obtained by the foregoing statistics is added to obtain the total data use time. For each data set type, the total number of times of using in the past one month is divided by the total number of days of working days to obtain the average number of times of using per day. The total time of using each data set type in the past one month is divided by the total time of using all data sets to obtain the percentage of the data set type use time in the total time. The two calculation results, i.e. the average number of times of using per day and the data use time percentage, are comprehensively considered to determine the data use frequency.

[0078] In one embodiment of the present application, the data key ratio includes the following steps:

[0079] The relevant business targets are collected from the strategic planning documents, annual reports and business department plans of the enterprise, the collected targets are classified and arranged, the collected targets are classified according to the time dimension, including short-term targets, medium-term targets and long-term targets, each business target is decomposed into a plurality of measurable sub-targets, and the key success factors of each sub-target are determined; a hierarchical structure model is constructed, the highest layer is the overall target of the enterprise, the middle layer is the sub-targets of each business field, and the bottom layer is the key success factors of each sub-target, the targets of different layers and the corresponding time attributes are marked, the marked data for the training set are used to train the hierarchical structure model, and the trained model is deployed to the server;

[0080] The dependency relationship between the targets of different layers is determined, the target dependency relationship is analyzed, the target sequence which plays a decisive role in the overall target is found out as the key target path in combination with the key path method;

[0081] The collected data is mapped with each target and key success factor in the target hierarchy, it is judged whether the corresponding key success factor is located on the key target path, and the node number of the factor located on the key target path and the total node number of the key target path are counted, and the data key ratio is obtained by dividing the two.

[0082] Data key ratio calculation formula:

[0083]

[0084] wherein, the data key ratio, the node number located on the key target path, the total node number of the key target path.

[0085] Specifically, according to the time span, the target is divided into short-term within one year, medium-term one to three years, long-term more than three years, for example, a target is "the sales growth of this quarter is 10%", which belongs to the short-term goal; "launch 3 new products in the next 2 years" is a medium-term goal; "become the top three in the industry market share within 5 years" is a long-term goal, detailed records are made for each target, including target content, business field, time attribute and other information, forming a preliminary target list, for each business target, it is decomposed into multiple measurable sub-targets. For example, for the target of "increasing market share", it can be decomposed into sub-targets such as "increasing the number of new customers", "improving customer loyalty", "expanding new sales channels", etc. For each sub-target, the key success factors required to achieve the target are analyzed. For example, for the sub-target of "increasing the number of new customers", the key success factors may include "advertising effect", "sales team lead conversion rate", "product competitiveness", etc. A hierarchical model is constructed, the highest layer is the overall target of the enterprise, the middle layer is the sub-targets of each business field, and the bottom layer is the key success factors of each sub-target. The ownership relationship between each layer is clear in the model, the time attribute of different layers of targets is marked, the relevant data is collected for the training set, including historical business data and market trend data, the labeled data is trained on the hierarchical model, and the trained model is deployed to the server for use in the actual business environment; the dependency relationship between different levels of targets is analyzed, that is, whether the realization of a target depends on the completion of other targets. For example, "launching new products" may depend on "research and development success" and "production process improvement" targets, etc. Combined with the key path method, the target sequence that plays a decisive role in the overall target is found as the key target path from numerous target sequences, the collected data is mapped to each target and key success factor in the target hierarchy, and the position of the key success factor in the key target path is determined by comparing the key success factor and the target in the key target path. The number of nodes of the factor located on the key target path and the total number of nodes of the key target path are counted, and the data key ratio is obtained by dividing the two.

[0086] In one embodiment of the present application, the data sensitivity degree comprises the following steps:

[0087] The data information types are divided into personal identity information, financial information, health information and business secrets, a sensitivity base value of different data information types is defined, a data volume factor is assigned according to a specified rule to a proportion value of a data volume of different data information types in a total data volume, a risk degree factor is assigned according to a severity degree of leakage and theft of different data information types, and an ease of use factor is assigned according to an ease of access and use of different data information types, and the sensitivity base value of different data information types is normalized and combined with the data volume factor, the risk degree factor and the ease of use factor to obtain a data sensitivity degree of a corresponding data information type;

[0088] Data sensitivity degree calculation formula:

[0089]

[0090] wherein, represents the data sensitivity degree, represents the sensitivity base value, represents the minimum sensitivity base value, represents the maximum sensitivity base value, represents the data volume factor, represents the risk degree factor, represents the ease of use factor, , and represent corresponding weight coefficients.

[0091] Specifically, the data is explicitly divided into four types of personal identity information, financial information, health information and business secrets, and a sensitivity base value is defined for each data information type according to the general sensitive characteristics of the data, for example, the minimum sensitivity is set to 1 and the maximum sensitivity is set to 10, assuming that the personal identity information sensitivity base value is 7, the financial information is 9, the health information is 8, and the business secret is 10, which reflects the sensitivity of the data type itself without being affected by other factors; determine the proportion of the amount of data of different data information types in the total amount of data. This proportion can be obtained through data analysis and statistics, and these proportion values are converted into data quantity factors according to certain proportion rules, for example, the proportion value can be directly used as the data quantity factor, the severity of the leakage and misuse of different data information types is evaluated, and the risk degree factor is allocated, for example, between 0.1-1, assuming that the personal identity information risk degree factor is 0.7, the financial information is 0.9, the health information is 0.8, and the business secret is 1, the ease of access and use of different data information types is analyzed to allocate the ease of use factor, which is allocated to each data information type within a certain numerical range, such as 1 to 10, the larger the value, the easier to access and use, assuming that the personal identity information ease of use factor is 6, the financial information is 4, the health information is 3, and the business secret is 2, the sensitivity base value of different data information types is normalized for joint calculation with other factors, and the normalized sensitivity base value is combined with the data quantity factor, the risk degree factor and the ease of use factor to obtain the data sensitivity of the corresponding data information type; wherein, the value of a is 0.3, the value of b is 0.4, and the value of c is 0.3.

[0092] In one embodiment of the present application, the value evaluation index value comprises the following steps:

[0093] The data usage frequency, data key ratio and data sensitivity are added to obtain the value evaluation index value, and the value evaluation index value calculation formula is:

[0094]

[0095] wherein, represents the value evaluation index value, represents the data usage frequency, represents the data key ratio, represents the data sensitivity;

[0096] The value evaluation index values of different data types are collected, sorted in ascending order by using the ascending function in the sorting of the spreadsheet software, and the three-level division standards are determined according to the data distribution after sorting, and the 33% and 67% quantiles are used as the division points;

[0097] According to the determined division standard, the value evaluation index values of different data types are divided into levels:

[0098] The value evaluation index values between 0%-33% quantile are first-level data;

[0099] The value evaluation index values between 34%-67% quantile are second-level data;

[0100] The value evaluation index values between 68%-100% quantile are third-level data.

[0101] Specifically, the value evaluation index values of different data types are entered into the spreadsheet software, the column where the value evaluation index values are located is selected, the'sorting' button in the 'data' tab is clicked, the 'ascending' sorting method is selected in the pop-up dialog box, and then the 33% and 67% quantiles of the sorted data are calculated, and the calculated 33% and 67% quantiles are used as the division points of the three levels, the first-level data represents high value, high sensitivity and high frequency of use; the second-level data represents medium value, medium sensitivity and medium frequency of use; the third-level data represents low value, low sensitivity and low frequency of use, and the value evaluation index values of each data type are determined according to the determined value evaluation index values and the level division standard.

[0102] In one embodiment of the present application, the processed data is sent to the cloud storage resource layer for storage, the data is distributed according to the value evaluation index value, and the data is controlled and backed up regularly according to the preset access control backup strategy, including the following steps:

[0103] The first-level data corresponding to structured data uses high-performance database services, the second-level data uses standard database services, and the third-level data uses low-cost database services; the first-level data corresponding to unstructured data is stored in the high-performance layer of object storage, the second-level data is stored in the standard layer of object storage, and the third-level data is stored in the low-cost layer of object storage; the first-level data corresponding to semi-structured data uses high-performance NoSQL database, the second-level data uses standard NoSQL database, and the third-level data uses low-cost NoSQL database;

[0104] At the same time, the first-level data corresponding to all data types is set with strict access control and high-frequency backup, the second-level data is set with medium-intensity access control and regular backup, and the third-level data is set with basic access control and low-frequency backup.

[0105] Specifically, a suitable high-performance relational database is selected, the selected database is installed and configured on a high-performance server, initialization settings are made, including creating a database instance, allocating storage space, configuring memory and CPU resources, etc., existing first-level structured data is migrated from the original storage location to the newly deployed high-performance database, database user groups with different permissions are created, specific access permissions are assigned to each user group, a high-frequency backup strategy is configured, such as performing a full backup every day, while performing an incremental backup every few hours; a standard relational database suitable for business needs is selected, the database is installed and configured on a standard configuration server, necessary initialization settings are made, such as setting the character set, creating a database user, etc., second-level structured data is migrated to the standard database, a user group with medium-strength access permissions is created, such as limiting access to a specific IP address segment, encrypting some sensitive fields, etc. Access control measures, configure a regular backup strategy, such as performing a full backup once a week; install and configure the database on a server resource with lower cost, make basic initialization settings, migrate third-level structured data to the low-cost database, set basic access control, such as a simple username and password verification mechanism, configure a low-frequency backup strategy, such as performing a full backup once a month.

[0106] The above examples are only used to illustrate the technical method of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present application.

Claims

1. A cloud server-based data storage management system, characterized in that, Includes the following modules: Data acquisition module: Deploys structured data acquisition units, unstructured data acquisition units, and semi-structured data acquisition units to acquire data; Data classification module: Constructs a classification model to classify the data types within the three collection units; Data processing and analysis module: After analyzing and calculating the data usage frequency, key ratio, and sensitivity of the three data types respectively, the values ​​are added together to obtain the value evaluation index value of the corresponding data type. Data storage and backup module: Sends the processed data to the cloud storage resource layer for storage, allocates the data according to the value evaluation index, and controls and backs up the data periodically according to the preset access control and backup policies; The key ratio of the data includes the following steps: Relevant business objectives are collected from the company's strategic planning documents, annual reports, and business department plans. These objectives are then categorized and organized according to a time dimension, including short-term, medium-term, and long-term objectives. Each business objective is broken down into multiple measurable sub-objectives, and the key success factors for each sub-objective are identified. A hierarchical model is constructed, with the company's overall objectives at the top, the sub-objectives for each business area in the middle, and the key success factors for each sub-objective at the bottom. Objectives at different levels and their corresponding time attributes are labeled. The labeled training data is used to train the hierarchical model, and the trained model is deployed to a server. Determine the dependencies between objectives at each level, analyze these dependencies, and use the key-guide method to identify the sequence of objectives that plays a decisive role in the overall objective as the key objective path. The collected data is mapped to each target and key success factor in the target hierarchy. It is determined whether the corresponding key success factor is located on the key target path. The number of nodes on the key target path and the total number of nodes on the key target path are counted and divided to obtain the data key ratio. Formula for calculating key ratios of data: in, Indicates the key ratio of the data. This indicates the number of nodes located on the critical target path. This indicates the total number of nodes on the critical target path.

2. The cloud server-based data storage management system according to claim 1, characterized in that, The deployment of structured data acquisition units, unstructured data acquisition units, and semi-structured data acquisition units to acquire data includes the following steps: The data acquisition module is constructed from three types of data acquisition units. The structured data acquisition unit connects to the enterprise's existing business database and captures the structured data generated by business operations in real time through the database interface. The unstructured data acquisition unit deploys a file crawler program to traverse and collect documents, images, and audio resources in the enterprise's internal file servers and cloud storage. The semi-structured data acquisition unit uses a corresponding parser to extract key data elements from configuration information and log files stored in a specified format, and incorporates the semi-structured data into a unified acquisition system. The specified format refers to XML and JSON formats.

3. The cloud server-based data storage management system according to claim 1, characterized in that, The classification model is constructed to classify data within the three types of collection units, including the following steps: Clean and format structured data; The text is converted into numerical vectors using a pre-trained Word2Vec model; multi-scale features are extracted from images using a convolutional neural network, followed by fusion and normalization operations; and time-domain, frequency-domain, and time-frequency-domain features are extracted from audio and then dimensionality-reduced and fused using principal component analysis. Flatten the JSON data in the acquired semi-structured data; convert the XML data into a table format; Semi-automatic annotation tools are used to annotate the processed data, extract features from the annotated data, including fields of structured data, numerical representations of unstructured data, and key fields of semi-structured data. A data type classification model is built using a neural network-based framework, the extracted features are input into the model for training, and the trained model is deployed to the server for real-time data classification.

4. The cloud server-based data storage management system according to claim 1, characterized in that, The frequency of data usage includes the following steps: The value evaluation indicators are calculated separately for structured data, unstructured data, and semi-structured data. The value evaluation indicators include data usage frequency, data key ratio, and data sensitivity. Within the past month's working days, the number of times and time that different types of datasets were accessed, viewed, processed, and used in other ways were counted. The total number of times and the total time of use were then added together to obtain the data usage frequency. The data usage frequency was obtained by combining the total number of times the data was used by the total number of days and the data usage time by the total time. Formula for calculating data usage frequency: in, Indicates the frequency of data usage. Indicates the number of times it has been called. This indicates the number of times the content has been viewed. Indicates the number of times it was processed. Indicates the number of times other methods were used. This indicates the total number of working days in the past month. Indicates the time when it was invoked. Indicates the time when it was viewed. Indicates the time it was processed. Indicate the time when other methods are used. This indicates the total number of working days in the past month.

5. A cloud server-based data storage management system according to claim 1, characterized in that, The data sensitivity assessment includes the following steps: Data information types are categorized into personal identity information, financial information, health information, and trade secrets. Sensitivity baseline values ​​are defined for different data information types. Data volume factors are assigned based on the proportion of each data information type in the total data volume according to specified rules. Risk factors are assigned based on the severity of leakage and misuse of different data information types. Usability factors are assigned based on the ease of access and use of different data information types. The sensitivity baseline values ​​for different data information types are normalized and then combined with the data volume factor, risk factor, and usability factor to obtain the data sensitivity of the corresponding data information type. Formula for calculating data sensitivity: in, Indicates the degree of data sensitivity. This represents the baseline sensitivity value. This represents the basic minimum value of sensitivity. This represents the base maximum value of sensitivity. Indicates the data volume factor. Indicates the risk factor. Indicates the ease of use factor. , and This represents the corresponding weighting coefficient.

6. A cloud server-based data storage management system according to claim 1, characterized in that, The value evaluation index includes the following steps: The value assessment index is obtained by adding the data usage frequency, the data key ratio, and the data sensitivity. The formula for calculating the value assessment index is as follows: in, Indicates the value of the evaluation index. Indicates the frequency of data usage. Indicates the key ratio of the data. Indicates the degree of data sensitivity; Collect value evaluation index values ​​of different data types, sort them in ascending order using the sorting function of spreadsheet software, and determine the three-level division criteria based on the distribution of sorted data, using the 33% and 67% quantiles as the dividing points. Based on the established classification labels, the value evaluation index values ​​of different data types are classified into different levels: Values ​​of the evaluation index between the 0% and 33% percentiles are used as primary data. Values ​​of the evaluation indexes between the 34% and 67% percentiles are used as secondary data; Evaluation values ​​between the 68% and 100% percentiles are used as third-level data.

7. A cloud server-based data storage management system according to claim 1, characterized in that, The process of sending the processed data to the cloud storage resource layer for storage, allocating the data according to value evaluation indicators, and periodically controlling and backing up the data according to a preset access control and backup strategy includes the following steps: For structured data, high-performance database services are used for Level 1 data, standard database services are used for Level 2 data, and low-cost database services are used for Level 3 data. For unstructured data, Level 1 data is stored in the high-performance layer of object storage, Level 2 data is stored in the standard layer of object storage, and Level 3 data is stored in the low-cost layer of object storage. For semi-structured data, high-performance NoSQL databases are used for Level 1 data, standard NoSQL databases are used for Level 2 data, and low-cost NoSQL databases are used for Level 3 data. At the same time, strict access control and high-frequency backup are set for all data types corresponding to Level 1 data, medium-intensity access control and regular backup are set for Level 2 data, and basic access control and low-frequency backup are set for Level 3 data.

Images