Storage management method and system based on data encryption

Abstract

The invention relates to the technical field of data storage, and discloses a storage management method and system based on data encryption, and the method comprises the steps: obtaining a model sample set, building a feature thermodynamic diagram based on the model sample set, determining a key feature cluster based on the feature thermodynamic diagram, carrying out the decoupling of the key feature cluster according to an auto-encoder, determining a model data set, and storing the model data set in a database. To-be-stored classification data are determined based on a multi-head self-attention model, each piece of to-be-stored classification data is layered based on analytic hierarchy process, a judgment matrix is constructed, and corresponding encryption strategies are adopted for all the to-be-stored classification data based on a data comprehensive score to determine encrypted data. The temporarily stored encrypted data is moved to the target sub-memory according to the data verification result, the backup mode of the target sub-memory is determined based on the storage occupancy rate, whether the backup mode is adjusted or not is judged based on the data retrieval condition, and the stability and reliability of data storage management are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a storage management method and system based on data encryption. Background Art

[0002] With the rapid development of information technology and the deepening of digitalization, all types of data continue to grow at an unprecedented rate, covering a wide range of areas, including personal information, corporate business data, government data, medical records, and financial transaction information. The effective storage and management of data is crucial for personal privacy protection and business operational decision-making. However, due to the wide range of data sources and diverse types, and different data have different attributes, existing data storage lacks an intelligent classification mechanism. It usually relies on manual classification, which is difficult to adapt to the dynamic changes of data, resulting in chaotic data classification and unclear levels. It not only increases the difficulty of data retrieval and management and reduces data utilization efficiency, but may also cause data misuse due to classification errors. In addition, there are a large number of security threats such as malicious attacks and illegal intrusions during the storage process. Data is easily stolen, tampered with or deleted, causing data leakage or damage, thereby affecting data backup and leading to the risk of data loss.

[0003] Therefore, it is necessary to design a storage management method and system based on data encryption to solve the problems existing in the current technology. Summary of the Invention

[0004] In view of this, the present invention proposes a storage management method and system based on data encryption, which aims to solve the problem of relying on manual classification, being difficult to adapt to dynamic changes in data, resulting in chaotic data classification and unclear hierarchy, and security threats in the storage process causing data leakage or damage, thereby affecting data backup and leading to data loss.

[0005] In one aspect, the present invention provides a storage management method based on data encryption, comprising: Acquire all data to be stored and preprocess each data to be stored to determine target data to be stored, acquire a model sample set and construct a feature heat map based on the model sample set; Determine a key feature cluster based on the feature heat map, decouple the key feature cluster according to the autoencoder to determine a model data set, build a multi-head self-attention model based on the model data set, and determine the classification data to be stored based on the multi-head self-attention model; Based on hierarchical analysis, each classified data to be stored is layered and a judgment matrix is ​​constructed. The judgment matrix is ​​checked for consistency to determine the comprehensive score of the data. Based on the comprehensive score of the data, a corresponding encryption strategy is adopted for all the classified data to be stored to determine the encrypted data. All encrypted data with the same classification attribute are temporarily stored in a sub-memory, and data verification is performed in the sub-memory. Based on the result of the data verification, the temporarily stored encrypted data is moved to the target sub-memory; Obtain the storage occupancy rate of the target sub-memory, determine the backup mode of the target sub-memory based on the storage occupancy rate, obtain the data retrieval status of the target sub-memory under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or the adjustment result.

[0006] Furthermore, when all the data to be stored are acquired and each data to be stored is pre-processed to determine the target data to be stored, the following steps are included: Perform data cleaning on all data to be stored, including processing missing values, outliers, and duplicate values, and perform data conversion on all cleaned data to be stored, including data standardization and data normalization, verifying the integrity and consistency of each data to be stored after data conversion, and determining whether it conforms to the storage format; If the verification passes and complies with the storage format, all the data to be stored are determined as the target data to be stored; If the verification fails or does not conform to the storage format, the data to be stored that fails the verification or does not conform to the storage format will be retained and prompted for manual verification, and the remaining data to be stored will be determined as the target data to be stored.

[0007] Furthermore, when obtaining a model sample set and constructing a feature heat map based on the model sample set, the method includes: Determine all feature dimensions of the model sample set, and determine the distribution probability of each feature dimension in each category, and determine the entropy value of each feature dimension based on the distribution probability, wherein the entropy value is proportional to the distribution difference; A matrix is ​​constructed with the feature dimension as the horizontal axis and the category as the vertical axis. The feature heat map is constructed based on the data visualization library and the matrix. The color depth of each cell in the feature heat map is determined by the entropy value of the corresponding feature in the category.

[0008] Furthermore, when determining key feature clusters based on the feature heat map and decoupling the key feature clusters according to the autoencoder to determine the model data set, the method includes: Determine a classification feature based on the color depth and calculate a feature variance of the classification feature, and determine the key feature cluster based on the feature variance; Determine the central eigenvector of each category, determine the cosine similarity based on the central eigenvectors of any two categories, obtain the average value of the cosine similarities between all two categories, and determine the confusion category based on the average value; Construct the autoencoder, which includes a three-layer encoder and a three-layer decoder symmetrical to the three-layer encoder, and the input layer dimension of each encoder is consistent with the dimension of the key feature cluster. Based on the autoencoder, the key feature clusters contained in the confusion category are decoupled, and the sample data of the decoupled key feature clusters are determined as the model data set.

[0009] Furthermore, when a multi-head self-attention model is constructed based on the model data set, and the classified data to be stored is determined based on the multi-head self-attention model, the method includes: Dividing the model dataset into a model training set and a model test set, selecting an initial multi-head model based on Transformer, training the initial multi-head model according to the model training set, and substituting the model test set into the trained initial multi-head model for testing; If the test value of the currently trained initial multi-head model is greater than or equal to the test value of the previously trained initial multi-head model, the training is stopped, and the currently trained initial multi-head model is determined as the multi-head self-attention model; If the test value of the currently trained initial multi-head model is less than the test value of the previously trained initial multi-head model, adjust the learning rate of the currently trained initial multi-head model and continue training until the test value of the currently trained initial multi-head model is greater than or equal to the test value of the previously trained initial multi-head model; Substitute all target data to be stored into the multi-head self-attention model to determine the classified data to be stored.

[0010] Furthermore, when each of the classified data to be stored is stratified based on hierarchical analysis and a judgment matrix is ​​constructed, and a consistency check is performed on the judgment matrix to determine a comprehensive score of the data, and when a corresponding encryption strategy is adopted for all the classified data to be stored based on the comprehensive score of the data to determine the encrypted data, the method includes: Determine a hierarchical structure based on the hierarchical analysis, the hierarchical structure including a target layer, a criterion layer, and a solution layer, the target layer determines the comprehensive score of the data, the criterion layer determines the data attributes of the classified data to be stored, and the solution layer determines the role carrier of the data attributes, the data attributes including data size, data structure, data association, data compression, and data source; Constructing the judgment matrix based on the data attributes, calculating the weight vector and the maximum eigenvalue based on the judgment matrix, determining the consistency index and the average consistency index based on the maximum eigenvalue, determining the consistency ratio based on the ratio of the consistency index to the average consistency index, and determining the data comprehensive score of each classified data to be stored based on the consistency ratio; When the data comprehensive score is greater than the data comprehensive score threshold, the encryption strategy is determined to be a composite encryption strategy, and the encrypted data is determined according to the composite encryption strategy; When the data comprehensive score is less than or equal to the data comprehensive score threshold, the encryption strategy is determined to be an independent encryption strategy, and the encrypted data is determined according to the independent encryption strategy.

[0011] Furthermore, when all encrypted data of the same classification attribute are temporarily stored in a sub-memory, data verification is performed in the sub-memory, and the temporarily stored encrypted data are moved to a target sub-memory according to the result of the data verification, the method includes: The data verification includes security detection and structure verification. When the security detection passes, the structure verification is performed; When the security check passes and the structure check passes, the data check succeeds; otherwise, the data check fails. When the data verification succeeds, the encrypted data stored in the sub-memory is moved to the target sub-memory.

[0012] Furthermore, when obtaining the storage occupancy of the target sub-storage and determining the backup mode of the target sub-storage based on the storage occupancy, the method includes: When the storage occupancy rate satisfies a first condition, full backup is determined as the backup mode of the target sub-storage; When the storage occupancy rate satisfies a second condition, differential backup is determined as the backup mode of the target sub-storage; When the storage occupancy rate satisfies a third condition, determining incremental backup as the backup mode of the target sub-storage; The backup levels of the full backup, differential backup and incremental backup are successively lowered.

[0013] Furthermore, obtaining data retrieval status of the target sub-storage in the backup mode and determining whether to adjust the backup mode based on the data retrieval status include: When the backup mode of the target sub-storage is the differential backup or incremental backup, data retrieval information of the target sub-storage is obtained, the data retrieval information including the number of data retrievals and the data viewing duration; when the data retrieval information satisfies the data retrieval condition, it is determined to adjust the backup mode; otherwise, it is determined not to adjust the backup mode; When it is determined to adjust the backup mode, the backup level of the differential backup or incremental backup is increased by one level.

[0014] Compared with the prior art, the present invention has the following advantages: at the classification management level, intelligent data classification is achieved through feature heat maps and multi-head self-attention models, replacing the manual classification mode, accurately capturing the dynamic features and associated attributes of data, reducing the risk of classification confusion and unclear hierarchy, improving data retrieval efficiency and management stability, adapting to the rapid growth and dynamic change requirements of data, and ensuring the security and reliability of data storage management. The comprehensive data score is formed based on hierarchical analysis and consistency verification, and differentiated encryption strategies are implemented for different data. The encryption strength is enhanced to resist malicious attacks and illegal intrusions, while avoiding the waste of resources caused by excessive data encryption. While ensuring data security, it strikes a balance with storage efficiency, effectively reducing the probability of data leakage and damage, and strengthening privacy protection and data integrity. Temporary storage verification ensures the accuracy of data from classification to storage, avoids the risk of data damage caused by erroneous storage, and improves the reliability of data storage management. The backup mode is dynamically determined based on the storage occupancy rate and adjusted in real time based on the data retrieval situation, achieving accurate allocation of backup resources, ensuring the availability of storage backups, reducing the risk of data loss, and thus ensuring the reliability and stability of data storage management.

[0015] On the other hand, the present application also provides a data encryption-based storage management system for applying the above-mentioned data encryption-based storage management method, including: An acquisition and analysis module is configured to acquire all data to be stored and pre-process each data to be stored to determine target data to be stored, acquire a model sample set and construct a feature heat map based on the model sample set; a data classification module configured to determine key feature clusters based on the feature heat map, decouple the key feature clusters according to the autoencoder to determine a model data set, construct a multi-head self-attention model based on the model data set, and determine the classification data to be stored based on the multi-head self-attention model; a data processing module configured to stratify each of the classified data to be stored based on hierarchical analysis and construct a judgment matrix, perform a consistency check on the judgment matrix to determine a comprehensive data score, adopt a corresponding encryption strategy for all the classified data to be stored based on the comprehensive data score to determine encrypted data, temporarily store all encrypted data with the same classification attribute in a sub-memory, perform data verification in the sub-memory, and move the temporarily stored encrypted data to a target sub-memory based on the result of the data verification; The storage management module is configured to obtain the storage occupancy rate of the target sub-memory, determine the backup mode of the target sub-memory based on the storage occupancy rate, obtain the data retrieval status of the target sub-memory under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or the adjustment result.

[0016] It is understandable that the above-mentioned storage management method and system based on data encryption have the same beneficial effects, which will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings: Figure 1 A flowchart of a storage management method based on data encryption provided by an embodiment of the present invention; Figure 2 This is a functional block diagram of a storage management system based on data encryption provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, unless there is a conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0019] See Figure 1 As shown, in some embodiments of the present application, a storage management method based on data encryption includes: S100: Acquire all data to be stored and pre-process each data to be stored to determine target data to be stored, acquire a model sample set and construct a feature heat map based on the model sample set.

[0020] S200: Determine key feature clusters based on the feature heat map, decouple the key feature clusters according to the autoencoder to determine the model data set, build a multi-head self-attention model based on the model data set, and determine the classification data to be stored based on the multi-head self-attention model.

[0021] S300: Based on hierarchical analysis, each classified data to be stored is layered and a judgment matrix is ​​constructed, and the judgment matrix is ​​checked for consistency to determine the comprehensive score of the data. Based on the comprehensive score of the data, corresponding encryption strategies are adopted for all classified data to be stored to determine the encrypted data. All encrypted data with the same classification attributes are temporarily stored in a sub-memory, and data verification is performed in the sub-memory. According to the results of the data verification, the temporarily stored encrypted data is moved to the target sub-memory.

[0022] S400: Obtain the storage occupancy of the target sub-memory, determine the backup mode of the target sub-memory based on the storage occupancy, obtain the data retrieval status of the target sub-memory in the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or the adjustment result.

[0023] Specifically, in the process of using crawlers and other technologies to obtain all the data to be stored, network fluctuations and transmission processes may cause risks such as data changes to the data to be stored. All the data to be stored are preprocessed to remove the impact of network environment transmission and fluctuations. At the same time, the model sample set is used to construct a feature heat map. The model sample set includes 100 model sample data, including text data, video data, audio data, etc. Each type of data is divided into different fields. For example: video data includes: education field, medical field and technology field, etc. Based on these samples, a model sample set is established to construct a feature heat map. The feature heat map intuitively displays the distribution pattern of data features and the degree of correlation between features through color depth. Based on the feature heat map, key feature clusters are extracted and decoupled with the help of autoencoders. Complex features are decomposed into independent basic feature dimensions to form a structured model data set. The multi-head self-attention model is trained based on the model data set. The multi-head self-attention model is a model based on the attention mechanism. It captures the correlation features and dependencies of different dimensions of data through multiple parallel attention heads, thereby realizing automatic classification of the target data to be stored, getting rid of the dependence on manual operation. By combining the feature heat map with the multi-head self-attention model, it can dynamically capture data feature changes and automatically generate accurate classification results, avoiding the problems of traditional classification confusion and unclear hierarchy, and improving the efficiency of data retrieval and storage management. After classification by the multi-head self-attention model, hierarchical analysis is used to stratify each classified data to be stored according to dimensions such as data size and data structure. A judgment matrix is ​​constructed to quantify the weight of each level. The consistency test ensures the logical self-consistency of the weight distribution, and finally generates a comprehensive data score.

[0024] It can be understood that the data comprehensive score reflects the comprehensive indicators of the classified data to be stored. The larger the data comprehensive score, the more representative the classified data to be stored is and the more comprehensive the data content covered. Different encryption strategies are adopted based on the data comprehensive score. Composite encryption algorithms and lightweight encryption algorithms are used for different data comprehensive scores to balance the security and efficiency of storage management. The encrypted data is temporarily stored in the sub-memory for integrity and consistency verification. After eliminating errors in the transmission or encryption process, it is transferred to the target sub-memory that matches the classification attributes. While ensuring data security, it avoids resource waste caused by excessive encryption. In addition, the verification of the sub-memory further filters out abnormal data, reduces the risk of tampering and theft during data transmission, and reduces the hidden dangers of data misuse and leakage. The backup mode is selected according to the storage occupancy of the target sub-storage. The storage occupancy reflects the storage level of the target sub-storage. The higher the storage occupancy, the more encrypted data is stored in the target sub-storage. The backup mode of the target sub-storage is dynamically determined based on the storage occupancy. In addition, the backup mode is dynamically adjusted according to the data retrieval situation, which improves data availability in high-frequency access scenarios, balances storage costs and data security, reduces the risk of data loss caused by rigid backup strategies, and ensures efficient management of the entire data life cycle. In some embodiments of the present application, when all the data to be stored are obtained and each data to be stored is pre-processed to determine the target data to be stored, it includes: performing data cleaning on all the data to be stored, data cleaning includes processing missing values, abnormal values ​​and duplicate values, and performing data conversion on all the cleaned data to be stored, data conversion includes data standardization and data normalization, verifying the integrity and consistency of each data to be stored after data conversion, and judging whether it conforms to the storage format. If the verification passes and conforms to the storage format, all the data to be stored are determined as the target data to be stored. If the verification fails or does not conform to the storage format, the data to be stored that fails the verification or does not conform to the storage format will be retained and reminded for manual verification, and the remaining data to be stored will be determined as the target data to be stored.

[0025] Specifically, data cleaning is first used to improve the basic quality of data. Missing value processing avoids analysis bias due to incomplete data. Outlier processing eliminates interference from data that deviates from the normal range. Duplicate value processing is used to eliminate data redundancy. Data conversion uses data standardization and data normalization to eliminate differences in data magnitudes, facilitating the output of subsequent models. Data integrity is then verified to ensure that no key information is missing and that data logic is consistent. Data after data cleaning and data conversion is more reliable. At the same time, it is checked whether it meets the storage format specifications. If it meets the conditions, the data to be stored will be determined as the target data to be stored. If it does not meet the conditions, it will be retained and targeted through manual verification for correction. The remaining data to be stored that has passed the verification and meets the storage format is determined as the target data to be stored, reducing subsequent storage redundancy. At the same time, unqualified data to be stored are eliminated, saving storage space and ensuring the integrity of the target data to be stored.

[0026] In some embodiments of the present application, when obtaining a model sample set and constructing a feature heat map based on the model sample set, it includes: determining all feature dimensions of the model sample set, and determining the distribution probability of each feature dimension in each category, determining the entropy value of each feature dimension based on the distribution probability, the entropy value is proportional to the distribution difference, constructing a matrix with the feature dimension as the horizontal axis and the category as the vertical axis, constructing a feature heat map based on the data visualization library and the matrix, and the color depth of each cell in the feature heat map is determined by the entropy value of the corresponding feature in the category.

[0027] Specifically, the model sample set includes 100 model sample data, including text, video, and audio data. Each data type is further categorized into different domains. All feature dimensions of the model sample set are determined. Feature dimensions refer to specific information dimensions within the model sample set that reflect the data's categorical properties. Examples include file size ranges, modification timestamp patterns, and metadata keywords. These information dimensions are key to distinguishing different data categories. Using the entropy method and distribution probability, the entropy value of each feature dimension is calculated, and the distribution probability of each feature dimension in the i-th category is determined. Higher entropy values ​​indicate greater differences in the distribution of features across categories. For example, "data suffixes" are extremely unevenly distributed across text and image categories. Higher entropy values ​​indicate stronger ability to distinguish between different categories. By calculating their entropy values, we can determine their distribution differences across categories and identify features that are valuable for classification. In classification tasks, categories are the different types to be distinguished. For example, for text data, image data, and audio data, the "educational," "science and technology," and "medical" categories need to be distinguished, thereby identifying key features that can distinguish them. Preliminary screening is achieved by calculating the entropy value of each feature dimension. The higher the entropy value, the greater the distribution difference of the feature dimension in different categories, and the stronger the discrimination. According to the entropy value result, a matrix is ​​constructed with the feature dimension as the horizontal axis and the category as the vertical axis. With the help of the data visualization library (Matplotlib library), a feature heat map is generated, and the color depth is used to represent the entropy value of the corresponding feature in the category. Generally speaking, the color gradient (such as from blue to red) shows the size of the cell value and intuitively presents the strength of the association between the feature and the category. The depth of the cell color in the feature heat map is proportional to the entropy value of the corresponding position. The darker the color, the higher the entropy value, reflecting that the feature has a stronger discriminatory ability in the category.

[0028] It can be understood that by establishing a feature heat map to visualize the feature discrimination, the distribution probability represented by the abstract entropy value is converted into a matrix distribution of color depth, so that the distribution differences of the feature dimensions in each category are clear at a glance, which facilitates the rapid positioning of high-discrimination features (darker areas) and low-discrimination features (lighter areas), reduces the complexity of manual analysis, and improves the efficiency of feature screening. In addition, the entropy value is used to accurately quantify the distribution differences of features between categories, and the visual comparison of the feature heat map is used to enhance the perception of the differences, avoiding the deviation of a single evaluation method, ensuring that the screened features have objective discrimination capabilities, and making the subsequent feature decoupling targeted, thereby improving the efficiency and accuracy of model training.

[0029] In some embodiments of the present application, when determining key feature clusters based on feature heat maps and decoupling the key feature clusters according to an autoencoder to determine a model data set, it includes: determining classification features based on color depth and calculating the feature variance of the classification features, determining key feature clusters based on the feature variance, determining the central feature vector of each category, and determining the cosine similarity based on the central feature vectors of any two categories, and obtaining the average value of the cosine similarities between all two categories, determining the confusion category based on the average value, constructing an autoencoder, the autoencoder includes a 3-layer encoder and a 3-layer decoder symmetrical to the 3-layer encoder, and the input layer dimension of each encoder is consistent with the dimension of the key feature cluster, decoupling the key feature clusters contained in the confusion category based on the autoencoder, and determining the sample data of the decoupled key feature cluster as the model data set.

[0030] Specifically, the classification features are preliminarily located based on the color depth of the feature heat map. The higher the entropy value, the greater the distribution difference of the feature in different categories, and it has basic distinguishing ability. The feature dimensions with higher entropy values ​​are then determined as classification features. For example, if 20 feature dimensions are included, the calculated entropy values ​​are distributed between 0.2 and 1.8. The theoretical maximum value of the entropy value is log2(k), where k is the number of categories. For example, the maximum entropy value of 8 categories of data is 3, so the top 50%, that is, the features with entropy values ​​≥ 1.0, are taken as classification features. The feature variance of these classification features is further calculated to measure the degree of fluctuation of the features in the sample. Classification features with feature variance greater than 0.6 are determined as key feature clusters. These classification features not only have certain differences in distribution between categories, but also have fluctuations in their own values. For example, "data size" fluctuates in different ranges of 100MB-2GB and 10KB-10MB in video data and text data, respectively. The feature variance determines all specific values ​​of the data size in the model sample set, forming a numerical sequence, such as [5MB, 12MB, 8MB, 20MB]. Add all the values ​​in the sequence and divide by the total number of samples to obtain the mean of the classification feature. The total value of the sequence is 45MB, and the number of samples is 4, so the mean is 11.25MB. Calculate the difference between the feature value and the mean of each sample one by one, square these differences (for example, the difference between 5MB and the mean is -6.25MB, which is 39.06 after square). Then add all the squared values ​​to get the total sum. The result is the feature variance of the classification feature. The larger the feature variance, the more dramatic the fluctuation of the classification feature in different samples, and the more it can reflect the differences between different categories of data. Therefore, classification features with feature variance greater than 0.6 are identified as key feature clusters. This can further enhance the classification discrimination and provide an accurate input foundation for subsequent model training. In addition, calculate the central eigenvector of each category (the mean of the feature vectors of all samples in a category, which represents the overall feature trend of the category). For example, the mean of the "resolution" feature vector of the "image data in the education field" sample is (1920, 1080). This mean vector is the central eigenvector of "resolution" for "image data in the education field". Cosine similarity is used to quantify the similarity between the center vectors of any two categories (the closer the value is to 1, the more similar the category features are). The average cosine similarity between all pairs of categories is taken. If the average value is greater than 0.6, the category is considered confused. A confused category refers to a category combination with an average cosine similarity between the two pairs of categories greater than 0.6. Such categories exhibit high similarity in key feature clusters. For example, "Word data" and "PDF data" have similar distributions in the "text encoding" feature. Directly using these for training will lead to model classification confusion and require further decoupling. It can be understood that the autoencoder with a symmetrical structure of 3-layer encoder and 3-layer decoder is constructed. The 3-layer encoder is responsible for compressing high-dimensional key feature clusters into low-dimensional space, while the 3-layer decoder attempts to reconstruct the original features from the low-dimensional space. For the key feature clusters contained in the confusion category, the autoencoder removes redundant associations through low-dimensional mapping and retains discriminative differences to achieve feature decoupling. The sample data of the decoupled key feature clusters eliminates confusion interference and retains the core classification features, finally forming a model data set. The model data set contains screened and decoupled feature samples, which provides accurate basic input for model training.

[0031] In some embodiments of the present application, when constructing a multi-head self-attention model based on a model data set and determining the classified data to be stored based on the multi-head self-attention model, it includes: dividing the model data set into a model training set and a model test set, selecting an initial multi-head model based on Transformer, training the initial multi-head model according to the model training set, and substituting the model test set into the trained initial multi-head model for testing; if the test value of the currently trained initial multi-head model is greater than or equal to the test value of the previously trained initial multi-head model, the training is stopped, and the currently trained initial multi-head model is determined as the multi-head self-attention model; if the test value of the currently trained initial multi-head model is less than the test value of the previously trained initial multi-head model, the learning rate of the currently trained initial multi-head model is adjusted and training is continued until the test value of the trained initial multi-head model is greater than or equal to the test value of the previously trained initial multi-head model, and all target data to be stored are substituted into the multi-head self-attention model to determine the classified data to be stored.

[0032] Specifically, Transformer is a deep learning framework based on the self-attention mechanism that can construct an initial multi-head model. The initial multi-head model contains a model structure with multiple parallel attention heads. Each attention head independently calculates feature weights, aiming to capture complex relationships in the data and thus improve classification capabilities. The model dataset is divided into a model training set and a model test set, typically in a ratio of 7:3, to ensure the model's generalization ability. The model training set is used to train the initial multi-head model, while the model test set is used to evaluate the performance of the trained model. By training the initial multi-head model, it can accurately output the classification judgment of each target data to be stored, avoiding human judgment errors and thus improving the efficiency of storage management. The initial multi-head model is trained using data from the model training set. During each training session, the initial multi-head model attempts to learn patterns and relationships in the data to improve its prediction or classification capabilities. After each training session, the model is tested using data from the model test set. Test metrics include precision and recall, which are used to measure model performance. If the current post-training test value is less than the previous test value, it indicates that the model's performance has declined. In this case, the model's learning rate needs to be adjusted, and then training should continue. This will help the model steadily approach the global optimal solution. If the current post-training test value is greater than or equal to the previous test value, it indicates that the model's performance has improved or remained stable. Iterative training can be stopped, and it is considered that the model has reached a satisfactory performance level. At this point, the currently trained initial multi-head model is determined as the multi-head self-attention model, and all target data to be stored is substituted into the multi-head self-attention model to determine the classified data to be stored. This improves the accuracy of the classification of the target data to be stored and ensures the stability and reliability of storage management.

[0033] In some embodiments of the present application, when each to-be-stored classified data is stratified based on hierarchical analysis and a judgment matrix is ​​constructed, and a consistency check is performed on the judgment matrix to determine a data comprehensive score, and when a corresponding encryption strategy is adopted for all the to-be-stored classified data based on the data comprehensive score to determine the encrypted data, the method includes: determining a hierarchical structure based on the hierarchical analysis, the hierarchical structure includes a target layer, a criterion layer, and a scheme layer, the target layer determines the data comprehensive score, the criterion layer determines the data attributes of the to-be-stored classified data, the scheme layer determines the role carrier of the data attributes, the data attributes include data size, data structure, data association, data compression, and data source, constructing a judgment matrix based on the data attributes, calculating a weight vector and a maximum eigenvalue based on the judgment matrix, determining a consistency index and an average consistency index based on the maximum eigenvalue, and determining a consistency ratio based on the ratio of the consistency index to the average consistency index, determining a data comprehensive score for each to-be-stored classified data based on the consistency ratio, when the data comprehensive score is greater than a data comprehensive score threshold, determining the encryption strategy as a composite encryption strategy, and determining the encrypted data based on the composite encryption strategy, and when the data comprehensive score is less than or equal to the data comprehensive score threshold, determining the encryption strategy as an independent encryption strategy, and determining the encrypted data based on the independent encryption strategy.

[0034] Specifically, the target layer is used to calculate the comprehensive score of the data. The criterion layer includes five data attributes: data size (storage resource usage), data structure (complexity), data association (dependence on other data), data compression (redundancy), and data source (credibility). The solution layer corresponds to the specific classified data to be stored, forming a three-level analysis framework of the target layer, criterion layer, and solution layer. The importance of each data attribute in the criterion layer is compared two by two, and a judgment matrix is ​​constructed using the 1-9 scaling method (the matrix element aᵢⱼ represents the importance of data attribute i relative to data attribute j). The consistency index CI is calculated by determining the weight vector (reflecting the influence ratio of each data attribute) and the maximum eigenvalue λmax. Combined with the average random consistency index RI (when the data attribute is 5, RI is preferably 1.12), the consistency ratio CR is obtained. If CR<0.1, the judgment matrix satisfies the consistency judgment logic without contradiction, and the weight is valid. Otherwise, the judgment matrix is ​​modified, and the weight vector is used to weight the score of each data attribute to determine the comprehensive score of the data. Only when the classified data to be stored exhibits high risk characteristics (such as very large data volume, complex structure, high correlation, low redundancy and reliable source) in the five data attributes of data size, data structure, data association, data compression and data source, the data is If the comprehensive score is greater than the set data comprehensive score threshold of 0.9, a composite encryption strategy is triggered, thereby focusing on the representative and covered classified data to be stored. The composite encryption strategy includes at least two encryptions of AES-128, AES-256, ECC, and Kyber. When the data comprehensive score is less than or equal to the data comprehensive score threshold, the encryption strategy is determined to be an independent encryption strategy. The independent encryption strategy includes one encryption of AES-128, AES-256, ECC, and Kyber. The judgment matrix is ​​converted into computable weights and scores, avoiding the bias of human experience and resolving the priority conflicts of multiple data attributes in decision-making (such as balancing the influence of "data size" and "data source"), ensuring dynamic encryption of different classified data to be stored, thereby achieving a dynamic balance of security needs and ensuring the stability and flexibility of storage management.

[0035] In some embodiments of the present application, when all encrypted data with the same classification attributes are temporarily stored in a sub-memory, and data verification is performed in the sub-memory, and the temporarily stored encrypted data is moved to a target sub-memory according to the result of the data verification, it includes: data verification includes security detection and structure verification. When the security detection passes, the structure verification is performed. When the security detection passes and the structure verification passes, the data verification is successful. Otherwise, the data verification fails. When the data verification succeeds, the encrypted data stored in the sub-memory is moved to the target sub-memory.

[0036] In some embodiments of the present application, when obtaining the storage occupancy of the target sub-memory and determining the backup mode of the target sub-memory based on the storage occupancy, it includes: when the storage occupancy meets the first condition, full backup is determined as the backup mode of the target sub-memory; when the storage occupancy meets the second condition, differential backup is determined as the backup mode of the target sub-memory; when the storage occupancy meets the third condition, incremental backup is determined as the backup mode of the target sub-memory, and the backup levels of full backup, differential backup and incremental backup decrease in sequence.

[0037] Specifically, all encrypted data with the same classification attributes is temporarily stored in a sub-storage. Identical classification attributes represent the same classification result, for example: all video data in the education field, all text data in the science field, etc. These encrypted data with the same classification attributes reflect the encryption results of a single classification scenario. The data verification phase achieves dual protection for encrypted data through a progressive logic from security testing to structural verification. Security testing focuses on data security, including virus signature matching, abnormal permission flagging, and encryption integrity verification (such as hash value comparison), while structural verification focuses on data integrity, checking data format compliance (such as field length and check digits) and logical relevance (such as the consistency of related data indexes). Only when both conditions are met will the encrypted data be migrated from the temporary sub-storage to the target sub-storage, forming a dual barrier of secure access and structural compliance. The backup mode is dynamically adapted based on the storage occupancy rate. The first condition is that the storage occupancy rate is greater than 70%. In this case, a full backup is used (all data in the target sub-storage is completely copied). The second condition is that the storage occupancy rate is less than or equal to 70% and greater than 30%. A differential backup is used. The third condition is that the occupancy rate is less than or equal to 30%, and an incremental backup is used (only data added / modified since the last backup is backed up). The layered verification mechanism first filters malicious data (security detection) and then corrects format errors (structural verification). This allows the target sub-storage to store data while meeting security and availability, reducing the risk of abnormalities during subsequent data calls. When storing large amounts of encrypted data, full backup is used to ensure data recoverability. When storing relatively small amounts of encrypted data, differential / incremental backups are used to balance backup integrity and efficiency, improving the stability of storage management and resource utilization.

[0038] In some embodiments of the present application, the data retrieval status of the target sub-storage in the backup mode is obtained, and whether to adjust the backup mode is determined based on the data retrieval status, including: when the backup mode of the target sub-storage is differential backup or incremental backup, the data retrieval information of the target sub-storage is obtained, and the data retrieval information includes the number of data retrievals and the data viewing time. When the data retrieval information meets the data retrieval conditions, it is determined to adjust the backup mode; otherwise, it is determined not to adjust the backup mode; when it is determined to adjust the backup mode, the backup level of the differential backup or incremental backup is increased by one level.

[0039] Specifically, when the target sub-storage utilizes differential or incremental backup, data retrieval information is recorded in real time. This data retrieval information includes the number of data retrievals and the data viewing duration. The number of data retrievals represents the total number of encrypted data accesses per unit time (e.g., 24 hours), and the data viewing duration represents the duration of encrypted data access after each retrieval. Based on this, data retrieval conditions are set as the basis for adjusting the triggering backup mode: the number of data retrievals within a 24-hour period must be greater than or equal to 5 times, and the data viewing duration must be greater than or equal to 30 seconds (the cumulative data viewing duration for all data retrievals). If the data retrieval conditions are met, the encrypted data stored in the target sub-storage is determined to be high-access data, requiring an upgrade in backup level. Backup levels increase in order from incremental backup, differential backup, to full backup, with upgrading the level to a higher level. If the target sub-storage's backup mode is already full backup, the highest backup level, then this full backup is maintained. Dynamically adjust the backup mode based on data retrieval information, concentrate storage and computing resources on highly accessed encrypted data, reduce redundant consumption, and meet the real-time needs of frequently accessed data. Frequently retrieved encrypted data is usually related to the core business. Improving its backup level can reduce the risk of data loss caused by incremental / differential backups (for example, full backups can be restored independently without being affected by the integrity of previous backups), thereby enhancing the adaptability and stability of data storage management.

[0040] In summary, the beneficial effects of the present invention are as follows: at the classification management level, intelligent classification of data is achieved through feature heat maps and multi-head self-attention models, replacing the manual classification mode, accurately capturing the dynamic features and associated attributes of data, reducing the risk of classification confusion and unclear hierarchy, improving data retrieval efficiency and management stability, adapting to the rapid growth and dynamic change requirements of data, and ensuring the security and reliability of data storage management. The data comprehensive score is formed based on hierarchical analysis and consistency test, and differentiated encryption strategies are implemented for different data. It not only strengthens the encryption strength of data to resist malicious attacks and illegal intrusions, but also avoids the waste of resources caused by excessive encryption of data. While ensuring data security, it strikes a balance with storage efficiency, effectively reducing the probability of data leakage and damage, and strengthening privacy protection and data integrity. Temporary storage verification ensures the accuracy of data from classification to storage, avoids the risk of data damage caused by erroneous storage, and improves the reliability of data storage management. The backup mode is dynamically determined based on the storage occupancy rate and adjusted in real time based on the data retrieval situation, achieving accurate allocation of backup resources, ensuring the availability of storage backups, reducing the risk of data loss, and thus ensuring the reliability and stability of data storage management.

[0041] In another preferred embodiment based on the above embodiment, refer to Figure 2 As shown, this embodiment provides a storage management system based on data encryption, which is used to apply the above-mentioned storage management method based on data encryption, including: The acquisition and analysis module is configured to obtain all the data to be stored and preprocess each data to be stored to determine the target data to be stored, obtain the model sample set and construct a feature heat map based on the model sample set.

[0042] The data classification module is configured to determine key feature clusters based on the feature heat map, decouple the key feature clusters according to the autoencoder to determine the model data set, build a multi-head self-attention model based on the model data set, and determine the classification data to be stored based on the multi-head self-attention model.

[0043] The data processing module is configured to stratify each classified data to be stored based on hierarchical analysis and construct a judgment matrix, and perform a consistency check on the judgment matrix to determine the comprehensive score of the data. Based on the comprehensive score of the data, a corresponding encryption strategy is adopted for all classified data to be stored to determine the encrypted data, all encrypted data with the same classification attributes are temporarily stored in a sub-memory, and data verification is performed in the sub-memory. According to the result of the data verification, the temporarily stored encrypted data is moved to the target sub-memory.

[0044] The storage management module is configured to obtain the storage occupancy rate of the target sub-storage, determine the backup mode of the target sub-storage based on the storage occupancy rate, obtain the data retrieval status of the target sub-storage in the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or the adjustment result.

[0045] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0046] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0047] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0048] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A storage management method based on data encryption, characterized in that: include: Acquire all data to be stored and preprocess each data to be stored to determine target data to be stored, acquire a model sample set and construct a feature heat map based on the model sample set; Determine a key feature cluster based on the feature heat map, decouple the key feature cluster according to the autoencoder to determine a model data set, build a multi-head self-attention model based on the model data set, and determine the classification data to be stored based on the multi-head self-attention model; Based on hierarchical analysis, each classified data to be stored is layered and a judgment matrix is ​​constructed. The judgment matrix is ​​checked for consistency to determine the comprehensive score of the data. Based on the comprehensive score of the data, a corresponding encryption strategy is adopted for all the classified data to be stored to determine the encrypted data. All encrypted data with the same classification attribute are temporarily stored in a sub-memory, and data verification is performed in the sub-memory. Based on the result of the data verification, the temporarily stored encrypted data is moved to the target sub-memory; Obtain the storage occupancy rate of the target sub-memory, determine the backup mode of the target sub-memory based on the storage occupancy rate, obtain the data retrieval status of the target sub-memory under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or the adjustment result.

2. The storage management method based on data encryption according to claim 1, characterized in that: When all the data to be stored are acquired and each data to be stored is pre-processed to determine the target data to be stored, the following steps are included: Perform data cleaning on all data to be stored, including processing missing values, outliers, and duplicate values, and perform data conversion on all cleaned data to be stored, including data standardization and data normalization, verifying the integrity and consistency of each data to be stored after data conversion, and determining whether it conforms to the storage format; If the verification passes and complies with the storage format, all the data to be stored are determined as the target data to be stored; If the verification fails or does not conform to the storage format, the data to be stored that fails the verification or does not conform to the storage format will be retained and prompted for manual verification, and the remaining data to be stored will be determined as the target data to be stored.

3. The storage management method based on data encryption according to claim 2, characterized in that: When obtaining a model sample set and constructing a feature heat map based on the model sample set, it includes: Determine all feature dimensions of the model sample set, and determine the distribution probability of each feature dimension in each category, and determine the entropy value of each feature dimension based on the distribution probability, wherein the entropy value is proportional to the distribution difference; A matrix is ​​constructed with the feature dimension as the horizontal axis and the category as the vertical axis. The feature heat map is constructed based on the data visualization library and the matrix. The color depth of each cell in the feature heat map is determined by the entropy value of the corresponding feature in the category.

4. The storage management method based on data encryption according to claim 3, characterized in that: When determining a key feature cluster based on the feature heat map and decoupling the key feature cluster according to the autoencoder to determine a model data set, the method includes: Determine a classification feature based on the color depth and calculate a feature variance of the classification feature, and determine the key feature cluster based on the feature variance; Determine the central eigenvector of each category, determine the cosine similarity based on the central eigenvectors of any two categories, obtain the average value of the cosine similarities between all two categories, and determine the confusion category based on the average value; Construct the autoencoder, which includes a three-layer encoder and a three-layer decoder symmetrical to the three-layer encoder, and the input layer dimension of each encoder is consistent with the dimension of the key feature cluster. Based on the autoencoder, the key feature clusters contained in the confusion category are decoupled, and the sample data of the decoupled key feature clusters are determined as the model data set.

5. The storage management method based on data encryption according to claim 4, characterized in that: When a multi-head self-attention model is constructed based on the model data set, and classified data to be stored is determined based on the multi-head self-attention model, the method includes: Dividing the model dataset into a model training set and a model test set, selecting an initial multi-head model based on Transformer, training the initial multi-head model according to the model training set, and substituting the model test set into the trained initial multi-head model for testing; If the test value of the currently trained initial multi-head model is greater than or equal to the test value of the previously trained initial multi-head model, the training is stopped, and the currently trained initial multi-head model is determined as the multi-head self-attention model; If the test value of the currently trained initial multi-head model is less than the test value of the previously trained initial multi-head model, adjust the learning rate of the currently trained initial multi-head model and continue training until the test value of the currently trained initial multi-head model is greater than or equal to the test value of the previously trained initial multi-head model; Substitute all target data to be stored into the multi-head self-attention model to determine the classified data to be stored.

6. The storage management method based on data encryption according to claim 5, characterized in that: When each of the classified data to be stored is layered based on hierarchical analysis and a judgment matrix is ​​constructed, a consistency check is performed on the judgment matrix to determine a comprehensive score of the data, and a corresponding encryption strategy is adopted for all the classified data to be stored based on the comprehensive score of the data to determine the encrypted data, the method includes: Determine a hierarchical structure based on the hierarchical analysis, the hierarchical structure including a target layer, a criterion layer, and a solution layer, the target layer determines the comprehensive score of the data, the criterion layer determines the data attributes of the classified data to be stored, and the solution layer determines the role carrier of the data attributes, the data attributes including data size, data structure, data association, data compression, and data source; Constructing the judgment matrix based on the data attributes, calculating the weight vector and the maximum eigenvalue based on the judgment matrix, determining the consistency index and the average consistency index based on the maximum eigenvalue, determining the consistency ratio based on the ratio of the consistency index to the average consistency index, and determining the data comprehensive score of each classified data to be stored based on the consistency ratio; When the data comprehensive score is greater than the data comprehensive score threshold, the encryption strategy is determined to be a composite encryption strategy, and the encrypted data is determined according to the composite encryption strategy; When the data comprehensive score is less than or equal to the data comprehensive score threshold, the encryption strategy is determined to be an independent encryption strategy, and the encrypted data is determined according to the independent encryption strategy.

7. The storage management method based on data encryption according to claim 6, characterized in that: When all encrypted data with the same classification attribute are temporarily stored in a sub-memory, data verification is performed in the sub-memory, and the temporarily stored encrypted data is moved to a target sub-memory according to a result of the data verification, the method includes: The data verification includes security detection and structure verification. When the security detection passes, the structure verification is performed; When the security check passes and the structure check passes, the data check succeeds; otherwise, the data check fails. When the data verification succeeds, the encrypted data stored in the sub-memory is moved to the target sub-memory.

8. The storage management method based on data encryption according to claim 7, characterized in that: When obtaining the storage occupancy of the target sub-storage and determining the backup mode of the target sub-storage based on the storage occupancy, the method includes: When the storage occupancy rate satisfies a first condition, full backup is determined as the backup mode of the target sub-storage; When the storage occupancy rate satisfies a second condition, differential backup is determined as the backup mode of the target sub-storage; When the storage occupancy rate satisfies a third condition, determining incremental backup as the backup mode of the target sub-storage; The backup levels of the full backup, differential backup and incremental backup are successively lowered.

9. The storage management method based on data encryption according to claim 8, characterized in that: Acquiring data retrieval status of the target sub-storage in the backup mode, and determining whether to adjust the backup mode based on the data retrieval status, includes: When the backup mode of the target sub-storage is the differential backup or incremental backup, data retrieval information of the target sub-storage is obtained, the data retrieval information including the number of data retrievals and the data viewing duration; when the data retrieval information satisfies the data retrieval condition, it is determined to adjust the backup mode; otherwise, it is determined not to adjust the backup mode; When it is determined to adjust the backup mode, the backup level of the differential backup or incremental backup is increased by one level.

10. A data encryption-based storage management system, used for applying the data encryption-based storage management method according to any one of claims 1 to 9, characterized in that: include: An acquisition and analysis module is configured to acquire all data to be stored and pre-process each data to be stored to determine target data to be stored, acquire a model sample set and construct a feature heat map based on the model sample set; a data classification module configured to determine key feature clusters based on the feature heat map, decouple the key feature clusters according to the autoencoder to determine a model data set, construct a multi-head self-attention model based on the model data set, and determine the classification data to be stored based on the multi-head self-attention model; a data processing module configured to stratify each of the classified data to be stored based on hierarchical analysis and construct a judgment matrix, perform a consistency check on the judgment matrix to determine a comprehensive data score, adopt a corresponding encryption strategy for all the classified data to be stored based on the comprehensive data score to determine encrypted data, temporarily store all encrypted data with the same classification attribute in a sub-memory, perform data verification in the sub-memory, and move the temporarily stored encrypted data to a target sub-memory based on the result of the data verification; The storage management module is configured to obtain the storage occupancy rate of the target sub-memory, determine the backup mode of the target sub-memory based on the storage occupancy rate, obtain the data retrieval status of the target sub-memory under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or the adjustment result.

Images