A data encryption-based storage management method and system

Abstract

The application relates to the technical field of data storage, and discloses a storage management method and system based on data encryption, which comprises the following steps: acquiring a model sample set and constructing a feature heat map based on the model sample set; determining a key feature cluster based on the feature heat map; determining a model data set by decoupling the key feature cluster according to a self-encoder; determining to-be-stored classified data based on a multi-head self-attention model; stratifying each to-be-stored classified data based on analytic hierarchy process and constructing a judgment matrix; determining encrypted data by adopting a corresponding encryption strategy on the basis of a data comprehensive score; moving temporarily stored encrypted data to a target sub-memory according to the result of data checking; determining a backup mode of the target sub-memory based on a storage occupancy rate; and judging whether to adjust the backup mode based on a data retrieval condition. The application ensures the stability and reliability of data storage management.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and more specifically, to a storage management method and system based on data encryption. Background Technology

[0002] With the rapid development of information technology and the deepening of digitalization, all types of data are growing at an unprecedented rate, covering multiple fields such as personal information, enterprise business data, government data, medical records, and financial transaction information. Effective storage and management of data are crucial for protecting personal privacy and for effective business decision-making.

[0003] However, due to the wide range of data sources, diverse types, and different attributes of different data, existing data storage lacks an intelligent classification mechanism. It usually relies on manual classification, which is difficult to adapt to the dynamic changes of data, resulting in chaotic data classification and unclear hierarchy. This not only increases the difficulty of data retrieval and management and reduces data utilization efficiency, but may also lead to misuse of data due to classification errors. Furthermore, there are a large number of security threats such as malicious attacks and illegal intrusions during the storage process. Data is easily stolen, tampered with, or deleted, causing data leakage or damage, which affects data backup and leads to the risk of data loss.

[0004] Therefore, it is necessary to design a storage management method and system based on data encryption to solve the problems existing in the current technology. Summary of the Invention

[0005] In view of this, the present invention proposes a storage management method and system based on data encryption, which aims to solve the problems of relying on manual classification, which is difficult to adapt to dynamic changes in data, resulting in chaotic data classification and unclear hierarchy, as well as security threats during storage that cause data leakage or damage, thereby affecting data backup and leading to data loss.

[0006] In one aspect, the present invention proposes a storage management method based on data encryption, comprising:

[0007] Acquire all data to be stored and preprocess each data to be stored to determine the target data to be stored; acquire the model sample set and construct a feature heatmap based on the model sample set.

[0008] Key feature clusters are determined based on the feature heatmap, and the key feature clusters are decoupled according to the autoencoder to determine the model dataset. A multi-head self-attention model is constructed based on the model dataset, and the classification data to be stored is determined based on the multi-head self-attention model.

[0009] Based on hierarchical analysis, each category of data to be stored is hierarchically divided and a judgment matrix is ​​constructed. The judgment matrix is ​​then subjected to a consistency check to determine the comprehensive data score. Based on the comprehensive data score, an appropriate encryption strategy is adopted for all category data to be stored to determine encrypted data. All encrypted data with the same category attribute are temporarily stored in a sub-memory, and data verification is performed in the sub-memory. Based on the result of the data verification, the temporarily stored encrypted data is moved to the target sub-memory.

[0010] Obtain the storage occupancy rate of the target sub-memory, determine the backup mode of the target sub-memory based on the storage occupancy rate, obtain the data retrieval status of the target sub-memory under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or adjustment result.

[0011] Furthermore, when acquiring all the data to be stored and preprocessing each piece of data to determine the target data to be stored, the process includes:

[0012] All data to be stored is cleaned, including handling missing values, outliers and duplicate values. The cleaned data is then transformed, including data standardization and data normalization. The integrity and consistency of each piece of data to be stored after the transformation are verified, and it is determined whether it conforms to the storage format.

[0013] If the verification is successful and the data conforms to the storage format, then all data to be stored will be identified as the target data to be stored.

[0014] If the verification fails or does not conform to the storage format, the data to be stored that fails the verification or does not conform to the storage format will be retained and a manual verification will be requested. The remaining data to be stored will be determined as the target data to be stored.

[0015] Furthermore, when acquiring the model sample set and constructing a feature heatmap based on the model sample set, the process includes:

[0016] Determine all feature dimensions of the model sample set, and determine the distribution probability of each feature dimension in each category. Based on the distribution probability, determine the entropy value of each feature dimension, wherein the entropy value is proportional to the distribution difference.

[0017] A matrix is ​​constructed with the feature dimension as the horizontal axis and the category as the vertical axis. The feature heatmap is constructed based on the data visualization library and the matrix. The color depth of each cell in the feature heatmap is determined by the entropy value of the corresponding feature in that category.

[0018] Furthermore, when determining key feature clusters based on the feature heatmap, and decoupling the key feature clusters according to the autoencoder to determine the model dataset, the process includes:

[0019] Based on the color depth, classification features are determined and the feature variance of the classification features is calculated. Based on the feature variance, the key feature clusters are determined.

[0020] Determine the central feature vector of each category, and determine the cosine similarity based on the central feature vectors of any two categories. Obtain the average value of the cosine similarity between all pairs of categories, and determine the confused category based on the average value.

[0021] The autoencoder is constructed, which includes a 3-layer encoder and a 3-layer decoder symmetrical to the 3-layer encoder. The input layer dimension of each encoder is consistent with the dimension of the key feature cluster. Based on the autoencoder, the key feature clusters contained in the confusion category are decoupled, and the sample data of the decoupled key feature clusters are determined as the model dataset.

[0022] Furthermore, when constructing a multi-head self-attention model based on the model dataset, and determining the classification data to be stored based on the multi-head self-attention model, the process includes:

[0023] The model dataset is divided into a model training set and a model test set. An initial multi-head model is selected based on Transformer. The initial multi-head model is trained according to the model training set, and the model test set is substituted into the trained initial multi-head model for testing.

[0024] If the test value of the current trained initial multi-head model is greater than or equal to the test value of the previous trained initial multi-head model, then training is stopped, and the current trained initial multi-head model is determined as the multi-head self-attention model.

[0025] If the test value of the current trained initial multi-head model is less than the test value of the previous trained initial multi-head model, then adjust the learning rate of the current trained initial multi-head model and continue training until the test value of the current trained initial multi-head model is greater than or equal to the test value of the previous trained initial multi-head model.

[0026] Substitute all target data to be stored into the multi-head self-attention model to determine the classification data to be stored.

[0027] Furthermore, when stratifying each category of data to be stored based on hierarchical analysis and constructing a judgment matrix, and performing a consistency check on the judgment matrix to determine the comprehensive data score, and then applying an appropriate encryption strategy to all category data to be stored based on the comprehensive data score to determine encrypted data, the process includes:

[0028] Based on the hierarchical analysis, a hierarchical structure is determined, which includes a target layer, a criterion layer, and a scheme layer. The target layer determines the comprehensive score of the data, the criterion layer determines the data attributes of the classified data to be stored, and the scheme layer determines the carrier of the data attributes. The data attributes include data size, data structure, data association, data compression, and data source.

[0029] The judgment matrix is ​​constructed based on the data attributes. The weight vector and the maximum eigenvalue are calculated based on the judgment matrix. The consistency index and the average consistency index are determined based on the maximum eigenvalue. The consistency ratio is determined based on the ratio of the consistency index and the average consistency index. The comprehensive data score of each classified data to be stored is determined based on the consistency ratio.

[0030] When the overall data score is greater than the overall data score threshold, the encryption strategy is determined to be a composite encryption strategy, and the encrypted data is determined according to the composite encryption strategy.

[0031] When the overall data score is less than or equal to the overall data score threshold, the encryption strategy is determined to be an independent encryption strategy, and the encrypted data is determined according to the independent encryption strategy.

[0032] Furthermore, when temporarily storing all encrypted data with the same classification attributes in a sub-memory, performing data verification in the sub-memory, and moving the temporarily stored encrypted data to the target sub-memory based on the data verification result, the process includes:

[0033] The data verification includes security checks and structural checks. When the security checks pass, the structural checks are performed.

[0034] If the security test and the structural verification pass, the data verification is successful; otherwise, the data verification fails.

[0035] When the data verification is successful, the encrypted data stored in the sub-memory is moved to the target sub-memory.

[0036] Furthermore, when obtaining the storage occupancy rate of the target sub-memory and determining the backup mode of the target sub-memory based on the storage occupancy rate, the process includes:

[0037] When the storage occupancy rate meets the first condition, a full backup is determined as the backup mode for the target sub-storage.

[0038] When the storage occupancy rate meets the second condition, differential backup is determined as the backup mode of the target sub-storage.

[0039] When the storage occupancy rate meets the third condition, incremental backup is determined as the backup mode for the target sub-storage.

[0040] The backup levels for full backup, differential backup, and incremental backup decrease sequentially.

[0041] Furthermore, when obtaining the data retrieval status of the target sub-storage in the backup mode, and determining whether to adjust the backup mode based on the data retrieval status, the process includes:

[0042] When the backup mode of the target sub-storage is differential backup or incremental backup, the data retrieval information of the target sub-storage is obtained. The data retrieval information includes the number of data retrievals and the data viewing duration. When the data retrieval information meets the data retrieval conditions, it is determined to adjust the backup mode; otherwise, it is determined not to adjust the backup mode.

[0043] When it is determined that the backup mode should be adjusted, the backup level of the differential backup or incremental backup will be increased by one level.

[0044] Compared with existing technologies, the beneficial effects of this invention are as follows: At the classification management level, intelligent classification of data is achieved through feature heatmaps and multi-head self-attention models, replacing the manual classification mode. This accurately captures the dynamic characteristics and correlation attributes of data, reduces the risk of classification confusion and unclear hierarchy, improves data retrieval efficiency and management stability, adapts to the needs of rapid data growth and dynamic changes, and ensures the security and reliability of data storage management. Based on hierarchical analysis and consistency checks, a comprehensive data score is formed, realizing differentiated encryption strategies for different types of data. This strengthens encryption strength to resist malicious attacks and illegal intrusions while avoiding resource waste caused by excessive encryption. It achieves a balance between data security and storage efficiency, effectively reducing the probability of data leakage and damage, and strengthening privacy protection and data integrity. Temporary storage verification ensures the accuracy of data from classification to storage, avoiding risks such as data corruption caused by incorrect storage, and improving the reliability of data storage management. Furthermore, the backup mode is dynamically determined based on storage occupancy and adjusted in real time according to data retrieval, achieving precise allocation of backup resources, ensuring the availability of storage backups, reducing the risk of data loss, and thus ensuring the reliability and stability of data storage management.

[0045] On the other hand, this application also provides a data encryption-based storage management system for applying the above-mentioned data encryption-based storage management method, including:

[0046] The data acquisition and analysis module is configured to acquire all data to be stored, preprocess each data to be stored to determine the target data to be stored, acquire a model sample set, and construct a feature heatmap based on the model sample set.

[0047] The data classification module is configured to determine key feature clusters based on the feature heatmap, decouple the key feature clusters according to the autoencoder to determine the model dataset, construct a multi-head self-attention model based on the model dataset, and determine the classification data to be stored based on the multi-head self-attention model.

[0048] The data processing module is configured to hierarchically classify each type of data to be stored based on hierarchical analysis and construct a judgment matrix, perform consistency checks on the judgment matrix to determine the comprehensive data score, apply corresponding encryption strategies to all types of data to be stored based on the comprehensive data score to determine encrypted data, temporarily store all encrypted data with the same classification attributes in a sub-memory, perform data verification in the sub-memory, and move the temporarily stored encrypted data to the target sub-memory according to the result of the data verification.

[0049] The storage management module is configured to obtain the storage occupancy rate of the target sub-storage, determine the backup mode of the target sub-storage based on the storage occupancy rate, obtain the data retrieval status of the target sub-storage under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or adjustment result.

[0050] It is understandable that the aforementioned data encryption-based storage management method and system have the same beneficial effects, and will not be elaborated further here. Attached Figure Description

[0051] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0052] Figure 1 A flowchart illustrating a data encryption-based storage management method provided in an embodiment of the present invention;

[0053] Figure 2 This is a functional block diagram of a data encryption-based storage management system provided in an embodiment of the present invention. Detailed Implementation

[0054] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0055] See Figure 1 As shown in some embodiments of this application, a storage management method based on data encryption includes:

[0056] S100: Obtain all data to be stored and preprocess each data to be stored to determine the target data to be stored, obtain the model sample set and construct a feature heatmap based on the model sample set.

[0057] S200: Determine key feature clusters based on feature heatmaps, decouple key feature clusters based on autoencoders to determine model datasets, construct multi-head self-attention models based on model datasets, and determine classification data to be stored based on multi-head self-attention models.

[0058] S300: Based on hierarchical analysis, each category of data to be stored is hierarchically divided and a judgment matrix is ​​constructed. The consistency of the judgment matrix is ​​checked to determine the comprehensive score of the data. Based on the comprehensive score of the data, the corresponding encryption strategy is adopted for all category data to be stored to determine the encrypted data. All encrypted data with the same category attribute are temporarily stored in the sub-storage, and data verification is performed in the sub-storage. Based on the result of the data verification, the temporarily stored encrypted data is moved to the target sub-storage.

[0059] S400: Obtain the storage occupancy rate of the target sub-memory, determine the backup mode of the target sub-memory based on the storage occupancy rate, obtain the data retrieval status of the target sub-memory under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or adjustment result.

[0060] Specifically, in the process of using web crawlers and other technologies to acquire all the data to be stored, network fluctuations and transmission processes may cause data modifications. Therefore, all the data to be stored is preprocessed to remove the impact of network transmission and fluctuations. Simultaneously, a feature heatmap is constructed using a model sample set. The model sample set includes 100 model sample data, including text data, video data, and audio data. Each type of data is further divided into different domains; for example, video data includes data from the education, medical, and technology fields. Based on these samples, a model sample set is built to construct the feature heatmap. The feature heatmap visually displays the distribution patterns of data features and the degree of correlation between features through color intensity. Based on feature heatmaps, key feature clusters are extracted and decoupled using an autoencoder, breaking down complex features into independent basic feature dimensions to form a structured model dataset. A multi-head self-attention model is then trained on this dataset. The multi-head self-attention model, an attention-based model, uses multiple parallel attention heads to capture the correlation features and dependencies across different dimensions of data, thereby achieving automatic classification of the target data to be stored. This eliminates reliance on manual operation. By combining feature heatmaps with the multi-head self-attention model, changes in data features can be dynamically captured, automatically generating accurate classification results. This avoids the problems of chaotic and unclear hierarchical classification found in traditional methods, improving the efficiency of data retrieval and storage management. After classification by the multi-head self-attention model, hierarchical analysis is used to stratify each category of data to be stored according to dimensions such as data size and data structure. A judgment matrix is ​​constructed to quantify the weights of each level. Consistency checks ensure the logical consistency of the weight allocation, ultimately generating a comprehensive data score.

[0061] Understandably, the comprehensive data score reflects the overall indicators of the categorized data to be stored. The higher the comprehensive data score, the more representative the categorized data is and the more comprehensive the data content is. Different encryption strategies are adopted based on the comprehensive data score. Composite encryption algorithms and lightweight encryption algorithms are used for different comprehensive data scores to balance the security and efficiency of storage management. The encrypted data is first temporarily stored in a sub-storage for integrity and consistency verification. After eliminating errors in the transmission or encryption process, it is then transferred to the target sub-storage that matches the classification attributes. While ensuring data security, this avoids the waste of resources caused by excessive encryption. Furthermore, the verification of the sub-storage further filters abnormal data, reduces the risk of tampering and theft during data transmission, and reduces the risk of data misuse and leakage. The backup mode is selected based on the storage occupancy rate of the target sub-memory. The storage occupancy rate reflects the storage capacity of the target sub-memory. A higher storage occupancy rate indicates that more encrypted data is stored in the target sub-memory. The backup mode of the target sub-memory is dynamically determined based on the storage occupancy rate. Furthermore, the backup mode is dynamically adjusted based on data retrieval status, which improves data availability in high-frequency access scenarios, balances storage costs and data security, reduces the risk of data loss due to rigid backup strategies, and ensures efficient management of the entire data lifecycle.

[0062] In some embodiments of this application, when acquiring all data to be stored and preprocessing each piece of data to determine the target data to be stored, the process includes: cleaning all data to be stored, including handling missing values, outliers, and duplicate values; transforming all cleaned data to be stored, including data standardization and data normalization; verifying the integrity and consistency of each piece of data to be stored after data transformation; determining whether it conforms to the storage format; if the verification passes and it conforms to the storage format, then all data to be stored is determined as the target data to be stored; if the verification fails or it does not conform to the storage format, then the data to be stored that fails the verification or does not conform to the storage format is retained and a manual verification is requested; and the remaining data to be stored is determined as the target data to be stored.

[0063] Specifically, the process begins with data cleaning to improve the basic quality of the data. Missing value handling prevents analytical biases caused by incomplete data, outlier handling eliminates interference from data deviating from the normal range, and duplicate value handling removes data redundancy. Data transformation, through standardization and normalization, eliminates differences between different data volumes, facilitating subsequent model output. Data integrity is then verified to ensure no critical information is missing and that the data logic is consistent. Data cleaned and transformed is more reliable. Simultaneously, it is checked for compliance with storage format specifications. If compliant, the data is designated as target data for storage; otherwise, it is retained and manually verified for targeted correction. The remaining verified data that conforms to the storage format is designated as target data for storage, reducing subsequent storage redundancy and saving storage space by eliminating unqualified data, thus ensuring the integrity of the target data for storage.

[0064] In some embodiments of this application, when acquiring a model sample set and constructing a feature heatmap based on the model sample set, the process includes: determining all feature dimensions of the model sample set and determining the distribution probability of each feature dimension in each category; determining the entropy value of each feature dimension based on the distribution probability, wherein the entropy value is proportional to the distribution difference; constructing a matrix with the feature dimension as the horizontal axis and the category as the vertical axis; constructing a feature heatmap based on a data visualization library and the matrix; wherein the color depth of each cell in the feature heatmap is determined by the entropy value of the corresponding feature in that category.

[0065] Specifically, the model sample set includes 100 model sample data, including text data, video data, and audio data, each type of data is further divided into different domains. All feature dimensions of the model sample set are determined. Feature dimensions refer to specific information dimensions in the model sample set that reflect the classification attributes of the data. Examples include: file size range, modification timestamp patterns, metadata keywords, etc. These information dimensions are key bases for distinguishing different categories of data. Using the entropy method and based on probability distribution, the entropy value of each feature dimension is calculated to determine the probability distribution of each feature dimension in the i-th category. The higher the entropy value, the greater the distribution difference of the feature in different categories. For example, the distribution of "data suffixes" is extremely uneven in text and image categories, resulting in a higher entropy value and a stronger ability to distinguish different categories. By calculating their entropy values, we can determine their distribution differences in different categories, and thus filter out features valuable for classification. In classification tasks, categories are the different types to be distinguished. For example, the results of text data, image data, and audio data need to be distinguished as "education domain," "technology domain," and "medical domain," thereby filtering out the key features that can distinguish them. Preliminary screening is achieved by calculating the entropy value of each feature dimension. The higher the entropy value, the greater the distribution difference of the feature dimension in different categories, and the stronger the discrimination. Based on the entropy results, a matrix is ​​constructed with the feature dimension as the horizontal axis and the category as the vertical axis. A feature heatmap is generated using a data visualization library (Matplotlib library), and the entropy value of the corresponding feature in the category is represented by the color depth. Generally speaking, the color gradient (such as from blue to red) shows the size of the cell value, which intuitively presents the correlation strength between the feature and the category. The depth of the cell color in the feature heatmap is directly proportional to the entropy value at the corresponding position. The darker the color, the higher the entropy value, reflecting that the feature has a stronger discrimination ability in the category.

[0066] Understandably, by establishing a feature heatmap to visualize feature discrimination, the abstract probability distribution represented by entropy values ​​is transformed into a matrix distribution of varying color depths. This makes the differences in feature dimension distribution across categories immediately apparent, facilitating the rapid identification of high-discrimination features (darker color areas) and low-discrimination features (lighter color areas). This reduces the complexity of manual analysis, improves the efficiency of feature selection, and precisely quantifies the differences in feature distribution across categories using entropy values. The visual contrast of feature heatmaps enhances the perception of these differences, avoiding the bias of a single evaluation method. This ensures that the selected features possess objective discriminative power, making subsequent feature decoupling more targeted, thereby improving the efficiency and accuracy of model training.

[0067] In some embodiments of this application, when determining key feature clusters based on feature heatmaps and decoupling the key feature clusters according to the autoencoder to determine the model dataset, the process includes: determining classification features based on color depth and calculating the feature variance of the classification features; determining key feature clusters based on the feature variance; determining the center feature vector of each category; determining the cosine similarity based on the center feature vectors of any two categories; obtaining the average value of the cosine similarity between all pairs of categories; determining the confused category based on the average value; constructing an autoencoder, which includes a 3-layer encoder and a 3-layer decoder symmetrical to the 3-layer encoder, wherein the input layer dimension of each encoder is consistent with the dimension of the key feature cluster; decoupling the key feature clusters contained in the confused category based on the autoencoder; and determining the sample data of the decoupled key feature clusters as the model dataset.

[0068] Specifically, based on the color depth of the feature heatmap, classification features are initially located. Higher entropy values ​​indicate greater differences in feature distribution across different categories, demonstrating basic discriminative ability. These feature dimensions with higher entropy values ​​are then identified as classification features. For example, if there are 20 feature dimensions with calculated entropy values ​​between 0.2 and 1.8, the theoretical maximum entropy value is log2(k), where k is the number of categories. If the maximum entropy value for 8 categories is 3, then the top 50%, i.e., features with entropy values ​​≥ 1.0, are taken as classification features. Further calculation of the feature variance of these classification features is used to measure the degree of feature fluctuation in the samples. Classification features with feature variance greater than 0.6 are identified as key feature clusters. These classification features not only exhibit differences in distribution across categories but also fluctuate in their own values. For example, "data size" shows different ranges in video data (100MB-2GB) and text data (10KB-10MB). The feature variance determines all specific values ​​of data size in the model sample set, forming a numerical sequence, such as [5MB, 12MB, 8MB, 20MB]. The mean of the classification feature is obtained by summing all the values ​​in the sequence and dividing by the total number of samples. In this case, the sequence sum is 45MB, and the number of samples is 4, so the mean is 11.25MB. The difference between the feature value of each sample and the mean is calculated, and these differences are squared (e.g., the difference between 5MB and the mean is -6.25MB, which squares to 39.06). All squared values ​​are then summed to obtain the feature variance. A larger feature variance indicates more significant fluctuations in the classification feature across different samples, better reflecting the differences between different categories of data. Therefore, classification features with a feature variance greater than 0.6 are identified as key feature clusters, which can further enhance the classification's discriminative power and provide an accurate input basis for subsequent model training. Furthermore, the central feature vector of each category is calculated (the mean of the feature vectors of all samples in a category, representing the overall feature trend of that category). For example, the mean of the "resolution" feature vector for the "image data in the education field" samples is (1920, 1080), and this mean vector is the central feature vector for the "resolution" of the "image data in the education field". Cosine similarity is used to quantify the similarity between the center vectors of any two categories (the closer to 1, the more similar the category features). The average cosine similarity between all pairwise categories is taken. If the average is >0.6, the category is considered a confusing category. A confusing category is a combination of categories where the average cosine similarity between any two categories is >0.6. These categories show high similarity in key feature clusters; for example, "Word data" and "PDF data" have similar distributions in the "text encoding" feature. Directly using them for training would lead to model classification confusion and requires further decoupling processing.

[0069] Understandably, an autoencoder with a symmetrical structure of a 3-layer encoder and a 3-layer decoder is constructed. The 3-layer encoder is responsible for compressing high-dimensional key feature clusters into a low-dimensional space, while the 3-layer decoder attempts to reconstruct the original features from the low-dimensional space. For key feature clusters contained in confused categories, the autoencoder removes redundant associations through low-dimensional mapping and retains discriminative differences to achieve feature decoupling. The sample data of the decoupled key feature clusters is then freed from confusion interference and retains the core classification features, ultimately forming a model dataset. The model dataset contains filtered and decoupled feature samples, providing accurate basic input for model training.

[0070] In some embodiments of this application, when constructing a multi-head self-attention model based on a model dataset and determining the classification data to be stored based on the multi-head self-attention model, the process includes: dividing the model dataset into a model training set and a model test set; selecting an initial multi-head model based on Transformer; training the initial multi-head model according to the model training set; and testing the initial multi-head model by substituting the model test set into the trained initial multi-head model. If the test value of the currently trained initial multi-head model is greater than or equal to the test value of the previously trained initial multi-head model, training is stopped, and the currently trained initial multi-head model is determined as a multi-head self-attention model. If the test value of the currently trained initial multi-head model is less than the test value of the previously trained initial multi-head model, the learning rate of the currently trained initial multi-head model is adjusted, and training continues until the test value of the currently trained initial multi-head model is greater than or equal to the test value of the previously trained initial multi-head model. Finally, all target data to be stored is substituted into the multi-head self-attention model to determine the classification data to be stored.

[0071] Specifically, Transformer is a deep learning framework based on self-attention mechanisms. It can build an initial multi-head model, which contains a model structure with multiple parallel attention heads. Each attention head independently calculates feature weights, aiming to capture complex relationships in the data and thus improve classification capabilities. The model dataset is divided into a training set and a test set, typically in a 7:3 ratio, to ensure the model's generalization ability. The training set is used to train the initial multi-head model, while the test set is used to evaluate the performance of the trained model. By training the initial multi-head model, the classification judgment for each target data to be stored can be accurately output, avoiding human judgment errors and improving storage management efficiency. The initial multi-head model is trained using data from the training set. In each training iteration, the initial multi-head model attempts to learn patterns and relationships in the data to improve its prediction or classification capabilities. After each training iteration, the model is tested using data from the test set. Test metrics include accuracy and recall to measure the model's performance. If the current post-training test value is lower than the previous test value, it indicates a decline in model performance. In this case, the model's learning rate needs to be adjusted, and training should continue to help the model steadily approach the global optimum. If the current post-training test value is greater than or equal to the previous test value, it indicates that the model's performance has improved or remained stable. Iterative training can then be stopped, assuming the model has reached a satisfactory performance level. At this point, the initial multi-head model after training is designated as a multi-head self-attention model, and all target data to be stored is substituted into this model to determine the classification data to be stored. This improves the accuracy of classifying the target data to be stored, ensuring the stability and reliability of storage management.

[0072] In some embodiments of this application, when stratifying each category of data to be stored based on hierarchical analysis and constructing a judgment matrix, and performing a consistency check on the judgment matrix to determine the comprehensive data score, and then applying an appropriate encryption strategy to all category of data to be stored based on the comprehensive data score to determine encrypted data, the process includes: determining a hierarchical structure based on hierarchical analysis, the hierarchical structure including a target layer, a criterion layer, and a scheme layer; the target layer determining the comprehensive data score; the criterion layer determining the data attributes of the category of data to be stored; and the scheme layer determining the carrier of the data attributes, the data attributes including data size, data structure, data association, data compression, and data source; constructing a judgment matrix based on the data attributes; calculating a weight vector and a maximum eigenvalue based on the judgment matrix; determining a consistency index and an average consistency index based on the maximum eigenvalue; determining a consistency ratio based on the ratio of the consistency index and the average consistency index; determining the comprehensive data score of each category of data to be stored based on the consistency ratio; when the comprehensive data score is greater than a threshold, the encryption strategy is determined to be a composite encryption strategy, and encrypted data is determined based on the composite encryption strategy; when the comprehensive data score is less than or equal to the threshold, the encryption strategy is determined to be an independent encryption strategy, and encrypted data is determined based on the independent encryption strategy.

[0073] Specifically, the target layer is used to calculate the comprehensive score of the data, the criteria layer includes five data attributes: data size (storage resource consumption), data structure (complexity), data association (dependence on other data), data compression (redundancy), and data source (credibility), and the scheme layer corresponds to the specific data to be stored, forming a three-level analysis framework of target layer, criteria layer and scheme layer. The importance of each data attribute in the criteria layer is compared pairwise, and a judgment matrix is ​​constructed using the 1-9 scaling method (matrix element aᵢⱼ represents the importance of data attribute i relative to data attribute j). The consistency index CI is calculated by determining the weight vector (reflecting the proportion of influence of each data attribute) and the maximum eigenvalue λmax. Combined with the average random consistency index RI (preferably 1.12 when there are 5 data attributes), the consistency ratio CR is obtained. If CR < 0.1, the judgment matrix satisfies the consistency judgment logic without contradiction, and the weights are valid; otherwise, the judgment matrix is ​​modified. The scores of each data attribute are weighted using the weight vector to determine the overall data score. Only when the data to be stored exhibits high-risk characteristics (such as extremely large data volume, complex structure, high correlation, low redundancy, and reliable source) in all five data attributes (data size, data structure, data association, data compression, and data source), is the data considered valid. If the overall score is greater than the set data overall score threshold of 0.9, a composite encryption strategy is triggered, thereby focusing on the representative and comprehensive data to be stored. The composite encryption strategy includes at least two of the following encryption methods: AES-128, AES-256, ECC, and Kyber. When the data overall score is less than or equal to the data overall score threshold, an independent encryption strategy is determined, which includes one of the following encryption methods: AES-128, AES-256, ECC, and Kyber. The judgment matrix is ​​transformed into calculable weights and scores, avoiding the bias of human experience and resolving the priority conflict of multiple data attributes in decision-making (such as balancing the impact of "data size" and "data source"). This ensures dynamic encryption of different data categories to be stored, thereby achieving a dynamic balance of security requirements and guaranteeing the stability and flexibility of storage management.

[0074] In some embodiments of this application, when all encrypted data with the same classification attributes are temporarily stored in a sub-memory and data verification is performed in the sub-memory, and the temporarily stored encrypted data is moved to the target sub-memory based on the data verification result, the following steps are included: the data verification includes security detection and structure verification. When the security detection passes, the structure verification is performed. When both the security detection and the structure verification pass, the data verification is successful; otherwise, the data verification fails. When the data verification is successful, the encrypted data stored in the sub-memory is moved to the target sub-memory.

[0075] In some embodiments of this application, when obtaining the storage occupancy rate of the target sub-memory and determining the backup mode of the target sub-memory based on the storage occupancy rate, the process includes: when the storage occupancy rate meets a first condition, a full backup is determined as the backup mode of the target sub-memory; when the storage occupancy rate meets a second condition, a differential backup is determined as the backup mode of the target sub-memory; and when the storage occupancy rate meets a third condition, an incremental backup is determined as the backup mode of the target sub-memory, with the backup levels of full backup, differential backup, and incremental backup decreasing sequentially.

[0076] Specifically, all encrypted data with the same classification attribute is temporarily stored in a sub-storage. The same classification attribute represents the same classification result; for example, all data are video data in the education field, and all data are text data in the technology field. This collection of encrypted data with the same classification attribute reflects the encryption result of a single classification. The data verification process achieves dual protection for encrypted data through a progressive logic from security detection to structural verification. Security detection focuses on data security, including virus signature matching, abnormal permission marking, and encryption integrity verification (such as hash value comparison), while structural verification emphasizes data integrity, checking data format standardization (such as field length and check bits) and logical coherence (such as consistency of related data indexes). Encrypted data is only migrated from the temporary sub-storage to the target sub-storage when both conditions are met, forming a dual barrier of security access and structural compliance. The backup mode is dynamically adapted based on storage occupancy. The first condition is that the storage occupancy is greater than 70%, in which case a full backup (completely copying all data in the target sub-storage) is used. The second condition is that the storage occupancy is less than or equal to 70% but greater than 30%, in which case differential backup is used. The third condition is that the occupancy is less than or equal to 30%, in which case incremental backup is used (only backing up data added / modified since the last backup). The layered verification mechanism first filters malicious data (security detection) and then corrects format errors (structural verification), so that the target sub-storage can meet both security and availability requirements while storing data, reducing the risk of anomalies during subsequent data retrieval. When storing a large amount of encrypted data, full backup ensures data recoverability, while when storing a relatively small amount of encrypted data, differential / incremental backup is used to balance backup integrity and efficiency, improving the stability and resource utilization of storage management.

[0077] In some embodiments of this application, when obtaining the data retrieval status of the target sub-memory in backup mode and determining whether to adjust the backup mode based on the data retrieval status, the process includes: when the backup mode of the target sub-memory is differential backup or incremental backup, obtaining the data retrieval information of the target sub-memory, which includes the number of data retrievals and the data viewing duration; when the data retrieval information meets the data retrieval conditions, determining to adjust the backup mode; otherwise, determining not to adjust the backup mode; and when determining to adjust the backup mode, upgrading the backup level of differential backup or incremental backup by one level.

[0078] Specifically, when the target sub-storage uses differential or incremental backup, data retrieval information is recorded in real time. This information includes the number of data retrievals and the data viewing duration. The number of data retrievals represents the total number of times encrypted data is accessed within a unit of time (e.g., 24 hours), and the data viewing duration represents the duration during which encrypted data is accessed and read after each retrieval. Based on this, data retrieval conditions are set as the basis for adjusting the backup mode trigger. Specifically, the number of data retrievals within 24 hours must be greater than or equal to 5, and the data viewing duration must be greater than or equal to 30 seconds (the cumulative data viewing duration for all data retrievals). If these conditions are met, the encrypted data stored in the target sub-storage is identified as high-access data, and the backup level must be increased. The backup level increases sequentially from incremental backup to differential backup to full backup, with each level upgrade switching to the next higher level. If the target sub-storage's backup mode is already at the highest backup level (full backup), then this full backup is maintained. The backup mode is dynamically adjusted based on data retrieval information, concentrating storage and computing resources on frequently accessed encrypted data, reducing redundancy consumption, meeting the real-time requirements of frequently accessed data, and the frequently retrieved encrypted data is usually associated with the core business. Upgrading its backup level can reduce the risk of data loss caused by incremental / differential backups (such as full backups can be independently restored without being affected by the integrity of previous backups), and enhance the adaptability and stability of data storage management.

[0079] In summary, the beneficial effects of this invention are as follows: At the classification management level, intelligent data classification is achieved through feature heatmaps and multi-head self-attention models, replacing the manual classification mode. This accurately captures dynamic features and related attributes of data, reducing the risk of classification confusion and unclear hierarchy, improving data retrieval efficiency and management stability, adapting to the needs of rapid data growth and dynamic changes, and ensuring the security and reliability of data storage management. Based on hierarchical analysis and consistency checks, a comprehensive data score is formed, enabling differentiated encryption strategies for different types of data. This strengthens encryption to resist malicious attacks and illegal intrusions while avoiding resource waste caused by excessive encryption. It achieves a balance between data security and storage efficiency, effectively reducing the probability of data leakage and damage, and strengthening privacy protection and data integrity. Temporary storage verification ensures the accuracy of data from classification to storage, avoiding risks such as data corruption caused by incorrect storage, thus improving the reliability of data storage management. Furthermore, the backup mode is dynamically determined based on storage occupancy and adjusted in real time according to data retrieval conditions, achieving precise allocation of backup resources, ensuring the availability of storage backups, reducing the risk of data loss, and ultimately ensuring the reliability and stability of data storage management.

[0080] In another preferred embodiment based on the above embodiments, see [reference] Figure 2 As shown, this embodiment provides a data encryption-based storage management system for applying the above-described data encryption-based storage management method, including:

[0081] The data acquisition and analysis module is configured to acquire all data to be stored, preprocess each piece of data to determine the target data to be stored, acquire the model sample set, and construct a feature heatmap based on the model sample set.

[0082] The data classification module is configured to determine key feature clusters based on feature heatmaps, decouple the key feature clusters based on autoencoders to determine the model dataset, construct a multi-head self-attention model based on the model dataset, and determine the classification data to be stored based on the multi-head self-attention model.

[0083] The data processing module is configured to hierarchically classify each category of data to be stored based on hierarchical analysis and construct a judgment matrix, perform consistency checks on the judgment matrix to determine the comprehensive data score, apply corresponding encryption strategies to all category data to be stored based on the comprehensive data score to determine encrypted data, temporarily store all encrypted data with the same classification attributes in a sub-storage, perform data verification in the sub-storage, and move the temporarily stored encrypted data to the target sub-storage according to the data verification result.

[0084] The storage management module is configured to obtain the storage occupancy rate of the target sub-storage, determine the backup mode of the target sub-storage based on the storage occupancy rate, obtain the data retrieval status of the target sub-storage under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or adjustment result.

[0085] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A storage management method based on data encryption, characterized in that, include: Acquire all data to be stored and preprocess each data to be stored to determine the target data to be stored; acquire the model sample set and construct a feature heatmap based on the model sample set. Key feature clusters are determined based on the feature heatmap, and the key feature clusters are decoupled according to the autoencoder to determine the model dataset. A multi-head self-attention model is constructed based on the model dataset, and the classification data to be stored is determined based on the multi-head self-attention model. Based on hierarchical analysis, each category of data to be stored is hierarchically divided and a judgment matrix is ​​constructed. The judgment matrix is ​​then subjected to a consistency check to determine the comprehensive data score. Based on the comprehensive data score, an appropriate encryption strategy is adopted for all category data to be stored to determine encrypted data. All encrypted data with the same category attribute are temporarily stored in a sub-memory, and data verification is performed in the sub-memory. Based on the result of the data verification, the temporarily stored encrypted data is moved to the target sub-memory. Obtain the storage occupancy rate of the target sub-storage, determine the backup mode of the target sub-storage based on the storage occupancy rate, obtain the data retrieval status of the target sub-storage under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or adjustment result; When acquiring a model sample set and constructing a feature heatmap based on the model sample set, the process includes: Determine all feature dimensions of the model sample set, and determine the distribution probability of each feature dimension in each category. Based on the distribution probability, determine the entropy value of each feature dimension, wherein the entropy value is proportional to the distribution difference. A matrix is ​​constructed with the feature dimension as the horizontal axis and the category as the vertical axis. The feature heatmap is constructed based on the data visualization library and the matrix. The color depth of each cell in the feature heatmap is determined by the entropy value of the corresponding feature in that category. When determining key feature clusters based on the feature heatmap, and decoupling the key feature clusters according to the autoencoder to determine the model dataset, the process includes: Based on the color depth, classification features are determined and the feature variance of the classification features is calculated. Based on the feature variance, the key feature clusters are determined. Determine the central feature vector of each category, and determine the cosine similarity based on the central feature vectors of any two categories. Obtain the average value of the cosine similarity between all pairs of categories, and determine the confused category based on the average value. The autoencoder is constructed, which includes a 3-layer encoder and a 3-layer decoder symmetrical to the 3-layer encoder. The input layer dimension of each encoder is consistent with the dimension of the key feature cluster. Based on the autoencoder, the key feature clusters contained in the confusion category are decoupled, and the sample data of the decoupled key feature clusters are determined as the model dataset. When constructing a multi-head self-attention model based on the model dataset, and determining the classification data to be stored based on the multi-head self-attention model, the process includes: The model dataset is divided into a model training set and a model test set. An initial multi-head model is selected based on Transformer. The initial multi-head model is trained according to the model training set, and the model test set is substituted into the trained initial multi-head model for testing. If the test value of the current trained initial multi-head model is greater than or equal to the test value of the previous trained initial multi-head model, then training is stopped, and the current trained initial multi-head model is determined as the multi-head self-attention model. If the test value of the current trained initial multi-head model is less than the test value of the previous trained initial multi-head model, then adjust the learning rate of the current trained initial multi-head model and continue training until the test value of the current trained initial multi-head model is greater than or equal to the test value of the previous trained initial multi-head model. Substitute all target data to be stored into the multi-head self-attention model to determine the classification data to be stored.

2. The storage management method based on data encryption according to claim 1, characterized in that, When acquiring all data to be stored and preprocessing each piece of data to determine the target data to be stored, the process includes: All data to be stored is cleaned, including handling missing values, outliers and duplicate values. The cleaned data is then transformed, including data standardization and data normalization. The integrity and consistency of each piece of data to be stored after the transformation are verified, and it is determined whether it conforms to the storage format. If the verification is successful and the data conforms to the storage format, then all data to be stored will be identified as the target data to be stored. If the verification fails or does not conform to the storage format, the data to be stored that fails the verification or does not conform to the storage format will be retained and a manual verification will be requested. The remaining data to be stored will be determined as the target data to be stored.

3. The storage management method based on data encryption according to claim 2, characterized in that, When performing hierarchical analysis to stratify each category of data to be stored and construct a judgment matrix, and then performing a consistency check on the judgment matrix to determine the comprehensive data score, and applying an appropriate encryption strategy to all category data to be stored based on the comprehensive data score to determine encrypted data, the process includes: Based on the hierarchical analysis, a hierarchical structure is determined, which includes a target layer, a criterion layer, and a scheme layer. The target layer determines the comprehensive score of the data, the criterion layer determines the data attributes of the classified data to be stored, and the scheme layer determines the carrier of the data attributes. The data attributes include data size, data structure, data association, data compression, and data source. The judgment matrix is ​​constructed based on the data attributes. The weight vector and the maximum eigenvalue are calculated based on the judgment matrix. The consistency index and the average consistency index are determined based on the maximum eigenvalue. The consistency ratio is determined based on the ratio of the consistency index and the average consistency index. The comprehensive data score of each classified data to be stored is determined based on the consistency ratio. When the overall data score is greater than the overall data score threshold, the encryption strategy is determined to be a composite encryption strategy, and the encrypted data is determined according to the composite encryption strategy. When the overall data score is less than or equal to the overall data score threshold, the encryption strategy is determined to be an independent encryption strategy, and the encrypted data is determined according to the independent encryption strategy.

4. The storage management method based on data encryption according to claim 3, characterized in that, When temporarily storing all encrypted data with the same classification attribute in a sub-memory, performing data verification in the sub-memory, and moving the temporarily stored encrypted data to the target sub-memory based on the data verification result, the process includes: The data verification includes security checks and structural checks. When the security checks pass, the structural checks are performed. If the security test and the structural verification pass, the data verification is successful; otherwise, the data verification fails. When the data verification is successful, the encrypted data stored in the sub-memory is moved to the target sub-memory.

5. The storage management method based on data encryption according to claim 4, characterized in that, When acquiring the storage occupancy rate of the target sub-memory and determining the backup mode of the target sub-memory based on the storage occupancy rate, the process includes: When the storage occupancy rate meets the first condition, a full backup is determined as the backup mode for the target sub-storage. When the storage occupancy rate meets the second condition, differential backup is determined as the backup mode of the target sub-storage. When the storage occupancy rate meets the third condition, incremental backup is determined as the backup mode for the target sub-storage. The backup levels for full backup, differential backup, and incremental backup decrease sequentially.

6. The storage management method based on data encryption according to claim 5, characterized in that, When obtaining the data retrieval status of the target sub-storage in the backup mode, and determining whether to adjust the backup mode based on the data retrieval status, the process includes: When the backup mode of the target sub-storage is differential backup or incremental backup, the data retrieval information of the target sub-storage is obtained. The data retrieval information includes the number of data retrievals and the data viewing duration. When the data retrieval information meets the data retrieval conditions, it is determined to adjust the backup mode; otherwise, it is determined not to adjust the backup mode. When it is determined that the backup mode should be adjusted, the backup level of the differential backup or incremental backup will be increased by one level.

7. A data encryption-based storage management system, used to apply the data encryption-based storage management method as described in any one of claims 1-6, characterized in that, include: The data acquisition and analysis module is configured to acquire all data to be stored, preprocess each data to be stored to determine the target data to be stored, acquire a model sample set, and construct a feature heatmap based on the model sample set. The data classification module is configured to determine key feature clusters based on the feature heatmap, decouple the key feature clusters according to the autoencoder to determine the model dataset, construct a multi-head self-attention model based on the model dataset, and determine the classification data to be stored based on the multi-head self-attention model. The data processing module is configured to hierarchically classify each type of data to be stored based on hierarchical analysis and construct a judgment matrix, perform consistency checks on the judgment matrix to determine the comprehensive data score, apply corresponding encryption strategies to all types of data to be stored based on the comprehensive data score to determine encrypted data, temporarily store all encrypted data with the same classification attributes in a sub-memory, perform data verification in the sub-memory, and move the temporarily stored encrypted data to the target sub-memory according to the result of the data verification. The storage management module is configured to obtain the storage occupancy rate of the target sub-storage, determine the backup mode of the target sub-storage based on the storage occupancy rate, obtain the data retrieval status of the target sub-storage under the backup mode, determine whether to adjust the backup mode based on the data retrieval status, and complete the storage based on the backup mode or adjustment result.

Images