A data archiving processing method and system based on a shutdown system

Abstract

The present invention discloses a data archiving processing method and system based on a shutdown system. The method includes: classifying data using a multi-dimensional classification model based on deep learning according to the data type, access frequency, and business importance of the shutdown system to obtain a classified data set and its priority label; inputting the classified data set into a combined compression and encryption processing model based on a neural network to obtain a compressed and encrypted data packet; distributing the compressed and encrypted data packet to a distributed storage system, and using an index construction algorithm based on a graph database to generate a data storage path index to obtain a distributed storage index table; and verifying the data integrity using a blockchain-based archiving verification technology according to the distributed storage index table to generate an archiving verification report. By using the embodiments of the present invention, intelligent classification, effective compression, and secure storage of data in the shutdown system can be achieved, ultimately ensuring the integrity and traceability of the data.

Description

Technical Field

[0001] The present invention belongs to the technical field of data archiving, and in particular to a data archiving processing method and system based on a shutdown system. Background Art

[0002] With the rapid development of information technology, a large amount of data is widely generated and used in various industries. Many companies have accumulated a large amount of historical data in the course of their operations, and the management and storage of this data has gradually become a major challenge facing enterprises. In industries such as finance, medical care, and energy, the data is not only huge in volume, but also covers sensitive information, which is related to the security and business compliance of the enterprise. Therefore, how to efficiently and securely archive the data in the shutdown system has become an important issue that needs to be solved urgently. Traditional data archiving methods usually use static classification and simple storage solutions, which often cannot meet the requirements of modern enterprises for efficiency, security, and flexibility of data storage. In these methods, the classification and processing of data often rely on manual intervention, resulting in inefficient data archiving and difficulty in adapting to rapidly changing business needs. Summary of the invention

[0003] The purpose of the present invention is to provide a data archiving processing method and system based on a shutdown system to address the deficiencies in the prior art, and to achieve intelligent classification, effective compression and secure storage of data in the shutdown system, ultimately ensuring the integrity and traceability of the data.

[0004] An embodiment of the present application provides a data archiving processing method based on shutting down a system, the method comprising:

[0005] According to the data type, access frequency and business importance of the shut-down system, a multi-dimensional classification model based on deep learning is used to classify the data. The multi-dimensional classification model dynamically divides the archiving priority of the data through attention technology and adaptive weight allocation algorithm to obtain the classified data set and its priority label;

[0006] Inputting the classified data set into a compression and encryption joint processing model based on a neural network to perform data compression and encryption, wherein the joint processing model combines a quantized compression algorithm and homomorphic encryption technology to achieve efficient compression while ensuring data security, thereby obtaining a compressed and encrypted data packet;

[0007] Distribute the compressed and encrypted data packets to a distributed storage system, and generate a data storage path index using an index construction algorithm based on a graph database, wherein the index construction algorithm optimizes data retrieval efficiency and storage load balancing through dynamic hash mapping and a distributed consistency protocol to obtain a distributed storage index table;

[0008] According to the distributed storage index table, use the blockchain-based archiving verification technology to verify data integrity. Among them, the archiving verification technology combines smart contracts and distributed ledger technology to monitor the data storage status and access records in real time, generate an archiving verification report, and ensure the reliability and traceability of data archiving.

[0009] Optionally, according to the data type, access frequency, and business importance of the shutdown system, use a multi-dimensional classification model based on deep learning to classify the data. Among them, the multi-dimensional classification model dynamically divides the archiving priority of the data through attention technology and an adaptive weight allocation algorithm, and obtains the classified data set and its priority label, including:

[0010] According to the data type in the shutdown system, use a data acquisition framework based on edge computing to obtain multi-source data in real time, and through an adaptive data cleaning algorithm, filter out noise and fill in missing values in the data to generate a preliminary standardized data set;

[0011] For the preliminary standardized data set, use a multi-dimensional classification model based on deep learning, combine the data type, access frequency, and business importance, extract multi-dimensional features of the data, and capture the correlation between different dimensional features through multi-head attention technology to generate a preliminary feature representation;

[0012] For the preliminary feature representation, use a priority division method based on the adaptive weight allocation algorithm, combine the access frequency and business importance of the data, dynamically allocate the archiving priority of the data, and optimize the accuracy and reasonableness of the weight allocation through attention technology to generate a preliminary priority label;

[0013] For the preliminary priority label, use a data classification method based on the clustering algorithm, combine the data type and priority label, divide the data into different categories, and ensure the accuracy and consistency of the classification through dynamic threshold adjustment technology to generate the classified data set and its priority label.

[0014] Optionally, input the classified data set into a combined compression and encryption processing model based on a neural network for data compression and encryption. Among them, the combined processing model combines a quantization compression algorithm and homomorphic encryption technology to achieve efficient compression while ensuring data security, and obtains a compressed and encrypted data packet, including:

[0015] For the classified data set, use a quantization compression algorithm based on a neural network to convert high-precision data into a low-precision representation, and dynamically adjust the quantization precision through an adaptive quantization threshold adjustment technology to generate preliminary compressed data on the premise of minimizing data information loss;

[0016] For the preliminary compressed data, an encryption method based on homomorphic encryption technology is adopted. Combining the priority tags and security requirements of the data, the data is encrypted. Through lightweight key management technology, the efficiency and security of the encryption process are ensured, and preliminary encrypted data is generated;

[0017] For the preliminary encrypted data, a joint optimization method based on neural networks is adopted. Combining the quantization compression algorithm and homomorphic encryption technology, the compression and encryption parameters are dynamically adjusted. Through the multi-objective optimization algorithm, the compression efficiency and encryption security are balanced, and preliminary compressed and encrypted data packets are generated;

[0018] For the preliminary compressed and encrypted data packets, a data integrity verification method based on hash check is adopted to ensure that the data is not damaged during the compression and encryption process. Through the feedback correction technology, the compression and encryption parameters are dynamically adjusted, and the final compressed and encrypted data packets are generated.

[0019] Optionally, when distributing the compressed and encrypted data packets to the distributed storage system, an index construction algorithm based on the graph database is used to generate a data storage path index. Among them, the index construction algorithm optimizes the data retrieval efficiency and storage load balance through dynamic hash mapping and distributed consistency protocol, and obtains a distributed storage index table, including:

[0020] For the compressed and encrypted data packets, a distribution method based on the distributed storage system is adopted. Combining the priority tags of the data and the load status of the storage nodes, the data storage path is planned. Through the dynamic hash mapping algorithm, the balance of data distribution is ensured, and a preliminary storage path plan is generated;

[0021] For the preliminary storage path plan, an index construction method based on the graph database is adopted. The data storage path is abstracted into nodes and edges in the graph structure. Through the distributed consistency protocol, the consistency and reliability of the index are ensured, and a preliminary index structure is generated;

[0022] For the preliminary index structure, an index optimization method based on dynamic load balancing is adopted. Combining the real-time load status of the storage nodes and the data access frequency, the index structure is dynamically adjusted. Through the adaptive hash mapping technology, the data retrieval efficiency and storage load balance are optimized, and an optimized index structure is generated;

[0023] For the optimized index structure, an index table generation method based on visualization technology is adopted. The index structure is mapped into a distributed storage index table. Through the real-time monitoring technology, the accuracy and consistency of the index table are ensured, and the final distributed storage index table is generated.

[0024] Optionally, based on the distributed storage index table, a blockchain-based archival verification technology is used to verify data integrity. The archival verification technology combines smart contracts and distributed ledger technology to monitor the data storage status and access records in real time, generate an archival verification report, and ensure the reliability and traceability of data archiving, including:

[0025] Based on the distributed storage index table, a blockchain-based archival verification technology is adopted, and smart contracts are combined to verify the data storage status. Through the hash check algorithm, it is ensured that the data has not been damaged or tampered with during storage, and a preliminary integrity verification result is generated.

[0026] For the preliminary integrity verification result, a recording method based on distributed ledger technology is adopted to write the data storage status and access records into the blockchain. Through the consensus technology, the immutability and traceability of the records are ensured, and a preliminary distributed ledger record is generated.

[0027] For the distributed ledger record, a real-time monitoring method based on smart contracts is adopted, combined with the data access frequency and storage status to detect abnormal behaviors. Through the anomaly detection algorithm, a preliminary monitoring report is generated.

[0028] For the preliminary monitoring report, a report generation method based on natural language generation technology is adopted to integrate the integrity verification result, the distributed ledger record, and the monitoring report into an archival verification report. Through the visualization technology, a final archival verification report is generated.

[0029] Another embodiment of the present application provides a data archiving processing system based on a shutdown system. The system includes:

[0030] A classification module, which is used to classify data according to the data type, access frequency, and business importance of the shutdown system by using a multi-dimensional classification model based on deep learning. The multi-dimensional classification model dynamically divides the archival priority of the data through the attention technology and the adaptive weight allocation algorithm, and obtains the classified data set and its priority label.

[0031] A processing module, which is used to input the classified data set into a combined compression and encryption processing model based on a neural network for data compression and encryption. The combined processing model combines the quantization compression algorithm and the homomorphic encryption technology to achieve efficient compression while ensuring data security, and obtains a compressed and encrypted data packet.

[0032] An indexing module, which is used to distribute the compressed and encrypted data packets to a distributed storage system, and generates a data storage path index by using an index construction algorithm based on a graph database. Among them, the index construction algorithm optimizes the data retrieval efficiency and storage load balance through dynamic hash mapping and a distributed consistency protocol, and obtains a distributed storage index table;

[0033] An archiving module, which is used to verify the data integrity according to the distributed storage index table by using an archiving verification technology based on blockchain. Among them, the archiving verification technology combines smart contracts and distributed ledger technology to monitor the data storage status and access records in real time, generates an archiving verification report, and ensures the reliability and traceability of data archiving.

[0034] Another embodiment of the present application provides a storage medium in which a computer program is stored. Among them, the computer program is set to execute the method described in any one of the above when running.

[0035] Another embodiment of the present application provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is set to run the computer program to execute the method described in any one of the above.

[0036] Compared with the prior art, a data archiving processing method based on a shutdown system provided by the present invention classifies data by using a multi-dimensional classification model based on deep learning according to the data type, access frequency, and business importance of the shutdown system, and obtains a classified data set and its priority label; inputs the classified data set into a compression and encryption joint processing model based on a neural network to obtain a compressed and encrypted data packet; distributes the compressed and encrypted data packet to a distributed storage system, and generates a data storage path index by using an index construction algorithm based on a graph database to obtain a distributed storage index table; verifies the data integrity according to the distributed storage index table by using an archiving verification technology based on blockchain, and generates an archiving verification report, so as to realize the intelligent classification, effective compression, and secure storage of data in the shutdown system, and finally ensure the integrity and traceability of the data. Description of the Drawings

[0037] Figure 1 It is a hardware structure block diagram of a computer terminal for a data archiving processing method based on a shutdown system provided by an embodiment of the present invention;

[0038] Figure 2 It is a flow schematic diagram of a data archiving processing method based on a shutdown system provided by an embodiment of the present invention;

[0039] Figure 3Schematic diagram of a data archiving and processing system based on a shutdown system provided by an embodiment of the present invention. Detailed implementation manners

[0040] The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0041] An embodiment of the present invention first provides a data archiving and processing method based on a shutdown system. This method can be applied to electronic devices, such as computer terminals, specifically ordinary computers, etc.

[0042] Below, taking running on a computer terminal as an example, it will be described in detail. Figure 1 Hardware structure block diagram of a computer terminal for a data archiving and processing method based on a shutdown system provided by an embodiment of the present invention. As Figure 1 shown, the computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the memory may include a non-volatile storage medium and an internal memory.

[0043] The non-volatile storage medium can store an operating system and computer programs. The computer programs include program instructions. When the program instructions are executed, the processor can execute any data archiving and processing method based on a shutdown system.

[0044] The processor is used to provide computing and control capabilities to support the operation of the entire computer device.

[0045] The internal memory provides an environment for the operation of the computer programs in the non-volatile storage medium. When the computer programs are executed by the processor, the processor can execute any data archiving and processing method based on a shutdown system.

[0046] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that Figure 1 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0047] It should be understood that the processor can be a Central Processing Unit (CPU), and the processor can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0048] See Figure 2 , embodiments of the present invention provide a data archiving processing method based on a shutdown system, which may include the following steps:

[0049] S201, classify the data using a multi-dimensional classification model based on deep learning according to the data type, access frequency, and business importance of the shutdown system. Among them, the multi-dimensional classification model dynamically divides the archiving priority of the data through an attention technique and an adaptive weight allocation algorithm, and obtains the classified data set and its priority label;

[0050] In this process, the system uses deep learning algorithms to identify different types of data by analyzing the characteristics of the data, including the structure, content, and usage frequency of the data. The multi-dimensional classification model focuses on the key characteristics of the data through the attention technique, such as the access frequency in historical use and the importance to the business process. At the same time, the adaptive weight allocation algorithm dynamically adjusts the weights of these characteristics according to real-time feedback, so that for high-value or high-frequency accessed data, the model can give a higher degree of attention, thereby automatically forming the archiving priority of the data.

[0051] This deep learning-based classification method can significantly improve the efficiency and accuracy of data archiving. By dynamically dividing the archiving priority of the data, the system ensures that important and high-frequency data can be preferentially processed and securely stored, avoiding the loss of important information. In addition, this intelligent classification helps to reasonably utilize storage resources, reduce unnecessary storage overhead, and then improve the flexibility and response ability of the entire data management. Combining the application of the attention mechanism and the adaptive algorithm, the system can continuously learn and adapt to new data characteristics and business requirements, achieving long-term optimization and improvement.

[0052] Specifically, according to the data types in the shutdown system, a data acquisition framework based on edge computing can be adopted to obtain multi-source data in real time. Through an adaptive data cleaning algorithm, the data is filtered for noise and missing values are filled to generate a preliminary standardized data set.

[0053] In this step, the system will adopt a data acquisition framework based on edge computing to achieve real-time capture of various types of data in the shutdown system. The introduction of edge computing brings data processing closer to the data source, reducing latency and improving acquisition efficiency. The system uses an adaptive data cleaning algorithm to automatically identify and filter noise in the data, such as outliers and incorrect inputs, while implementing a missing value filling strategy to fill missing data by using the mean or median of surrounding data, thus ensuring that the generated data set has high integrity and accuracy.

[0054] This way of data acquisition and cleaning can maximize data quality, providing a solid foundation for subsequent data analysis and classification. By effectively filtering and cleaning data at the acquisition stage, the system can reduce data redundancy, improve the efficiency of subsequent processing, and thus effectively shorten the overall data processing cycle. A high-quality standardized data set is more conducive to the training of deep learning models, thereby improving the classification accuracy of multi-dimensional classification models.

[0055] In this step, the system will first deploy edge computing nodes to obtain multi-source data in the shutdown system. The advantage of edge computing is that it can move data processing tasks as close as possible to the source of data generation, thereby reducing network latency and improving the real-time nature of data acquisition. The system designs a data acquisition framework that can support the access of different data sources, including sensor data, user input, and historical records. For example, when the system collects real-time monitoring data from multiple sensors (such as temperature, humidity, flow rate, etc.), it can collect this data in a streaming manner to the edge node for aggregation and preliminary processing.

[0056] After the data is acquired, the system will use an adaptive data cleaning algorithm to process the obtained data. The algorithm analyzes the characteristics of the data set to automatically identify and filter noise. For example, during the acquisition process, if the system finds that certain sensor data is abnormal (such as being higher or lower than the preset reasonable range), then this data will be marked as noise and filtered out. At the same time, for missing values, the algorithm will fill the missing data through interpolation methods or statistical methods (such as mean filling, median filling, etc.) to ensure the integrity and reliability of the generated preliminary standardized data set.

[0057] After completing data cleaning, the generated preliminary standardized dataset will be formatted into a consistent structure for subsequent processing. This structure usually adopts a common data format (such as CSV or JSON), ensuring that each record contains necessary fields, such as data type, timestamp, and collection source, etc. This process of data standardization not only improves the consistency of data but also lays a good foundation for subsequent analysis. A high-quality dataset is the key to the performance of subsequent deep learning models.

[0058] For the preliminary standardized dataset, a multi-dimensional classification model based on deep learning is adopted. Combining data type, access frequency, and business importance, multi-dimensional features of the data are extracted. Through the multi-head attention technology, the correlation relationships between features of different dimensions are captured to generate a preliminary feature representation.

[0059] In this step, the system will apply a multi-dimensional classification model based on deep learning to the preliminary standardized dataset for feature extraction. Combining data type, access frequency, and business importance, the model will analyze the multi-dimensional features of each piece of data and capture the complex relationships between features through the multi-head attention mechanism. For example, some data may be important under high-frequency access conditions, while other data has its unique value in key business processes. Through multi-head attention, the model can ensure that these information are fully considered to generate a more expressive feature representation.

[0060] The main significance of this step lies in enhancing the model's ability to understand data features. By using deep learning models and multi-head attention mechanisms, the system can better identify potential rules and relationships in the data, which is crucial for subsequent classification decisions. The generated preliminary feature representation reflects the true value of the data, enabling the model to make a more reasonable classification priority division on a multi-dimensional basis, improving the overall classification accuracy and reliability.

[0061] After inputting the preliminary standardized dataset into a multi-dimensional classification model based on deep learning, the system will first embed the data and transform it into a tensor form suitable for model processing. At this time, the model will analyze and extract potential multi-dimensional features according to data type, access frequency, and business importance. For example, the system may analyze the access behavior data of users, identify the usage habits of different user groups for specific data, and thus extract features related to user behavior.

[0062] Next, the system will use the multi-head attention mechanism to process the extracted multi-dimensional features. The multi-head attention mechanism can focus on the relationships between data features from different perspectives, enhancing the model's expressive power. For example, for a set of features, some of which are directly related to business importance, while others may better reflect the usage frequency of the data. This mechanism allows the model to process this information in parallel in multiple "heads" and gradually learn the importance of each feature in different contexts.

[0063] Finally, after being processed by the multi-head attention, the model will generate preliminary feature representations. These feature representations can be regarded as multi-dimensional comprehensive descriptions of each piece of data, containing the representation information of the data and the evaluation of its importance. These feature descriptions will provide the necessary information basis for the subsequent dynamic division of priorities, ensuring the rationality and accuracy of the final classification.

[0064] For the preliminary feature representations, a priority division method based on the adaptive weight allocation algorithm is adopted. Combining the access frequency and business importance of the data, the archival priorities of the data are dynamically assigned. Through the attention technology, the accuracy and rationality of the weight allocation are optimized to generate preliminary priority labels;

[0065] In this step, the system will apply a method based on the adaptive weight allocation algorithm to the preliminary generated feature representations to divide the priorities of the data. The system dynamically adjusts their weights by analyzing the access frequency and business importance of each piece of data and assigns priorities to the data based on these weights. The application of the attention technology enables the system to focus on the features most relevant to data archiving, thus achieving a more reasonable division of archival priorities and ensuring that high-value data is processed quickly.

[0066] Through this dynamic priority division, the system significantly improves the decision-making efficiency of data archiving, ensures that important information can be processed and stored in a timely manner, and reduces the potential business losses caused by information delays. At the same time, accurate priority allocation can not only optimize the use of storage resources but also enhance the system's response ability to user needs, increasing the flexibility and adaptability of the business.

[0067] Based on the preliminary feature representations, the system will use the adaptive weight allocation algorithm to dynamically calculate the archival priority of each piece of data. This process will first combine the access frequency and business importance of the data and use their respective weights as inputs to the algorithm. For example, some data may occupy a higher priority because they are frequently accessed, while other data may also be given a higher weight because they play an important role in key decisions.

[0068] During the process of dynamically calculating weights, the system also utilizes attention technology to optimize the weight allocation of features. By focusing on the influence of different features in data archiving, the model can more accurately assign priorities. The system will traverse all features, identify those directly related to the archiving decision, and assign corresponding attention scores to them, ultimately calculating the comprehensive priority label for each piece of data. For example, if the access frequency of a certain piece of data surges during a specific period, the model will automatically detect this change and dynamically increase its priority.

[0069] Finally, the generated preliminary priority labels will serve as a guide for subsequent classification, ensuring that high-importance and high-frequency data are processed in a timely manner. This process not only guarantees the flexibility of the algorithm but also ensures the scientific nature of priority allocation, laying a guiding foundation for the effectiveness of the entire data management system.

[0070] For the preliminary priority labels, a data classification method based on clustering algorithms is adopted. Combining the data type and the priority label, the data is divided into different categories. Through dynamic threshold adjustment technology, the accuracy and consistency of classification are ensured, and the classified data set and its priority label are generated.

[0071] In this step, the system will use a data classification method based on clustering algorithms to divide the data into different categories according to the preliminary priority labels and data types. The clustering algorithm can automatically group similar data based on data characteristics and priorities. This classification not only considers the business importance of the data but also affects subsequent data management strategies. The dynamic threshold adjustment technology ensures that during the classification process, appropriate thresholds can be adaptively set for different categories to optimize the classification effect.

[0072] The key to this step is to ensure the accuracy and consistency of data classification, thus laying a good foundation for subsequent data archiving and processing. By reasonably classifying the data, the system can target data archiving and storage, avoiding complex data management and processing procedures, and improving the efficiency of data processing. This classification method also facilitates subsequent analysis and retrieval, making the use of data more flexible and efficient.

[0073] In this step, the system will use a method based on clustering algorithms to combine the adopted preliminary priority labels with the data types for automatic data classification. The system first selects a clustering algorithm suitable for the current data set (such as K-Means, hierarchical clustering, etc.) and applies it to the data set that meets certain conditions to discover the internal connections and similarities among the data. For example, the system may cluster all the data marked as high priority to identify their commonalities and thus divide the data into different groups.

[0074] As the clustering progresses, the system also adopts dynamic threshold adjustment technology to ensure the accuracy and consistency of the classification results. The dynamic threshold is set based on the real-time evaluation of the classification effect. The system continuously monitors the clustering results and dynamically adjusts the threshold according to the number of data and the feature similarity within different categories to optimize the classification results. For example, if the number of data in a certain category is too small, the system may lower the clustering threshold of this category to include more data and ensure that important information is not fragmented.

[0075] Finally, after the processing of the clustering algorithm and dynamic threshold adjustment, the system will generate a classified data set and its priority labels. This result not only provides a basis for subsequent data archiving and storage but also enables a good structure for future data retrieval and analysis, making the overall data management more efficient and systematic.

[0076] S202, input the classified data set into a neural network-based combined compression and encryption processing model for data compression and encryption. Among them, the combined processing model realizes efficient compression while ensuring data security by combining quantization compression algorithms and homomorphic encryption technology, and obtains a compressed and encrypted data packet.

[0077] In this step, the classified data set is input into a neural network-based combined compression and encryption processing model for data compression and encryption. The combined processing model will utilize the combination of quantization compression algorithms and homomorphic encryption technology to ensure that the integrity and security of the data are not lost while compressing the data volume. The quantization compression algorithm reduces the precision of the data, converting high-precision data into low-precision representations, thereby reducing the storage space occupied by the data. Homomorphic encryption technology enables encrypted data to remain operable during processing, that is, calculations can still be performed in the encrypted state, avoiding the risk of exposing the original data during the exchange of sensitive data. This effectively compresses the data without loss and accelerates the efficiency of data transmission and storage.

[0078] This combined compression and encryption processing mechanism has important practical significance. On the one hand, efficient data compression can significantly reduce storage costs and bandwidth requirements. Especially when dealing with large-scale data, it can quickly improve the data transmission efficiency and storage capacity. On the other hand, ensuring data privacy and security through homomorphic encryption can effectively prevent data from being stolen or tampered with during processing, ensuring the security of data during storage and transmission. The implementation of this mechanism enables enterprises to not only ensure the security of data when dealing with data in shut-down systems but also improve the data processing efficiency, enhancing the overall performance and usability of the system.

[0079] Specifically, for the classified data set, a quantization compression algorithm based on neural network can be adopted to convert high-precision data into low-precision representation. Through the adaptive quantization threshold adjustment technology, the quantization precision is dynamically adjusted to generate preliminary compressed data on the premise of ensuring the minimum loss of data information.

[0080] In this step, the system quantizes and compresses the classified data set. The core of the quantization compression algorithm lies in converting high-precision data (such as floating-point numbers) into low-precision representation (such as fixed-point numbers), thereby effectively reducing the storage space of data. For example, a high-precision measurement value may need to be stored using 32-bit floating-point numbers, while through quantization, it can be compressed into 16-bit or 8-bit fixed-point numbers, thus saving storage overhead. This compression method is not limited to numerical data and can also be applied to data in multiple fields such as images and audio, ensuring that data processing is not restricted due to excessive storage costs.

[0081] In this process, the system uses the adaptive quantization threshold adjustment technology to dynamically adjust the quantization precision according to the characteristics of each data feature and its distribution in the data set. For example, for some important features, the system may choose a higher quantization precision, while for features with less impact on the business, a lower quantization precision is adopted. In this way, the system can effectively find the best balance point between the compression rate and information loss, maximizing the preservation of data information integrity.

[0082] Through quantization compression, the system can significantly reduce the storage space occupied by data, thereby improving the processing efficiency when facing massive data. Especially in scenarios where long-term data needs to be stored or real-time data analysis is required, the quantization technology can reduce the storage burden and improve the response speed of the system. At the same time, the strategy of dynamically adjusting the quantization precision ensures that the basic validity of the data is not affected during compression, which is crucial for subsequent data analysis and processing, maintaining the availability and accuracy of the data, and ensuring the flexibility and efficiency of the entire system in data management.

[0083] In this step, the system first receives the classified data set and applies a quantization compression algorithm based on neural network to each data feature. The algorithm aims to convert high-precision data (such as floating-point numbers) into low-precision representation (such as fixed-point numbers) to reduce the storage space occupied by data. For example, assume that the temperature data recorded by a certain sensor is in floating-point form (such as 23.456°C). After quantization compression, this data can be converted into a lower-precision fixed-point number form (such as 23°C), which not only reduces the storage requirement but also improves the efficiency of subsequent data processing.

[0084] To ensure that the information loss during the compression process is minimized, the system adopts an adaptive quantization threshold adjustment technique, which monitors the changes in data characteristics in real time and automatically adjusts the quantization accuracy according to the characteristics of different data. For example, for some feature data with large variation amplitudes, the system may choose to retain a higher quantization accuracy, while for data with smaller variations, the quantization accuracy can be reduced. This dynamic adjustment mechanism enables the system to flexibly optimize the compression strategy according to the actual situation, ensuring that the generated compressed data can efficiently store while retaining the key information.

[0085] Finally, the data after quantization compression is aggregated to form a preliminary set of compressed data. These compressed data not only perform excellently in saving storage space but also lay the foundation for subsequent encryption steps. The low-precision representation of the compressed data ensures the efficiency of storage and transmission, enabling the subsequent processing steps to proceed more quickly and overall enhancing the system's response speed and processing capacity.

[0086] For the preliminary compressed data, an encryption method based on homomorphic encryption technology is adopted. Combining the priority tags and security requirements of the data, the data is encrypted. Through lightweight key management technology, the efficiency and security of the encryption process are ensured, and preliminary encrypted data is generated;

[0087] In this step, the system will perform homomorphic encryption on the preliminary data after quantization compression. Different from traditional encryption methods, homomorphic encryption allows operations on encrypted data without first decrypting it, which keeps the data secure during the processing. During implementation, the system will apply different encryption strategies to different types of data according to the priority tags and security requirements of the data. For example, for highly sensitive data, the system may choose a stronger encryption algorithm, while for low-sensitivity data, a more basic encryption method is adopted to optimize the processing efficiency while meeting the security requirements.

[0088] In addition, lightweight key management technology plays an important role in this process. The system will generate and manage the keys required during the encryption process, ensuring that the distribution and storage of the keys meet security standards while providing efficient key operations. For example, a hash algorithm can be used to generate the corresponding hash value for the key, thereby verifying its validity without exposing the key. This process not only protects the security of the data but also improves the flexibility and response speed of the system in data encryption processing.

[0089] Homomorphic encryption bridges the gap between data security and processing capabilities, allowing users to perform necessary computations while maintaining data privacy. Such technological applications are particularly suitable for use in cloud computing or multi-party collaboration environments, effectively avoiding security risks brought about by data leakage. Additionally, through lightweight key management, the encryption process of the system can be carried out efficiently, enabling users to enjoy a smooth operation experience while receiving security protection. Ultimately, the initially encrypted data provides security for subsequent data storage and retrieval, ensuring integrity and reliability during the data archiving process.

[0090] In this step, the system will apply homomorphic encryption technology to encrypt the initial compressed data. The core advantage of this technology lies in allowing computations to be performed on the encrypted data without decryption, thereby protecting data privacy. For example, for a set of data containing users' sensitive information, using homomorphic encryption can ensure that the original content of the data will not be leaked during processing, while still being able to perform necessary computations such as statistics or analysis. This makes the technology particularly suitable for scenarios where sensitive data needs to be securely processed in a cloud computing environment.

[0091] During the encryption process, the system will dynamically select a suitable homomorphic encryption strategy by combining the priority tags and security requirements of each piece of data. For example, for highly sensitive financial transaction data, the system may choose to adopt a complex encryption process based on encryption algorithms (such as Paillier or RGH) to ensure the security of the data after encryption. At the same time, lightweight key management technology can improve the efficiency of the encryption process, ensuring the efficiency of key generation, distribution, and use. For example, the system can use a random number generator to generate encryption keys and adopt hash technology to verify the validity and security of the keys, thereby greatly reducing the operational and maintenance complexity.

[0092] Ultimately, the data after homomorphic encryption will form an initial encrypted data packet, ensuring security and privacy during storage and transmission. The encryption process at this stage provides a secure and reliable data foundation for the subsequent joint optimization stage, ensuring that the data will not be threatened by damage or leakage during processing, thereby enhancing the overall security architecture of the system.

[0093] For the initial encrypted data, a joint optimization method based on neural networks is adopted, combining quantization compression algorithms and homomorphic encryption technology, dynamically adjusting compression and encryption parameters, and through a multi-objective optimization algorithm, balancing compression efficiency and encryption security to generate an initial compressed and encrypted data packet;

[0094] In this step, the system will perform joint optimization on the preliminarily encrypted data, aiming to balance the compression efficiency of the data and the security of encryption. The joint optimization method based on neural network can dynamically adjust the parameters of quantization compression and encryption algorithm by learning the characteristics of historical data. For example, the system can analyze the characteristics of the current data and its access pattern, and identify which data should adopt a higher compression ratio to save storage, while which data needs more refined encryption to ensure security.

[0095] By introducing a multi-objective optimization algorithm, the system can weigh between compression efficiency and encryption strength and select the parameter setting that best suits the characteristics of the current data. This algorithm usually considers multiple objective outputs simultaneously. For example, it can achieve the optimal compression ratio while ensuring data integrity, and dynamically enhance the encryption strength when necessary to adapt to changes in external risks. This intelligent optimization strategy ensures the flexibility of data during processing and can adapt to various operating conditions in real time.

[0096] Through this joint optimization, the system can ensure the security of information while guaranteeing the efficient compression of data, reducing the potential risk of data leakage. This flexible adjustment ability not only adapts to various types of data requirements but also provides scalability for future data processing. Therefore, the generated preliminarily compressed and encrypted data packets not only save storage space but also improve the security and reliability of data during transmission and storage, playing an important role in promoting the efficiency of the entire data processing system.

[0097] In this step, the system will apply a joint optimization method based on neural network to the preliminarily encrypted data. This method aims to dynamically adjust the quantization compression and homomorphic encryption parameters by learning and analyzing data characteristics to achieve a balance between compression efficiency and encryption security. Specifically, the system can use a trained neural network model to evaluate the characteristics of the current data and determine the most suitable compression and encryption parameters based on this.

[0098] For example, the system may analyze the access frequency, sensitivity, and compression effect of different data types. For frequently accessed low-sensitivity data, the system can choose a higher compression ratio to improve storage efficiency, while for sensitive data, it gives priority to enhancing encryption strength. In this process, combined with the application of the multi-objective optimization algorithm, the system can handle multiple optimization objectives simultaneously, such as compression ratio, processing speed, and encryption strength. Through this multi-level optimization, the system can find the best balance among different processing requirements, making the processing process more efficient and secure.

[0099] Finally, the generated preliminary compressed and encrypted data packet achieves excellent storage performance while ensuring data integrity and security. This data packet provides an effective basis for subsequent integrity verification steps, ensuring that the entire data processing flow reaches an optimal solution between efficiency and security, and improving the adaptability of the system in data management.

[0100] For the preliminary compressed and encrypted data packet, a data integrity verification method based on hash check is adopted to ensure that the data is not damaged during the compression and encryption process. Through the feedback correction technology, the compression and encryption parameters are dynamically adjusted to generate the final compressed and encrypted data packet.

[0101] In this step, the system will verify the data integrity of the preliminary compressed and encrypted data packet to ensure that the data has not been damaged or tampered with during the entire compression and encryption process. To this end, the system will adopt a method based on hash check. First, a hash operation is performed on the preliminary data packet to generate a hash value of a fixed length, which is used to represent the integrity of the data. When the data packet is created, the system will compare the hash value with the original uncompressed and unencrypted data to verify whether the data remains complete and secure during the processing. This process is crucial, especially when dealing with sensitive data, and can effectively prevent risks caused by data loss or tampering.

[0102] At the same time, the system will also implement the feedback correction technology to dynamically adjust the compression and encryption parameters. By analyzing the results of the hash check, the system can identify potential problems in data processing. For example, if the hash values do not match, the system will trigger a feedback mechanism to automatically adjust the quantization compression ratio or encryption intensity to seek solutions. This mechanism enables the system to self-correct and optimize during the data processing process, thus ensuring that the finally generated data packet is both secure and efficient.

[0103] This integrity verification process provides a strong security guarantee for data processing, ensuring that the data will not be damaged or tampered with during the compression and encryption process, and laying a solid foundation for subsequent data transmission and storage. By implementing the hash check and dynamic feedback correction technology, the system can achieve efficient self-adjustment, making the data processing process more flexible and adaptable. This not only enhances the security of the data but also improves the reliability of the entire data archiving process, providing strong support for the subsequent management and use of the data.

[0104] In this step, the system uses a data integrity verification method based on hash checking to ensure that the initially compressed and encrypted data packets are not damaged during processing. Hash checking is performed by generating a hash value of a fixed length for the data content, and this hash value can uniquely represent the data. When a data packet is created, the system calculates its hash value and compares it with the hash value of the original data during subsequent processing to confirm its integrity and consistency. This process can effectively prevent data from being damaged or tampered with due to operation errors or external attacks.

[0105] In addition, if the result of the hash check shows that the data is abnormal or inconsistent, the system will immediately enable the feedback correction technology. This technology will analyze the possible reasons for the hash mismatch and correct the compression and encryption parameters according to the dynamically adjusted results. For example, assuming that the hash value of a data packet does not match the original value, the system can reduce the compression ratio to retain more information or increase the encryption strength to enhance the security of the data. This feedback mechanism ensures the integrity and security of the data under any circumstances and improves the robustness of the system.

[0106] Finally, after integrity verification and necessary adjustments, the system will generate the final compressed and encrypted data packet. This data packet can ensure the stability and security during storage and transmission, laying a solid foundation for subsequent data management and use. This step not only enhances the security guarantee of the data but also improves the reliability of the entire data processing process, ensuring the efficient connection of all links and enabling users to perform data operations with more confidence.

[0107] S203. Distribute the compressed and encrypted data packet to the distributed storage system, and generate a data storage path index using an index construction algorithm based on a graph database. Among them, the index construction algorithm optimizes the data retrieval efficiency and storage load balance through dynamic hash mapping and a distributed consistency protocol to obtain a distributed storage index table.

[0108] In this method, first, the compressed and encrypted data packet is transmitted to the distributed storage system to ensure the high availability and reliability of the data. At this time, the system generates a data storage path index using an index construction algorithm based on a graph database. Specifically, this algorithm implements a dynamic hash mapping technique to distribute the data on different storage nodes, thereby avoiding a certain node becoming a bottleneck due to overloading. At the same time, using the distributed consistency protocol, the system ensures the data consistency in all storage nodes to avoid data redundancy or inconsistency problems and guarantees the efficiency and stability of data access. The finally formed storage path index not only covers the location information of data storage but also provides efficient support for subsequent data retrieval.

[0109] By distributing the compressed and encrypted data packets to a distributed storage system and generating corresponding storage path indexes, the data access efficiency and availability are greatly improved. In summary, the index construction algorithm based on the graph database optimizes the data retrieval efficiency, enabling fast and accurate positioning of frequently read data and enhancing the user experience. In addition, the combination of dynamic hash mapping and distributed consistency protocol ensures the performance of the overall system in load balancing, guaranteeing the security and effectiveness of data during storage. This method is applicable to large-scale data storage and management scenarios, especially enterprise information systems involving sensitive information.

[0110] Specifically, for compressed and encrypted data packets, a distribution method based on a distributed storage system can be adopted. Combining the priority tags of the data and the load status of the storage nodes, plan the data storage path. Through the dynamic hash mapping algorithm, ensure the balance of data distribution and generate a preliminary storage path plan.

[0111] In this step, the system uses a distribution method based on distributed storage, combining the priority tags of the data and the load status of the storage nodes to plan the data storage path. First, the system analyzes each compressed and encrypted data packet and evaluates its priority tag to ensure that important data can be preferentially stored on storage nodes with stronger performance. When determining the selection of storage nodes, the system also monitors the load status of each storage node in real time to avoid performance degradation caused by overloading of a certain node.

[0112] This dynamic planning method of the storage path ensures the balanced distribution of data on different storage nodes, thus optimizing the data access performance. At the same time, the comprehensive consideration of priority tags enables key business data to obtain a faster response speed, enhancing the overall performance and reliability of the system. Through this method, enterprises can effectively manage and store large-scale data, ensure that important data is always available, and improve the data processing efficiency and scalability.

[0113] In this step, the system first analyzes the received compressed and encrypted data packets and extracts their priority tags. The priority tags are usually based on the importance, access frequency, and security requirements of the data. For example, the priority of financial transaction data may be higher, while the priority of log files may be lower. The system assigns a storage node to each data packet. The selection of the storage node takes into account not only the priority of the data but also the load conditions of each current node. In this way, the system can preferentially send important data packets to nodes with lower loads to ensure the fast access and processing of data.

[0114] To achieve an even distribution of data storage paths, the system uses a dynamic hash mapping algorithm. This algorithm can dynamically calculate hash values based on the characteristics of data packets and distribute the data packets to the corresponding storage nodes. For example, suppose there are three storage nodes A, B, and C, and data packets X, Y, and Z to be stored. The system will calculate the hash values of these data packets and select the most suitable node for storage according to the load situation. This dynamic calculation and selection mechanism ensures the uniform distribution of data in the storage system and avoids performance bottlenecks caused by overloading of certain nodes.

[0115] Finally, the generated preliminary storage path plan will include the target storage node for each data packet and its storage path information. This information will be recorded for subsequent data retrieval and management. Through this method, the system not only achieves the efficiency of data storage but also improves the overall system performance, ensuring scalability and reliability in large-scale data processing scenarios.

[0116] For the preliminary storage path plan, an index construction method based on a graph database is adopted. The data storage path is abstracted as nodes and edges in the graph structure, and through a distributed consistency protocol, the consistency and reliability of the index are ensured to generate a preliminary index structure;

[0117] In this step, the system constructs an index of the preliminary storage path in a graph database. Specifically, the system will regard the storage path as a graph structure containing multiple nodes and edges for real-time access and query of data. Each node represents a storage location, and the edge represents the connection relationship between storage nodes. By partitioning these structures, the system can effectively organize and manage the storage path and improve the data retrieval speed. At the same time, combined with the distributed consistency protocol, the reliability of the index is guaranteed to avoid possible data inconsistency in a distributed environment.

[0118] Through the index construction method of the graph database, the management of the data storage path becomes more efficient and flexible. The formation of the graph structure enables data queries to be performed through effective graph traversal algorithms, thus significantly improving the data retrieval speed. In addition, the implementation of the distributed consistency protocol ensures the reliability of the index structure, prevents index inconsistency problems caused by node failures, and improves the fault tolerance and availability of the system. This method provides an innovative solution for data management and retrieval in a dynamic storage environment.

[0119] In this step, the system converts the preliminary storage path plan into an index structure in the graph database. Specifically, each storage node is regarded as a node in the graph, and the connection relationship between nodes (i.e., the data storage path) is regarded as an edge in the graph. The advantage of this graph structure is that it can intuitively display the association between the storage path and the data, making storage management clearer and more efficient.

[0120] During the implementation process, the system will extract the information of storage nodes and construct the connections between nodes. For example, if data packet X is stored in node A and data packet Y needs to be read from node B, then the system will form an edge from A to B in the graph. This structure enables the path where data is stored to be quickly found during data query, significantly reducing the time for data retrieval. In addition, the system will use a distributed consensus protocol (such as Paxos or Raft) to manage the index, ensuring the consistency of index information among various nodes. During each index update, the protocol will guarantee the coordination and consistency of modification operations, thus avoiding data inconsistency problems caused by node failures or network delays.

[0121] As a result, the generated preliminary index structure can ensure that the overall index information is reliable and consistent. The implementation of this graph structure not only improves the visualization of the data storage path but also provides important support for subsequent data access, greatly enhancing the efficiency and accuracy of data retrieval.

[0122] For the preliminary index structure, an index optimization method based on dynamic load balancing is adopted. Combining the real-time load status of storage nodes and the data access frequency, the index structure is dynamically adjusted. Through adaptive hash mapping technology, the data retrieval efficiency and storage load balancing are optimized to generate an optimized index structure;

[0123] In this step, the system adopts an index optimization method based on dynamic load balancing to adjust the preliminary index structure. First, the system will monitor the real-time load status of each storage node and the access frequency of each piece of data to determine which nodes currently have a high load and which data is frequently accessed. Based on this information, the system will dynamically optimize the index structure so that frequently accessed data can be preferentially located on nodes with a lower load to achieve the balance of storage load and the improvement of data retrieval efficiency.

[0124] This way of dynamically adjusting the index structure can effectively disperse the storage pressure and prevent the performance of the entire system from being affected due to overloading of several storage nodes. In addition, through adaptive hash mapping technology, data retrieval becomes faster and more efficient, especially when facing large datasets with frequent access, greatly enhancing the user experience. To sum up, this optimization process not only improves the storage performance but also ensures the reasonable utilization of resources, providing strong support for the sustainable development of the system.

[0125] In this step, the system implements a dynamic load balancing optimization method for the preliminary index structure. The system first monitors the real-time load status of the storage node, including CPU usage, memory usage, and the current amount of data access requests. For example, when the CPU load of node A is high and node B is relatively idle, the system will recognize that node B should receive more data storage requests. Through such dynamic adjustments, the system can effectively avoid performance degradation caused by overload on some nodes.

[0126] At the same time, the system will also optimize the index structure based on the frequency of data access. For frequently accessed data, the system will adjust its storage path so that it is stored on nodes with lighter loads, thereby improving access speed. For example, if a specific data file has been read many times recently, the system can choose to migrate it from a node with higher load to a node with lower load, in order to balance the overall load while ensuring access speed.

[0127] Finally, the index structure was optimized through adaptive hash mapping technology. This process ensures fast data retrieval and consistency, and achieves significant results in balancing storage load. The optimized index structure enables data to be retrieved quickly when accessed, thereby improving the user experience, especially in scenarios with high concurrent access.

[0128] For the optimized index structure, an index table generation method based on visualization technology is adopted to map the index structure into a distributed storage index table. Through real-time monitoring technology, the accuracy and consistency of the index table are ensured to generate the final distributed storage index table.

[0129] In this step, the system maps the optimized index structure to a distributed storage index table for further data management and retrieval. Through visualization technology, users can view and manage the index structure in a graphical way, which is convenient for understanding the storage path and data distribution. The system will present the visualization results as an easy-to-read and understand table or graph, showing the status of each storage node, the stored data items, and the access frequency. At the same time, through real-time monitoring technology, the system can continuously detect the accuracy and consistency of the index table to ensure that there will be no errors or omissions in the data access process.

[0130] By displaying the index structure in a visual way, users can not only intuitively understand the status of the storage system, but also quickly identify potential problems, such as excessive load on a storage node or abnormal data access. In addition, the introduction of real-time monitoring technology ensures the continuous update and accuracy of the index table, thereby improving the efficiency and reliability of data retrieval. This method improves the manageability and ease of use of the storage system, promotes the security and efficiency of data management, and enables users to use storage resources more conveniently.

[0131] In this step, the system converts the optimized index structure into a visual distributed storage index table. Visualization presents the status of storage nodes and data storage paths in a graphical way, enabling managers to easily understand and monitor the overall operation of the storage system. Specifically, the system may use some visualization tools to present information such as the load of nodes, the data stored, and their priorities as clear charts or graphs, facilitating decision-making and operation by users.

[0132] During implementation, the system integrates real-time monitoring technology to continuously track the status of each storage node. This includes not only the load of the nodes but also the access frequency and storage status of the data. For example, if the load of a certain node remains too high, the system will quickly issue an alarm and mark the node in the visual interface for managers to pay attention to. This real-time monitoring enables managers to adjust the storage strategy immediately, thereby improving the stability and effectiveness of data storage.

[0133] Finally, the generated distributed storage index table will provide a solid foundation for the management of the system and the effective access to data. Its accuracy and consistency ensure that there is no delay or error during the execution of data retrieval operations, enhancing the user experience. Through this method, users can conveniently monitor the usage of storage resources, make timely decisions, and greatly enhance the manageability and efficiency of the system.

[0134] S204, verify the data integrity according to the distributed storage index table by using the blockchain-based archival verification technology, wherein the archival verification technology combines smart contracts and distributed ledger technology to monitor the data storage status and access records in real time, generate an archival verification report, and ensure the reliability and traceability of data archiving.

[0135] In this method, based on the distributed storage index table, the system uses the blockchain-based archival verification technology to verify the integrity of data. Specifically, the system will transmit the information in the distributed storage index table to the blockchain network and combine smart contracts to verify the data storage status in real time. Smart contracts are automatically executed contract codes that can automatically execute corresponding operations when specific conditions are met. Through smart contracts, the system can check the integrity of each data entry during storage to ensure that the data has not been tampered with or damaged. At the same time, by using the distributed ledger technology of the blockchain, the system can also record all data storage statuses and access records, ensuring the immutability of these records and providing a reliable basis for subsequent auditing and tracing.

[0136] By leveraging blockchain technology for data integrity verification, the security and reliability of data during storage and access are ensured. This method greatly enhances data traceability, enabling rapid backtracking to the true state of data in case of data issues, providing strong support for timely problem handling. Additionally, this technical architecture combining smart contracts and distributed ledgers makes the data verification process efficient, transparent, and automated, reducing the need for manual operations and enhancing the overall system security and management efficiency. This mechanism has particularly important application value for industries with a large amount of sensitive data, such as finance and healthcare.

[0137] Specifically, according to the distributed storage index table, blockchain-based archiving verification technology can be adopted, combined with smart contracts to verify the data storage status. Through the hash verification algorithm, it is ensured that the data has not been damaged or tampered with during storage, generating a preliminary integrity verification result.

[0138] In this step, the system first extracts relevant data storage information from the distributed storage index table and generates a unique hash value for each piece of data. The hash verification algorithm converts the content of the data into a fixed-length hash value, and any change in the data will cause a change in the hash value. Therefore, this method can effectively detect the consistency and integrity of the data. The system will use smart contracts to compare these hash values with the storage status to determine whether the data has been tampered with or damaged during storage.

[0139] The implementation of this step ensures the integrity and security of the data. Once a hash value mismatch is detected, the system will immediately trigger an alarm and record the specific abnormal event, providing clues for subsequent traceability and handling. This method makes data storage management more transparent and reliable, providing strong protection for data, especially suitable for business scenarios that require a high level of security, such as financial transactions and the storage of personal privacy data.

[0140] In this step, the system first extracts the detailed information of each piece of data from the distributed storage index table, including the identity identifier, storage location, and current hash value of the data. This process is usually carried out simultaneously with data upload, and the system records the hash value of each data object for subsequent integrity verification. The hash value generation algorithm, such as SHA-256, calculates the data content and returns a fixed-length string, ensuring that even a minor modification will cause a significant change in the hash value. This enables the system to monitor the integrity of the data in real-time during storage, and if data tampering is detected, the inconsistent situation can be immediately identified.

[0141] Next, the system will compare the hash value of each piece of data with its storage record. Through smart contracts, this verification process can be seamlessly and automatically executed, following the pre-defined contract rules. For example, the smart contract will set that every time data is accessed, the system automatically calculates the current hash value of the data and compares it with the original hash value stored on the blockchain. If the two are consistent, the system will record this verification result and generate a preliminary integrity verification result; if they are inconsistent, an alarm will be triggered, the data will be marked as a potential risk, and further review will be required.

[0142] Through this process, the system ensures the integrity and security of the data since its creation. This not only improves the transparency of data management but also enhances users' trust in data storage security. Especially when dealing with sensitive information (such as medical data or financial records), using blockchain technology for verification can effectively prevent data tampering and achieve a high level of security.

[0143] For the preliminary integrity verification results, a recording method based on distributed ledger technology is adopted to write the data storage status and access records into the blockchain. Through consensus technology, the immutability and traceability of the records are ensured, and preliminary distributed ledger records are generated;

[0144] In this step, the system deposits the preliminary integrity verification results and the relevant data storage status and access records into the blockchain. Specifically, the system will call the distributed ledger technology to record each data access and its status on the blockchain. By using consensus mechanisms (such as PoW, PoS, etc.), the validity and consistency of each record are ensured, thus preventing data inconsistency caused by malicious tampering. In this way, the storage status and access records of all data will form a complete and immutable historical track.

[0145] Through this recording method, the system provides strong traceability, and any data access and change can be traced and audited subsequently. This is particularly important for industries with high legal, compliance, and auditing requirements (such as banking, insurance, etc.). At the same time, due to the decentralized nature of the blockchain, the storage and management of data become more transparent, enhancing users' trust in the system. At the same time, this method also helps to achieve efficient data governance and security compliance.

[0146] In this step, the system integrates the information related to the preliminary integrity verification results into the blockchain to record the data storage status and access behavior. Specifically, the system will use the distributed ledger technology to write each data verification, storage status, its corresponding hash value, timestamp, and visitor information into the blockchain. Through such records, all access and verification information will be retained on the blockchain in a transparent and immutable form, ensuring that future auditing and tracing work can rely on reliable evidence.

[0147] With a consensus mechanism (such as Proof of Work or Proof of Stake), the validity and integrity of each record will be agreed upon among network nodes. When a data access request occurs, the smart contract will automatically trigger the recording mechanism and add the newly generated state data to the transaction pool awaiting confirmation. Only after being verified and confirmed by multiple nodes will these records be added to the blockchain. In this way, the system ensures the authenticity and credibility of data records, and any administrator or relevant personnel can view and verify the data access and storage status at any time, thus ensuring the transparency of the entire system.

[0148] For example, assume a financial trading system is processing user transaction data. When each transaction is stored, the system records the specific details of the transaction (such as the amount, sender and recipient accounts, etc.) and calculates the hash value of the transaction through the hash algorithm, and then writes the information into the blockchain. This not only ensures that the transaction records are accurate and cannot be tampered with, but also provides a complete evidence chain for subsequent audits, enabling quick verification in case of any disputes.

[0149] For distributed ledger records, adopt a real-time monitoring method based on smart contracts, combine data access frequency and storage status, detect abnormal behaviors, and generate a preliminary monitoring report through anomaly detection algorithms;

[0150] In this step, the system uses smart contracts to achieve real-time monitoring, combines data access frequency and storage status, and dynamically detects abnormal behaviors. The smart contract can set a conditional trigger mechanism. For example, if the access frequency of a certain piece of data suddenly increases, or the load status of a certain storage node is abnormal, the system will automatically execute the preset detection tasks for in-depth analysis. In this way, the system can timely capture potential security threats or abnormal operations.

[0151] The implementation of this process improves the security of the system, enabling it to quickly respond to possible security risks. By timely detecting and handling abnormal behaviors, the risk of data leakage or tampering can be effectively reduced. In addition, the monitoring report can provide valuable information for system administrators, contributing to subsequent system optimization and security management decisions. This intelligent monitoring method not only effectively enhances data security but also provides a perfect data protection solution for enterprises.

[0152] In this step, the system analyzes the records in the distributed ledger by leveraging the real-time monitoring capabilities of smart contracts to identify potential abnormal behaviors. This process involves monitoring the data access frequency and conducting real-time checks on the storage status. For example, whether the access times of a specific data entry exceed a preset threshold within a certain period, or whether the load condition of a data storage node is abnormal. These monitoring pointers can be dynamically set based on historical data to form a comprehensive monitoring standard.

[0153] Smart contracts automatically run on the blockchain network and trigger corresponding monitoring actions based on set conditions. For example, if it is observed that the access frequency of a certain storage node suddenly and significantly increases within a short period, the smart contract will immediately trigger a predefined anomaly detection algorithm to conduct in-depth analysis on this node to determine whether this access is normal. If suspicious behavior is detected, the system will generate a preliminary monitoring report, detailing information such as the nature, time, and possible impacts of the anomaly.

[0154] Such a monitoring mechanism not only enhances the security of the system but also provides real-time basis for management decisions. For example, when a certain storage node is continuously accessed multiple times within a short period and the access behavior is significantly different from the historical access pattern, the system can immediately feedback this information to the administrator. The administrator can quickly take response measures according to the preliminary monitoring report, such as further investigation or temporarily freezing the relevant data to prevent the expansion of potential security risks.

[0155] For the preliminary monitoring report, a report generation method based on natural language generation technology is adopted to integrate the integrity verification results, distributed ledger records, and monitoring report into an archived verification report, and generate the final archived verification report through visualization technology.

[0156] In this step, the system integrates the preliminary monitoring report, integrity verification results, and distributed ledger records to generate a comprehensive archived verification report. The generation of this report relies on natural language generation (NLG) technology. The system will automatically write logical and readable report content based on the existing data information and monitoring results. During the report generation process, the NLG algorithm will convert complex technical information into easy-to-understand language, enabling non-technical personnel to clearly understand the storage status and security of the data. At the same time, the system presents the key data and analysis results in the report in the form of charts or other visualizations through visualization technology to enhance the readability and information transmission effect of the report.

[0157] This automated report generation mechanism greatly improves the efficiency of data management and reduces the workload of manually writing reports. By integrating information from multiple sources into a clear archived verification report, decision-makers can quickly grasp the status, integrity, and security of data storage, and thus make timely responses and adjustments. This approach not only improves the transparency of the report but also promotes information sharing within the organization, helping different departments to better collaborate and communicate. In addition, reports with visual elements are more attractive, capable of intuitively showing data anomalies or potential risks, and providing intuitive decision-making support for managers.

[0158] In this step, the system combines the preliminary monitoring report, integrity verification results, and records of the distributed ledger, and uses natural language generation (NLG) technology to create a comprehensive archived verification report. This process involves extracting relevant information from different data sources, integrating it, and transforming it into readable natural language text. NLG technology can analyze data and automatically generate structured content, such as by describing the storage status of data, the results of integrity verification, and anomalies found in monitoring, to provide a comprehensive situation report for management.

[0159] During the report generation process, the system also uses visualization technology to enhance the effect of information transmission. Data can be presented in the form of charts, images, or dashboards. For example, a bar chart can be used to show the change in data access frequency in the past month, or a pie chart can be used to display the status distribution of different data items. This visual presentation not only makes the report content more rich and easy to understand but also helps decision-makers quickly obtain key data.

[0160] The final generated archived verification report will be output in PDF or web page format and stored in a secure storage location for relevant departments to access. For example, in a hospital management system, when relevant departments need to view the storage and access records of patient data, they can quickly access this report and find the integrity verification results, storage status, and any abnormal behaviors found in monitoring of patient data. This automated report generation method greatly improves work efficiency and transparency, making data management more scientific and efficient.

[0161] It can be seen that, according to the data type, access frequency, and business importance of the shutdown system, a multi-dimensional classification model based on deep learning is used to classify the data, obtaining the classified data set and its priority label; the classified data set is input into a combined compression and encryption processing model based on a neural network to obtain a compressed and encrypted data packet; the compressed and encrypted data packet is distributed to a distributed storage system, and an index construction algorithm based on a graph database is used to generate a data storage path index, obtaining a distributed storage index table; according to the distributed storage index table, a blockchain-based archiving verification technology is used to verify the data integrity, generating an archiving verification report, so as to realize the intelligent classification, effective compression, and secure storage of the data in the shutdown system, and ultimately ensure the integrity and traceability of the data.

[0162] Another embodiment of the present invention provides a data archiving processing system based on a shutdown system. Refer to Figure 3 , the system may include:

[0163] A classification module 301, configured to classify data according to the data type, access frequency, and business importance of the shutdown system by using a multi-dimensional classification model based on deep learning. Among them, the multi-dimensional classification model dynamically divides the archiving priority of the data through an attention technology and an adaptive weight allocation algorithm, obtaining the classified data set and its priority label;

[0164] A processing module 302, configured to input the classified data set into a combined compression and encryption processing model based on a neural network for data compression and encryption. Among them, the combined processing model realizes efficient compression while ensuring data security by combining a quantization compression algorithm and a homomorphic encryption technology, obtaining a compressed and encrypted data packet;

[0165] An index module 303, configured to distribute the compressed and encrypted data packet to a distributed storage system and generate a data storage path index by using an index construction algorithm based on a graph database. Among them, the index construction algorithm optimizes the data retrieval efficiency and storage load balance through dynamic hash mapping and a distributed consistency protocol, obtaining a distributed storage index table;

[0166] An archiving module 304, configured to verify the data integrity according to the distributed storage index table by using a blockchain-based archiving verification technology. Among them, the archiving verification technology monitors the data storage status and access records in real time by combining a smart contract and a distributed ledger technology, generating an archiving verification report to ensure the reliability and traceability of data archiving.

[0167] It can be seen that, according to the data type, access frequency, and business importance of the shutdown system, a multi-dimensional classification model based on deep learning is used to classify the data, obtaining the classified data set and its priority label; the classified data set is input into a combined compression and encryption processing model based on a neural network to obtain a compressed and encrypted data packet; the compressed and encrypted data packet is distributed to a distributed storage system, and an index construction algorithm based on a graph database is used to generate a data storage path index, obtaining a distributed storage index table; according to the distributed storage index table, a blockchain-based archiving verification technology is used to verify the data integrity, generating an archiving verification report, thereby enabling intelligent classification, effective compression, and secure storage of the data in the shutdown system, and ultimately ensuring the integrity and traceability of the data.

[0168] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, and the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0169] Specifically, in this embodiment, the above storage medium can be configured to store a computer program for executing the following steps:

[0170] S201, according to the data type, access frequency, and business importance of the shutdown system, use a multi-dimensional classification model based on deep learning to classify the data. Among them, the multi-dimensional classification model dynamically divides the archiving priority of the data through an attention technology and an adaptive weight allocation algorithm, obtaining the classified data set and its priority label;

[0171] S202, input the classified data set into a combined compression and encryption processing model based on a neural network for data compression and encryption. Among them, the combined processing model realizes efficient compression while ensuring data security by combining a quantization compression algorithm and a homomorphic encryption technology, obtaining a compressed and encrypted data packet;

[0172] S203, distribute the compressed and encrypted data packet to a distributed storage system, and use an index construction algorithm based on a graph database to generate a data storage path index. Among them, the index construction algorithm optimizes the data retrieval efficiency and storage load balance through dynamic hash mapping and a distributed consistency protocol, obtaining a distributed storage index table;

[0173] S204, according to the distributed storage index table, use a blockchain-based archiving verification technology to verify the data integrity. Among them, the archiving verification technology monitors the data storage status and access records in real time by combining a smart contract and a distributed ledger technology, generating an archiving verification report to ensure the reliability and traceability of data archiving.

[0174] It can be seen that according to the data type, access frequency, and business importance of the shut-down system, a multi-dimensional classification model based on deep learning is used to classify the data, obtaining the classified data set and its priority label; the classified data set is input into a combined compression and encryption processing model based on a neural network to obtain a compressed and encrypted data packet; the compressed and encrypted data packet is distributed to a distributed storage system, and an index construction algorithm based on a graph database is used to generate a data storage path index, obtaining a distributed storage index table; according to the distributed storage index table, a blockchain-based archiving verification technology is used to verify the data integrity, generating an archiving verification report, so as to realize the intelligent classification, effective compression, and secure storage of the data in the shut-down system, and finally ensure the integrity and traceability of the data.

[0175] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0176] Specifically, the above electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0177] Specifically, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0178] S201, according to the data type, access frequency, and business importance of the shut-down system, use a multi-dimensional classification model based on deep learning to classify the data. Among them, the multi-dimensional classification model dynamically divides the archiving priority of the data through an attention technology and an adaptive weight allocation algorithm, obtaining the classified data set and its priority label;

[0179] S202, input the classified data set into a combined compression and encryption processing model based on a neural network for data compression and encryption. Among them, the combined processing model realizes efficient compression while ensuring data security by combining a quantization compression algorithm and a homomorphic encryption technology, obtaining a compressed and encrypted data packet;

[0180] S203, distribute the compressed and encrypted data packet to a distributed storage system, and use an index construction algorithm based on a graph database to generate a data storage path index. Among them, the index construction algorithm optimizes the data retrieval efficiency and storage load balance through dynamic hash mapping and a distributed consistency protocol, obtaining a distributed storage index table;

[0181] S204. Verify the data integrity using the blockchain-based archiving verification technology according to the distributed storage index table. Among them, the archiving verification technology combines smart contracts and distributed ledger technology to monitor the data storage status and access records in real time, generate an archiving verification report, and ensure the reliability and traceability of data archiving.

[0182] It can be seen that according to the data type, access frequency, and business importance of the shutdown system, a multi-dimensional classification model based on deep learning is used to classify the data, obtaining the classified data set and its priority label; the classified data set is input into a combined compression and encryption processing model based on a neural network to obtain a compressed and encrypted data packet; the compressed and encrypted data packet is distributed to the distributed storage system, and an index construction algorithm based on a graph database is used to generate a data storage path index, obtaining a distributed storage index table; according to the distributed storage index table, the data integrity is verified using the blockchain-based archiving verification technology to generate an archiving verification report, thereby enabling intelligent classification, effective compression, and secure storage of the data in the shutdown system, and ultimately ensuring the integrity and traceability of the data.

[0183] The above has detailed the structure, features, and function effects of the present invention according to the illustrated embodiments. The above is only the preferred embodiment of the present invention, but the present invention is not limited to the scope defined by the drawings. Any changes made according to the concept of the present invention, or equivalent embodiments modified to equivalent changes, still within the spirit covered by the specification and the drawings, shall be within the protection scope of the present invention.

Claims

1. A data archiving and processing method based on a shutdown system, characterized in that, The method includes: Classify the data using a multi-dimensional classification model based on deep learning according to the data type, access frequency, and business importance of the shutdown system. Among them, the multi-dimensional classification model dynamically divides the archival priority of the data through attention technology and an adaptive weight allocation algorithm, obtaining the classified data set and its priority label; Among them, according to the data type in the shutdown system, use a data acquisition framework based on edge computing to obtain multi-source data in real time, and through an adaptive data cleaning algorithm, filter out noise and fill in missing values for the data to generate a preliminary standardized data set; For the preliminary standardized data set, use a multi-dimensional classification model based on deep learning, combine the data type, access frequency, and business importance, extract multi-dimensional features of the data, and through the multi-head attention technology, capture the correlation relationship between different dimensional features to generate a preliminary feature representation; For the preliminary feature representation, use a priority division method based on the adaptive weight allocation algorithm, combine the access frequency and business importance of the data, dynamically allocate the archival priority of the data, and through attention technology, optimize the accuracy and rationality of weight allocation to generate a preliminary priority label; For the preliminary priority label, use a data classification method based on the clustering algorithm, combine the data type and priority label, divide the data into different categories, and through the dynamic threshold adjustment technology, ensure the accuracy and consistency of classification to generate the classified data set and its priority label; Input the classified data set into a combined compression and encryption processing model based on a neural network for data compression and encryption. Among them, the combined processing model combines the quantization compression algorithm and the homomorphic encryption technology to achieve efficient compression while ensuring data security, obtaining a compressed and encrypted data packet; Distribute the compressed and encrypted data packet to a distributed storage system, and use an index construction algorithm based on a graph database to generate a data storage path index. Among them, the index construction algorithm optimizes the data retrieval efficiency and storage load balance through dynamic hash mapping and a distributed consistency protocol, obtaining a distributed storage index table; According to the distributed storage index table, use an archival verification technology based on blockchain to verify the data integrity. Among them, the archival verification technology combines smart contracts and distributed ledger technology to monitor the data storage status and access records in real time, generating an archival verification report to ensure the reliability and traceability of data archiving.

2. The method according to claim 1, wherein The step of inputting the classified data set into a combined compression and encryption processing model based on a neural network for data compression and encryption. Among them, the combined processing model combines the quantization compression algorithm and the homomorphic encryption technology to achieve efficient compression while ensuring data security, obtaining a compressed and encrypted data packet, includes: For the classified data set, use a quantization compression algorithm based on a neural network to convert high-precision data into a low-precision representation, and through an adaptive quantization threshold adjustment technology, dynamically adjust the quantization precision to generate preliminary compressed data on the premise of minimizing data information loss; For the preliminary compressed data, an encryption method based on homomorphic encryption technology is adopted. Combining the priority tags and security requirements of the data, the data is encrypted. Through lightweight key management technology, the efficiency and security of the encryption process are ensured, and preliminary encrypted data is generated; For the preliminary encrypted data, a joint optimization method based on neural networks is adopted. Combining quantization compression algorithms and homomorphic encryption technology, the compression and encryption parameters are dynamically adjusted. Through multi-objective optimization algorithms, the compression efficiency and encryption security are balanced, and preliminary compressed and encrypted data packets are generated; For the preliminary compressed and encrypted data packets, a data integrity verification method based on hash verification is adopted to ensure that the data is not damaged during compression and encryption. Through feedback correction technology, the compression and encryption parameters are dynamically adjusted, and final compressed and encrypted data packets are generated.

3. The method according to claim 2, wherein When distributing the compressed and encrypted data packets to the distributed storage system, an index construction algorithm based on graph databases is used to generate data storage path indexes. Among them, the index construction algorithm optimizes data retrieval efficiency and storage load balancing through dynamic hash mapping and distributed consistency protocols, and obtains a distributed storage index table, including: For the compressed and encrypted data packets, a distribution method based on the distributed storage system is adopted. Combining the priority tags of the data and the load status of the storage nodes, the data storage path is planned. Through the dynamic hash mapping algorithm, the balance of data distribution is ensured, and a preliminary storage path plan is generated; For the preliminary storage path plan, an index construction method based on graph databases is adopted. The data storage path is abstracted into nodes and edges in the graph structure. Through the distributed consistency protocol, the consistency and reliability of the index are ensured, and a preliminary index structure is generated; For the preliminary index structure, an index optimization method based on dynamic load balancing is adopted. Combining the real-time load status of the storage nodes and the data access frequency, the index structure is dynamically adjusted. Through adaptive hash mapping technology, the data retrieval efficiency and storage load balancing are optimized, and an optimized index structure is generated; For the optimized index structure, an index table generation method based on visualization technology is adopted. The index structure is mapped to a distributed storage index table. Through real-time monitoring technology, the accuracy and consistency of the index table are ensured, and a final distributed storage index table is generated.

4. The method according to claim 3, wherein According to the distributed storage index table, a blockchain-based archival verification technology is used to verify data integrity. Among them, the archival verification technology combines smart contracts and distributed ledger technology to monitor the data storage status and access records in real time, generates an archival verification report, and ensures the reliability and traceability of data archiving, including: According to the distributed storage index table, a blockchain-based archival verification technology is adopted. Combining smart contracts, the data storage status is verified. Through the hash verification algorithm, it is ensured that the data is not damaged or tampered with during storage, and a preliminary integrity verification result is generated; For the preliminary integrity verification result, a recording method based on distributed ledger technology is adopted. The data storage status and access records are written into the blockchain. Through consensus technology, the immutability and traceability of the records are ensured, and a preliminary distributed ledger record is generated. For distributed ledger records, a real-time monitoring method based on smart contracts is adopted. By combining data access frequency and storage status, abnormal behaviors are detected, and a preliminary monitoring report is generated through an anomaly detection algorithm. For the preliminary monitoring report, a report generation method based on natural language generation technology is adopted. The integrity verification results, distributed ledger records, and monitoring report are integrated into an archived verification report, and the final archived verification report is generated through visualization technology.

5. A data archiving and processing system based on a shutdown system, characterized in that, The system includes: A classification module, which is used to classify data by using a multi-dimensional classification model based on deep learning according to the data type, access frequency, and business importance of the shutdown system. Among them, the multi-dimensional classification model dynamically divides the archival priority of data through attention technology and an adaptive weight allocation algorithm, and obtains the classified data set and its priority label. Among them, according to the data type in the shutdown system, a data acquisition framework based on edge computing is adopted to obtain multi-source data in real time. Through an adaptive data cleaning algorithm, noise filtering and missing value filling are performed on the data to generate a preliminary standardized data set. For the preliminary standardized data set, a multi-dimensional classification model based on deep learning is adopted. By combining the data type, access frequency, and business importance, multi-dimensional features of the data are extracted. Through multi-head attention technology, the correlation relationship between different dimensional features is captured to generate a preliminary feature representation. For the preliminary feature representation, a priority division method based on an adaptive weight allocation algorithm is adopted. By combining the access frequency and business importance of the data, the archival priority of the data is dynamically allocated. Through attention technology, the accuracy and rationality of weight allocation are optimized to generate a preliminary priority label. For the preliminary priority label, a data classification method based on a clustering algorithm is adopted. By combining the data type and priority label, the data is divided into different categories. Through dynamic threshold adjustment technology, the accuracy and consistency of classification are ensured to generate the classified data set and its priority label. A processing module, which is used to input the classified data set into a combined compression and encryption processing model based on a neural network for data compression and encryption. Among them, the combined processing model realizes efficient compression while ensuring data security by combining quantization compression algorithm and homomorphic encryption technology, and obtains a compressed and encrypted data packet. An indexing module, which is used to distribute the compressed and encrypted data packets to a distributed storage system, and generate a data storage path index by using an index construction algorithm based on a graph database. Among them, the index construction algorithm optimizes data retrieval efficiency and storage load balancing through dynamic hash mapping and distributed consistency protocol, and obtains a distributed storage index table. An archiving module, which is used to verify the data integrity by using blockchain-based archiving verification technology according to the distributed storage index table. Among them, the archiving verification technology combines smart contracts and distributed ledger technology to monitor the data storage status and access records in real time, generates an archiving verification report, and ensures the reliability and traceability of data archiving.

6. The system according to claim 5, wherein The processing module is specifically used for: For the classified data set, a quantization compression algorithm based on neural network is adopted to convert high-precision data into low-precision representation. Through the adaptive quantization threshold adjustment technology, the quantization precision is dynamically adjusted to generate preliminary compressed data on the premise of ensuring the minimum loss of data information; For the preliminary compressed data, an encryption method based on homomorphic encryption technology is adopted. Combining the priority label and security requirements of the data, the data is encrypted. Through the lightweight key management technology, the efficiency and security of the encryption process are ensured to generate preliminary encrypted data; For the preliminary encrypted data, a joint optimization method based on neural network is adopted. Combining the quantization compression algorithm and homomorphic encryption technology, the compression and encryption parameters are dynamically adjusted. Through the multi-objective optimization algorithm, the compression efficiency and encryption security are balanced to generate preliminary compressed and encrypted data packets; For the preliminary compressed and encrypted data packets, a data integrity verification method based on hash check is adopted to ensure that the data is not damaged during the compression and encryption process. Through the feedback correction technology, the compression and encryption parameters are dynamically adjusted to generate the final compressed and encrypted data packets.

7. A storage medium, characterized in that, A computer program is stored in the storage medium, wherein the computer program is set to execute the method according to any one of claims 1-4 when running.

8. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is set to run the computer program to execute the method according to any one of claims 1-4.

Images