A lightweight blockchain data security management method and system

Abstract

The present invention provides a lightweight blockchain data security management method and system, which relates to the field of data processing technology. The method includes: preprocessing the original transaction data on the blockchain to obtain processed data, and losslessly compressing the processed data to obtain compressed data; constructing a distributed data processing architecture based on the compressed data, and using smart contracts to automatically perform data verification, screening, and preliminary analysis tasks to obtain a preliminary analysis data set; encrypting the sensitive parts of the preliminary analysis data set, and performing data statistics and pattern recognition to generate encrypted data. The present invention achieves efficient, secure management and rapid recovery of blockchain data through strategies such as lightweight processing, smart contract automation, encryption protection, real-time monitoring, and automatic optimization, effectively improving the efficiency and security of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for securely managing lightweight blockchain data. Background Art

[0002] Traditional blockchain data management technologies, when faced with large-scale transaction data, require complex data verification, storage, and transmission operations, which can slow data processing and affect the system's real-time responsiveness. While these methods guarantee data security to a certain extent, there is still room for improvement in terms of efficient data processing.

[0003] For example, traditional blockchain data management technologies can lag behind in security monitoring. Due to the complexity of data processing and verification processes, the system may not be able to identify and respond to all abnormal behaviors and potential security threats in real time, providing opportunities for malicious nodes to exploit.

[0004] As blockchain applications continue to expand, data security management requirements are also evolving. However, traditional approaches lack sufficient flexibility and scalability, making it difficult to quickly adapt to new security threats and challenges. This can lead to systems being overly vulnerable to new attacks. Regarding privacy protection, over-reliance on a single encryption technology or privacy protection strategy can lead to unsatisfactory privacy protection in certain scenarios and fail to fully meet users' diverse data privacy needs. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a lightweight blockchain data security management method and system, which improves the speed and efficiency of data processing.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] In a first aspect, a method for securely managing lightweight blockchain data is provided, the method comprising:

[0008] Preprocess the original transaction data on the blockchain to obtain processed data, and then perform lossless compression on the processed data to obtain compressed data;

[0009] Based on the compressed data, a distributed data processing architecture based on blockchain is constructed, and smart contracts are used to automatically perform data verification, screening, and preliminary analysis tasks to obtain a preliminary analysis data set.

[0010] Encrypt the sensitive parts of the data set after preliminary analysis, perform data statistics and pattern recognition, and generate encrypted data;

[0011] Based on the encrypted data, a security monitoring model is built to analyze blockchain transaction data in real time, identify abnormal behavior and potential security threats, and automatically optimize and adjust monitoring plans and parameters based on the monitored data characteristics and behavior patterns;

[0012] Establish a data recovery and disaster backup plan to restore data from the backup node when the security monitoring model detects data loss or damage.

[0013] Furthermore, the original transaction data on the blockchain is pre-processed to obtain processed data, and the processed data is losslessly compressed to obtain compressed data, including:

[0014] Obtain original blockchain transaction data through blockchain network nodes. Each block contains transaction records and timestamps.

[0015] Perform pre-processing operations on the original blockchain transaction data, including data cleaning and data formatting, to obtain processed data;

[0016] Initialize an empty dictionary based on the processed data to store the string patterns that appear during the compression process;

[0017] Scan the processed data, reading one character at a time, and search the initialized dictionary for the final matching string; if a matching string is found, replace the original string with the position and matching length of the corresponding matching string in the dictionary to form a compressed code; if no matching string is found, output the current character;

[0018] After each replacement is completed, the new string pattern is added to the dictionary and the dictionary is updated;

[0019] Scan the remaining processed data and repeat the character reading, string matching, replacement and encoding and dictionary update process. When the processed data is scanned and processed, the code and characters output during the compression process are combined to obtain the compressed data.

[0020] Furthermore, based on the compressed data, a distributed data processing architecture based on blockchain is constructed, and smart contracts are used to automatically perform data verification, screening, and preliminary analysis tasks to obtain a preliminary analysis data set, including:

[0021] Store the compressed data in a specific storage area on the blockchain and determine the number and configuration of nodes in the distributed data processing architecture;

[0022] A corresponding number of data processing nodes are configured on the blockchain. Each node has data decompression, processing, and analysis functions, and distributes compressed data stored in a specific area to each data processing node. A preset smart contract is configured on the blockchain, which includes data verification, screening, and preliminary analysis logic.

[0023] Initialize the smart contract, set parameters, status, and trigger conditions, including a specific time and when data reception is complete. When the trigger conditions are met, the blockchain network automatically executes on each data processing node, decompressing the compressed data to obtain the decompressed data.

[0024] Verify the decompressed data. After verification, filter the information that meets the requirements from the decompressed data to obtain filtered data.

[0025] Conduct preliminary analysis on the screened data, including calculating statistical indicators and identifying data patterns, to obtain preliminary analysis results, including statistical indicators, data patterns, outliers, data quality assessment and business indicator information;

[0026] Integrate statistical indicators, data patterns, outliers, data quality assessments, and business indicator information to obtain a preliminary analysis data set.

[0027] Furthermore, the sensitive parts of the data set after preliminary analysis are encrypted, and data statistics and pattern recognition are performed to generate encrypted data, including:

[0028] After preliminary analysis, the data set is checked item by item, and sensitive data is screened according to the preset sensitive data identification standards;

[0029] Encrypt sensitive data to generate encrypted sensitive data, and replace the corresponding part in the original data set with the encrypted sensitive data to construct a partially encrypted data set;

[0030] Re-statistic the encrypted data set, calculate new statistical indicators, and update data distribution information;

[0031] Analyze partially encrypted data sets to identify new patterns, association rules, or abnormal behaviors in the data;

[0032] Integrate statistical indicators, updated data distribution information, and new patterns, association rules, or abnormal behavior information in the identified data into a partially encrypted dataset to generate encrypted data that contains the key information and statistical features of the original data.

[0033] Furthermore, we analyze some of the encrypted data sets to identify new patterns, association rules, or abnormal behaviors in the data, including:

[0034] Extract feature variables from partially encrypted data sets, including non-encrypted features and representative features of processed encrypted data;

[0035] According to the characteristics of the feature variables and the analysis objectives, the cosine similarity is used to calculate the similarity between the feature vectors in the partially encrypted data set;

[0036] Based on the similarity between each eigenvector, cluster analysis is performed on part of the encrypted data set. During the clustering process, the data points are divided into different clusters according to the size of the similarity between each eigenvector, and the center point of the cluster and the statistical information of the data within the cluster are determined. The clustering results are obtained, including the division of clusters, the center point of the cluster, and the statistical information of the data within the cluster;

[0037] Analyze the clustering results to identify new patterns, association rules, or abnormal behaviors in the data; new patterns are manifested as associations between data features within a specific cluster; association rules are obtained by analyzing the dependencies between data within a cluster; and abnormal behaviors are manifested as clusters that are different from the data points.

[0038] Furthermore, based on the encrypted data, a security monitoring model is constructed to analyze blockchain transaction data in real time, identify abnormal behaviors and potential security threats, and automatically optimize and adjust monitoring plans and parameters based on the monitored data characteristics and behavior patterns, including:

[0039] Based on the encrypted data, identify features relevant to anomaly detection and security threat identification, including transaction frequency, transaction amount distribution, and transaction patterns, to construct a feature representation set for the encrypted data.

[0040] Determine a security monitoring model based on the feature representation set, and train the security monitoring model using the historical encrypted data and the corresponding feature representation set to obtain a trained security monitoring model;

[0041] The encrypted blockchain transaction data collected in real time is converted into a feature representation set and input into the trained security monitoring model for real-time analysis. By comparing the current transaction behavior with the normal transaction pattern, abnormal behavior that is inconsistent with normal transaction behavior is identified. Based on the abnormal behavior, potential security threats are analyzed, including fraudulent transactions and money laundering activities, and identification results that include abnormal behavior and potential security threats are generated.

[0042] Based on the identification results, the monitoring plan is adjusted, including adding new features, optimizing security monitoring model parameters, and changing the monitoring frequency.

[0043] Second, a lightweight blockchain data security management system, including:

[0044] The acquisition module is used to pre-process the original transaction data on the blockchain to obtain processed data, and then perform lossless compression on the processed data to obtain compressed data. Based on the compressed data, a distributed data processing architecture based on the blockchain is constructed, and smart contracts are used to automatically perform data verification, screening and preliminary analysis tasks to obtain a preliminary analysis data set. The sensitive parts of the preliminary analysis data set are encrypted, and data statistics and pattern recognition are performed to generate encrypted data.

[0045] The processing module is used to build a security monitoring model based on the encrypted data to analyze blockchain transaction data in real time, identify abnormal behaviors and potential security threats, and automatically optimize and adjust monitoring plans and parameters based on the monitored data characteristics and behavior patterns; establish a data recovery and disaster backup plan, and restore data from the backup node when the security monitoring model detects data loss or damage.

[0046] According to a third aspect, a computing device includes:

[0047] one or more processors;

[0048] The storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method.

[0049] In a fourth aspect, a computer-readable storage medium stores a program, which implements the method when executed by a processor.

[0050] The above solution of the present invention includes at least the following beneficial effects:

[0051] By preprocessing and losslessly compressing raw transaction data on the blockchain, the burden of data storage and transmission is effectively reduced. This step not only reduces system resource consumption but also improves the speed and efficiency of data processing, enabling the system to respond more quickly to transaction requests and queries. A distributed data processing architecture based on blockchain is constructed, utilizing smart contracts to automate data verification, screening, and preliminary analysis tasks. This decentralized approach not only increases the automation of data processing but also enhances the robustness of the system, enabling it to maintain normal operation even in the event of a single node failure.

[0052] Sensitive portions of the data set after preliminary analysis are encrypted to ensure data security during transmission and storage. This measure effectively prevents data leakage and unauthorized access, protecting user privacy. A security monitoring model is built to analyze blockchain transaction data in real time, enabling timely detection and identification of abnormal behavior and potential security threats. By automatically optimizing and adjusting monitoring plans and parameters, the system can continuously adapt to new security threats and challenges, enhancing overall security protection capabilities. A data recovery and disaster backup plan is established so that when the security monitoring model detects data loss or corruption, it can quickly restore data from backup nodes. This measure ensures data integrity and availability, reducing the risks and losses caused by data loss or corruption. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1This is a flow chart of a method for securely managing lightweight blockchain data provided by an embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of a lightweight blockchain data security management system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0056] like Figure 1 As shown, an embodiment of the present invention proposes a method for secure management of lightweight blockchain data, the method comprising the following steps:

[0057] Step 11: pre-process the original transaction data on the blockchain to obtain processed data, and then perform lossless compression on the processed data to obtain compressed data;

[0058] Step 12: Based on the compressed data, a distributed data processing architecture based on blockchain is constructed, and smart contracts are used to automatically perform data verification, screening, and preliminary analysis tasks to obtain a preliminary analysis data set.

[0059] Step 13: encrypt the sensitive parts of the data set after preliminary analysis, perform data statistics and pattern recognition, and generate encrypted data;

[0060] Step 14: Based on the encrypted data, a security monitoring model is constructed to analyze blockchain transaction data in real time, identify abnormal behavior and potential security threats, and automatically optimize and adjust monitoring plans and parameters based on the monitored data characteristics and behavior patterns.

[0061] Step 15: Establish a data recovery and disaster backup plan. When the security monitoring model detects data loss or damage, restore the data from the backup node.

[0062] In this embodiment of the present invention, preprocessing and lossless compression effectively reduce the volume of raw transaction data, alleviating the storage burden on blockchain nodes and enabling the system to more efficiently manage large amounts of data. The compressed data consumes less bandwidth during transmission, speeding up data transmission, reducing network latency, and improving the user experience.

[0063] Distributed data processing architectures can fully leverage the computing power of multiple nodes to process data in parallel, significantly improving data processing efficiency. The automated execution of smart contracts ensures fairness and transparency in data verification, screening, and preliminary analysis, enhancing data credibility. Automated processing reduces manual intervention and labor costs while also improving the accuracy and consistency of data processing.

[0064] Encrypting sensitive data ensures data security during storage and transmission, effectively protecting user privacy. Encrypted data can still be used for data statistics and pattern recognition, providing strong support for subsequent security monitoring and data analysis.

[0065] The security monitoring model analyzes transaction data in real time, promptly identifying abnormal behavior and potential security threats, and providing the system with timely warning and response mechanisms. The security monitoring model automatically optimizes and adjusts monitoring plans and parameters based on detected data characteristics and behavioral patterns, ensuring the system remains in optimal security.

[0066] Data recovery and disaster backup solutions ensure that even in the event of data loss or corruption, data can be quickly restored from backup nodes, ensuring data integrity and availability. The backup mechanism enhances the system's fault tolerance, allowing the system to quickly recover and continue normal operation in the face of unexpected situations.

[0067] In a preferred embodiment of the present invention, the above step 11, pre-processing the original transaction data on the blockchain to obtain processed data, and losslessly compressing the processed data to obtain compressed data, may include:

[0068] Step 111: Obtain original blockchain transaction data through blockchain network nodes. Each block contains transaction records and timestamps.

[0069] Step 112: Preprocessing the original blockchain transaction data, including data cleaning and data formatting, to obtain processed data.

[0070] Step 113: Initialize an empty dictionary based on the processed data to store the string patterns that appear during the compression process;

[0071] Step 114: Scan the processed data, reading one character at a time, and search the initialized dictionary for the final matching string. If a matching string is found, the original string is replaced with the position and matching length of the corresponding matching string in the dictionary to form a compressed code. If no matching string is found, the current character is output.

[0072] Step 115, after each replacement is completed, the new string pattern is added to the dictionary and the dictionary is updated;

[0073] Step 116, scan the remaining processed data, repeat the character reading, string matching, replacement and encoding and dictionary update process, when the processed data is scanned and processed, combine the code and characters output during the compression process to obtain compressed data.

[0074] In this embodiment of the present invention, in step 111, a set of blockchain network nodes is provided, each of which stores a portion or all of the blockchain data. By accessing these nodes, the original blockchain transaction data can be obtained. Each block contains a transaction record and a timestamp.

[0075] Step 112 cleans the original blockchain transaction data to remove invalid or redundant data, such as null values and duplicate records. The data is then formatted to arrange the transaction records and timestamps in a predetermined format to obtain processed data.

[0076] Step 113: Initialize an empty dictionary to store the string patterns and their corresponding codes that appear during the compression process. The dictionary key is the string pattern, and the value is the unique code of the pattern in the dictionary (using the insertion order of the string pattern in the dictionary as the code).

[0077] Step 114: Set the currently scanned position as p and initialize p = 0. Each time a character is read from the processed data, denoted as c. The longest character string pattern with c as the prefix is searched in the dictionary, denoted as m.

[0078] In step 115, if a matching string m is found, the original string is replaced with the dictionary encoding and matching length of the matching string to form a compressed encoding. That is, the dictionary encoding and matching length of the matching string are output. If no matching string is found, the current character c is directly output.

[0079] Step 116, after each replacement is completed, the new string pattern (i.e., the character or character combination currently scanned) is added to the dictionary and the dictionary is updated. The scanning position is updated, and the remaining processed data is scanned again, and the character reading, string matching, replacement and encoding and dictionary updating process are repeated. When the processed data is scanned and processed, the code and character output during the compression process are combined to obtain the compressed data.

[0080] Assume that the original blockchain transaction data is cleaned and formatted to obtain processed data R = "abcdabc".

[0081] Initialize dictionary d = {};

[0082] The scanning and compression operations from step 113 to step 116 are as follows:

[0083] Read 'a', no match, output 'a', update dictionary d = {'a': 1};

[0084] Read 'b', no match, output 'b', update dictionary d = {'a': 1, 'b': 2};

[0085] Read 'c', no match, output 'c', update dictionary d = {a': 1, b': 2, c': 3};

[0086] Read 'd': no match, output 'd', update dictionary d = {a': 1, b': 2, c': 3, d': 4}.

[0087] Read 'a': there is a match ('a'), output (1, 1), and do not update the dictionary;

[0088] Read 'b': There is a match ('b'), output (2, 1), and do not update the dictionary;

[0089] Read 'c': There is a match ('c'), output (3, 1), and the dictionary is not updated.

[0090] Get the compressed data, D=[a ′,′ b ′ ,c ′ c ′ d ′ ,(1,1),(2,1),(3,1)].

[0091] Obtaining data directly from blockchain network nodes ensures the data source is reliable and cannot be tampered with. The latest transaction data can be obtained promptly, providing a foundation for subsequent data processing and analysis. Data cleansing removes invalid, redundant, or erroneous data, improving data quality. Data formatting unifies data from different sources or formats. Using a lossless compression algorithm, repetitive or predictable portions of processed data are represented using shorter codes, reducing data size. Compressed data takes up less storage space, reducing storage costs. Compressed data requires less time to transmit, increasing data transmission speed. Because of lossless compression, compressed data can be completely restored to its original form without any information loss. During the compression process, the dictionary is continuously updated as new string patterns emerge, allowing the compression algorithm to adapt to different data characteristics and improve compression efficiency.

[0092] In a preferred embodiment of the present invention, step 12, based on the compressed data, constructs a distributed data processing architecture based on blockchain, and utilizes smart contracts to automatically perform data verification, screening, and preliminary analysis tasks to obtain a preliminary analyzed data set, which may include:

[0093] Step 122: Storing the compressed data in a specific storage area on the blockchain and determining the number and configuration of nodes in the distributed data processing architecture;

[0094] Step 123: Configure a corresponding number of data processing nodes on the blockchain. Each node has data decompression, processing, and analysis capabilities, and distributes the compressed data stored in a specific area to each data processing node. Configure a preset smart contract on the blockchain. The smart contract includes data verification, screening, and preliminary analysis logic.

[0095] Step 124: Initialize the smart contract, set parameters, status, and trigger conditions, including a specific time and when data reception is complete. When the trigger conditions are met, the blockchain network automatically executes on each data processing node, decompresses the compressed data, and obtains the decompressed data.

[0096] Step 125: Verify the decompressed data. After verification, filter the decompressed data to obtain the filtered data.

[0097] Step 126 , performing preliminary analysis on the filtered data, including calculating statistical indicators and identifying data patterns, to obtain preliminary analysis results, including statistical indicators, data patterns, outliers, data quality assessment, and business indicator information;

[0098] Integrate statistical indicators, data patterns, outliers, data quality assessments, and business indicator information to obtain a preliminary analysis data set.

[0099] In this embodiment of the present invention, in step 122, the compressed data is stored in a specific storage area on the blockchain. The number of nodes and configuration parameters (such as CPU, memory, storage, etc.) of the distributed data processing architecture are determined based on the data scale, processing requirements, and network performance.

[0100] Step 123: Configure data processing nodes on the blockchain. Each node is equipped with data decompression, processing, and analysis capabilities. Distribute storage areas to each data processing node so they can access the compressed data. Design and deploy a smart contract that includes data validation, filtering, and preliminary analysis logic. The smart contract's trigger conditions include specific time periods and data reception completion events.

[0101] Step 124 sets the smart contract's parameters and status, including the storage address, number of nodes, and a specific time. The initial state is "waiting for trigger." When the blockchain reaches a specific time or all data processing nodes report that data reception is complete, the smart contract is triggered. The smart contract automatically decompresses the data on each data processing node, generating the decompressed data.

[0102] Step 125 verifies the decompressed data, checking its integrity, consistency, and validity. If verification passes, the process continues; otherwise, an error is logged and processing terminates. The decompressed data is filtered for information that meets the requirements, yielding filtered data. Filtering criteria can be customized based on business needs, such as specific field values and data ranges.

[0103] Step 126: Perform a preliminary analysis of the filtered data, including calculating statistical indicators (e.g., mean, standard deviation), identifying data patterns (e.g., trends, periodicity), and detecting outliers. Data quality assessments are performed, such as for completeness, accuracy, and timeliness. Business indicator information, such as transaction volume and user activity, is extracted. The statistical indicators, data patterns, outliers, data quality assessments, and business indicator information are integrated to produce a preliminarily analyzed dataset.

[0104] Consider a transaction dataset on a blockchain, containing information such as transaction amounts, transaction times, and the addresses of both parties. Compressed transaction data is stored in a specific storage area, and five data processing nodes are identified. These five nodes are configured on the blockchain, and a smart contract is deployed. The contract contains logic for verifying the validity of the transaction data, filtering out transactions with amounts greater than 1,000, calculating the average transaction amount, and detecting abnormal transactions. The smart contract is set to trigger automatically at dawn each day, or immediately after all nodes receive the data. When the contract executes, the nodes decompress the data and process it. After decompression, the contract verifies the data's integrity (for example, by checking the hash value) and filters out transactions with amounts greater than 1,000. Preliminary analysis is performed on the filtered data, such as calculating the average transaction amount, identifying temporal trends in transaction amounts, and detecting abnormally high transaction amounts. These analysis results are then integrated into a preliminarily analyzed dataset.

[0105] Leveraging the blockchain's immutability, we ensure the secure storage of compressed data. Through rational node configuration, we achieve effective resource allocation and utilization, improving processing efficiency. A corresponding number of data processing nodes are configured on the blockchain, each equipped with data decompression, processing, and analysis capabilities. Compressed data is distributed to each data processing node, ensuring that each node receives a complete data set. Processing data across distributed nodes improves processing speed and reliability, reducing the risk of single points of failure. The introduction of smart contracts automates data processing, reducing human intervention and errors. Trigger conditions can be set based on actual needs, enabling flexible and timely data processing. The automated execution of smart contracts simplifies the data decompression process and improves processing efficiency. Verification and screening ensure that processed data is of high quality and meets requirements. Accurately filtering information based on business needs improves the relevance and practicality of data. Automating preliminary analysis through smart contracts improves the efficiency and accuracy of data analysis.

[0106] In a preferred embodiment of the present invention, the above step 13, encrypting the sensitive portion of the data set after preliminary analysis, performing data statistics and pattern recognition to generate encrypted data, may include:

[0107] Step 131 , checking the data set after preliminary analysis item by item, and screening sensitive data according to preset sensitive data identification standards;

[0108] Step 132: encrypt the sensitive data to generate encrypted sensitive data, and replace the corresponding part of the original data set with the encrypted sensitive data to construct a partially encrypted data set;

[0109] Step 133: re-statistic the partially encrypted data set, calculate new statistical indicators, and update data distribution information;

[0110] Step 134 , analyzing the partially encrypted data set to identify new patterns, association rules, or abnormal behaviors in the data;

[0111] Step 135 , integrating the statistical indicators, updated data distribution information, and the identified new patterns, association rules, or abnormal behavior information in the data into the partially encrypted data set, generating encrypted data containing the key information and statistical features of the original data.

[0112] In this embodiment of the present invention, step 131 presets sensitive data identification criteria based on business needs, such as data items containing personal privacy information, trade secrets, financial data, etc. Each data item in the dataset after preliminary analysis is checked to determine whether it is sensitive data. Data items that meet the sensitive data identification criteria are screened out to form a sensitive dataset.

[0113] Assume that after preliminary analysis, the dataset contains information such as user ID, name, age, income, and purchase history. According to the sensitive data identification standard, name, income, and purchase history are considered sensitive data, so these fields will be filtered out to form a sensitive dataset.

[0114] In step 132, an encryption algorithm, such as AES, is selected based on the data security requirements. Each item in the sensitive data set is encrypted to generate encrypted sensitive data. The encrypted sensitive data replaces the corresponding portion in the original data set to construct a partially encrypted data set.

[0115] In step 133, a statistical method, such as mean, is selected based on the data characteristics and analysis requirements. Statistics are re-performed on the partially encrypted data set to calculate new statistical indices. Based on the new statistical indices, the data distribution information is updated to form an updated data distribution description.

[0116] In step 134, a data analysis method, such as cluster analysis, is selected based on the data characteristics and analysis objectives. The partially encrypted data set is analyzed to identify new patterns, association rules, or abnormal behaviors within the data. The new patterns, association rules, or abnormal behaviors identified in the analysis are extracted to form an analysis result set.

[0117] In step 135, the statistical indicators, updated data distribution information, and analysis results are integrated into the partially encrypted dataset. The integrated dataset is formatted to generate encrypted data that contains the key information and statistical features of the original data while protecting the security of sensitive data.

[0118] By examining each dataset and screening for sensitive data, we ensure that subsequent encryption processing targets information that truly requires protection, thereby improving data security. Screening for sensitive data helps clarify data protection priorities. Encrypting sensitive data effectively prevents unauthorized access and leakage, protecting personal privacy and trade secrets. The encrypted sensitive data replaces the corresponding portion of the original dataset to construct a partially encrypted dataset, protecting sensitive information while maintaining data integrity. Re-analyzing the partially encrypted dataset reveals the impact of encryption on data distribution, ensuring the accuracy and timeliness of statistical indicators. The updated data distribution information provides a reliable foundation for subsequent data analysis and pattern recognition. Analyzing the partially encrypted dataset can identify new patterns, association rules, or abnormal behavior in the data, providing strong support for business decision-making. By uncovering hidden patterns in the data, the value of the data can be enhanced. Integrating statistical indicators, updated data distribution information, and identified new patterns, association rules, or abnormal behavior into the partially encrypted dataset creates encrypted data, enabling a comprehensive display of information. The encrypted data retains the key information and statistical features of the original data while protecting the security of sensitive data.

[0119] In another preferred embodiment of the present invention, the above step 134 of analyzing the partially encrypted data set to identify new patterns, association rules, or abnormal behaviors in the data may include:

[0120] Step 1344 , extracting feature variables from the partially encrypted data set, including non-encrypted features and representative features of the processed encrypted data;

[0121] Step 1345 , based on the characteristics of the feature variables and the analysis objectives, the similarity between the feature vectors in the partially encrypted data set is calculated using cosine similarity;

[0122] Step 1346: Perform cluster analysis on the partially encrypted data set based on the similarity between the eigenvectors. During the clustering process, the data points are divided into different clusters based on the similarity between the eigenvectors. The cluster centers and statistical information of the data within the clusters are determined, and the cluster results are obtained, including the cluster division, cluster centers, and statistical information of the data within the clusters.

[0123] Step 1347 , analyze the clustering results to identify new patterns, association rules, or abnormal behaviors in the data; new patterns are manifested as associations between data features within a specific cluster; association rules are obtained by analyzing dependencies between data within a cluster; and abnormal behaviors are manifested as clusters that are different from the data points.

[0124] In an embodiment of the present invention, step 1344 extracts the identified non-encrypted fields from the data set to form a new data set, namely a non-encrypted data set. This data set only contains unencrypted fields and their corresponding values. The fields in the non-encrypted data set are used as feature variables to construct a non-encrypted feature set. This feature set is a list or set that contains the names of all non-encrypted fields. In the process of reviewing the data set, it is also determined which fields are encrypted based on the annotations, descriptions or data dictionaries of the fields. These fields usually contain sensitive information such as user passwords, transaction amounts, personal identity information, etc. Based on the nature of the encrypted data and the analysis objectives, the principal component analysis (PCA) method is selected. The encrypted data is used as input data and the PCA algorithm is applied for feature extraction. The covariance matrix of the encrypted data is calculated, and the principal components are extracted by solving the eigenvalues and eigenvectors. The calculation formula of the covariance matrix is: Among them, C hj represents the covariance between the hth and jth features in the dataset; M represents the number of samples in the dataset; k represents the index variable; X[k,h] represents the centering value of the kth sample on the hth feature; and X[k,j] represents the centering value of the kth sample on the jth feature. These principal components are the representative features of the encrypted data. Based on the PCA results, the first k principal components are selected (the value of k is determined based on the contribution rate of the eigenvalues) as the representative features of the encrypted data, and an encrypted data feature set is constructed. This feature set is a list or set containing the names or numbers of all selected principal components. The non-encrypted feature set and the encrypted data feature set are combined to form a complete feature variable set. This feature variable set contains the names or numbers of all feature variables used for analysis.

[0125] Step 1345, for each data point in the data set, construct a feature vector based on its feature variable set. According to the characteristics of the feature variables and the analysis objectives, cosine similarity is selected as the similarity measurement method. The similarity between each feature vector is calculated using the cosine similarity formula. Select an appropriate clustering algorithm (such as K-means) and initialize the clustering parameters (such as the number of clusters, threshold, etc.). Based on the similarity between each feature vector, cluster analysis is performed on the partially encrypted data set. During the clustering process, the data points are divided into different clusters according to the size of the similarity of each feature vector, and the center point of each cluster and the statistical information of the data within the cluster (such as mean, variance, etc.) are determined. The clustering results include the division of clusters, the center point of the cluster, and the statistical information of the data within the cluster. By observing the data features within a specific cluster, new patterns in the data are identified. These new patterns are manifested as correlation relationships between the data features within a specific cluster. Use an association rule mining algorithm (such as Apriori) to analyze the dependency between the data within the cluster and obtain association rules. Specifically:

[0126] Define the minimum support threshold required for an itemset to be considered a frequent itemset. Support represents the frequency of an itemset in all transactions. Define the minimum confidence threshold required for an association rule to be considered a strong rule. Generate all possible itemsets from the data in the cluster as candidate itemsets. Iterate over each transaction in the cluster and calculate the support of each candidate itemset as Among them, u(Q) represents the support of item set Q, which is a value between 0 and 1, reflecting the frequency of item set Q in the cluster; represents the number of transactions in cluster V that include item set Q as a subset; V represents the total number of transactions in the cluster; and T represents a specific transaction. Remove candidate itemsets with support below the minimum support, retaining the frequent itemsets. Generate all possible association rules from the frequent itemsets. Compare the data characteristics of each cluster and identify clusters that differ from the majority of data points. These clusters represent anomalous behavior or unusual data points.

[0127] Suppose a partially encrypted dataset contains user shopping behavior data. The non-encrypted features include the user's age, gender, and shopping time. The encrypted data features are representative features extracted from the user's shopping history using principal component analysis. Age, gender, shopping time, and encrypted shopping history features are extracted from the dataset. Cosine similarity is used to calculate the similarity between each user's feature vector. The K-means algorithm is used to cluster the users, dividing the data points into clusters based on similarity, such as a cluster of young female users and a cluster of middle-aged male users. It is found that the young female user cluster has a higher proportion of cosmetics and clothing purchases, indicating a new shopping pattern. In the middle-aged male user cluster, users who purchase electronics and sporting goods also purchase alcoholic beverages, forming an association rule. It is found that users in one cluster shop at different times than in other clusters, primarily late at night, indicating anomalous shopping behavior.

[0128] Cluster analysis can reveal new patterns hidden in data. These patterns may represent new trends, patterns, or associations within the data, providing strong support for data-driven decision-making. Mining association rules helps understand dependencies within data, thereby discovering potential connections between different variables and providing new perspectives for data analysis and prediction. Timely detection of abnormal behavior is crucial for areas such as risk prevention and control and fraud identification. Cluster analysis can accurately identify outliers or clusters within data, providing a strong basis for timely countermeasures. Analyzing partially encrypted datasets not only unlocks the intrinsic value of data but also provides a scientific basis for its further utilization and protection, thereby enhancing its overall value.

[0129] In another preferred embodiment of the present invention, the calculation formula of cosine similarity is:

[0130]

[0131] Where cos(A, B) represents the cosine similarity between feature vector A and feature vector B; n represents the number of feature vectors; i represents the index variable; w i，A represents the weight of the feature vector A on the i-th feature; f i represents the value of the feature vector A on the i-th feature; w i，B represents the weight of the feature vector B on the i-th feature; s i Represents the value of vector B on the i-th feature.

[0132] In the embodiment of the present invention, the feature vectors A and B are defined to have the same dimension, that is, to contain the same number of features. For each feature vector, obtain its weight w on each feature. i，A and w i，B and the corresponding eigenvalue fi and s i Initialize the accumulators, where the numerator accumulator is used to accumulate the sum of the numerator part, with an initial value of 0. The denominator A accumulator is used to accumulate the sum of the squares of the denominator part of the A vector, with an initial value of 0. The denominator B accumulator is used to accumulate the sum of the squares of the denominator part of the B vector, with an initial value of 0. For each feature i (from 1 to n), calculate the weighted value w of the A vector i，A ·f i , represents the weighted eigenvalue of the eigenvector A on the i-th feature. Calculate the weighted value of the B vector, w i.B ·s i , which represents the weighted eigenvalue of feature vector B at the i-th feature. Multiply the weighted value of vector A by the weighted value of vector B and accumulate the result in the numerator accumulator. Calculate the square of the weighted value of vector A and accumulate the result in the denominator accumulator A. Calculate the square of the weighted value of vector B and accumulate the result in the denominator accumulator B.

[0133] Take the square root of the value in the denominator A accumulator to get the modulus (or length) of vector A. Take the square root of the value in the denominator B accumulator to get the modulus (or length) of vector B. The value in the numerator accumulator is the numerator in the formula. The product of the modulus of vector A and the modulus of vector B is the denominator in the formula. Dividing the numerator by the denominator gives the cosine similarity value, cos(A, B).

[0134] The value of cosine similarity is a number between -1 and 1. A value close to 1 means that the two vectors are similar, a value close to -1 means that the two vectors are not similar, and a value close to 0 means that the two vectors have no obvious similarity.

[0135] The feature weight w in the formula i，A and w i，B , which makes it possible to adjust the contribution of a feature in the similarity calculation based on its importance. This is particularly useful for processing datasets with features of varying importance. Cosine similarity performs well in high-dimensional spaces and is not affected by the curse of dimensionality. It focuses on the direction of the vector rather than its length, and therefore accurately reflects the similarity between vectors even in high-dimensional data. Since cosine similarity measures similarity by calculating the angle between vectors, it is not affected by the dimension of the eigenvalue. This allows comparisons between features of different dimensions.

[0136] In a preferred embodiment of the present invention, step 14, based on the encrypted data, constructs a security monitoring model to analyze blockchain transaction data in real time, identify abnormal behaviors and potential security threats, and automatically optimize and adjust monitoring plans and parameters based on the monitored data characteristics and behavior patterns. This may include:

[0137] Step 141: Determine features related to anomaly detection and security threat identification based on the encrypted data, including transaction frequency, transaction amount distribution, and transaction pattern, to construct a feature representation set for the encrypted data.

[0138] Step 142: Determine a security monitoring model based on the feature representation set, and train the security monitoring model using the historical encrypted data and the corresponding feature representation set to obtain a trained security monitoring model.

[0139] Step 143: Convert the encrypted blockchain transaction data collected in real time into a feature representation set and input it into the trained security monitoring model for real-time analysis. By comparing the current transaction behavior with the normal transaction pattern, abnormal behavior that is inconsistent with the normal transaction behavior is identified. Based on the abnormal behavior, potential security threats are analyzed, including fraudulent transactions and money laundering activities, to generate an identification result that includes abnormal behavior and potential security threats.

[0140] Step 144 , based on the identification results, adjust the monitoring plan, including adding new features, optimizing security monitoring model parameters, and changing the monitoring frequency.

[0141] In this embodiment of the present invention, in step 141, features relevant to anomaly detection and security threat identification are determined based on domain knowledge and historical data analysis. Statistical methods (such as mean, variance, and standard deviation) and machine learning techniques (such as feature importance scoring) are used to assess the effectiveness of these features. Selected features include transaction frequency, transaction amount distribution, and transaction patterns. The selected features are combined into a feature vector to represent each transaction or set of transactions.

[0142] Step 142: Determine the security monitoring model based on the characteristics of blockchain transaction data and the requirements for security monitoring. Assume that a support vector machine (SVM) is selected as the security monitoring model. Extract features from historical encrypted data and construct a feature representation set. Ensure that the training dataset contains a sufficient number of normal transaction samples and known abnormal transaction samples (such as fraudulent transactions and money laundering activities). Use the training dataset to train the SVM model to obtain a trained security monitoring model. Use cross-validation (such as k-fold cross-validation) to evaluate the performance of the model. Calculate indicators such as accuracy, recall, and precision.

[0143] Step 143: Collect encrypted blockchain transaction data in real time. Extract features from each transaction data and construct a new feature vector. Input the new feature vector into the trained security monitoring model for real-time analysis, and output the probability of the current transaction behavior being normal or abnormal. Compare the probability with a preset threshold (such as 0.5) to determine whether the transaction is normal. If the probability is > 0.5, the transaction is considered abnormal; otherwise, the transaction is considered normal. Based on the characteristics and patterns of abnormal behavior, further analyze potential security threats, such as fraudulent transactions and money laundering activities. The identified abnormal behavior and potential security threats are presented in the form of reports or alerts. The report contains detailed information about the abnormal transaction, the time of identification, the type of potential threat, etc.

[0144] Step 144, conduct a detailed analysis of the identification results and count the types, frequencies, and distributions of abnormal behaviors. Analyze the performance of the model on different types of transactions to identify possible weaknesses or areas for improvement. Based on the identification results, add new features to capture more abnormal behavior patterns. Evaluate the effectiveness of existing features and remove redundant or invalid features to simplify the model and improve efficiency. Use grid search to find the final parameter combination in the parameter space. Cross-validate each parameter combination to evaluate model performance. Adjust the monitoring frequency and granularity based on the frequency and severity of abnormal behavior. For example, increase the monitoring frequency (such as once per minute) when abnormal behavior occurs frequently, and reduce the monitoring frequency (such as once per hour) under normal circumstances to save resources. Apply the adjusted feature set, model parameters, and monitoring frequency to the security monitoring system. Continuously monitor model performance and make further adjustments and optimizations as needed.

[0145] Carefully selected features can more effectively capture abnormal patterns and security threats in transaction data, improving the accuracy and efficiency of the model. Well-trained security monitoring models can accurately identify abnormal behavior and security threats in transaction data in real time, providing strong support for timely countermeasures. Real-time analysis of transaction data enables timely detection and reporting of abnormal behavior and security threats, providing strong assurance for the secure operation of blockchain systems. Automatically optimizing and adjusting monitoring plans and parameters continuously improves the accuracy and efficiency of security monitoring models, adapting to the ever-changing security threat landscape and providing strong support for the long-term secure operation of blockchain systems. Furthermore, by reducing unnecessary monitoring and feature processing, computing resources can be conserved, reducing operating costs.

[0146] In a preferred embodiment of the present invention, the above step 15, establishing a data recovery and disaster backup plan, and restoring data from a backup node when the security monitoring model detects data loss or damage, may include:

[0147] In an embodiment of the present invention, a monitoring frequency is set (such as every hour, every day), and criteria for determining data loss or damage are defined (such as file size changes, checksum mismatches, etc.). When an anomaly is detected, the model triggers an alarm. Full backup is combined with incremental backup, and data is backed up to backup nodes regularly (such as full backup every day and incremental backup every hour). Select backup nodes that are geographically dispersed and have stable networks to ensure that at least one node is available in the event of a disaster. Backup data should be stored in encrypted form, and strict access rights should be set to ensure data security.

[0148] Develop automated recovery scripts to automatically retrieve and restore data from backup nodes when alerts from the security monitoring model are received. Regularly conduct recovery tests to ensure the effectiveness of the recovery scripts and verify the integrity of the backup data. Prioritize recovery based on business importance to ensure that critical business data is restored first.

[0149] The security monitoring model is integrated with the backup system. When data loss or corruption is detected, the backup data verification and recovery process is automatically triggered. All monitoring, backup, and recovery operation logs are recorded. Relevant personnel (such as administrators and operation and maintenance personnel) are trained in data security, backup and recovery, and other aspects. Regular emergency drills for data loss or corruption are conducted to verify the integrity and effectiveness of the plan. The performance of the backup and recovery process is continuously monitored to evaluate the effectiveness of the plan and promptly identify and resolve problems. As technology develops, the security monitoring model, backup technology, and recovery mechanism are continuously updated to improve data security and recovery efficiency.

[0150] like Figure 2 As shown, an embodiment of the present invention further provides a lightweight blockchain data security management system, including:

[0151] The acquisition module is used to pre-process the original transaction data on the blockchain to obtain processed data, and then perform lossless compression on the processed data to obtain compressed data. Based on the compressed data, a distributed data processing architecture based on the blockchain is constructed, and smart contracts are used to automatically perform data verification, screening and preliminary analysis tasks to obtain a preliminary analysis data set. The sensitive parts of the preliminary analysis data set are encrypted, and data statistics and pattern recognition are performed to generate encrypted data.

[0152] The processing module is used to build a security monitoring model based on the encrypted data to analyze blockchain transaction data in real time, identify abnormal behaviors and potential security threats, and automatically optimize and adjust monitoring plans and parameters based on the monitored data characteristics and behavior patterns; establish a data recovery and disaster backup plan, and restore data from the backup node when the security monitoring model detects data loss or damage.

[0153] An embodiment of the present invention further provides a computing device comprising: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the above-described method. All implementations in the above-described method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0154] The embodiment of the present invention further provides a computer-readable storage medium storing instructions, which, when executed on a computer, causes the computer to execute the above-described method. All implementations in the above-described method embodiment are applicable to this embodiment and can achieve the same technical effects.

[0155] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for secure management of lightweight blockchain data, characterized in that: The method comprises: S11, pre-processing the original transaction data on the blockchain to obtain processed data, and losslessly compressing the processed data to obtain compressed data; S12, based on the compressed data, builds a distributed data processing architecture based on blockchain, and uses smart contracts to automatically perform data verification, screening, and preliminary analysis tasks to obtain a preliminary analysis data set; S13, encrypting the sensitive parts of the data set after preliminary analysis, performing data statistics and pattern recognition, and generating encrypted data; S14, based on the encrypted data, builds a security monitoring model to analyze blockchain transaction data in real time, identify abnormal behaviors and potential security threats, and automatically optimize and adjust monitoring plans and parameters based on the monitored data characteristics and behavior patterns; S15, establish a data recovery and disaster backup plan. When the security monitoring model detects data loss or damage, restore the data from the backup node; In S13, the sensitive parts of the data set after preliminary analysis are encrypted, and data statistics and pattern recognition are performed to generate encrypted data, including: S131, check the data set after preliminary analysis item by item, and filter sensitive data according to the preset sensitive data identification standards; S132, encrypting the sensitive data to generate encrypted sensitive data, and replacing the corresponding part of the original data set with the encrypted sensitive data to construct a partially encrypted data set; S133, performing statistics on the partially encrypted data set again, calculating new statistical indicators, and updating data distribution information; S134, analyzing the partially encrypted data set to identify new patterns, association rules, or abnormal behaviors in the data; S135, integrating the statistical indicators, updated data distribution information, and the identified new patterns, association rules, or abnormal behavior information in the data into the partially encrypted data set to generate encrypted data containing key information and statistical features of the original data; S134 also includes: S1344, extracting feature variables from the partially encrypted data set, including non-encrypted features and representative features of the processed encrypted data; S1345, calculating the similarity between each feature vector in the partially encrypted data set using cosine similarity based on the characteristics of the feature variables and the analysis target; S1346: Perform cluster analysis on the encrypted data set based on the similarity between the eigenvectors. During the clustering process, the data points are divided into different clusters based on the similarity between the eigenvectors. The cluster centers and statistical information of the data within the clusters are determined, and a clustering result is obtained, including the cluster division, the cluster centers, and statistical information of the data within the clusters. S1347, analyzing the clustering results to identify new patterns, association rules, or abnormal behaviors in the data; wherein new patterns are manifested as associations between data features within a specific cluster; association rules are obtained by analyzing dependencies between data within the cluster; abnormal behaviors are manifested as clusters that are different from the data points.

2. The method for secure management of lightweight blockchain data according to claim 1, characterized in that: Preprocess the original transaction data on the blockchain to obtain processed data, and then perform lossless compression on the processed data to obtain compressed data, including: Obtain original blockchain transaction data through blockchain network nodes. Each block contains transaction records and timestamps. Perform pre-processing operations on the original blockchain transaction data, including data cleaning and data formatting, to obtain processed data; Initialize an empty dictionary based on the processed data to store the string patterns that appear during the compression process; Scan the processed data, reading one character at a time, and search the initialized dictionary for the final matching string; if a matching string is found, replace the original string with the position and matching length of the corresponding matching string in the dictionary to form a compressed code; if no matching string is found, output the current character; After each replacement is completed, the new string pattern is added to the dictionary and the dictionary is updated; Scan the remaining processed data and repeat the character reading, string matching, replacement and encoding and dictionary update process. When the processed data is scanned and processed, the code and characters output during the compression process are combined to obtain the compressed data.

3. The method for secure management of lightweight blockchain data according to claim 2, characterized in that: Based on the compressed data, a distributed data processing architecture based on blockchain is constructed, and smart contracts are used to automatically perform data verification, screening, and preliminary analysis tasks to obtain a preliminary analysis data set, including: Store the compressed data in a specific storage area on the blockchain and determine the number and configuration of nodes in the distributed data processing architecture; A corresponding number of data processing nodes are configured on the blockchain. Each node has data decompression, processing, and analysis functions, and distributes compressed data stored in a specific area to each data processing node. A preset smart contract is configured on the blockchain, which includes data verification, screening, and preliminary analysis logic. Initialize the smart contract, set parameters, status, and trigger conditions, including a specific time and when data reception is complete. When the trigger conditions are met, the blockchain network automatically executes on each data processing node, decompressing the compressed data to obtain the decompressed data. Verify the decompressed data. After verification, filter the information that meets the requirements from the decompressed data to obtain filtered data. Conduct preliminary analysis on the screened data, including calculating statistical indicators and identifying data patterns, to obtain preliminary analysis results, including statistical indicators, data patterns, outliers, data quality assessment and business indicator information; Integrate statistical indicators, data patterns, outliers, data quality assessments, and business indicator information to obtain a preliminary analysis data set.

4. The method for secure management of lightweight blockchain data according to claim 3, characterized in that: Based on the encrypted data, a security monitoring model is built to analyze blockchain transaction data in real time, identify abnormal behavior and potential security threats, and automatically optimize and adjust monitoring plans and parameters based on the monitored data characteristics and behavior patterns, including: Based on the encrypted data, identify features relevant to anomaly detection and security threat identification, including transaction frequency, transaction amount distribution, and transaction patterns, to construct a feature representation set for the encrypted data. Determine a security monitoring model based on the feature representation set, and train the security monitoring model using the historical encrypted data and the corresponding feature representation set to obtain a trained security monitoring model; The encrypted blockchain transaction data collected in real time is converted into a feature representation set and input into the trained security monitoring model for real-time analysis. By comparing the current transaction behavior with the normal transaction pattern, abnormal behavior that is inconsistent with normal transaction behavior is identified. Based on the abnormal behavior, potential security threats are analyzed, including fraudulent transactions and money laundering activities, and identification results that include abnormal behavior and potential security threats are generated. Based on the identification results, the monitoring plan is adjusted, including adding new features, optimizing security monitoring model parameters, and changing the monitoring frequency.

5. A lightweight blockchain data security management system, which implements the method according to any one of claims 1 to 4, characterized in that: include: The acquisition module is used to pre-process the original transaction data on the blockchain to obtain processed data, and then perform lossless compression on the processed data to obtain compressed data; Based on the compressed data, a distributed data processing architecture based on blockchain is constructed, and smart contracts are used to automatically perform data verification, screening, and preliminary analysis tasks to obtain a preliminary analysis data set. Encrypt the sensitive parts of the data set after preliminary analysis, perform data statistics and pattern recognition, and generate encrypted data; The processing module is used to build a security monitoring model based on the encrypted data to analyze blockchain transaction data in real time, identify abnormal behavior and potential security threats, and automatically optimize and adjust monitoring plans and parameters based on the monitored data characteristics and behavior patterns; Establish a data recovery and disaster backup plan to restore data from the backup node when the security monitoring model detects data loss or damage.

6. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, which implements the method according to any one of claims 1 to 4 when executed by a processor.

Images