Data life cycle management system in trusted data space

Abstract

The invention discloses a data life cycle management system in a trusted data space, which belongs to the technical field of data management, and comprises the following steps of: counting data from different sources, recording various change operations of the data and generating a data log; performing preprocessing and privacy judgment on the data to obtain the type and sensitive identification of the data; the method comprises the following steps: acquiring various change operations of data, and constructing a data change graph in a directed acyclic graph form; the change operation of the data is monitored by adopting an isolated forest algorithm, and the detection of the abnormal operation is realized; privacy protection is carried out when the data is queried through a preset self-adaptive noise mechanism; performing credibility evaluation on the data according to a preset credibility evaluation index system, and dynamically adjusting the size of the set noise based on the obtained credibility score of the data; according to the method, the acquisition timestamp of the data is obtained and is compared with the current UTC time for judgment, so that a judgment result is obtained, and the data is archived or destroyed.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a data lifecycle management system in a trusted data space. Background Art

[0002] With the rapid development of information technology, data has become one of the most important assets in modern society. This data is growing explosively and comes from a wide range of sources, including collection from IoT devices and user interactions. It has diverse formats and complex structures, making it difficult to integrate and manage.

[0003] Data changes frequently during its life cycle, and traditional management methods make it difficult to accurately record and track data, which makes it difficult to trace data problems and affects reliability and security.

[0004] In addition, the need for data credibility assessment is urgent but difficult. Data is interfered with by multiple factors. Existing assessments are mostly based on a single indicator, which cannot fully and accurately reflect the true credibility and lack a dynamic adjustment mechanism.

[0005] Traditional data management technologies focus on storage and retrieval, with insufficient consideration of lifecycle management, privacy protection, and credibility assessment; existing privacy protection technologies have defects, and methods such as differential privacy affect data availability and are mostly static; data credibility assessment technologies are immature, with single indicators, and are difficult to cope with complex environments. Summary of the Invention

[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a data lifecycle management system in a trusted data space to solve the problems raised in the above background technology.

[0007] The object of the present invention can be achieved through the following technical solution: A data lifecycle management system in a trusted data space, comprising: The data statistics module is used to collect statistics on data from different sources, record various data changes and generate data logs; obtain the data type and sensitive identification by preprocessing the data and making privacy judgments; The data analysis and monitoring module is used to traverse data logs to obtain various data change operations and construct a data change map in the form of a directed acyclic graph. At the same time, the isolation forest algorithm is used to monitor data change operations and detect abnormal operations. The data security and assessment module is used to protect the privacy of data when it is queried by adopting a preset adaptive noise mechanism. At the same time, it evaluates the credibility of the data according to a preset credibility assessment indicator system and dynamically adjusts the noise level based on the credibility score of the obtained data. The data destruction and archiving module is used to traverse the data log to obtain the data collection timestamp, and compare it with the current UTC time to obtain the judgment result; based on the judgment result, the data is archived or destroyed; among them, UTC time is coordinated universal time.

[0008] Preferably, data is pre-processed and privacy judgment is performed, including: Clean the data by using hash functions to construct the data fingerprint, identify data with the same data fingerprint, and remove duplicate data; Standardize the format of the collected data and generate a unique blockchain address as a data identifier for the standardized data; When implementing privacy judgment, the preset private information is obtained and matched with the data. If the match is successful, it means that the data is private information and is uploaded to the private chain for storage; if the match fails, it means that the data is public information and is uploaded to the public chain for storage; private information and public information constitute sensitive identification; Among them, privacy information includes personal privacy information and protected health information.

[0009] Preferably, the method for constructing the data change map is as follows: When constructing the data change graph G=(V,E), traverse each change operation in the data log and obtain the change operation from data state Di to data state Dj, which is recorded as edge , o is the data change operation, and Di and Dj are added to the node V to get ; Where n is the total number of data states; i and j represent the indexes of the data states, i=1,2,...,n, j=1,2,...,n, and i≠j.

[0010] Preferably, monitoring the data change operation includes: Extract features from the data change map and obtain the feature vector Q={X1,X2,...,Xm}, where m is the number of features; Randomly select a feature Xk and a split value q; where k≤m and is a positive integer; q is any value in the range of Xk; According to the split value q, the dataset C is divided into two parts, including C1 and C2. C1 represents the sample set with Xk≥q, and C2 represents the sample set with Xk<q. The dataset C is the set of edges in the data change graph. Recursively perform the above split on C1 and C2 until there is only one sample in each subset; Repeat the above process to construct multiple isolated trees until an isolation forest is formed.

[0011] Preferably, the method is characterized in that monitoring the data change operation further includes: For each sample x, calculate the sample anomaly score; The calculation formula is: ; Where s(x) is the anomaly score of sample x; h(x) is the path length of sample x in the isolated tree; E(h(x)) is the average path length; c(m) is the normalization factor used for normalization processing; Set the anomaly threshold to H. If the anomaly score s(x) ≥ H, it means that the change operation corresponding to sample x is an abnormal operation, and an abnormal alarm is immediately issued and the risk response mechanism is activated to protect data; otherwise, it means that the change operation corresponding to sample x is a normal operation.

[0012] Preferably, the process of protecting privacy when data is queried includes: Obtain the query function when the data is queried, and add Laplace noise to the query function based on the preset adaptive noise mechanism, making it impossible for attackers to infer the data as private information; The expression for adding Laplace noise to the query function is: ; in, for sensitivity; is the privacy budget parameter; f(D) is the query function; is the query function after adding noise; Lap is the Laplace distribution.

[0013] Preferably, the process of implementing credibility assessment on data includes: Set an evaluation cycle, obtain credibility evaluation indicators based on the preset credibility evaluation indicator system, evaluate all quantitative credibility evaluation indicators through a weighted summation method, and ultimately obtain the overall credibility score of the data within the evaluation cycle; the evaluation cycle can be set to daily or monthly; The calculation formula is: ; Where KD is the credibility score, is the weight coefficient corresponding to the credibility evaluation index; For the credibility assessment indicators; , L is the number of credibility evaluation indicators; Based on the credibility score of the acquired data, the credibility score of the data is mapped to the privacy budget parameter to dynamically control the size of the noise; The calculation formula is: ; Where, is the maximum noise scale, is the minimum noise scale.

[0014] Preferably, the implementation of the data destruction and archiving module includes: Traverse the data log to obtain the data collection timestamp, and compare it with the current UTC time to obtain the difference T between the data collection timestamp and the current UTC time. If T ≥ T1, the data is expired and needs to be destroyed. If T1 > T ≥ T2, the data is dormant and needs to be archived to save storage space. T1 is the destruction threshold, T2 is the archiving threshold, and T1 > T2. By archiving dormant data, the data is saved on long-term storage media and the data identifier is retained for future query and retrieval; By destroying expired data, overwriting it multiple times with random data or specific pattern data, and generating and verifying a destruction certificate; By formula Generate a proof of destruction, where is the hash value of expired data; t is the destruction timestamp; is the destruction method; Signed by a trusted third party to ensure the authenticity and authority of the destruction certificate; is the proof of destruction; Hash is the hash function; By calculating the information entropy of expired data before and after destruction, we can obtain H1 and H2. , it means that the data has approached a random state and cannot be restored, which means that the verification has passed; otherwise, the expired data will continue to be overwritten for multiple rounds; where H1 is the information entropy of the expired data, and H2 is the information entropy of the data after destruction; At the same time, the proof of destruction will be stored in the blockchain to form an unalterable record.

[0015] Compared with the existing solutions, the present invention achieves the following beneficial effects: The present invention manages data from different sources in a unified manner, records various data change operations in detail, and clearly presents the data change process by constructing a data change map, thereby improving the management and utilization value of data. This invention uses advanced algorithms and technologies to monitor abnormal data operations and protect data privacy. It also performs dynamic credibility assessment and noise adjustment on data based on a scientific credibility assessment indicator system to further enhance privacy protection and data availability. Finally, it performs reasonable archiving or destruction based on the data collection timestamp, thereby ensuring the credibility, security and privacy of the data throughout its life cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention will be further described below with reference to the accompanying drawings.

[0017] Figure 1 This is a module structure diagram of a data lifecycle management system in a trusted data space proposed by the present invention. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] like Figure 1 As shown, the present invention is a data lifecycle management system in a trusted data space, including: a data statistics module, a data analysis and monitoring module, a data security and evaluation module, and a data destruction and archiving module; The data statistics module is used to collect statistics on data from different sources, record various data changes and generate data logs; obtain the data type and sensitive identification by preprocessing the data and making privacy judgments; The data from different sources include, but are not limited to, raw data collected from various sources such as sensors, databases, and file systems; Data logs include but are not limited to data content, collection timestamps, and various data change operations; various data change operations include but are not limited to data cleaning, data conversion, and data sharing; Implement data pre-processing and privacy assessment, including: Clean the data by using a hash function (such as SHA-256) to construct a data fingerprint, identify data with the same fingerprint, and remove duplicate data; Standardize the format of the collected data and generate a unique blockchain address as a data identifier for the standardized data; When implementing privacy judgment, the preset private information is obtained and matched with the data. If the match is successful, it means that the data is private information and is uploaded to the private chain for storage; if the match fails, it means that the data is public information and is uploaded to the public chain for storage; private information and public information constitute sensitive identification; Among them, privacy information includes personal privacy information and protected health information.

[0020] The data analysis and monitoring module is used to traverse data logs to obtain various data change operations and construct a data change graph G = (V, E) in the form of a directed acyclic graph. At the same time, the isolation forest algorithm is used to monitor data change operations and detect abnormal operations. When constructing the data change graph, traverse each change operation in the data log and obtain the change operation from data state Di to data state Dj, which is recorded as edge , o is the data change operation, and Di and Dj are added to the node V to get ; Where n is the total number of data states; i and j represent the index of the data state, i=1,2,...,n, j=1,2,...,n, and i≠j; Monitor data changes, including: Extract features from the data change graph, including but not limited to the access frequency of each data state, the type and number of change operations for each data state, and obtain a feature vector Q = {X1, X2, ..., Xm}, where m is the number of features; Randomly select a feature Xk and a split value q; where k≤m and is a positive integer; q is any value in the range of Xk; According to the split value q, the dataset C is divided into two parts, including C1 and C2. C1 represents the sample set with Xk≥q, and C2 represents the sample set with Xk<q. The dataset C is the set of edges in the data change graph. Recursively perform the above split on C1 and C2 until there is only one sample in each subset; Repeat the above process to construct multiple isolated trees until an isolation forest is formed; For each sample x, calculate the sample anomaly score; The calculation formula is: ; Where s(x) is the anomaly score of sample x; h(x) is the path length of sample x in the isolation tree. The shorter the path length, the more likely the sample is an outlier. E(h(x)) is the average path length. c(m) is the normalization factor used for normalization processing. Set the anomaly threshold to H. If the anomaly score s(x) ≥ H, it means that the change operation corresponding to sample x is an abnormal operation, and an abnormality alarm is immediately issued and the risk response mechanism is activated to protect data. Otherwise, it means that the change operation corresponding to sample x is a normal operation. The specific value of the anomaly threshold is determined by those skilled in the art based on big data experience. It should be noted that abnormal alarms trigger the initiation of risk response mechanisms, including but not limited to immediately freezing relevant data access rights and initiating a security investigation process, to further ensure data security and compliance.

[0021] The data security and assessment module uses a preset adaptive noise mechanism to protect the privacy of data when it is queried. At the same time, it evaluates the credibility of the data according to a preset credibility assessment indicator system and dynamically adjusts the noise level based on the credibility score of the obtained data. When protecting the privacy of data when it is queried, obtain the query function when the data is queried, and add Laplace noise to the query function according to the preset adaptive noise mechanism, so that the attacker cannot infer the data as private information; Among them, the adaptive noise mechanism dynamically sets the size of Laplace noise according to the characteristics of the dataset and the query type, further improving privacy protection and data availability; The expression for adding Laplace noise to the query function is: ; in, Sensitivity measures the sensitivity of the query function to small changes in data; is the privacy budget parameter that controls the amount of noise added; f(D) is the query function, which is a mathematical identifier for performing calculations or statistics on a dataset, such as count queries, sum queries, and average queries; is the query function after adding noise; Lap is the Laplace distribution; When conducting credibility assessment on data, an assessment cycle is set, credibility assessment indicators are obtained according to the preset credibility assessment indicator system, and all quantitative credibility assessment indicators are evaluated by weighted summation method to finally obtain the overall credibility score of the data within the assessment cycle; wherein, the assessment cycle is set to be daily or monthly; The calculation formula is: ; Where KD is the credibility score, The weight coefficient corresponding to the credibility evaluation index, the specific value is set by experts in this field according to actual needs; For the credibility assessment indicators; , L is the number of credibility evaluation indicators; Based on the credibility score of the acquired data, the credibility score of the data is mapped to the privacy budget parameter to dynamically control the size of the noise; The calculation formula is: ; Where, is the maximum noise scale, is the minimum noise scale; It should be noted that the credibility assessment indicators include data integrity, data consistency, and privacy protection strength; by periodically monitoring and processing the data, quantitative credibility assessment indicators are obtained; For example, data integrity refers to whether there are missing values or invalid values in the data set; periodically traverse the data log to count the number of valid records and the total number of records in the collected data, and calculate the ratio of the number of valid records to the total number of records to obtain the integrity of the data set within the period; wherein, the number of valid records in the collected data is the number of records without missing values or invalid values in the collected data; the total number of records refers to the total number of records in the collected data; In an embodiment of the present invention, dynamically adjusting the size of Laplace noise based on the data credibility score is an effective method to seek the optimal balance between privacy protection and data availability. By designing a dynamic adjustment strategy, the practical value of the data can be maximized while protecting data privacy.

[0022] The data destruction and archiving module is used to traverse the data log to obtain the data collection timestamp, compare it with the current UTC time, and obtain the judgment result; based on the judgment result, the data is archived or destroyed. The UTC time is Coordinated Universal Time. By traversing the data log to obtain the data collection timestamp, and comparing it with the current UTC time, the difference T between the data collection timestamp and the current UTC time is obtained. If T≥T1, the data is expired and needs to be destroyed; if T1>T≥T2, the data is dormant and needs to be archived to save storage space. Among them, T1 is the destruction threshold, T2 is the archiving threshold, and T1>T2. The specific value is determined by experts in this field through multiple experimental tests; By archiving dormant data, the data is saved on long-term storage media and the data identifier is retained for future query and retrieval; By destroying expired data, overwriting it multiple times with random data or specific pattern data, and generating and verifying a destruction certificate; By formula Generate a proof of destruction, where is the hash value of expired data; t is the destruction timestamp; is the destruction method; Signed by a trusted third party to ensure the authenticity and authority of the destruction certificate; is the proof of destruction; Hash is the hash function; By calculating the information entropy of expired data before and after destruction, we can obtain H1 and H2. , it means that the data has approached a random state and cannot be restored, which means that the verification has passed; otherwise, the expired data will continue to be overwritten for multiple rounds; where H1 is the information entropy of the expired data, and H2 is the information entropy of the data after destruction. The specific calculation formula is the existing Shannon entropy calculation method; At the same time, the proof of destruction will be stored in the blockchain to form an unalterable record.

[0023] In the several embodiments provided by the present invention, it should be understood that the disclosed system can be implemented in other ways. For example, the embodiments of the invention described above are merely illustrative. For example, the division of modules is only a logical function division, and other division methods may be used in actual implementation.

[0024] Modules described as separate components may or may not be physically separate, and components displayed as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these modules may be selected to achieve the purpose of this embodiment based on actual needs.

[0025] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0026] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0027] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A data lifecycle management system in a trusted data space, characterized in that: include: The data statistics module is used to collect statistics on data from different sources, record various data changes and generate data logs; Obtain the data type and sensitive identification by pre-processing and privacy judgment of the data; The data analysis and monitoring module is used to traverse data logs to obtain various data change operations and construct a data change map in the form of a directed acyclic graph. At the same time, the isolation forest algorithm is used to monitor data change operations and detect abnormal operations. The data security and assessment module is used to protect the privacy of data when it is queried by adopting a preset adaptive noise mechanism. At the same time, it evaluates the credibility of the data according to a preset credibility assessment indicator system and dynamically adjusts the noise level based on the credibility score of the obtained data. The data destruction and archiving module is used to traverse the data log to obtain the data collection timestamp, and compare it with the current UTC time to obtain the judgment result; based on the judgment result, the data is archived or destroyed; among them, UTC time is coordinated universal time.

2. A data lifecycle management system in a trusted data space according to claim 1, characterized in that: Implement data pre-processing and privacy assessment, including: Clean the data by using hash functions to construct the data fingerprint, identify data with the same data fingerprint, and remove duplicate data; Standardize the format of the collected data and generate a unique blockchain address as a data identifier for the standardized data; When implementing privacy judgment, the preset private information is obtained and matched with the data. If the match is successful, it means that the data is private information and is uploaded to the private chain for storage; if the match fails, it means that the data is public information and is uploaded to the public chain for storage; private information and public information constitute sensitive identification; Among them, privacy information includes personal privacy information and protected health information.

3. The data lifecycle management system in a trusted data space according to claim 2, characterized in that: The method of constructing the data change map is as follows: When constructing the data change graph G=(V,E), traverse each change operation in the data log and obtain the change operation from data state Di to data state Dj, which is recorded as edge , o is the data change operation, and Di and Dj are added to the node V to get ; Where n is the total number of data states; i and j represent the indexes of the data states, i=1,2,...,n, j=1,2,...,n, and i≠j.

4. The data lifecycle management system in a trusted data space according to claim 1, characterized in that: Monitor data changes, including: Extract features from the data change map and obtain the feature vector Q={X1,X2,...,Xm}, where m is the number of features; Randomly select a feature Xk and a split value q; where k≤m and is a positive integer; q is any value in the range of Xk; According to the split value q, the dataset C is divided into two parts, including C1 and C2. C1 represents the sample set with Xk≥q, and C2 represents the sample set with Xk<q. The dataset C is the set of edges in the data change graph. Recursively perform the above split on C1 and C2 until there is only one sample in each subset; Repeat the above process to construct multiple isolated trees until an isolation forest is formed.

5. The data lifecycle management system in a trusted data space according to claim 4, characterized in that: Monitoring of data changes also includes: For each sample x, calculate the sample anomaly score; The calculation formula is: ; Where s(x) is the anomaly score of sample x; h(x) is the path length of sample x in the isolated tree; E(h(x)) is the average path length; c(m) is the normalization factor used for normalization processing; Set the anomaly threshold to H. If the anomaly score s(x) ≥ H, it means that the change operation corresponding to sample x is an abnormal operation, and an abnormal alarm is immediately issued and the risk response mechanism is activated to protect data; otherwise, it means that the change operation corresponding to sample x is a normal operation.

6. The data lifecycle management system in a trusted data space according to claim 1, characterized in that: The process of protecting privacy when data is queried includes: Obtain the query function when the data is queried, and add Laplace noise to the query function based on the preset adaptive noise mechanism, making it impossible for attackers to infer the data as private information; The expression for adding Laplace noise to the query function is: ; in, for sensitivity; is the privacy budget parameter; f(D) is the query function; is the query function after adding noise; Lap is the Laplace distribution.

7. The data lifecycle management system in a trusted data space according to claim 6, characterized in that: The process of conducting credibility assessment on data includes: Set an evaluation cycle, obtain credibility evaluation indicators based on the preset credibility evaluation indicator system, evaluate all quantitative credibility evaluation indicators through a weighted summation method, and ultimately obtain the overall credibility score of the data within the evaluation cycle; the evaluation cycle can be set to daily or monthly; The calculation formula is: ; Where KD is the credibility score, is the weight coefficient corresponding to the credibility evaluation index; For the credibility assessment indicators; , L is the number of credibility evaluation indicators; Based on the credibility score of the acquired data, the credibility score of the data is mapped to the privacy budget parameter to dynamically control the size of the noise; The calculation formula is: ; Where, is the maximum noise scale, is the minimum noise scale.

8. The data lifecycle management system in a trusted data space according to claim 7, characterized in that: The implementation of the data destruction and archiving module includes: Traverse the data log to obtain the data collection timestamp, and compare it with the current UTC time to obtain the difference T between the data collection timestamp and the current UTC time. If T ≥ T1, the data is expired and needs to be destroyed. If T1 > T ≥ T2, the data is dormant and needs to be archived to save storage space. T1 is the destruction threshold, T2 is the archiving threshold, and T1 > T2. By archiving dormant data, the data is saved on long-term storage media and the data identifier is retained for future query and retrieval; By destroying expired data, overwriting it multiple times with random data or specific pattern data, and generating and verifying a destruction certificate; By formula Generate a proof of destruction, where is the hash value of expired data; t is the destruction timestamp; is the destruction method; Signed by a trusted third party to ensure the authenticity and authority of the destruction certificate; is the proof of destruction; Hash is the hash function; By calculating the information entropy of expired data before and after destruction, we can obtain H1 and H2. , it means that the data has approached a random state and cannot be restored, which means that the verification has passed; otherwise, the expired data will continue to be overwritten for multiple rounds; where H1 is the information entropy of the expired data, and H2 is the information entropy of the data after destruction; At the same time, the proof of destruction will be stored in the blockchain to form an unalterable record.

Images