Dynamic self-adaptive data life cycle management system and method

Abstract

The invention relates to the technical field of data life cycle management, and discloses a dynamic self-adaptive data life cycle management system and method.The dynamic self-adaptive data life cycle management method comprises the steps that a data object and a business lineage log thereof are logically bound, and a lineage entropy value is calculated in a non-semantic counting mode based on the log; according to the method, a data management path which does not depend on an external complex rule engine to carry out event analysis is established, so that a management strategy can be generated by business process records of data in an endogenous and deterministic manner; and meanwhile, for the data which is subjected to high-level control due to accidental high-risk events, after the risk of the data is actually dissipated, automatic falling of the control level is realized, and the coupling degree and the maintenance complexity of the whole management system are reduced.

Description

Technical Field

[0001] This invention relates to a dynamic adaptive data lifecycle management system and method, belonging to the field of data lifecycle management technology. Background Technology

[0002] In current data management practices, to ensure that data is handled in a manner commensurate with its business value and compliance risks throughout its lifecycle, a common technical strategy is to establish a centralized strategy center. This strategy center responds to events generated by external business systems through a pre-defined rules engine, and then issues corresponding management instructions to data objects. This approach of separating strategy logic from data entities provides a clear and executable framework for data management and has become a common technical practice in the industry.

[0003] With increasingly complex and dynamic business processes, this management approach has revealed its inherent technical limitations in practical applications. For example, to maintain the accuracy of decision-making, the strategy center needs to subscribe to and parse event streams generated by an increasing number of heterogeneous systems such as trading, risk control, auditing, and legal departments. This directly leads to a combinatorial increase in rule definitions and a large amount of system adaptation and maintenance work. The complexity and coupling of the entire management system also increase. Any change in upstream business systems may cause unexpected adjustments to the strategy logic. Its delayed response and high maintenance costs limit the further improvement of overall data management efficiency.

[0004] While the industry has attempted to introduce more powerful rule engines or event correlation analysis technologies, these improvements essentially still rely on an external system to interpret the data's business status. This often results in increased complexity for the central system without altering its operational mechanism. Analysis reveals the following limitations of existing technologies: 1. Their management logic is based on passive responses to external events, lacking an intrinsic mechanism where management strategies are directly driven by the data's own business history records; 2. Their decision-making relies on semantic interpretation of events generated by different business systems, the completeness and accuracy of which are difficult to guarantee in a dynamic business environment; 3. The increase in system complexity is directly correlated with the number and complexity of integrated business systems, lacking a method to effectively abstract business processes to maintain architectural simplicity. Therefore, the technical problem this invention aims to solve is how to provide a solution that does not rely on complex external rule engines for event parsing but can adaptively determine lifecycle management strategies based directly on the data object's own business history records. Summary of the Invention

[0005] This invention provides a dynamic and adaptive data lifecycle management system and method, the main purpose of which is to solve the problem that existing technologies rely on external complex rule engines and cannot allow data to endogenously and adaptively determine its management strategy based on its own historical footprint.

[0006] To achieve the above objectives, the present invention provides a dynamic adaptive data lifecycle management system, the system comprising:

[0007] The binding module is configured to logically bind data objects to their business lineage logs, which record the history of business operations on data objects by multiple business systems.

[0008] The entropy calculation module is configured to calculate an entropy vector consisting of breadth entropy, depth entropy, and sensitivity entropy that characterize the business operation history in each evaluation period by non-semantic counting and string matching on the business lineage log.

[0009] The strategy determination module is configured to use the entropy vector as a unique index to look up and determine a lifecycle strategy identifier in a preset static strategy mapping table;

[0010] The policy execution module is configured to execute the corresponding lifecycle policy in the storage and access control unit based on the lifecycle policy identifier; and the entropy calculation module is also configured to establish an active clock for the sensitivity entropy increment caused by sensitive words associated with user permission roles recorded in the business lineage log when calculating sensitivity entropy, and follow the following rule: when the active clock is not reset by access events of data objects by users belonging to user permission roles within a preset period, the sensitivity entropy increment is automatically reduced.

[0011] Preferably, the entropy calculation module is specifically configured to: calculate the breadth entropy of the number of independent business systems that have contacted the data object and are recorded in the business lineage log; and calculate the depth entropy of the total number of business operation history entries in the business lineage log.

[0012] Preferably, the system further includes: a sensitive word discovery module, configured to identify the user's permission level initiating the data query operation, and when the user's permission level is identified as a preset high permission level, to determine whether the number of query results of the query operation is lower than a preset rarity threshold, and in response to the determination that it is, to identify the keyword contained in the query operation as a sensitive word for calculating the sensitivity entropy.

[0013] Preferably, the system further includes: a status broadcast receiving module, configured to receive and store a business entity status associated with a business entity identifier; and an entropy calculation module, further configured to, before calculating the entropy vector, if it is determined that a data object is associated with a business entity identifier, directly correct the calculation result of the entropy vector based on the business entity status.

[0014] Preferably, the entropy calculation module is also configured to process historical log entries in the business lineage log according to the following time-weighted rules to calculate breadth entropy and depth entropy: ,in, The entropy value is the time-weighted value. The total number of historical log entries. For log entries The basic entropy contribution value, For the current assessment time, For log entries Recording time, It is a pre-set time decay constant.

[0015] Preferably, the policy execution module is configured to initiate the entropy calculation module and the policy determination module each time an access request for a data object is triggered, to perform a lifecycle policy determination and update. The system also includes an arbitration gateway module, located between the entropy calculation module and the policy determination module, configured to intercept the entropy vector before the policy determination module performs the lookup, and determine whether the entropy vector conforms to one or more preset consistency rules used to define the normal entropy vector mode. In response to the determination being negative, a preset exception handling process is triggered instead of passing the entropy vector to the policy determination module.

[0016] Preferably, the arbitration gateway module is also configured to automatically generate and update consistency rules based on statistical analysis of multiple entropy vectors output by the entropy calculation module during historical normal operation.

[0017] Preferably, the system further includes a log verification module, configured to use a hash chain mechanism to ensure the immutability of the business lineage log, wherein each newly added business lineage log entry contains the hash value of its previous log entry, and the log verification module verifies the integrity of the hash chain before the entropy calculation module performs the calculation.

[0018] Preferably, the static policy mapping table is a data structure that directly maps a combined range of breadth entropy, depth entropy, and sensitivity entropy to a unique lifecycle policy identifier.

[0019] A dynamic and adaptive data lifecycle management method, comprising:

[0020] Logically bind the data object to its business lineage log, which records the history of business operations on the data object by multiple business systems;

[0021] Within each evaluation period, an entropy vector representing the breadth entropy, depth entropy, and sensitivity entropy of the business lineage log is calculated using non-semantic counting and string matching. The steps for calculating sensitivity entropy include: establishing an activity clock for the sensitivity entropy increment caused by sensitive words associated with user permission roles recorded in the business lineage log, following the rule that: when the activity clock is not reset by user access events to data objects belonging to user permission roles within a preset period, the sensitivity entropy increment is automatically reduced; using the entropy vector as a unique index, a lifecycle policy identifier is searched and determined in a preset static policy mapping table; and based on the lifecycle policy identifier, the corresponding lifecycle policy is executed in the storage and access control unit.

[0022] Compared with the prior art, the beneficial effects of the present invention are:

[0023] 1. By logically binding data objects to their business lineage logs and calculating lineage entropy values ​​(breadth entropy, depth entropy, etc.) representing their historical complexity based on these logs, and then determining the current strategy by finding a preset strategy mapping relationship based on the entropy values, a data management path that does not rely on semantic understanding of specific business content is established. The triggering of management behavior no longer requires an external system to parse the business meaning of each operation and match complex rules, but directly responds to the objective fact of the data's own business footprint accumulating in breadth and frequency. This transforms the judgment of data value and risk from a method that previously relied on a continuously updated and error-prone external knowledge base to an intrinsic deterministic response method directly driven by the data's own history.

[0024] 2. This invention associates preset sensitive words used to calculate sensitivity entropy with specific user permission roles. When a data object's entropy value increases due to containing a sensitive word, the system continuously checks whether a user belonging to that associated role accesses the data within a preset period. If no such access event occurs, the contribution of the sensitive word to the entropy value is reduced or removed. This mechanism treats the inaction of high-privilege roles responsible for specific risks as an effective business signal, allowing data placed under high-level control due to occasional high-risk events to have its control level reduced after the risk has actually dissipated. This avoids the problem of unnecessary long-term resource occupation caused by management strategies only being able to upgrade in one direction.

[0025] 3. Before determining the lifecycle strategy based on the calculated entropy values ​​of multiple lineages, these entropy values ​​are first treated as an entropy vector to determine whether they conform to the preset consistency rules used to define the normal entropy vector mode. If the determination is negative, a preset exception handling process is triggered instead of continuing to determine the strategy. This approach adds a verification step based on historical statistical patterns to the core strategy determination path. It utilizes the process information generated by the entropy calculation process itself, enabling the system to not only adapt to normal business flows but also identify abnormal patterns in upstream business activities that may lead to illogical entropy value combinations and intervene before erroneous management strategies are applied.

[0026] 4. This invention also provides a method for correcting lineage entropy calculation. The system receives and stores the state of a business entity associated with a certain business entity identifier. When calculating the lineage entropy value for a data object, if it is determined that the data object is related to the business entity identifier, the calculation result of the entropy value will be directly corrected based on the stored business entity state. This introduces a macro-input channel from the management level to a data set for the original judgment mechanism that was entirely based on the history of individual data. This allows a holistic decision for a specific business to be transmitted to the underlying data management strategy in an instant without having to operate on each data object individually, solving the problem of slow response due to relying solely on changes in the history of individual data. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the overall functional architecture and core processing flow of the system of the present invention;

[0028] Figure 2 This is an example diagram illustrating the evolution of the entropy vector of the data object over time according to the present invention.

[0029] Figure 3 This is a schematic diagram illustrating the interaction relationships between the core functional modules of the system of this invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0031] The present invention provides a dynamic adaptive data lifecycle management system and method. In a specific application scenario, such as transaction data management and compliance auditing in financial institutions, the system is deployed to manage the lifecycle of a large number of transaction record data objects. Functionally, the system can be divided into a binding module, an entropy calculation module, a strategy determination module, and a strategy execution module. Their workflow is as follows: the binding module first establishes a logical association between the data object and its business lineage log; the entropy calculation module periodically reads the log and quantifies its state into an entropy vector using a non-semantic counting method; the strategy determination module then uses this entropy vector to query the corresponding lifecycle strategy identifier in a preset static strategy mapping table; finally, the strategy execution module performs specific storage or access control operations on the data object based on the identifier. This process begins at the start of the business process. At the outset, when a new financial transaction generates a data object, such as a JSON-formatted transaction record, to address the management decision-making difficulties caused by the separation of data from its business context, the binding module creates a logically related business lineage log for that data object. This log is designed as an independent data set to sequentially record the history of every operation performed on the data object by different business systems. For example, when the transaction data is called by the risk analysis engine 5ms after its generation, a record containing a timestamp, business system ID, operation type, and object ID, such as (timestamp: a timestamp value accurate to milliseconds, system_id: RiskEngine-A, action, Assessment, object_id, Data-007), will be written to its business lineage log. This association method provides objective and traceable factual basis for subsequent management decisions. To ensure that the business lineage log is complete and tamper-free before being used for decision-making, the system can be configured with a log verification module. This module runs a hash chain mechanism, where each newly generated log entry contains the hash value of the previous log entry, thus connecting all log entries into a chain structure. Before the entropy calculation module starts its calculation task, the log verification module will verify the integrity of the hash chain, and any inconsistencies caused by tampering will be detected.

[0032] The entropy calculation module is the core processing unit of the system. It evaluates the state of data through a quantitative statistical method, replacing the traditional approach of semantic understanding of complex business logic. This module can be configured to calculate the business lineage log of a specific data object at a preset period (e.g., every hour) or when the data is accessed, generating an entropy vector composed of three-dimensional components: breadth entropy, depth entropy, and sensitivity entropy. A specific calculation example is as follows: Assume that the business lineage log of data object Data-007 recorded a total of 12 operations during the evaluation period: 5 from TradingSys, 4 from RiskEngine-A, 1 from ClearingHouse-B, 1 from InternalAudit (operation type AUDIT_REVIEW), and 1 from LegalDept (operation type LEGAL_HOLD_APPLY). The specific processing steps of the entropy calculation module are as follows: First, calculate the breadth entropy. The module traverses all 12 log records, extracts the system_id field, and after deduplication, obtains the independent set of business systems (TradingSys, RiskEngine-A, ClearingHouse-B, InternalAudit ... The number of elements in the set (lAudit, LegalDept) is the breadth entropy value, which is 5 here. Second, the depth entropy is calculated by counting the total number of log entries, which is 12 here. Third, the sensitivity entropy is calculated. The module performs string matching on the system_id or action field in the log records based on a preset list of sensitive words (e.g., containing AUDIT, LEGAL). In this example, the InternalAudit record matches AUDIT, and the LEGAL_HOLD_APPLY record matches LEGAL. If the contribution value of each match is counted as 1, the calculated value of the sensitivity entropy is 2. Finally, the entropy calculation module outputs an entropy vector [5, 12, 2], which digitally represents the breadth of business, frequency of interaction, and risk correlation of the data object Data-007 at the current evaluation node.

[0033] To make entropy calculations better reflect the impact of recent business activities, time weighting can be introduced into the calculation of breadth and depth entropy. The calculation can follow the formula: In this formula, It is the time-weighted entropy value; This is the total number of log entries; It is the base entropy contribution value of a single log entry, usually set to 1; This is the current assessment time. It is the recording time of the log entry; It is the time decay constant. The value of this parameter determines the decay rate of the historical record weight. Its value can be calibrated experimentally. For example, if the business requires the weight of operation records from one week ago to decay to 50% of the initial value, the corresponding value can be calculated accordingly. The strategy determination module receives the entropy vector [5, 12, 2] and queries a pre-defined static strategy mapping table. This table is a data structure that maps the combined numerical range of the entropy vector to lifecycle strategy identifiers. For example, the table may contain the following mapping relationship: if the breadth entropy... And depth entropy And sensitivity entropy Then it is mapped to the policy IDPOLICY_ACTIVE_STANDARD; if the breadth entropy And depth entropy And sensitivity entropy Then it is mapped to the policy IDPOLICY_ARCHIVE_TIER1; if the sensitivity entropy If the value is 0, it is mapped to the policy ID POLICY_LEGAL_HOLD_WORM. According to this table, the entropy vector [5, 12, 2] matches the third rule. Therefore, the policy determination module outputs POLICY_LEGAL_HOLD_WORM. After receiving this identifier, the policy execution module performs the corresponding storage layer operation, such as migrating the data object Data-007 to the WORM storage medium and changing its access control policy. Considering that the data management policy upgraded due to occasional high-risk events may be fixed for a long time, the entropy calculation module also includes an active clock machine for sensitivity entropy. When a sensitive operation triggered by a specific user permission role (such as LegalDept) is logged and generates a sensitivity entropy increment, the system starts an independent preset period (e.g., 180 days) for that increment. If any user belonging to that role accesses the data object again within this period, the activity clock will be reset. Conversely, if no such access occurs by the end of the clock period, the system determines that the associated risk has been reduced, and will automatically reduce or remove the contribution of the log to the sensitivity entropy. This allows the control level of the data object to automatically decrease as the risk dissipates.

[0034] To adapt to the emergence of new risky terms in the business environment, the system can be equipped with a sensitive word discovery module. This module, upon detecting a data query initiated by a user marked with high privileges (such as an internal auditor) and the number of query results being lower than a preset rarity threshold (e.g., 5 results), will store the keywords in the query in a candidate pool. When multiple different high-privilege users use the same keyword in similar low-hit-rate queries, the term is verified and added to the official vocabulary list for calculating sensitivity entropy, allowing the system's risk identification capabilities to be dynamically updated. Simultaneously, to meet the management needs for real-time, batch policy changes to certain types of data, the system can also include a status broadcast receiving module. Administrators can use this module to publish statuses (e.g., LitigationActive) for specific business entities (e.g., ProductLine-Saturn). When processing any data object, the entropy calculation module will check whether its metadata is associated with that business entity; if associated, regardless of its metadata... Regardless of the content of its own business lineage logs, the entropy calculation module will directly correct the calculation results of its entropy vector based on the broadcast status. For example, it can set its sensitivity entropy to a preset maximum value, thereby quickly transmitting a management decision to all relevant data objects. To improve the reliability of automated decision-making, an arbitration gateway module can be placed between the entropy calculation module and the policy determination module. This module establishes consistency rules for defining normal entropy vector modes by statistically analyzing (e.g., clustering) the entropy vectors generated during normal operation. After receiving a new entropy vector output by the entropy calculation module, the arbitration gateway will first verify it according to these rules. If it finds that the entropy vector pattern is abnormal (e.g., extremely low breadth entropy and abnormally high depth entropy), it will determine that it does not conform to any known normal mode. At this time, a preset abnormal handling process will be triggered (e.g., issuing an alarm and applying a temporary security policy), instead of passing the suspicious entropy vector to the policy determination module. This adds a decision verification link based on historical patterns to the system.

[0035] Example 1: This example demonstrates the application of a technical solution in a specific business scenario. All technical features and processing procedures have been described in the detailed implementation. In the anti-money laundering compliance monitoring business of a large financial institution, a cross-border transfer data object D-42 for customer A with an amount of $5,000 is created. Its initial business lineage log contains only one record from the core transaction system. Based on this, the entropy calculation module calculates its initial entropy vector as [1, 1, 0]. The strategy determination module determines its lifecycle strategy as POLICY_ACTIVE_STANDARD based on the static strategy mapping table. Data object D-42 is in a standard online active state. Several months later, an investigation into a new type of trade finance fraud is launched, codenamed Alpha-Goose. A user belonging to the internal investigation department, marked by the system as having high privileges, uses Alpha-Goose as a keyword for the initial data retrieval. The number of results returned by this query is lower than the preset rarity threshold. Based on this, the sensitive word discovery module selects Alpha-Goose as a candidate sensitive word. The output of this module, namely the new sensitive words, is used as input for subsequent automated policy upgrades. As the investigation progressed, high-privilege users from multiple departments, including fraud analysis, risk management, and compliance audit, independently used the Alpha-Goose keyword for data queries, prompting the system to officially add the keyword to the sensitive word list. During this period, because Client A's counterparty was associated with the Alpha-Goose investigation target, investigators accessed data object D-42. This access and subsequent follow-up analysis by other departments increased the number of business lineage log entries for data object D-42. Its breadth entropy increased to 4 due to the increased number of departments contacted, and its depth entropy also increased accordingly. At the same time, its sensitivity entropy increased because the sensitive word Alpha-Goose, associated with high-privilege roles, was recorded in the log. In the next evaluation cycle, the entropy calculation module calculated a new entropy vector for data object D-42. This vector was mapped to the POLICY_LEGAL_HOLD_WORM policy in the policy mapping table, and data object D-42 was then migrated to immutable archive storage by the policy execution module.

[0036] In this business scenario, a management dilemma exists: while rapidly and comprehensively controlling all potentially risky data in the initial investigation phase, it's crucial to avoid permanent, costly over-control of a large amount of indirectly related data that is ultimately confirmed as risk-free after the investigation concludes. This technical solution addresses this issue through the collaborative operation of the active clock mechanism and entropy calculation mechanism. For data objects ultimately identified as core evidence, investigators continuously access and analyze them. Each access from the associated user's permission role resets the active clock bound to the sensitivity entropy increment, maintaining a high sensitivity entropy state and thus ensuring the stability of the POLICY_LEGAL_HOLD_WORM policy. However, for data object D-42 and thousands of other data items only associated in the initial screening, investigators, after confirming they pose no direct risk... After that, access was no longer granted; therefore, the activity clock bound to the sensitivity entropy increment generated by the term "Alpha-Goose" was not reset within the preset 180-day period; after the period expired, the sensitivity entropy contribution of this sensitive word to this part of the data was automatically reduced, and its overall entropy vector subsequently dropped; in subsequent evaluations, the lifecycle policy of these data objects was automatically downgraded from POLICY_LEGAL_HOLD_WORM to POLICY_ARCHIVE_TIER1, and this policy downgrade process did not require manual batch operations or configuration of external rules to trigger it; ultimately, a few core data objects related to the Alpha-Goose investigation were stably placed under the legally required retention policy, while the control level of a large number of non-involved data objects that were once associated was automatically reduced, returning to the regular archiving policy that matches their current business value.

[0037] Example 2: To verify the accuracy and response efficiency of strategy application in a dynamic business environment, a controlled experiment was conducted. This experiment built a data processing platform simulating a financial business environment. The core of the platform was a business event stream generator, which generated 100,000 independent business events over a simulated one-year period according to a preset statistical model. These events covered types such as routine transactions, customer complaints, internal audits, and legal investigations. The experiment set up two parallel systems: a control group using existing technology and an experimental group deploying the technical solution of this invention. Both systems received identical business event streams. The control group system was based on a central strategy engine that subscribed to business events and matched them with a rule base containing over 500 rules to determine data strategies. The experimental group system operated according to a specific implementation method using business lineage logs and lineage entropy calculation. The evaluation period for entropy calculation was set to 5 minutes, a value determined after balancing the real-time nature of data updates with the system's computational load, to ensure timely response after changes in business status.

[0038] During the experiment, two indicators were continuously monitored when the two systems processed the same event stream: the accuracy of policy application and the average time of policy correction. The accuracy of policy application was defined as the percentage of data objects with the correct lifecycle policy applied at the sampling time point to the total number of data objects. The average time of policy correction refers to the average time from the occurrence of a business event that changes the policy of a data object to the actual completion of the policy change by the system. Table 1 shows the data collected at four key time points during the experiment.

[0039] Table 1: Comparison of performance indicators between the experimental group and the control group.

[0040]

[0041] Experimental data (see Table 1) show that during the regular operation period, the experimental group's strategy application accuracy was higher than that of the control group. On day 92 of the experiment, when an internal audit event involving a large number of data objects was triggered, the accuracy of the control group dropped to 95.2% because the central rule engine experienced delays and some rule conflicts when handling concurrent events. However, the experimental group's accuracy remained at 99.5% due to its calculation method based on the history of individual data. On day 350 of the experiment, 90% of the data objects that had previously been placed under high-level control due to legal investigations had been de-escalated. The control group, lacking an automatic degradation mechanism, saw its strategy become disconnected from business realities, resulting in an accuracy drop to 88.6%. In contrast, the active clock mechanism in the experimental group was triggered, automatically reducing the sensitivity entropy increment of data that had not been accessed by legal affairs roles for a long time, causing its strategy to automatically revert to its previous state, and maintaining an accuracy of 99.1%.

[0042] Regarding the average time taken for policy modification, for policy upgrades, the control group's time was between 1.5 and 1.8 hours, including the transmission of the event bus and the matching execution of the rule engine; the experimental group's time was stable at 0.1 hours, consistent with the length of an entropy calculation and evaluation cycle. For policy downgrades, the control group required more than 72 hours of manual intervention to complete the policy change because the rule base did not have a preset corresponding scenario. In contrast, the experimental group's active clock mechanism automatically triggered the reduction of entropy values ​​in the next evaluation cycle after the preset cycle expired, thus causing the policy to automatically downgrade. The experimental data shows that, compared with the method based on an external rule engine, the method of using entropy calculation based on the data's own business lineage logs to determine the lifecycle policy exhibits higher policy application accuracy and lower policy modification time in the simulated dynamic business environment, while also reducing the maintenance requirements of the rule base.

[0043] Example 3: This example combines Figures 1 to 3 This describes a dynamic, adaptive data lifecycle management system and method, such as... Figure 1As shown, the process begins with an independent data object, such as a transaction record, which is logically associated with its business lineage log through a binding module. This log uses a hash chain mechanism and its immutability is ensured by a log verification module. The entropy calculation module is responsible for quantifying the business operation history recorded in the log, such as risk control calls and audit accesses, into an entropy vector composed of breadth entropy, depth entropy, and sensitivity entropy. Among them, the sensitive word discovery module can dynamically identify rare query keywords of high-privilege users to update the sensitive word list on which the entropy calculation module depends. The calculated entropy vector is passed to the arbitration gateway module, which verifies whether the entropy vector conforms to the preset consistency rules. If it does not conform, it initiates an exception handling process, such as issuing an alarm and applying a temporary security policy. If it conforms to the rules, it passes the entropy vector to the policy determination module. The policy determination module uses a static policy mapping table, with the entropy vector as an index, to look up and determine the lifecycle policy, and outputs the result to the policy execution module. This module performs specific storage, archiving, access control, and other operations on the data objects for which the lifecycle policy has been executed.

[0044] like Figure 2 As shown, the horizontal axis represents time (in days), and the vertical axis represents entropy value. The three curves in the figure represent the changing trends of breadth entropy, depth entropy, and sensitivity entropy within a 190-day period. It can be seen that depth entropy, as a reflection of the total number of historical business operation entries, shows a trend of growth followed by decline over time. Breadth entropy, as a reflection of the number of independent business systems that have come into contact with the data object, shows a relatively slow growth and remains stable in the later stages. Sensitivity entropy, on the other hand, shows slight fluctuations at specific points in time due to the matching of sensitive words. Overall, it intuitively demonstrates the dynamic quantification process of the business process complexity of the data object in different dimensions.

[0045] like Figure 3 As shown in the diagram, this diagram centers on a DataObject and illustrates how various modules collaborate around it: the binding module creates a business lineage log for logical associations of the DataObject. This log is protected by the log verification module through hash chain and digital signature verification. The entropy calculation module calculates breadth entropy, depth entropy, and sensitivity entropy based on the business lineage log. Its calculation process is also affected by the active clock mechanism's automatic decay of entropy values ​​and the sensitive word list updated by the sensitive word discovery module. The calculated entropy vector first undergoes anomaly detection by the arbitration gateway module and then is sent to the policy determination module. This module queries the static policy mapping table to determine the policy ID. The policy execution module adjusts the storage layer and access control layer of the DataObject based on this ID. In addition, a status broadcast receiving module can receive the status of external business entities and adjust the policies of related data objects in batches accordingly, realizing the rapid transmission of macro-management instructions.

[0046] Example 4: In a medical insurance claims management system deployment scenario, in order to enable the management method of the present invention to adapt to its business processes and data patterns, a dataset containing a total of 1 million archived claims cases from the past 24 months is used to perform offline calibration on the system. This dataset contains the entire business lineage log of each claims case from creation, initial review, investigation, payment to closure, as well as the lifecycle strategies that should be applied at different stages, such as active processing, investigation freeze, routine archiving or long-term archiving, as marked by senior claims underwriters based on historical business rules.

[0047] First, the time decay constant λ used in the time-weighted entropy calculation is calibrated. The goal of calibration is to ensure that the entropy calculation results reflect the timeliness of the impact of events within the business domain, i.e., the impact of a claims investigation event on its related data typically decreases within 6 months after its conclusion. The calibration procedure involves randomly selecting 10,000 cases from the dataset as a calibration sample and setting... The range of candidate values ​​is to and with Iterate through the data by step size; for each... Candidate values, the system uses a formula The entropy vectors of all data objects in the sample set at each historical time point are recalculated from their business lineage logs, and their lifecycle strategies are determined based on the static strategy mapping table. Subsequently, the system-determined strategies are compared with the correct strategies marked by the claims underwriters to calculate the lifecycle strategy for each data object. The strategy matching accuracy is determined by the value; ultimately, the strategy matching accuracy that is highest is selected. Candidate values, i.e. As the system operates in this business environment Parameters; secondly, the rarity threshold in the sensitive word discovery module is determined; the setting of this threshold is a trade-off between the sensitivity of new risk signal discovery and the false alarm rate; the calibration procedure is as follows: extract all query operation logs of all users marked as senior investigators in the dataset over the past 24 months, totaling approximately 50,000 logs; for each query log, record the number of results returned and generate a frequency distribution histogram of the number of query results; analyze the histogram to determine the cumulative probability of the distribution of the number of query results, and select the number of query results with a cumulative probability of 5% as the rarity threshold; in this scenario, it is calculated that 5% of query operations return less than or equal to 3 results, therefore, the rarity threshold is set to 3.

[0048] Finally, consistency rules for the arbitration gateway module are generated. The procedure involves using historical entropy vectors generated from all 1 million cases in the dataset during stable operation as training data. The dataset is processed using the K-means clustering algorithm, with the number of clusters K set to 5, to identify 5 typical normal entropy vector modes in this business scenario. After the algorithm is executed, 5 cluster centers are obtained, each a three-dimensional entropy vector, with one center at [8, 55, 2]. For each cluster, the standard deviations of all vectors within it are calculated in terms of breadth entropy, depth entropy, and sensitivity entropy. The standard deviations are [2.5, 15, 0.5], respectively. Based on this, a consistency rule is generated, that is, if an entropy vector is determined to conform to this mode, its breadth entropy component must be in the range of 8 ± (3 × 2.5), its depth entropy component must be in the range of 55 ± (3 × 15), and its sensitivity entropy component must be in the range of 2 ± (3 × 0.5). Five such rules generated based on five clusters are configured in the arbitration gateway module. After the above procedure, the key parameters in the system are all set with values ​​that match the data characteristics of the specific business environment, so that the deployment of the management system of the present invention in the medical insurance claims management scenario has a reproducible operating basis.

[0049] Example 5: This example illustrates the technical solution designed to ensure system operation under abnormal conditions and to provide reproducible configuration procedures for core decision-making components. When the system is deployed in a high-frequency trading data management environment, to address the situation where upstream business systems generate a large number of duplicate log entries due to faults, the binding module performs a pre-check upon receiving a new business lineage log write request. This pre-check compares the business system ID and operation type of the log entry to be written with the last existing log entry. If they match, a new independent log entry is not created; instead, a duplicate counter is incremented in the metadata of the last log entry. Correspondingly, when calculating deep entropy, the entropy calculation module uses a logarithmic compression algorithm to weight log entries containing duplicate counters. A log entry with a duplicate counter of 1000 contributes to the deep entropy as follows: That is, calculation 3 is performed, and this procedure suppresses non-business-related growth of entropy caused by upstream system anomalies.

[0050] To construct a static policy mapping table that matches the management requirements of a specific business domain, the creation process follows a standardized offline calibration procedure. First, business managers select a baseline sample set from the historical database containing at least 1000 data objects covering all known lifecycle states, and manually label each data object with its corresponding lifecycle policy identifier. Then, the system calculates the corresponding entropy vector for each data object in the baseline sample set. Finally, all entropy vectors are grouped according to their manually labeled policy identifiers, and statistical distribution analysis is performed on the vector data within each group across various dimensions to determine the entropy value interval boundaries corresponding to the policy. The upper and lower limits of the entropy value interval corresponding to the policy are set as the 5th and 95th percentiles of the distribution of the data in that group across that dimension. Through this procedure, a static policy mapping table can be generated for a new business environment.

[0051] Example 6: This example illustrates the initialization and secure operation procedures adopted by the system during the initial deployment phase when historical data accumulation is lacking, and when facing anomalies in business lineage log injection. When deploying the data management system of this invention in a brand-new business department, due to the lack of historical business lineage log data that can be used for parameter calibration, the system first runs in an initial secure mode. In this mode, the system adopts a set of preset general static policy mapping tables. This table has a narrow range definition for the entropy value range corresponding to low-risk policies and maps a broad range of medium-to-high entropy values ​​to a temporary policy that requires manual review. At the same time, the arbitration gateway module is configured with a set of basic consistency rules based on general business logic, such as when the sensitivity entropy is greater than zero, the breadth entropy should be greater than one. The system runs a 90-day data accumulation cycle in this initial secure mode to continuously collect business lineage logs. After the cycle ends, the accumulated and verified data is used as a benchmark sample set, and the above calibration procedures are executed to generate an operating parameter and policy mapping table suitable for the business environment. Subsequently, the system switches to normal operation mode.

[0052] To address the possibility of authorized business systems injecting forged records into the business lineage log, the log verification module, based on the hash chain mechanism, adds a signature verification procedure based on asymmetric encryption. Each business system authorized to write to the business lineage log is assigned a unique key pair. When a business system requests to write a log entry, its request body must contain a digital signature generated using its private key for the log content. Before performing hash chain integrity verification, the log verification module first verifies the digital signature using the public key corresponding to the business system. Only after the signature verification passes is the log entry allowed to be written and participate in subsequent entropy calculations; any write request that fails signature verification will be rejected and trigger a security alert.

[0053] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A dynamic adaptive data lifecycle management system, characterized in that, The system includes: The binding module is configured to logically bind data objects to their business lineage logs, which record the history of business operations on data objects by multiple business systems. The entropy calculation module is configured to calculate an entropy vector consisting of breadth entropy, depth entropy, and sensitivity entropy that characterize the business operation history in each evaluation period by non-semantic counting and string matching on the business lineage log. The strategy determination module is configured to use the entropy vector as a unique index to look up and determine a lifecycle strategy identifier in a preset static strategy mapping table; The policy execution module is configured to execute the corresponding lifecycle policy in the storage and access control unit based on the lifecycle policy identifier; and the entropy calculation module is also configured to establish an active clock for the sensitivity entropy increment caused by sensitive words associated with user permission roles recorded in the business lineage log when calculating sensitivity entropy, and follow the following rule: when the active clock is not reset by access events of data objects by users belonging to user permission roles within a preset period, the sensitivity entropy increment is automatically reduced.

2. The dynamic adaptive data lifecycle management system according to claim 1, characterized in that, The entropy calculation module is specifically configured to: calculate the breadth entropy of the number of independent business systems that have interacted with the data objects and are recorded in the business lineage log; and calculate the depth entropy of the total number of business operation history entries in the business lineage log.

3. The dynamic adaptive data lifecycle management system according to claim 1, characterized in that, The system also includes a sensitive word discovery module, which is configured to identify the user's permission level when initiating a data query operation, and when the user's permission level is identified as a preset high permission level, determine whether the number of query results of the query operation is lower than a preset rarity threshold, and in response to the determination that it is, identify the keywords contained in the query operation as a sensitive word for calculating the sensitivity entropy.

4. The dynamic adaptive data lifecycle management system according to claim 1, characterized in that, The system also includes: a status broadcast receiving module, configured to receive and store the status of a business entity associated with a business entity identifier; and an entropy calculation module, configured to, before calculating the entropy vector, if it is determined that the data object is associated with the business entity identifier, directly correct the calculation result of the entropy vector based on the business entity status.

5. A dynamic adaptive data lifecycle management system according to claim 1, characterized in that, The entropy calculation module is also configured to process historical log entries in the business lineage log according to the following time-weighted rules to calculate breadth and depth entropy: ,in, The entropy value is the time-weighted value. The total number of historical log entries. For log entries The basic entropy contribution value, For the current assessment time, For log entries Recording time, It is a pre-set time decay constant.

6. A dynamic adaptive data lifecycle management system according to claim 1, characterized in that, The policy execution module is configured to start the entropy calculation module and the policy determination module every time an access request for a data object is triggered, in order to perform a lifecycle policy determination and update. The system also includes an arbitration gateway module, which is located between the entropy calculation module and the policy determination module. It is configured to intercept the entropy vector before the policy determination module performs the lookup, and determine whether the entropy vector conforms to one or more preset consistency rules used to define the normal entropy vector mode. In response to the determination being negative, a preset exception handling process is triggered.

7. A dynamic adaptive data lifecycle management system according to claim 1, characterized in that, The system also includes a log verification module, which is configured to use a hash chain mechanism to ensure the immutability of the business lineage log. Each new business lineage log entry contains the hash value of its previous log entry, and the log verification module verifies the integrity of the hash chain before the entropy calculation module performs the calculation.

8. A dynamic adaptive data lifecycle management system according to claim 1, characterized in that, The static policy mapping table is a data structure that directly maps a range of combined values ​​of breadth entropy, depth entropy, and sensitivity entropy to a unique lifecycle policy identifier.

9. A dynamic adaptive data lifecycle management method, characterized in that, The methods include: Logically bind the data object to its business lineage log, which records the history of business operations on the data object by multiple business systems; Within each evaluation period, an entropy vector representing the breadth entropy, depth entropy, and sensitivity entropy of the business lineage log is calculated using non-semantic counting and string matching. The steps for calculating sensitivity entropy include: establishing an activity clock for the sensitivity entropy increment caused by sensitive words associated with user permission roles recorded in the business lineage log, following the rule that: when the activity clock is not reset by user access events to data objects belonging to user permission roles within a preset period, the sensitivity entropy increment is automatically reduced; using the entropy vector as a unique index, a lifecycle policy identifier is searched and determined in a preset static policy mapping table; and based on the lifecycle policy identifier, the corresponding lifecycle policy is executed in the storage and access control unit.

Images