Database reconstruction method and computer program product

Abstract

The application provides a database reconstruction method and a computer program product, wherein the database reconstruction method comprises: acquiring multi-source evidence corresponding to a target time of a database; the multi-source evidence comprises at least two of an audit log, a transaction log, a schema change record, a historical snapshot, an application log and a system trace event; generating a candidate state set based on a target schema version corresponding to the target time, a transaction causal relationship and a constraint set corresponding to the database; determining a reconstruction state of the database corresponding to the target time in the candidate state set based on evidence weights corresponding to the multi-source evidence; and outputting the reconstruction state.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a database reconstruction method and computer program product. Background Technology

[0002] Database systems are widely used in finance, government affairs, e-commerce, industrial control, cloud platforms, and internet business scenarios. During operation, databases may be affected by malicious deletion, malicious tampering, unauthorized access, ransomware encryption, replication anomalies, business errors, and system failures, leading to the loss of historical states or the unreliability of the current state. After an incident, it is often necessary to reconstruct the database's state at a specific point in time to facilitate business recovery, security forensics, loss assessment, accountability, and compliance review.

[0003] Existing database reconstruction schemes are difficult to adapt to the long-term evolution of databases, especially under conditions where multi-source evidence is incomplete or even conflicting, resulting in poor reliability and effectiveness of the reconstructed database state. Summary of the Invention

[0004] To address the existing technical problems, this application provides a database reconstruction method and computer program product that can improve the reliability of database state reconstruction.

[0005] In a first aspect, embodiments of this application provide a database reconstruction method, comprising: acquiring multi-source evidence corresponding to the database at a target time; the multi-source evidence including at least two of audit logs, transaction logs, schema change records, historical snapshots, application logs, and system tracing events; generating a candidate state set based on the target schema version, transaction causality, and constraint set corresponding to the database at the target time; determining the reconstruction state of the database at the target time based on the evidence weights corresponding to the multi-source evidence in the candidate state set; and outputting the reconstruction state.

[0006] Secondly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the database reconstruction method described in any embodiment of this application.

[0007] In the database reconstruction method provided in the above embodiments, multi-source evidence corresponding to the database at a target time is obtained. This multi-source evidence includes at least two of the following: audit logs, transaction logs, schema change records, historical snapshots, application logs, and system tracing events. A candidate state set is generated based on the target schema version, transaction causality, and the constraint set corresponding to the database at the target time. The reconstruction state of the database at the target time is determined from the candidate state set based on the evidence weights corresponding to the multi-source evidence. The reconstruction state is then output. Thus, even with incomplete audit logs and fragmented transaction logs, multi-source evidence can still be used to reconstruct the target state of the database. Combined with a schema version awareness mechanism, generating candidate states based on the target schema version at the target time allows for adaptation to the structural evolution of the database during long-term operation, avoiding mismatches between the schema version and the target time that hinder adaptation to the evolution process. Furthermore, the inclusion of transaction causality and the database constraint set in the generation of candidate states facilitates the introduction of correlation factors related to the database state, enabling the reasonable completion of missing fields and records. This method of selecting the most matching reconstruction state through evidence weighting is more accurate. Even under complex conditions such as incomplete audit logs, fragmented transaction logs, and schema evolution, it can still reconstruct the historical state of the target database, thus comprehensively improving the reliability of the reconstruction state.

[0008] The computer program product provided in the above embodiments belongs to the same concept as the corresponding database reconstruction method embodiments, and thus has the same technical effect as the corresponding database reconstruction method embodiments, which will not be repeated here. Attached Figure Description

[0009] Figure 1 This is a schematic flowchart of a database reconstruction method in one embodiment of this application; Figure 2 This is a flowchart illustrating a database reconstruction method in another embodiment of this application; Figure 3 This is a schematic diagram of the architecture of a database reconstruction system in one embodiment of this application. Detailed Implementation

[0010] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] In the following description, the phrase "some embodiments" refers to a subset of all possible embodiments. It should be noted that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0013] In the following description, the terms "first," "second," and "third" are used merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first," "second," and "third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0014] In their research on database reconstruction, the inventors of this application discovered that existing database reconstruction solutions primarily rely on backups, checkpoints, transaction logs, redo logs, undo logs, or change logs to restore the database to any point in time. These methods typically assume a complete log chain, a stable schema structure, and a clear replay order, making them suitable for disaster recovery scenarios. However, their recovery capabilities significantly decrease in scenarios involving corrupted logs, insufficient retention periods, missing cross-source information, and complex schema evolution.

[0015] Another type of method relies primarily on database audit logs, unified audit platforms, or attack scenario analysis systems to record and replay database operations. This type of method can answer the question "who performed what operation at what time," and is suitable for auditing, accountability, and compliance. However, it typically only records the top-level Structured Query Language (SQL), accessed objects, session attributes, or user information, lacking precise state information and making it difficult to directly obtain the complete state of the database at the target time.

[0016] Another type of method extracts database artifacts from database files, data pages, log fragments, memory images, or unallocated space to recover some historical data during the forensic process. While this type of method is helpful for post-hoc analysis, it typically lacks a unified mechanism for fusing multi-source evidence, a mechanism for adapting to pattern version evolution, and a mechanism for quantifying the reliability of the reconstructed results.

[0017] In practical applications, database historical state reconstruction faces at least the following problems: First, audit logs often only record top-level statements, failing to cover triggers, stored procedures, function calls, and some internal statements; Second, transaction logs, binary logs, write-ahead logs, redo logs, and undo logs may be incomplete due to retention periods, abnormal overwriting, access restrictions, or attacks; Third, database schemas undergo field additions, deletions, renamings, type changes, table splits, or table merges during their runtime, rendering old evidence uninterpretable under new schemas; Fourth, evidence from different sources exhibits time drift, semantic granularity differences, and content conflicts; Fifth, existing recovery solutions typically output a single recovery result, lacking confidence metrics at the field, row, and table levels; Sixth, existing log tamper-proofing solutions typically only prove that the original logs were not tampered with, failing to demonstrate the integrity and consistency of the reconstruction process itself.

[0018] Therefore, there is an urgent need for a new database reconstruction scheme that can perform high-confidence reconstruction of the database state at a target time, and output verifiable reconstruction results, under the condition that multi-source evidence is incomplete or even conflicting.

[0019] Based on this, embodiments of this application provide a database reconstruction method, such as... Figure 1 As shown, it may include: S10: Obtain multi-source evidence corresponding to the database at the target time; the multi-source evidence includes at least two of the following: audit logs, transaction logs, schema change records, historical snapshots, application logs, and system tracing events.

[0020] In one embodiment, the database can be a relational database, or a database that supports multiple models such as document model, key-value model, and graph model. The target time refers to the historical moment when the database state needs to be reconstructed, and the output reconstructed state is the state of the database at that target time.

[0021] In one embodiment, multi-source evidence refers to evidence that includes multiple sources of evidence, where the source of evidence corresponding to audit logs is the audit log source, the source of evidence corresponding to transaction logs is the transaction log source, the source of evidence corresponding to schema change records is the schema change source, the source of evidence corresponding to historical snapshots is the snapshot source, the source of evidence corresponding to application logs is the application log source, and the source of evidence corresponding to system tracing events is the system tracing.

[0022] For example, any one of the pieces of evidence from multiple sources. It can be represented as the following tuple: .

[0023] in, Indicates evidence item timestamp, Indicate the source of the evidence. Indicates the target object identifier. Indicates the operation type. This represents a parameter, predicate, or conditional expression. Indicates value information, front mirror or back mirror. Indicates the original mode version identifier of the evidence. Indicates the quality label of the evidence.

[0024] Among them, the source of the evidence It can be any one of the following: audit log source, transaction log source, schema change source, snapshot source, application log source, or system tracing source.

[0025] In one embodiment, the operation type It can include at least one of the following: reading, inserting, updating, deleting, schema changing, permission changing, session connection, and session disconnection.

[0026] S20: Generate a candidate state set based on the target mode version corresponding to the target time, the transaction causal relationship, and the constraint set corresponding to the database.

[0027] In one embodiment, the constraint set corresponding to the database includes at least one of the following: primary key constraint, foreign key constraint, unique constraint, NOT NULL constraint, and check constraint. For example, the constraint set is represented as: .

[0028] in, Represents the set of primary key constraints. Represents the set of foreign key constraints. Represents a unique constraint set. Represents a set of non-empty constraints. This indicates the set of constraints to be checked.

[0029] In some implementations, the constraint set may also include at least one extended constraint such as business rule constraints, enumeration value constraints, application layer parameter linkage constraints, operation timing constraints, and access permission constraints.

[0030] In one embodiment, the target schema version refers to the schema version of the database at the target time. The schema version of the database will evolve during operation, that is, there will be one or more schema versions of the database before the current time.

[0031] For example, a database schema version may evolve during runtime through actions such as adding or deleting fields, renaming fields, splitting fields, merging fields, splitting tables, merging tables, and changing data types. Without considering the schema evolution history, early evidence cannot be correctly interpreted within the schema version corresponding to the target time.

[0032] In one embodiment, transaction causality refers to the sequential execution logic between transactions in the database, and between transactions and data. For example, it can characterize the sequential dependencies, read-write dependencies, and timing constraints of transaction execution.

[0033] In one embodiment, the process of generating a candidate state set may include: generating candidate values ​​for missing fields or missing records based on primary key constraints, foreign key constraints, unique constraints, NOT NULL constraints, and check constraints, and using at least one piece of information from historical snapshots, transaction logs, and application logs to complete the missing fields or missing records.

[0034] S30: Based on the evidence weights corresponding to the multi-source evidence, determine the reconstruction state of the database at the target time in the candidate state set.

[0035] In one embodiment, each piece of evidence in the multi-source evidence has a corresponding evidence weight. Based on the evidence weight, the degree of support of the multiple pieces of evidence for each target state in the candidate state set, and the soft conflict penalty function and penalty coefficient of the target state, the reconstructed state of the database at the target time can be selected from the candidate state set.

[0036] In one embodiment, step S30 may include: determining a subset of objects based on an object filtering function; determining an evidence subgraph corresponding to the subset of objects based on the evidence dependency graph corresponding to the multi-source evidence; and determining the reconstruction state of the database at the target time based on the evidence weights corresponding to the evidence subgraphs in the candidate state set.

[0037] The subset of objects is determined by at least one of the following methods: table, partition, primary key range, session-affected object, or attack path-affected object.

[0038] For example, suppose the complete set of database objects is The subset of the target objects can be represented as: .

[0039] in, Filter functions for objects.

[0040] In some implementations, object filtering functions It can be determined based on at least one of the following conditions: specified table, specified partition, specified primary key range, specified transaction set, specified session affected objects, specified attack path affected objects, or specified business object set. During subset reconstruction, only those subsets are considered. The relevant evidence subgraph performs candidate generation, conflict resolution, state solving, and witness chain generation, thereby reducing the overhead of full database scanning and full playback.

[0041] S40; Output the reconstruction status.

[0042] In one embodiment, the method may further include: calculating field-level, row-level, and table-level confidence levels corresponding to the reconstructed state based on evidence coverage, constraint consistency, source reliability, schema interpretation consistency, and conflict degree. The reconstructed state, the confidence levels, and the evidence chain corresponding to the reconstructed state are then output.

[0043] The chain of evidence corresponding to the reconstructed state refers to at least one piece of evidence corresponding to the reconstructed state.

[0044] Thus, even with incomplete audit logs and fragmented transaction logs, multi-source evidence can still be used to reconstruct the target database state. Combined with a schema version awareness mechanism, candidate states are generated based on the target schema version at the target time. This adapts to the structural evolution of the database over long-term operation, avoiding the inability to adapt to the evolution process due to a mismatch between the schema version and the target time. Furthermore, by incorporating transaction causality and database constraint sets into the candidate state generation, it is beneficial to introduce correlation factors related to the database state, enabling the reasonable completion of missing fields and records. This method of selecting the most matching reconstructed state through evidence weighting results in higher accuracy. Even under complex conditions such as incomplete audit logs, fragmented transaction logs, and schema evolution, the historical state of the target database can be reconstructed, comprehensively improving the reliability of the reconstructed state.

[0045] In some embodiments, generating a candidate state set based on the target mode version corresponding to the target time, the transaction causality, and the constraint set corresponding to the database may include: Based on the target mode version and transaction causal relationship corresponding to the target time, a candidate state set is generated by solving the state solution method that satisfies the constraint set corresponding to the database.

[0046] In one embodiment, a target time is set. The database status is recorded as Unlike existing technologies that rely solely on log playback, this application's embodiments treat the state reconstruction at the target time as a candidate state generation and optimization problem under the combined constraints of multi-source evidence, pattern versions, and a set of constraints. Let the candidate state set be: .in, N It is a positive integer. Indicates the first N There are several candidate states.

[0047] In one embodiment, the generation of candidate states is based on: the operation objects and predicate ranges derived from audit logs, before-image and after-image information in transaction logs or write-ahead logs (WAL), known states in historical snapshots, parameters or business identifiers in application logs, process-object relationships in system tracing events, and the corresponding constraint set in the database. .

[0048] In one embodiment, during the solution process for the state-based constraint set corresponding to the database, for missing fields, candidate values ​​can be generated through constraint propagation and completion using associated objects. For example, the existence of records in the main table can be inferred through foreign key records, and the candidate range of field values ​​can be inferred through unique indexes.

[0049] For example, in the process of solving the state of the constraint set corresponding to the database, candidate values ​​are generated for missing fields or missing records based on primary key constraints, foreign key constraints, unique constraints, NOT NULL constraints and check constraints, and at least one of the information in historical snapshots, transaction logs and application logs is used to complete the missing fields or missing records.

[0050] Thus, the target time-based mode version can adapt to the structural evolution of the database during long-term operation, and the transaction causal relationship can filter illegal candidates with temporal logical contradictions. Even in cases of chaotic temporal sequences of multi-source evidence, the operation chain can still be accurately restored. Furthermore, through the constraint-driven solution mechanism, missing fields and missing records can be reasonably supplemented, comprehensively improving the reliability and accuracy of candidate states.

[0051] In some embodiments, such as Figure 2 As shown, step S30 may include: S31: Perform unified time calibration, semantic standardization and pattern version mapping based on the multi-source evidence to obtain a standardized evidence set; S32: Construct an evidence dependency graph based on the standardized evidence set; the evidence dependency graph includes at least transaction nodes, object nodes, field nodes, operation nodes, and source nodes; S33: Based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph, determine the reconstruction state of the database at the target time in the candidate state set.

[0052] In one embodiment, the execution mode version mapping may include: constructing a mode version timeline based on the data definition language change history, and determining the target mode version based on the target time; for evidence that is incompatible with the target mode version, conversion is performed using at least one of the transformation rules among field renaming, field splitting, field merging, type mapping, table migration, and index change mapping.

[0053] For example, at least one piece of evidence from the multi-source evidence that is inconsistent with the target pattern version is subjected to pattern conversion to map the multi-source evidence to the target pattern version.

[0054] In one embodiment, since different evidence sources generate time in different ways, there may be problems such as host clock offset, sampling delay, asynchronous writing and batch submission. Therefore, it is necessary to first perform unified time calibration on the evidence from multiple sources.

[0055] Performing a unified time calibration may include: calibrating the timestamps of the multi-source evidence based on the systematic time offset correction amount of the corresponding sources of the multi-source evidence.

[0056] For example, suppose one piece of evidence in a multi-source evidence... The original timestamp is The timestamp after time calibration is Then it can be expressed as: .

[0057] in, Indicate the source of evidence The systematic time offset correction amount, This indicates a local correction term. In some implementations, It can be obtained through at least one of the following methods: alignment of co-occurring events of the same transaction in multiple evidence sources, alignment using heartbeat events or commit events, and alignment using the mapping relationship between the database internal log sequence number and the external log time.

[0058] In one embodiment, performing semantic standardization after completing unified time calibration may include converting the multi-source evidence into a standardized semantic structure. For example, it may include mapping the multi-source evidence to a standardized semantic structure corresponding to a standardized event model.

[0059] For example, multiple sources of evidence, such as SQL statements, WAL records, redo records, binlog events, application call logs, and system file access logs, can be uniformly converted into a standardized semantic structure, such as the semantic structure of "object-operation-value-moment".

[0060] In one embodiment, the evidence dependency graph can be a table-level, row-level, and field-level evidence dependency graph. For example, the evidence dependency graph includes at least transaction nodes, object nodes, field nodes, operation nodes, and source nodes, with transaction nodes and object nodes connected via read edges, write edges, delete edges, schema change edges, or causal edges.

[0061] In one embodiment, the evidence dependency graph can be represented as: .

[0062] in, Represents a set of nodes. This represents the set of edges. The set of nodes... At least including: .

[0063] in, Represents a set of transaction nodes. Represents a collection of object nodes. Represents a collection of field nodes. Represents the set of operation nodes. This represents the set of source nodes.

[0064] The set of edges It includes at least read edges, write edges, delete edges, schema change edges, causal edges, and source-supporting edges.

[0065] In one embodiment, each piece of evidence in the evidence dependency graph corresponds to an evidence weight, which can be a composite weight, used to optimize the candidate state set and resolve conflicts to obtain the reconstructed state corresponding to the target time.

[0066] In this way, by time calibration, semantic standardization and pattern version mapping, unified time, unified semantics and unified pattern of multi-source evidence can be achieved, which is conducive to the subsequent establishment of a reliable evidence dependency graph. It can also standardize the calculation of weight indicators such as source reliability and pattern compatibility. Then, based on the evidence dependency graph, evidence items from different sources can be uniformly organized into a temporal, objectified and traceable evidence structure, so as to solve the target database state in the future.

[0067] In some embodiments, performing unified time calibration, semantic standardization, and pattern version mapping based on the multi-source evidence to obtain a standardized evidence set may include: Based on the systematic time offset correction amount of the corresponding sources of the multi-source evidence, the timestamps of the multi-source evidence are calibrated, and the multi-source evidence is converted into a standardized semantic structure; At least one piece of evidence from the multi-source evidence that is inconsistent with the target pattern version is subjected to pattern conversion in order to map the multi-source evidence to the target pattern version.

[0068] In one embodiment, performing mode conversion on at least one piece of evidence from the multi-source evidence that is inconsistent with the target mode version may include: Based on whether each piece of evidence in the multi-source evidence directly adapts to the target mode version, whether it can be adapted to the target mode version, and whether there is information loss during the transformation process, calculate the mode compatibility of each piece of evidence; Based on the pattern compatibility, at least one piece of evidence from the multi-source evidence is determined to be inconsistent with the target pattern version; A pattern transformation is performed on at least one of the pieces of evidence.

[0069] For example, schema conversion of the at least one piece of evidence may include: performing schema conversion on at least one piece of evidence from the multi-source evidence that is inconsistent with the target schema version based on at least one of the following mappings: field renaming mapping, field splitting mapping, field merging mapping, field type conversion mapping, table migration mapping, and index or primary key structure change mapping.

[0070] In one embodiment, before performing pattern conversion on at least one piece of evidence from the multi-source evidence that is inconsistent with the target pattern version, a pattern version sequence can be constructed on a unified timeline: .

[0071] in, Indicates the first Each mode version. For the target time The target mode version is determined using the following mapping function. : If and only if .

[0072] in, Indicates mode version The effective start time, Indicates mode version The effective start time.

[0073] In one embodiment, for any piece of evidence If its original mode version With target mode version If there is an inconsistency, and the evidence is determined to be inconsistent with the target schema version, then schema conversion is performed. For example, the compatibility function between evidence and schema version can be defined as follows: .

[0074] in, This indicates whether the evidence is directly compatible with the target mode version. This indicates whether the evidence can be adapted to the target mode version. This indicates whether there is information loss during the transformation process. , and All are non-negative coefficients.

[0075] In one embodiment, the process of performing schema transformation may include at least one of the following: field renaming mapping, field splitting mapping, field merging mapping, field type conversion mapping, table migration mapping, and index or primary key structure change mapping.

[0076] Thus, through the schema version awareness mechanism, it is possible to adapt to the structural evolution of the database during long-term operation, map various pieces of evidence to the schema version corresponding to the target moment of the required reconstruction state, thereby unifying the schema interpretation of multi-source evidence and improving the reliability of the reconstruction state.

[0077] In some embodiments, determining the reconstruction state of the database at the target time based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph within the candidate state set may include: The evidence weight corresponding to each piece of evidence is calculated based on the source reliability, time consistency, content completeness, pattern compatibility, and conflict penalty for each piece of evidence in the evidence dependency graph. Based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph, the reconstruction state of the database at the target time is determined in the candidate state set.

[0078] In one embodiment, the evidence weight is determined by the reliability of the source, the consistency of time, the completeness of the content, the pattern compatibility, and the conflict penalty, and a subset of evidence is selected to participate in the state solution based on the evidence weight.

[0079] For example, suppose the evidence The weight is ,but It can be represented as: .

[0080] in, Indicates the reliability of the source. Indicates time consistency. Indicates the completeness of the content. Indicates pattern compatibility. Indicates the degree of conflict penalty. 、 、 、 , These are the weighting coefficients corresponding to different parameters.

[0081] In one embodiment, source reliability It can be defined as: .

[0082] in, Indicates the basic credibility of the source type. Indicates the degree of integrity of the source. This indicates the degree to which the source has not been tampered with. 、 These are the weighting coefficients corresponding to different parameters.

[0083] In one embodiment, time consistency It can be defined as: .

[0084] in, For the calibrated evidence timestamp, The object corresponding to the object node Reference time, This is a time-scale parameter.

[0085] In one embodiment, pattern compatibility This can be achieved by the aforementioned compatibility function. Conclusion: Content completeness The conflict penalty can be calculated based on the number of fields included in the evidence, value mirror coverage, and transaction boundary integrity. It can be calculated based on the degree of contradiction between the evidence and other high-weight evidence, as well as the degree of conflict with the constraint set.

[0086] In this way, by using multi-dimensional data such as data sources, time, content integrity, table structure adaptation, and evidence conflicts, the credibility of evidence is transformed into a calculable quantitative value, so as to make the adjudication logic objective and reproducible, without relying on human subjective judgment. Furthermore, based on the reconstruction state of multi-source evidence weights, the weight of high-quality evidence can be increased and the weight of weak evidence can be automatically reduced, solving the defect that a single log cannot be completely reconstructed, thereby improving the reliability of the reconstruction state.

[0087] In some embodiments, determining the reconstruction state of the database at the target time based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph within the candidate state set may include: Based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph, and the degree of support of the multiple pieces of evidence for each target state in the candidate state set, the reconstruction state of the database at the target time is determined in the candidate state set. In the case where multiple pieces of evidence in the evidence dependency graph point to the same target state and the content of the multiple pieces of evidence is inconsistent, a conflict resolution is performed based on the pattern compatibility, source reliability, and satisfied database constraints corresponding to the multiple pieces of evidence, so as to retain one of the multiple pieces of evidence.

[0088] In one embodiment, the reconstruction state of the database at the target time can be determined from the candidate state set based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph, the degree of support of the multiple pieces of evidence for each target state in the candidate state set, the soft conflict penalty function of the target state, and the penalty coefficient.

[0089] For example, each state in the candidate state set can be used as the target state. The reconstruction state of the database at the target time is determined from the candidate state set, which can be based on an objective function, defined as follows: That is, output the function that makes the output... The group with the largest value and .

[0090] in, Present evidence For the target state The level of support Indicates the weight of evidence. Representing state The soft conflict penalty function, Let represent the penalty coefficient. Let the target state be... Candidate adoption variables are ,in, ,like This indicates that the evidence has been accepted. ;like This indicates that the evidence will not be accepted. . n Indicates the total number of pieces of evidence.

[0091] In one embodiment, the reconstruction state is determined based on an objective function, while simultaneously satisfying the following constraints: ; .

[0092] in, This represents a hard constraint that must be satisfied. This indicates the total number of hard constraints. This indicates a soft constraint that allows for a certain degree of relaxation. This indicates the total number of soft constraints.

[0093] In one embodiment, conflict resolution is performed based on the pattern compatibility, source reliability, and satisfied database constraints corresponding to the multiple pieces of evidence, to retain one of the multiple pieces of evidence. This may include: prioritizing the retention of evidence with higher pattern compatibility; if the pattern compatibility is the same, prioritizing the retention of evidence with higher source reliability; if the source reliability is the same, prioritizing the retention of evidence that satisfies more constraints in the constraint set.

[0094] In one embodiment, if the patterns cannot be distinguished based on the pattern compatibility, the source reliability, and the satisfied database constraints, multiple candidate states are retained, and the uncertainty is reflected in the subsequent confidence level.

[0095] Thus, based on the degree of support between evidence and candidate target states, and the penalty function, the reconstructed state that best matches the evidence and the target state at that moment can be accurately matched. The final reconstructed state is the equilibrium solution with the highest overall credibility of the evidence and the fewest violations of business rules. Furthermore, through the evidence conflict adjudication mechanism, the stability of the reconstruction results can be improved, ultimately enhancing the overall reliability of the reconstructed state.

[0096] In some embodiments, outputting the reconstruction status may include: Based on the evidence coverage, constraint consistency, source reliability, pattern interpretation consistency, and conflict degree corresponding to the reconstructed state, the field-level confidence level corresponding to the reconstructed state is calculated; Row-level confidence scores are obtained by aggregating the field-level confidence scores. The table-level confidence level is obtained by aggregating the row-level confidence levels. Output the reconstruction status, the field-level confidence level, the row-level confidence level, the table-level confidence level, and the evidence corresponding to the reconstruction status.

[0097] In one embodiment, confidence levels are defined at the field, row, and table levels, respectively. For fields... The reconstruction status, its field-level confidence level is defined as: .

[0098] in, For mapping functions, Indicates the coverage of evidence. Indicates the degree of constraint consistency. This indicates a high degree of support from a credible source. Represents the consistency of pattern interpretation. Indicates the degree of conflict. 、 、 、 、 、 These are the weighting coefficients corresponding to different parameters.

[0099] In one embodiment, the mapping function For the Sigmoid function: .

[0100] In one embodiment, for row Its row-level confidence can be defined as an aggregation of field-level confidence: .

[0101] in, Indicates line The corresponding set of fields.

[0102] In one embodiment, for table Its table-level confidence score can be defined as the aggregation of row-level confidence scores: .

[0103] in, Representation Table The corresponding set of rows.

[0104] In one embodiment, subset-level confidence can also be defined for subsets of objects and output together with field-level confidence, row-level confidence, and table-level confidence.

[0105] In one embodiment, the evidence corresponding to the reconstructed state can be output in the form of a chain of evidence.

[0106] Thus, by using field-level, row-level, and table-level confidence quantification mechanisms, a single recovery result is avoided. Field-level, row-level, and table-level confidence quantification enables the reconstruction state to be interpretable, which helps to improve the reliability of the reconstruction state.

[0107] In some embodiments, after outputting the reconstruction status, the following may also be included: Based on the operation summary, intermediate state summary and time identifier of each operation before the reconstruction state is output, a witness value is generated for each operation. All the witness values ​​are integrated into a witness chain and stored.

[0108] In one embodiment, to make the reconstruction process itself verifiable, a witness value is generated for each step of the reconstruction operation. Let the... Step operation is , No. The intermediate state corresponding to the step operation is Its witness value is defined as: .

[0109] in, Represents a cryptographic hash function. This indicates a splicing operation. Indicates the first Operation summary of each step Indicates the first Summary of intermediate states of each step operation. Indicates the first The time marker for each step operation This represents the witness value from the previous operation.

[0110] After all the reconstruction steps are completed, a witness chain consisting of all witness values ​​will be obtained: .

[0111] Thus, through this witness chain, it is possible to verify whether there are any missing steps in the reconstruction process, whether the intermediate state has been tampered with, whether the final result is consistent with the reconstruction steps, and whether the final result is traceable to the original evidence.

[0112] In some embodiments, the above-described database reconstruction method can be applied to a database reconstruction system, such as... Figure 3 As shown, the system includes an evidence acquisition module, a standardization processing module, a graph construction module, a candidate generation module, a decision-making module, a confidence assessment module, a witness generation module, and an output module. For example, each module can be deployed on the same physical device or multiple server nodes and communicate via a message bus or application programming interface (API) gateway.

[0113] The system includes: an evidence acquisition module for acquiring multi-source evidence from the database at a target time; a standardization processing module for performing unified time calibration, semantic standardization, and pattern version mapping based on the multi-source evidence to obtain a standardized evidence set; a graph construction module for constructing an evidence dependency graph based on the standardized evidence set; a candidate generation module for generating a candidate state set based on the target pattern version at the target time, the transaction causal relationship, and the constraint set corresponding to the database; and an output module for outputting the reconstructed state.

[0114] In one embodiment, the decision-making module can be used to determine the reconstruction state of the database at the target time based on the evidence weights corresponding to the multi-source evidence in the candidate state set. For example, based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph and the degree of support of the multiple pieces of evidence for each target state in the candidate state set, the reconstruction state of the database at the target time is determined in the candidate state set. Wherein, if multiple pieces of evidence in the evidence dependency graph point to the same target state and the content of the multiple pieces of evidence is inconsistent, a conflict decision is performed based on the pattern compatibility, source reliability, and satisfied database constraints corresponding to the multiple pieces of evidence, so as to retain one of the multiple pieces of evidence.

[0115] In one embodiment, the confidence assessment module is used to calculate the field-level, row-level, and table-level confidence levels corresponding to the reconstruction state; the witness generation module is used to generate a witness value corresponding to each operation based on the operation summary, intermediate state summary, and time identifier corresponding to each operation before the reconstruction state is output; and all the witness values ​​are integrated into a witness chain for storage.

[0116] As one possible implementation method, the following three embodiments are provided as explanations of specific implementation methods: Example 1: Reconstruction of an e-commerce order database after abnormal deletion.

[0117] An e-commerce platform's order database includes an order table, an order details table, a payment table, and a refund table. At one point, it was discovered that an attacker maliciously deleted data via an application interface, resulting in the loss of some order information. The system can obtain the following evidence: audit logs recording the top-level delete statement; WAL fragments, retaining only some transaction page changes; application access logs recording the user, interface, and parameters that initiated the request; Data Definition Language (DDL) change history, showing that a refund status field was added to the order table two months ago; a table-level snapshot from the previous hour; and system source events showing the call relationship between the database process and the application process.

[0118] In this embodiment, the system first determines the corresponding mode version at the target time based on the DDL historical construction mode version timeline. Then, it maps the deletion statements in the audit log to the field layout of the target mode, and further infers the primary key set of the deleted records by combining WAL fragments. Next, using the interface parameters saved in the application log and the foreign key constraints in the refund and payment tables, it completes some missing fields in the order records. For example, for the order remarks field, since the WAL fragment is not fully saved and there is no field value in the audit log, this invention calculates the support of multiple candidate values ​​through snapshots, application logs, field constraints, and source weights, and selects the candidate value with the highest score as the final reconstructed value. Simultaneously, a low but acceptable field-level confidence score is output for this field. Finally, the system outputs the reconstruction result of the target order record, field-level, row-level, and table-level confidence scores, and the complete witness chain.

[0119] In this embodiment, compared with the scheme that only relies on WAL replay, this embodiment can still complete the state reconstruction using audit logs, application logs and snapshots when the WAL is incomplete; compared with the scheme that only relies on audit logs, this embodiment can complete the missing information at the field level; compared with the scheme that only relies on anti-tampering logs, this embodiment can directly provide verifiable reconstruction results of the database state.

[0120] Example 2: Object subset directional reconstruction.

[0121] In another implementation, the target subset is limited to the primary key range of the order table affected by the attack session. Corresponding records. The system extracts evidence, constructs subgraphs, generates candidate states, and calculates witness chains only for objects within the scope of the primary key. In this implementation, if the total number of records in the table reaches hundreds of millions, but only a few hundred objects are affected, this embodiment only needs to perform targeted solving on a subset of the relevant objects, without having to perform a full reconstruction of the entire database, thereby significantly reducing resource consumption and recovery latency.

[0122] Example 3: Reconstruction of historical states in a pattern evolution scenario.

[0123] A user table in a business database undergoes the following schema evolution during operation: In the initial version, the fields were user ID, name, and mobile phone number; in the second version, an email field was added; in the third version, the mobile phone number was split into country code and local number fields; and in the fourth version, the name was split into surname and given name. If the target time is in the third version, but the evidence contains both second-version audit logs and fourth-version snapshots, existing techniques often struggle to provide a unified interpretation. This embodiment can convert second-version evidence to third version based on the schema version timeline, and then reverse-map the fourth-version snapshot to the third version, allowing evidence from different time periods to participate in state resolution under the same target schema. This embodiment significantly reduces the reconstruction failure rate caused by schema evolution.

[0124] In some implementations, the object granularity of an object node can be expanded from "table-row-field" to "collection-document-attribute", "namespace-key-value fragment", or "graph-node-attribute / edge", etc.

[0125] In another alternative implementation, a machine learning model can be introduced to generate prior candidate values ​​for fields with insufficient evidence coverage. However, the machine learning model is only used as a means of candidate generation, and the final state is still determined by database constraints, consistency rules, and evidence weights.

[0126] In another alternative implementation, the method of this application embodiment can be integrated with a disaster recovery system, a compliance audit platform, a security operation platform, or a database proxy layer to achieve online high-confidence reconstruction or offline evidence collection and reconstruction.

[0127] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the database reconstruction method as described in any embodiment of this application.

[0128] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the method described in any of the above embodiments.

[0129] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0130] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0131] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A database reconstruction method, characterized in that, include: Obtain multi-source evidence from the database at the target time; the multi-source evidence includes at least two of the following: audit logs, transaction logs, schema change records, historical snapshots, application logs, and system tracing events. A candidate state set is generated based on the target mode version corresponding to the target time, the transaction causal relationship, and the constraint set corresponding to the database. Based on the systematic time offset correction amount of the corresponding sources of the multi-source evidence, the timestamps of the multi-source evidence are calibrated, and the multi-source evidence is converted into a standardized semantic structure; Based on whether each piece of evidence in the multi-source evidence directly adapts to the target mode version, whether it can be adapted to the target mode version, and whether there is information loss during the transformation process, calculate the mode compatibility of each piece of evidence; Based on the pattern compatibility, at least one piece of evidence from the multi-source evidence is determined to be inconsistent with the target pattern version; The at least one piece of evidence is subjected to pattern transformation to map the multi-source evidence to the target pattern version, thereby obtaining a standardized evidence set; Construct an evidence dependency graph based on the standardized evidence set; The evidence dependency graph includes at least transaction nodes, object nodes, field nodes, operation nodes, and source nodes; Based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph, the reconstruction state of the database at the target time is determined in the candidate state set. Output the reconstruction status.

2. The database reconstruction method according to claim 1, characterized in that, The process of generating a candidate state set based on the target mode version corresponding to the target time, the transaction causal relationship, and the constraint set corresponding to the database includes: Based on the target mode version and transaction causal relationship corresponding to the target time, a candidate state set is generated by solving the state solution method that satisfies the constraint set corresponding to the database.

3. The database reconstruction method according to claim 1, characterized in that, The step of determining the reconstruction state of the database at the target time based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph within the candidate state set includes: The evidence weight corresponding to each piece of evidence is calculated based on the source reliability, time consistency, content completeness, pattern compatibility, and conflict penalty for each piece of evidence in the evidence dependency graph. Based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph, the reconstruction state of the database at the target time is determined in the candidate state set.

4. The database reconstruction method according to claim 3, characterized in that, The step of determining the reconstruction state of the database at the target time based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph within the candidate state set includes: Based on the evidence weights corresponding to multiple pieces of evidence in the evidence dependency graph, and the degree of support of the multiple pieces of evidence for each target state in the candidate state set, the reconstruction state of the database at the target time is determined in the candidate state set. In the case where multiple pieces of evidence in the evidence dependency graph point to the same target state and the content of the multiple pieces of evidence is inconsistent, a conflict resolution is performed based on the pattern compatibility, source reliability, and satisfied database constraints corresponding to the multiple pieces of evidence, so as to retain one of the multiple pieces of evidence.

5. The database reconstruction method according to claim 1, characterized in that, The output of the reconstructed state includes: Based on the evidence coverage, constraint consistency, source reliability, pattern interpretation consistency, and conflict degree corresponding to the reconstructed state, the field-level confidence level corresponding to the reconstructed state is calculated; Row-level confidence scores are obtained by aggregating the field-level confidence scores. The table-level confidence level is obtained by aggregating the row-level confidence levels. Output the reconstruction status, the field-level confidence level, the row-level confidence level, the table-level confidence level, and the evidence corresponding to the reconstruction status.

6. The database reconstruction method according to claim 1, characterized in that, After outputting the reconstructed state, the following is also included: Based on the operation summary, intermediate state summary and time identifier of each operation before the reconstruction state is output, a witness value is generated for each operation. All the witness values ​​are integrated into a witness chain and stored.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the database reconstruction method as described in any one of claims 1 to 6.

Images