Data blood relationship recording and training proving method, device, equipment and medium

Abstract

The invention relates to the technical field of distributed account books, can be applied to business scenes such as financial science and technology and medical health, and discloses a data consanguinity recording and training proving method, device and equipment and a medium. Constructing a data storage transaction and writing the data storage transaction into a distributed account book to form a data consanguinity record; acquiring a data abstract value set, a model configuration parameter and a training timestamp based on the batch identifier in a model training process, generating a training proof, and writing the training proof into a distributed account book to establish an associated index; and during verification, data integrity and training compliance verification is executed by using the association index to retrieve the data consanguinity record and the training proof. According to the method, the data blood relationship and the training proof are recorded, and the link association is established, so that the data use process and the training process are traceable and verifiable, and the data management transparency and the compliance credibility of model training can be improved.

Description

Technical Field

[0001] This invention relates to the field of distributed ledger technology, and in particular to a method, apparatus, device, and medium for data lineage recording and training proof. Background Technology

[0002] In the fintech business, business data is typically distributed across multiple systems and channels, including policy management systems, claims systems, risk control systems, and customer service systems. This data is frequently transferred and processed between different stages, forming complex data chains. Existing data lineage management largely relies on database logs, ETL process records, or manually maintained document lists. These methods struggle to cover cross-system data flows and lack a unified and reliable tracking mechanism. In the event of data quality disputes, disputes over the misuse of models, or regulatory audit requirements, existing lineage records often fail to provide complete, continuous, and reliable evidence. Furthermore, while data tagging is crucial in risk assessment, customer profiling, and claims classification, current technologies cannot trace the origin of data tags or provide proof of tag credibility, leading to difficulties in verifying the data foundation used by many models.

[0003] In the healthcare sector, medical data flows continuously between stages such as patient reception, examination, diagnosis, treatment, and follow-up, resulting in complex data processing links between different departments and business systems. Existing data management models typically rely on internal information system logs or manual documentation, lacking verifiable records of the entire data processing workflow. When healthcare institutions need to conduct data compliance reviews of diagnostic assistance models, risk warning models, or medical record analysis models, it is difficult to accurately determine which specific data was used during training, whether the data was used within the legally authorized scope, and whether the data processing link is consistent with the records. Furthermore, healthcare business data is more sensitive; if training records are incomplete or data is tampered with, it will be difficult to provide credible compliance audit evidence to regulators and patients. Summary of the Invention

[0004] The main objective of this invention is to provide a method, apparatus, device, and storage medium for data lineage recording and training proof, aiming to solve the technical problem that the existing technology lacks a unified compliance verification mechanism that can simultaneously form a credible data lineage record for the entire business data process and generate verifiable training proofs for the model training process, resulting in the inability to effectively audit and verify the compliance of data use and model training.

[0005] To achieve the above objectives, the present invention provides a method for data lineage recording and training proof, comprising: Acquire business data, extract attribute metadata of the business data according to a preset data parsing strategy, and determine the data digest value of the business data; A data notarization transaction containing the attribute metadata and data digest value is constructed, the data notarization transaction is broadcast to the consensus nodes of the distributed ledger network for consensus verification, and a data lineage record is generated in the distributed ledger network; During the training of the target model, the batch identifier of the current training batch is obtained, the set of data summary values ​​corresponding to the training data read in the current training batch is obtained from the data lineage record, and the model configuration parameters corresponding to the target model and the training timestamp of the current training batch are obtained. Generate training proof based on the batch identifier, data digest value set, model configuration parameters, and training timestamp; The training proof is written into the distributed ledger network, and an association index between the data lineage record and the target model is established in the distributed ledger network; In response to a verification request, data lineage records and training proofs are retrieved from the distributed ledger network based on the associated index to verify data integrity and model training compliance.

[0006] Furthermore, to achieve the above objectives, the present invention provides a data lineage recording and training verification device, comprising: The data parsing module is used to acquire business data, extract attribute metadata of the business data according to a preset data parsing strategy, and determine the data digest value of the business data. The evidence storage module is used to construct a data evidence storage transaction containing the attribute metadata and data digest value, broadcast the data evidence storage transaction to the consensus nodes of the distributed ledger network for consensus verification, and generate a data lineage record in the distributed ledger network. The training acquisition module is used to obtain the batch identifier of the current training batch during the training process of the target model, obtain the set of data summary values ​​corresponding to the training data read in the current training batch from the data lineage record, and obtain the model configuration parameters corresponding to the target model and the training timestamp of the current training batch. The proof generation module is used to generate training proofs based on the batch identifier, the set of data digest values, the model configuration parameters, and the training timestamp. An on-chain indexing module is used to write the training proof into the distributed ledger network and establish an association index between the data lineage record and the target model in the distributed ledger network; The verification module is used to respond to verification requests by retrieving data lineage records and training proofs from the distributed ledger network based on the associated index to verify data integrity and model training compliance.

[0007] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a data lineage recording and training verification program stored in the memory and executable on the processor, wherein when the data lineage recording and training verification program is executed by the processor, it implements the steps of the data lineage recording and training verification method as described above.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a non-volatile computer-readable storage medium storing a data lineage record and training proof program, wherein the data lineage record and training proof program, when executed by a processor, implements the steps of the data lineage record and training proof method as described above.

[0009] Beneficial Effects: This invention relates to the field of distributed ledger technology and can be applied to business scenarios such as fintech and healthcare. It discloses a method, apparatus, device, and medium for data lineage recording and training proof, comprising: acquiring business data and performing data parsing processing to extract attribute metadata and data digest values; constructing a data storage transaction containing attribute metadata and data digest values ​​and submitting it to a distributed ledger network to form a data lineage record; acquiring training batch information during model training and reading the set of data digest values ​​corresponding to the training data from the data lineage record, while simultaneously acquiring model configuration parameters and training timestamps; generating a training proof based on the batch identifier, the set of data digest values, the model configuration parameters, and the training timestamp, and writing the training proof into the distributed ledger network to establish an association index between the data lineage record and the target model; and responding to a verification request by retrieving the data lineage record and training proof from the distributed ledger network through the association index to perform data integrity and training compliance verification. This invention extracts business data metadata and generates data summary values, records data lineage in a distributed ledger network, generates training proofs during model training and establishes a connection with the data lineage records, making the data flow chain and the entire training process traceable and verifiable. In the verification phase, it can complete data integrity and training compliance checks based on on-chain records, thereby improving the transparency of data management and enhancing the credibility of the model training process. Attached Figure Description

[0010] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of an application environment for the data lineage recording and training proof method in one embodiment of the present invention; Figure 2 This is a flowchart illustrating an embodiment of the data lineage recording and training proof method of the present invention; Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the data lineage recording and training verification device of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0011] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0012] The data lineage recording and training proof method provided in this embodiment of the invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can obtain business data from the client and perform data parsing to extract attribute metadata and data digest values; construct a data storage transaction containing attribute metadata and data digest values ​​and submit it to the distributed ledger network to form a data lineage record; during model training, obtain training batch information and read the set of data digest values ​​corresponding to the training data from the data lineage record, while also obtaining model configuration parameters and training timestamps; generate training proofs based on batch identifiers, data digest value sets, model configuration parameters, and training timestamps, and write the training proofs to the distributed ledger network to establish an association index between the data lineage record and the target model; respond to verification requests and retrieve the data lineage record and training proofs from the distributed ledger network through the association index to perform data integrity and training compliance verification. This invention extracts business data metadata and generates data summary values, records data lineage in a distributed ledger network, and generates training proofs during model training and establishes a link with the data lineage records. This makes the data flow chain and the entire training process traceable and verifiable. During the verification phase, it can complete data integrity and training compliance checks based on on-chain records, thereby improving data management transparency and enhancing the credibility of the model training process. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0013] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the data lineage recording and training proof method provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0014] like Figure 2 As shown, the data lineage recording and training proof method proposed in this invention includes the following steps: S10, acquire business data, extract attribute metadata of the business data according to a preset data parsing strategy, and determine the data digest value of the business data; In this embodiment, business data is acquired from system interfaces, data streams, or file inputs, and its basic format is parsed by a data reading component. Business data may contain text fields, numeric fields, tag fields, or time fields. Different sources lead to structural differences, thus requiring a unified data parsing strategy. The parsing strategy includes field identification, format standardization, numeric conversion, tag extraction, and field classification rules, used to map raw fields into a well-structured set of fields. Field identification uses a preset field mapping table or semantic model to determine the meaning of fields; format standardization is used to unify date encoding, text encoding, and delimiters; numeric conversion processes characters and numbers into a unified numeric type; tag extraction can use keyword matching or semantic recognition; and field classification is used to group similar fields together with consistent attributes.

[0015] Attribute metadata is generated based on the parsing results and includes field name, field type, field semantic tag, source identifier, collection time, and verification status, used to express the data structure and business meaning. Field semantic tags are determined by the parsing strategy or semantic model; the source identifier is recorded through the data interface; the collection time is obtained from the data access timestamp; and the verification status determines the validity of a field based on its format and value rules. Once generated, attribute metadata is used to construct the data feature structure.

[0016] The data digest value is generated by the digest calculation module. It is calculated by concatenating field contents in a fixed order and inputting them into a digest algorithm to calculate fixed-length data. The digest algorithm can use a hash function or a combination of multiple digest segments. Different field types need to be uniformly encoded before digestion. Numerical fields need to handle fluctuation errors, while multimedia fields can be calculated using feature digests instead of bit digests. The structure of attribute metadata determines the organization of the digest content, ensuring the digest's verifiability. By acquiring data, applying parsing strategies, extracting attribute metadata, and calculating digest values, a stable data foundation chain can be formed for subsequent trusted recording and verification.

[0017] This embodiment enables business data to have structured expression capabilities and verifiable characteristics through a unified parsing strategy, attribute metadata structure, and summary calculation mechanism, achieving consistency and integrity in cross-system processing and providing a reliable foundation for subsequent evidence storage, auditing, and training verification stages.

[0018] S20, construct a data storage transaction containing the attribute metadata and data digest value, broadcast the data storage transaction to the consensus nodes of the distributed ledger network for consensus verification, and generate a data lineage record in the distributed ledger network; In this embodiment, attribute metadata and data digest values ​​are combined into a data storage transaction after structuring. A data storage transaction represents a record unit of data state, containing a parsable structure, a field content area, and verification-related fields. The structure determines the parsing method, the field content area carries the attribute metadata and data digest values, and the verification-related fields are used to subsequently verify whether the data has been tampered with. The generation process of a data storage transaction needs to ensure structural integrity, parsable content, and consistent field order so that different nodes can reproduce the same content.

[0019] After a data storage transaction is generated, it enters a broadcast process. The broadcast mechanism sends the transaction content to the consensus nodes in the distributed ledger network through the network communication module. The broadcast process does not rely on a single node, but distributes the transaction to multiple consensus nodes, making data recording not controlled by a single point. Consensus nodes refer to the nodes in the distributed ledger network that perform transaction reception, content verification, sorting, and confirmation. They typically have independent computing environments and can participate in the consensus process.

[0020] Consensus verification involves consensus nodes performing format verification, digest verification, structure verification, and uniqueness checks on data storage transactions. Format verification ensures the transaction structure conforms to the ledger's specified format; digest verification confirms the data digest value has not been modified; structure verification checks the field types and their order of combination; and uniqueness checks are used to identify duplicate or invalid transactions. After passing consensus verification, the data storage transaction is written to the ledger, and once written, the transaction cannot be modified irreversibly.

[0021] A data lineage record is a data association structure generated based on the evidence stored after a successful ledger write. It includes a set of data source fields, a summary reference structure, time information, and a relationship index, used to express the relationship between data and its source. The generation of the data lineage record depends on the ledger write result. Each record is bound to a unique transaction identifier and forms a traceable link through time sequence, summary reference, and source information, enabling subsequent training, validation, and data auditing to trace the data source path.

[0022] This embodiment encapsulates attribute metadata and summary values ​​into evidence-based transactions and writes them into a distributed ledger network, enabling cross-system data recording. This gives the data link traceable, verifiable, and tamper-proof characteristics, building a stable data lineage infrastructure for subsequent training records and audit verification.

[0023] S30, during the training process of the target model, obtain the batch identifier of the current training batch, obtain the set of data summary values ​​corresponding to the training data read in the current training batch from the data lineage record, and obtain the model configuration parameters corresponding to the target model and the training timestamp of the current training batch. In this embodiment, during the training of the target model, it is necessary to identify training batches from the training execution environment and establish a connection between batch identifiers and data lineage records. A batch identifier is a unique number generated by the training execution process for each training batch, used to distinguish different data reading processes within the training loop. Batch identifiers can originate from the internal event management structure, batch generator, or the iteration control module of the training framework, and their generation logic is typically related to the training iteration order, task time, or randomization parameters. After generating batch identifiers, the training system obtains the identifier information corresponding to the training data from the data reading subsystem. This identifier information is mapped to the data lineage records, thus allowing the location of the actual set of data summary values ​​read during the training process from the data lineage records through the identifier mapping structure. The set of data summary values ​​consists of multiple summary values, used to characterize the reliability of the training data. It is generally generated based on a hash function and used to verify that the data read during training has not been replaced or shifted.

[0024] To ensure the verifiability of the training process, model configuration parameters need to be obtained during the training process. These parameters can originate from the model definition file, training start instructions, or parameter structures loaded by the training controller. They include hierarchical structure, activation mode, loss structure, optimization control parameters, etc. These parameters remain stable during training and are used to represent the structure and hyperparameter conditions upon which the training is based.

[0025] Training timestamps represent the time when a training batch occurred. In the training environment, they can be directly generated by the time management module, logging system, or training scheduler. Training timestamps help determine whether training operations match the order in which data is used. Training timestamps can use second-level, millisecond-level, or any fixed precision, facilitating subsequent comparison of the training order. During training, the batch identifier, data summary value set, model configuration parameters, and training timestamps together constitute a batch-level training record, used for subsequent verification of whether the training behavior is consistent with the expected data flow.

[0026] This embodiment extracts batch identifiers, data summary value sets, model configuration parameters, and training timestamps during the training process to form comparable and traceable training records. This makes the training behavior verifiable in subsequent audits and helps to confirm that the input data, configuration conditions, and actual time sequence of the training batch have not been offset or replaced.

[0027] S40, generate training proof based on the batch identifier, data digest value set, model configuration parameters and training timestamp; In this embodiment, the process of generating training proofs based on batch identifiers, data digest value sets, model configuration parameters, and training timestamps belongs to the structured record generation process of training behavior, used to construct a verifiable chain of training evidence. The batch identifier originates from the batch management structure of the training scheduling environment, possessing uniqueness and traceability, and is used to identify the training epoch to which the training behavior belongs. When incorporating the batch identifier into the training proof, it is necessary to ensure that the identifier is consistent with the event sequence of the training execution process, enabling it to form a stable mapping with the index in the data lineage record.

[0028] The data digest value set consists of multiple data digests actually read in the training batch. These digests are generated by hash calculation logic and are stable and irreversible, serving to fix the data source in the generation of training proofs. When writing the data digest value set into the training proof, it is necessary to ensure that the order of the data digests in the set is consistent with the order of reading during training, so as to restore the reading trajectory of the training batch in the verification stage.

[0029] Model configuration parameters describe the structural definitions and hyperparameter configurations used in the training process, and are conditional attributes of the training behavior. Model configuration parameters can be contained in parameter dictionaries, structured vectors, or compressed parameter sets. Their scope can be expanded according to the training system's definition of parameter importance, including layer structure, activation settings, loss configuration, learning rate control strategies, etc. Including model configuration parameters in training demonstrates that it provides comparable conditional information for subsequent training validation.

[0030] Training timestamps are generated by the training scheduler or time management module when training occurs, and are used to indicate the point in time when the training action takes place. After being included in the proof-of-training process, the training timestamps are used to verify whether the training order matches the data usage order, and also to reconstruct the timeline of training actions across node environments.

[0031] When integrating batch identifiers, data digest value sets, model configuration parameters, and training timestamps, a structured data block needs to be constructed to stably express the relationships between multiple fields. This structured data block can adopt a hierarchical or flat structure, and improving verifiability by fixing the field order and format. After completing the structured data block, a corresponding hash value needs to be generated to produce a fixed-length digest representation, ensuring the training proof has verifiable consistency. To prevent the training proof from being illegally modified during transmission or storage, a digital signature mechanism is used to perform a signature operation on the hash value, giving the training proof verifiable integrity protection capabilities. The final generated training proof includes a structured data block, a hash value, and a digital signature, facilitating traceability and integrity verification in subsequent verification stages.

[0032] This embodiment integrates batch identifiers, data digest value sets, model configuration parameters, and training timestamps into training proof and adds signature protection, which can form a traceable and verifiable training behavior record, ensuring that the source of training data, training conditions, and training time can be accurately determined in the subsequent verification process to ensure that they are consistent with the original record.

[0033] S50, the training proof is written into the distributed ledger network, and an association index between the data lineage record and the target model is established in the distributed ledger network; In this embodiment, the process of writing the training proof into the distributed ledger network includes operations such as encapsulation, submission, verification, and recording. The training proof, after being structurally combined, is encapsulated into a ledger-recognizable data unit. The encapsulation structure can adopt binary format, key-value pair format, or hierarchical field format to meet the parsing requirements of multiple nodes. After encapsulation, the training proof is submitted to the ledger nodes, which perform hash verification, signature verification, and structure verification to ensure that the verified content is consistent with the original record generated during training. Upon successful verification, the ledger node adds the training proof to the ledger structure, generating a fixed-position index in the ledger for subsequent tracking and retrieval.

[0034] When establishing the association index between data lineage records and the target model, it is necessary to combine the storage location index of training proof, the data transaction index in the data lineage record, and the model identification information of the target model into a unique mapping. This mapping expresses the relationship between the training data source, training conditions, and model entities, enabling the ledger to synchronously record the connection between training actions and data sources. After the association index is generated, it is written to the ledger network in a structured manner, ensuring the immutability of the index content and obtaining a consistent view in the ledger's multi-node structure. The association index also serves as the entry point for subsequent verification queries, enabling training actions and data usage records to form a bidirectional retrieval path in the ledger structure.

[0035] This embodiment writes training proofs into the ledger and generates associated indexes, forming a verifiable link between training records, data sources, and model entities. This makes the training process traceable, verifiable, and tamper-proof, significantly improving data usage compliance and the credibility of training records.

[0036] S60, in response to the verification request, retrieve data lineage records and training proofs from the distributed ledger network based on the association index to verify data integrity and model training compliance.

[0037] In this embodiment, the process of responding to verification requests begins with receiving a verification request from an external system or auditing system. The verification request carries a target identifier to determine the data trajectory and training behavior that need to be verified. The target identifier can come from a business primary key, model identifier, or batch identifier, and is used to locate the associated index in the ledger structure. The associated index serves to establish a connection path between training proofs, data lineage records, and the target model in the ledger network. It can be understood as a reference structure in the ledger, through which the data source information and model operating conditions used in the training process can be quickly located.

[0038] After obtaining the target identifier, the data lineage record and training proof are located on the ledger node using the associated index. The data lineage record contains fields such as data digest value, data tag, data generation time, and transaction index, which express the source chain of the data before it is collected, parsed, and enters the training process. The training proof contains batch identifier, training timestamp, model configuration parameters, data digest value set, and hash verification content, which express the conditions and input states in the training process. The ledger network adopts a decentralized storage structure. Ledger nodes locate and retrieve storage locations using hash positioning and block numbers, ensuring that the retrieval operation has clear temporal order and is tamper-proof.

[0039] After the retrieval is completed, the data summary value in the data lineage record is compared with the summary value of the current target data to determine whether the data has been altered during collection, parsing, or transmission. Summary generation relies on hash operations, so any minor change will alter the summary result, making it naturally applicable for data integrity verification. Model configuration parameters and training timestamps are extracted from the training proof. Model configuration parameters may include model type, layer structure, training epochs, learning rate, or other key parameters, while the training timestamp indicates the actual time the training action occurred. Checking the parameter and time information allows us to determine whether the training action matches the expected configuration and whether the training sequence is reasonable.

[0040] After completing the summary verification and configuration check, data integrity verification conclusions and model training compliance verification conclusions are generated based on the verification results, and a unified verification report is further formed. The report includes the data chain status, training behavior status, and compliance conclusions, which serve as the basis for subsequent audits, supervision, or automated checks.

[0041] This embodiment obtains a complete data source chain and training condition record by retrieving data lineage records and training proofs based on the association index. This enables the auditing system to accurately determine whether the data has been tampered with and whether the training behavior follows the established norms, thereby improving the transparency of data use, the controllability of the training process, and the compliance verification capability.

[0042] In one embodiment, step S10 includes: S101, receiving business data items from multiple data systems; S102, parse each business data item according to the preset data parsing strategy and extract the data source system identifier; S103 records the data generation timestamp, data type, and sensitivity level for each business data item; S104, assign data labels and annotation information to each business data item; S105, Use a hash function to determine the data digest value for each business data item; S106, combine the data source system identifier, data generation timestamp, data type, sensitivity level, data tag, annotation information and data summary value to form attribute metadata.

[0043] In this embodiment, business data refers to the set of raw records generated during specific business operations, which may include transaction records, customer information records, claims records, device reporting records, etc., and exist in structured tables, semi-structured logs, or message formats. When collecting business data, the system can establish connections with multiple data systems through message queue subscriptions, file imports, interface pulls, etc. These data systems may include core business systems, risk management systems, customer relationship management systems, external service platforms, etc. To ensure consistency and controllability in processing, a unified access standard needs to be configured for data channels from different sources during the access phase, such as field encoding formats, time field precision, character sets, line breaks and delimiters, etc., so that subsequent parsing can be performed under the premise of unified input.

[0044] A pre-defined data parsing strategy guides the extraction of structural information and key fields from business data. This strategy can be maintained through configuration files, rule bases, or a visual configuration interface, and typically includes field mapping rules, data type conversion rules, time format conversion rules, data masking and hiding rules, and outlier handling rules. For example, transaction time fields from different systems can be uniformly converted to a standard timestamp format using the parsing strategy; amount fields can be uniformly converted to a unified currency unit; and text fields can be cleaned using rules to remove invalid characters. The parsing process can be executed sequentially by the parsing engine. First, it determines which parsing strategy to use based on the data source, then performs field splitting, type validation, and format conversion according to field-level rules, transforming the original business data items into an internally unified structure.

[0045] A business data item is the smallest processing unit extracted from the business data stream; it can be understood as a single transaction, a claim record, or a device report record. When receiving data, the system generates a unique internal identifier for each business data item, used for subsequent association with attribute metadata and data summary values. The data source system identifier identifies which data system the business data item originates from. This identifier can be in the form of a system code, system name, or system domain identifier. Source information can come from data channel configuration, request header fields, message subject names, etc. The parsing strategy extracts or derives the data source system identifier from the business data item using predefined rules and persists it.

[0046] The data generation timestamp records the time when a business data item is generated within a business scenario. This timestamp can originate from the business time field written by the business system or from the reception time recorded at the access gateway. The system can uniformly convert raw time fields into a standard timestamp format and record time zone information to facilitate time series analysis in cross-regional and multi-system environments. Data types describe the semantic category of business data items, such as transaction data, policy information, claims records, and behavioral events. The parsing strategy can categorize business data items into corresponding data types based on field combinations, interface sources, and topic names, facilitating subsequent classification and auditing according to different types.

[0047] Sensitivity levels characterize the degree of sensitivity of information contained in business data items. They can be categorized into multiple levels, such as public, internal, sensitive, and high-sensitivity. Sensitivity levels can be determined based on a field-level sensitivity rule engine. The engine matches field names, field content patterns, and business domain configurations to comprehensively assess whether business data items contain sensitive content such as identity information, health information, financial information, and location information, and labels data containing such content with a higher sensitivity level. The system can then associate sensitivity levels with access control policies, encryption policies, and storage policies to achieve subsequent compliance controls.

[0048] Data tags are used to categorize and label business data items from a business perspective, such as their business line, product type, customer segment, risk level, and channel source. Annotation information can be used to record additional explanatory content, such as tag generation rules, manual review comments, and quality inspection results. Data tags and annotation information can originate from existing fields in the business system or be generated in the data access platform through rule calculations or model inference. For example, abnormal behavior tags can be generated based on transaction behavior sequences, and case complexity tags can be generated based on claims fields. When parsing business data items, the system organizes the tags and annotation information into structured fields for unified management with other metadata.

[0049] Hash functions are used to map business data items to fixed-length data digest values, which are then used for subsequent data integrity verification. When calculating the data digest value, the system first serializes the business data items according to a predefined field order. Text, numerical, and time fields are uniformly encoded and formatted, then concatenated into a byte sequence. This byte sequence is then input into the hash function to calculate a fixed-length digest result. Hash functions have the characteristics of one-wayness and extremely low collision probability, meaning that if any field of a business data item is modified, the data digest value will change significantly. This can be used to detect whether data has been tampered with or lost in subsequent stages. To avoid digest differences caused by different field orders, the serialization stage must strictly adhere to a fixed field order and a fixed encoding format.

[0050] Attribute metadata is a comprehensive collection of information that provides a structured description of business data items. In this step, attribute metadata is formed by combining the data source system identifier, data generation timestamp, data type, sensitivity level, data tag, annotation information, and data summary value. The system can organize these fields into a unified structure, such as a key-value structure, row record structure, or document structure, and bind them to the internal identifier of the business data item. Attribute metadata can be stored in a metadata storage system, retrieval engine, or subsequently used in the transaction building module for writing to the distributed ledger. Through attribute metadata, each business data item possesses queryable source information, time information, semantic category, sensitivity level, and integrity verification criteria, providing fundamental support for subsequent data lineage record construction and training dataset auditing.

[0051] This embodiment receives business data items from multiple data systems, extracts the data source system identifier, data generation timestamp, data type, sensitivity level, data tag and annotation information based on a preset data parsing strategy, and combines the data digest value calculated by the hash function to form attribute metadata. This enables business data to have standardized source description and integrity marking before entering the subsequent processing chain, thereby providing searchable and verifiable basic information support for subsequent data lineage record generation, training data tracing and compliance auditing.

[0052] In one embodiment, step S20 above includes: S201, the attribute metadata and data digest value are encapsulated into a blockchain transaction structure; S202, Choose a private blockchain or a consortium blockchain as the distributed ledger network; S203, broadcast the data storage transaction to the consensus nodes of the distributed ledger network, and perform consensus verification among the consensus nodes through the consensus module; S204, add the data storage transaction that has passed consensus verification to the blockchain ledger; S205, generate a unique transaction index identifier for each of the data storage transactions; S206, Based on all verified data storage transactions and corresponding transaction index identifiers, a data lineage record is generated in the distributed ledger network.

[0053] In this embodiment, the attribute metadata and data digest value have already been formed in the previous stage. The current stage uses these two types of information as input to generate a data storage transaction that can enter the distributed ledger network. The attribute metadata includes the data source system identifier, data generation timestamp, data type, sensitivity level, tags, and annotation information, used to describe the key attributes of business data in terms of semantics, source, and security level. The data digest value is calculated using a hash function on the business data content, forming an irreversible short tag for integrity verification. To enter on-chain storage, the attribute metadata and data digest value need to be combined and organized into a blockchain transaction structure. The blockchain transaction structure can adopt key-value pairs, structured records, or serialized binary format, including a unique transaction identifier, timestamp field, attribute metadata field, digest field, and necessary signature fields. In implementation, the attribute metadata can first be mapped to a predefined set of fields, and then the data digest value can be appended to the end of the structure as a fixed-length field, converted into a byte array by a serialization component, preparing for subsequent signing and broadcasting.

[0054] A data storage transaction is a logical transaction unit within a blockchain system. It carries one or more attribute metadata and data digest values, representing a data storage event for a batch of business data. When generating a data storage transaction, the system can choose to assign one transaction to one piece of business data, or, in high-concurrency scenarios, aggregate multiple pieces of business data within the same time window into a single transaction to balance write frequency and on-chain storage overhead. The transaction must carry a transaction identifier, a source node identifier, and joint signature information to ensure accurate location and traceability in the ledger later.

[0055] In this scenario, the distributed ledger network employs either a private blockchain or a consortium blockchain architecture. Private blockchains are suitable for collaboration among multiple systems within a single organization, such as joint audits of underwriting, claims, and customer service systems within an insurance group. Consortium blockchains are suitable for sharing among multiple organizations, such as cross-organizational audit networks involving multiple insurance companies, reinsurance institutions, and regulatory authorities. During actual deployment, the list of participating organizations, node public keys, and access control policies can be pre-registered through a network configuration center. Then, at runtime, the type of blockchain to be written to for the current data storage transaction can be selected based on business needs; for example, a private blockchain can be used for internal compliance records, while a consortium blockchain can be used for records that need to be shared with external regulators.

[0056] Consensus nodes are computational nodes in a distributed ledger network that participate in consensus verification. Each node maintains a copy of the ledger and is responsible for checking the legitimacy of new transactions. The consensus module is a component that implements the consistency protocol within a node, supporting Byzantine fault-tolerant protocols, voting-based multi-round confirmation protocols, or leader election-based log replication protocols. After a data storage transaction is generated, the system broadcasts the transaction to all consensus nodes through the node's network interface. This broadcasting process can be implemented through message queues, multicast, or peer-to-peer relay. Upon receiving a transaction, each consensus node performs signature verification, data structure verification, field integrity verification, and permission checks locally to ensure the transaction's source is trustworthy, its structure meets expectations, and it does not violate the write policy. After multiple nodes complete local verification, the consensus module tallies the voting results according to a preset protocol. For example, if a certain number of nodes confirm the transaction's validity, the consensus verification is considered successful.

[0057] Data notarization transactions that pass consensus verification are packaged into new blocks and added to the blockchain ledger. The blockchain ledger is structured in blocks, each block containing a block header and a block body. The block body records multiple verified data notarization transactions, while the block header records metadata such as the hash of the previous block, timestamp, and consensus result. The transaction addition process is typically handled by the packaging node. After authorization from the consensus module, the packaging node selects a set of verified transactions from the transaction pool, constructs the block body, performs hash calculations and signing, and then broadcasts the new block to other nodes. Once verified, other nodes append the block to their local ledger. During this process, the content of the data notarization transactions is stored on the blockchain, and any subsequent changes to the original business data can be detected by comparing the data digest value.

[0058] To support subsequent data lineage analysis, a unique transaction index identifier needs to be generated for each data storage transaction. This transaction index identifier can be generated by combining fields such as block height, transaction sequence number within the block, and transaction hash, or it can use a separate index encoding rule. During generation, the block number and the transaction's offset position within the block can be encoded as a string or number to ensure uniqueness across the entire network. The transaction index identifier will be mapped to the internal identifier of the business data in the attribute metadata, enabling bidirectional data location from on-chain to off-chain. If it is necessary to audit the data source of a specific business record in the future, the system can quickly retrieve the corresponding data storage transaction in the ledger using the transaction index identifier.

[0059] Data lineage records are structures extracted and organized from a distributed ledger network based on all verified data storage transactions and their transaction index identifiers. These records represent the evolutionary relationships of data during collection, processing, and use. In this stage, by traversing transactions related to specific business domains in the ledger, attribute metadata and data summary values ​​are combined according to dimensions such as time, source system, and data type to construct a data flow link between different systems. In implementation, a time-ordered list of transaction indexes can be maintained for each business data internal identifier. Each item in the list points to a storage behavior on the chain, thus forming a traceable lineage chain. This lineage chain can be further associated with the datasets used in subsequent training batches, providing a basis for tracing the specific data source during the model training phase.

[0060] This embodiment encapsulates attribute metadata and data digest values ​​into a blockchain transaction structure. In a private or consortium blockchain environment, verification is performed by consensus nodes and consensus modules. Verified data storage transactions are written into the blockchain ledger, and a unique transaction index is generated for each transaction. Based on all verified transactions and their corresponding indexes, a data lineage record is generated in the distributed ledger network. This enables a trusted record and traceable representation of business data from access to on-chain storage, providing an immutable and verifiable data lineage foundation for subsequent data tracking and compliance auditing during the training process.

[0061] In one embodiment, step S30 above includes: S301, during the training process of the target model, the training batch start event is monitored through an event listener; S302, in response to the training batch start event, the identifier generator is invoked to generate the batch identifier of the current training batch; S303, parse the data identifier information of the training data read in the current training batch; S304, Based on the data identification information, query the corresponding data summary value from the data lineage record; S305, verify the integrity and validity of the data digest value, and combine the data digest values ​​that have passed the integrity and validity verification in a preset order to form a data digest value set; S306, Parse the model configuration parameters, including model type and hyperparameters, from the configuration file of the target model; S307, record the start time and end time of the current training batch, and select one of the start time and end time as the training timestamp.

[0062] In this embodiment, during the training of the target model, each training batch needs to be bound to its actual data and configuration conditions to form a traceable record. Therefore, the training process itself is not only a parameter update process but also a continuous generation process of audit information. The target model can be any machine learning or deep learning model used for risk assessment, pricing, fraud detection, health status prediction, etc., including multi-layer neural networks, ensemble tree models, sequence models, etc., and its parameters are typically updated iteratively within a unified training framework. The training process is divided into multiple training batches, each using a set of training data and a set of model configuration parameters, forming an independently describable training unit.

[0063] To automatically capture information about these training units during training, event listeners are deployed within the training framework. These event listeners connect to the training scheduling module or the data loading module to monitor training batch start events. A training batch start event can be defined as the moment the current model completes parameter updates for the previous batch and prepares to load the next batch of training data, or it can be defined as the moment the data loader completes reading the current batch of data and passes it to the model. In implementation, callback interfaces can be inserted at the training loop entry point or in the data iterator. The event listener registers callback functions, which are triggered when the training loop enters a new batch, thereby capturing the start time, current batch number, and associated model identifier for each training batch.

[0064] Upon detecting a training batch initiation event, the identifier generator is invoked to generate a batch identifier for the current training batch. The batch identifier needs to be unique throughout the entire training process to accurately locate a specific training batch during auditing and tracing. The identifier generator can construct batch identifiers based on multiple fields, such as the global identifier of the target model, the current training epoch number, the batch sequence number, and the current system time, using string concatenation or hash encoding to create a unique batch identifier. Alternatively, a distributed unique identifier algorithm can be introduced to ensure that batch identifiers generated by multiple training nodes in parallel training scenarios do not conflict. Once generated, the batch identifier is immediately written to the training log and memory structure and used consistently with subsequent data lineage records, training proofs, and other objects.

[0065] The training data read in the current training batch is typically read in batches from the underlying data storage or intermediate feature storage by the data loader. To establish a mapping between on-chain data lineage records and the data used in the current training batch, the data identification information of this training data needs to be parsed first. Data identification information can include fields such as the primary key of the original business record, the source system identifier, the data version number, the feature table name, and the row identifier. In implementation, one or more fields for identifying the source record can be retained in the training data. For example, during the feature engineering phase, the business primary key and data version can be passed as additional columns to the training samples. While reading the training data in the current training batch, these identifier columns are extracted to form a data identification information list. Alternatively, the data preparation module can generate a sample identifier list before data loading, mapping the internal labels of samples in the training batch to the identifier structure used in the data lineage records.

[0066] After obtaining the data identifier information, it is necessary to use the data lineage record to query the data digest value corresponding to the training data used in the current training batch. The data lineage record has already stored the attribute metadata and data digest value of each business data according to the transaction index identifier, and a query index can be built based on fields such as data source system identifier, business primary key, and generation timestamp. This step constructs query conditions based on the aforementioned data identifier information to retrieve the data lineage record in the distributed ledger network. In implementation, a query index structure synchronized with the on-chain transaction index can be maintained off-chain, or the associated data digest value can be retrieved directly based on the data identifier field through the on-chain smart contract interface. The search results are returned in the form of a list of data digest values, each corresponding to a specific training data piece that actually participated in the computation of the current training batch.

[0067] To ensure that the data used in the current training batch has not been tampered with or mismatched, the retrieved data digest values ​​need to be verified for integrity and validity. Integrity verification involves recalculating the digest values ​​of the training data in the current batch and comparing them one by one with the data digest values ​​retrieved from the data lineage record. If they match perfectly, the data is considered unchanged from its entry into the training process. Validity can be checked based on the data's usage status, such as verifying whether the data is within the permitted usage window, whether it has been marked for revocation, correction, or deletion, and whether it complies with sensitivity level control policies. In implementation, a status flag and validity period field can be attached to each data digest value. During the verification phase, these fields are read, and records that do not meet the usage conditions are directly removed. This process creates a list containing only complete and still valid data digest values.

[0068] To facilitate subsequent training proof generation, validated data digest values ​​are combined in a preset order to form a data digest value set. This preset order can be based on the sorting rules of the data identification information, such as sorting by data generation time, business primary key, or grouping by source system, or it can be arranged according to the order in which the training data appears in the current batch. In implementation, the data identification information and the digest value list can be sorted once before constructing the data digest value set, and then the digest values ​​are written into the set structure sequentially according to the sorting result. The data digest value set retains both the data range used in the current training batch and implicitly contains the data usage order, providing structured input for subsequent auditing and inference of training behavior.

[0069] The target model relies on model configuration parameters both before and during training. These parameters are typically stored in a separate configuration file. The configuration file can be in a structured format, such as JSON, YAML, or INI text, and contains information such as model type, network architecture description, loss function settings, optimization algorithm selection, learning rate, batch size, regularization coefficient, and maximum training epochs. The current step parses the model configuration parameters from the target model's configuration file. A configuration parsing component reads the file content and maps each field to a configuration object in memory. The model type distinguishes between image, text, and time-series models, while hyperparameters control the numerical behavior during training; for example, the learning rate affects the magnitude of parameter updates, the batch size affects gradient estimation noise, and the regularization coefficient constrains model complexity. After parsing, the model type and hyperparameters are saved in a stable field structure for later use in training validation.

[0070] Training timestamps are used to record the actual time information of the current training batch. The system obtains a time value as the start time when the training batch starts and another time value as the end time when the training batch completes. Both times can be obtained from the same time source, such as using a unified time synchronization service or a high-precision system clock. The current step, after recording the start and end times, selects one of these two points as the training timestamp. For example, the start time can be chosen to identify the trigger position of the training batch on the timeline, or the end time can be chosen to identify the completion time of the training batch. Alternatively, a combined structure including both start and end times can be used when more granular auditing is required. When used in conjunction with batch identifiers, data summary value sets, and model configuration parameters, training timestamps help auditors perform time-related analysis of training activities with external business events or strategy changes.

[0071] Through the above steps, this embodiment achieves traceability of data usage and restoration of configuration conditions at the granular level of the training process, providing a structurally complete, clearly sourced, and verifiable basic record for subsequent training proof generation and compliance auditing, thereby reducing the cost of obtaining evidence when the training process is questioned.

[0072] In one embodiment, step S40 above includes: S401, integrate the batch identifier, data digest value set and model configuration parameters to generate a combined data block; S402, the training timestamp is added to the combined data block to form a proof data block; S403, Use a hash function to determine the hash value of the proof data block; S404, Perform digital signature processing on the hash value to generate a digital signature; S405, the proof data block, hash value and digital signature are structured and assembled to generate training proof.

[0073] In this embodiment, the batch identifier, data summary value set, and model configuration parameters are first integrated to generate a combined data block. The batch identifier serves as a unique marker for the training batch, distinguishing different training periods. The data summary value set is compiled from the summary values ​​of the training data obtained in previous steps after querying and verifying their integrity, serving as proof of the training data's source. The model configuration parameters include the model type and key hyperparameters, representing the structured behavior of the training process. By orderly combining this set of key fields, the data source, model structure, and batch identity are bound to the same data entity, forming the smallest complete unit of the training logic.

[0074] Subsequently, training timestamps are added to the combined data blocks to form proof data blocks. The training timestamps are generated from the start or end time of the training batch and are used to record the precise time position of the training action. Embedding the timestamps into the combined data blocks ensures that the generated proof data blocks are temporally immutable.

[0075] Then, a hash function is used to process the proof data block, calculating a unique hash value. This hash value establishes an irreversible mapping from the proof data block to a fixed-length string, used to detect any subtle changes to the data block. The hash result is directly used in subsequent signing and verification processes; therefore, the choice of hash function typically satisfies collision resistance and high consistency requirements, such as using the SHA family.

[0076] Next, a digital signature is performed on the hash value. The digital signature is generated using the private key of the training environment or model owner, allowing external verifiers to verify authenticity using the corresponding public key. The signature binds the training proof to the identity of a specific entity, improving the attribution of training records. Because the signature only applies to the hash value, not the original data block, both verification efficiency and data structure scalability are maintained.

[0077] Finally, the proof data block, hash value, and digital signature are structurally assembled to form the training proof. This structured assembly ensures that different verification modules can parse the internal components of the training proof in field order, achieving universal parsing across systems. As the final verifiable credential, the training proof can be further written into a distributed ledger, forming a bidirectional index structure with data lineage records, supporting complete traceability of the model training path.

[0078] This embodiment generates a training proof by binding batch identifiers, data digest value sets, model configuration parameters, and training timestamps. A verifiable structure is constructed using hashing and digital signatures, ensuring the unalterable joint solidification of the input content, configuration content, and time information of the training batch. External systems can independently verify the authenticity of training behavior, the consistency of training data, and the integrity of model configuration by verifying the hash values ​​and signatures in the training proof. This achieves verifiability of the training process, reduces the audit risks associated with the black box nature of model training, and improves the traceability and credibility of the training process.

[0079] In one embodiment, step S50 above includes: S501, the training proof is encapsulated as a blockchain write transaction; S502, The blockchain transaction is submitted to the nodes of the distributed ledger network for consensus verification; S503, After the consensus verification is passed, the training proof is recorded in the blockchain ledger of the distributed ledger network; S504, Generate a storage location index for the training proof recorded in the blockchain ledger; S505, extract the data transaction index corresponding to the training data read in the current training batch from the data lineage record; S506, Obtain the model identification information of the target model; S507, Based on the storage location index of the training proof, the data transaction index, and the model identification information, generate an associated index; S508, the associated index is stored in the distributed ledger network.

[0080] In this embodiment, the process of writing the training proof to the distributed ledger network first involves encapsulating the training proof into a blockchain write transaction. The training proof itself is a structured data unit containing fields such as batch identifier, data digest value set, model configuration parameters, and training timestamp. Before being directly written to the blockchain, it needs to be converted into a data payload conforming to the target chain's protocol format. The encapsulation process typically includes operations such as serializing fields, filling in on-chain transaction header information, setting transaction type identifiers, and specifying the target ledger channel or chaincode name, enabling the transaction to be recognized and processed by nodes in the distributed ledger network. Through this encapsulation, the training proof is transformed from an application-layer data object into a blockchain transaction structure, achieving compatibility with the underlying consensus and storage mechanisms.

[0081] After a transaction is written to the blockchain, it is submitted to multiple nodes in the distributed ledger network for consensus verification. The distributed ledger network can be deployed on a private or consortium blockchain and consists of multiple nodes with verification and accounting capabilities. Consensus verification includes steps such as verifying the legality of the transaction format, signature and permission verification, and on-chain business rule verification, ensuring that the training proof is credible, complete, and does not violate predefined on-chain writing rules. Under different consensus mechanisms, nodes vote or sort transactions differently, but the goal is always to form a unified transaction confirmation result within the network.

[0082] After the consensus phase is complete, the training proof is written to the blockchain ledger. The blockchain ledger can be understood as an irreversible, ordered linked list structure composed of blocks, each recording a set of transaction data that has passed consensus. When the training proof is written, it is placed in the transaction list of the new block or the block currently to be written, forming an immutable record together with the hash value and time information of the previous block in the block header. Once the block is persisted, the existence of the training proof in the ledger can be verified through the hash chain, and the record cannot be modified or deleted externally without being detected.

[0083] After the training proof is written to the blockchain ledger, a storage location index needs to be generated. The storage location index can include a set of positioning fields such as block height, block hash, transaction hash, and the offset position of the transaction within the block. These fields are used to quickly locate the physical position of a specific training proof in the ledger during subsequent queries. During the generation process, different index combinations can be selected based on the characteristics of the underlying ledger implementation, such as using transaction hash to accelerate the direct location of a single proof, or using block height to enable batch scanning.

[0084] Simultaneously, it is necessary to extract the data transaction index corresponding to the training data read in the current training batch from the data lineage record. The data lineage record is a chain-like relationship structure built upon data being uploaded to the blockchain using information such as transaction indexes, field identifiers, source system identifiers, and timestamps. It indicates the flow path of business data between different processing stages. The data transaction index can be a unique identifier generated for each data storage transaction in the early stages, used to locate the original data write record. By traversing or retrieving the data lineage record using the training data identifier set of the current training batch, a data transaction index corresponding one-to-one with the data set used in this training is obtained, allowing the training proof to accurately point to the data storage trajectory involved in the training.

[0085] Next, the model identification information of the target model is obtained. This model identification information uniquely identifies the audited object and may include fields such as model name, model version number, deployment environment flag, and parameter file hash value. Model identification information is typically generated and stored in the model management system during the model registration or deployment phase. It is read from the management system or configuration storage when building relationships, ensuring that any subsequent verification request accurately maps the on-chain record to the specific running model instance.

[0086] After obtaining the storage location index, data transaction index, and model identifier information for the training proof, a related index is generated based on these three elements. The related index can be designed as a structured key-value pair, for example, using the model identifier information as the primary key and combining the training proof storage location index and a set of data transaction indices into a value. Alternatively, a reverse mapping structure can be used, with the data transaction index as the primary key, linking multiple model identifiers to their corresponding training proof locations. The generation process requires standardized encoding and formatting of the index fields so that different nodes in the distributed ledger network can parse and use them according to the same rules. Finally, the related index is written to the distributed ledger network. It can be recorded as a separate on-chain data structure in a dedicated index table or index contract, or stored as a key-value pair in a query-oriented on-chain state database. This allows subsequent verification modules to perform joint retrieval and cross-validation between data lineage records, training proofs, and model identifiers through the related index, all from a unified entry point.

[0087] This embodiment achieves on-chain traceable binding between training actions, data lineage records, and the target model by writing training proofs into a distributed ledger network and generating an association index containing an index of the training proof storage location, a data transaction index, and model identification information. Any subsequent verification request can directly locate the training proof and its corresponding data lineage trajectory in the distributed ledger network through the association index, eliminating reliance on offline logs or manual comparisons in distributed systems. This ensures unified auditing of data usage paths and the model training process while guaranteeing the immutability of records.

[0088] In one embodiment, step S60 above includes: S601, Receive a verification request containing the target identifier to be verified; S602, based on the association index, retrieve the data lineage record and training proof corresponding to the target identifier from the distributed ledger network; S603, extract the data digest value from the data lineage record, and determine the current data digest value of the current target data; S604, compare the current data digest value with the data digest value in the data lineage record, and generate a data integrity verification conclusion based on the comparison result; S605, extract model configuration parameters and training timestamp from the training proof, check the model configuration parameters according to the preset compliance policy, and verify whether the training timestamp meets the time series requirements. Based on the check results and verification results, generate a model training compliance verification conclusion. S606. Based on the data integrity verification conclusion and the model training compliance verification conclusion, generate a verification report.

[0089] In this embodiment, the system first receives a verification request. This request can be initiated through a regulatory interface, a compliance audit platform, or an internal monitoring system, and the request content includes a target identifier. The target identifier identifies the object to be verified and can be a model identifier, training batch identifier, business task identifier, or a combination of multiple fields. In implementation, it is generally implemented as a string or structured identifier, containing information such as model name, version number, business line code, and time window. The receiving process involves more than just receiving packets; it also requires parsing and format validation of the request. This includes verifying the target identifier's format for validity, the presence of required fields, and whether it matches the verification types supported by the current system. Target identifiers that pass validation are then converted into an internally unified identifier structure for consistent use during subsequent retrieval.

[0090] After obtaining the target identifier, the system retrieves the corresponding data lineage record and training proof from the distributed ledger network based on the association index. The association index is constructed on-chain in the preceding steps and internally uses key-value mapping to map the target identifier or model identifier into a structure containing a training proof storage location index and a data transaction index set. During retrieval, the system first accesses the on-chain state-oriented index storage, such as a state database or smart contract state table, and queries the associated entries using the target identifier to obtain the location parameters of the training proof and the data transaction index set related to this target. Subsequently, it uses the location parameters of the training proof to access the distributed ledger network, accessing the on-chain ledger by block height, transaction hash, or contract key-value to read the training proof content; simultaneously, it uses the data transaction index set to perform multiple location operations in the data lineage record to obtain the set of lineage entries related to the training process. The distributed ledger network here provides a consistent storage space across nodes. During the retrieval process, the ledger query function can be called through the light node or full node interface to ensure that the read training proof and data lineage record have not been tampered with.

[0091] After successfully retrieving the data lineage records, the system needs to extract data digest values ​​from these records and determine the current data digest value for the current target data. The data lineage records store digest values ​​generated when the data is uploaded to the blockchain. These digest values ​​are typically calculated using a pre-selected hash function, such as hashing the serialization result of business data items. The extraction operation requires traversing the data lineage entries associated with the target identifier, reading the data digest fields of the records, and organizing them into a comparable list or set of data digests. To determine whether the data in the current production environment is consistent with that during training, the current data digest value for the current target data also needs to be calculated. The current target data can be read from the business database, data lake, object storage, or cache system. During reading, the corresponding record is located based on the data identifier and source system identifier in the data lineage records. Then, using a parsing and hash calculation process completely consistent with the original evidence storage stage, the current data content is converted into the current data digest value, ensuring that the calculation rules for the two digest values ​​are consistent with the input format.

[0092] After obtaining the on-chain data digest value and the current data digest value, the system performs a comparison operation and generates a data integrity verification conclusion based on the comparison result. The comparison process can be carried out in a one-to-one correspondence manner, comparing the digest value associated with each data transaction index with the current data digest value. The comparison rule can be strict equality, or it can be extended to multiple checks based on the characteristics of the hash algorithm, such as performing multiple digest comparisons for different field combinations. If the digest values ​​of all corresponding entries are completely consistent, the system can mark the data integrity verification conclusion as passed; if any inconsistent entries exist, they are marked as failed in the conclusion, and the specific inconsistent data identifier and difference type are recorded. The data integrity verification conclusion can be represented in a structured result, organizing elements such as pass / fail flags, number of differences, difference details, and timestamps into a unified conclusion object, which facilitates the subsequent generation of a comprehensive report together with the training compliance conclusion.

[0093] The retrieval results from the training proof section are used to support the verification of model training compliance. The system extracts model configuration parameters and training timestamps from the training proof. Model configuration parameters cover the model structure type, input dimensions, loss function configuration, optimizer type, and hyperparameter settings used at the time of training, while the training timestamp reflects the start and end times or key time points of the training process. The extraction process requires parsing the structured fields in the training proof, mapping the configuration parameters and time information to internal objects or records respectively. Subsequently, the system checks the model configuration parameters according to a preset compliance policy. The preset compliance policy can be defined by the internal risk control department of financial and insurance institutions or the data governance department of healthcare institutions, and includes the allowed range of loss functions, parameter update strategy restrictions, regularization or constraint modules to be enabled when using sensitive features, and sample weight setting boundaries. During the check, the system compares each model configuration parameter with the rules in the compliance policy, generates violation entries for configuration items that do not comply with the rules, and records the violation type and related parameters.

[0094] The verification of training timestamps revolves around time series requirements. These requirements may include that training time must fall within a specified regulatory window, training activities must not precede data collection time, must not be later than a specific model deployment time, or the time interval between different training stages must meet a minimum safety interval. The system compares the training timestamp with data generation timestamps, model registration times, and business event times in the data lineage record to determine whether unapproved data was used during training, whether the authorized time window was exceeded, or whether the training sequence was inconsistent with business events. Training time verification results are generated based on preset time series rules and then integrated with parameter check results to form a model training compliance verification conclusion. This conclusion may include the overall compliance status, a list of failed rules, the parameters or time fields involved, and the referenced policy identifiers.

[0095] Finally, the system generates a verification report based on the data integrity verification conclusions and the model training compliance verification conclusions. The report generation process requires a structured summary of the conclusions from both dimensions, resulting in an information set that includes the target identifier, data integrity status, model training compliance status, detailed discrepancies, violation rule number, corresponding data lineage record references, corresponding training proof references, and the generation time. The report can be output as a JSON structure for inter-system communication or rendered as a highly readable document format for manual review or provision to regulatory authorities.

[0096] This embodiment, through a verification mechanism that combines on-chain recording and off-chain computation, can detect deviations between current data and on-chain records, and identify whether training configurations and scheduling violate established compliance rules. This significantly reduces compliance risks caused by data tampering, unauthorized training, and opaque model behavior, providing a unified, reliable, and automated verification basis for regulatory verification, self-verification, and cross-institutional collaboration in financial insurance and healthcare scenarios.

[0097] In one embodiment, a data lineage recording and training verification device is provided, which corresponds one-to-one with the data lineage recording and training verification method described in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the data lineage recording and training proof device of the present invention. The modules include a data parsing module 10, a proof recording module 20, a training acquisition module 30, a proof generation module 40, an on-chain indexing module 50, and a verification module 60. Detailed descriptions of each functional module are as follows: The data parsing module 10 is used to acquire business data, extract attribute metadata of the business data according to a preset data parsing strategy, and determine the data digest value of the business data. The evidence storage module 20 is used to construct a data evidence storage transaction containing the attribute metadata and data digest value, broadcast the data evidence storage transaction to the consensus nodes of the distributed ledger network for consensus verification, and generate a data lineage record in the distributed ledger network. The training acquisition module 30 is used to obtain the batch identifier of the current training batch during the training process of the target model, obtain the set of data summary values ​​corresponding to the training data read in the current training batch from the data lineage record, and obtain the model configuration parameters corresponding to the target model and the training timestamp of the current training batch. The proof generation module 40 is used to generate training proof based on the batch identifier, the data digest value set, the model configuration parameters, and the training timestamp. The on-chain index module 50 is used to write the training proof into the distributed ledger network and establish an association index between the data lineage record and the target model in the distributed ledger network; The verification module 60 is used to respond to a verification request by retrieving data lineage records and training proofs from the distributed ledger network based on the association index to verify data integrity and model training compliance.

[0098] In one embodiment, the data parsing module 10 is specifically used for: Receive business data items from multiple data systems; Each business data item is parsed according to a preset data parsing strategy, and the data source system identifier is extracted. Record the data generation timestamp, data type, and sensitivity level for each business data item; Assign data labels and annotation information to each business data item; Use a hash function to determine the data digest value for each business data item; The data source system identifier, data generation timestamp, data type, sensitivity level, data tag, annotation information, and data summary value are combined to form attribute metadata.

[0099] In one embodiment, the evidence storage recording module 20 is specifically used for: The attribute metadata and data digest values ​​are encapsulated into a blockchain transaction structure; Choose a private blockchain or a consortium blockchain as the distributed ledger network; The data storage transaction is broadcast to the consensus nodes of the distributed ledger network, and consensus verification is performed between the consensus nodes through the consensus module. Add the data storage transactions that have passed consensus verification to the blockchain ledger; A unique transaction index identifier is generated for each of the data storage transactions; Based on all verified data storage transactions and their corresponding transaction index identifiers, a data lineage record is generated in the distributed ledger network.

[0100] In one embodiment, the training acquisition module 30 is specifically used for: During the training of the target model, the training batch start event is monitored through an event listener; In response to the training batch start event, the identifier generator is invoked to generate the batch identifier for the current training batch; Parse the data identifier information of the training data read in the current training batch; Based on the data identification information, the corresponding data summary value is retrieved from the data lineage record; Verify the integrity and validity of the data digest values, and combine the data digest values ​​that have passed the integrity and validity verification in a preset order to form a data digest value set; Parse the model configuration parameters, including model type and hyperparameters, from the configuration file of the target model; Record the start and end times of the current training batch, and select one of the start and end times as the training timestamp.

[0101] In one embodiment, the proof generation module 40 is specifically used for: The batch identifier, data digest value set, and model configuration parameters are integrated to generate a combined data block; The training timestamp is added to the combined data block to form a proof data block; The hash value of the proof data block is determined using a hash function; The hash value is digitally signed to generate a digital signature; The proof data block, hash value, and digital signature are structured and assembled to generate the training proof.

[0102] In one embodiment, the on-chain index module 50 is specifically used for: The training proof is encapsulated into a blockchain and written into a transaction; The blockchain transaction is submitted to the nodes of the distributed ledger network for consensus verification. After consensus verification is passed, the training proof is recorded in the blockchain ledger of the distributed ledger network; Generate a storage location index for the training proofs recorded in the blockchain ledger; Extract the data transaction index corresponding to the training data read in the current training batch from the data lineage record; Obtain the model identification information of the target model; Based on the storage location index of the training proof, the data transaction index, and the model identification information, an associated index is generated; The associated index is stored in the distributed ledger network.

[0103] In one embodiment, the verification module 60 is specifically used for: Receive a verification request containing the identifier of the target to be verified; Based on the association index, retrieve the data lineage record and training proof corresponding to the target identifier from the distributed ledger network; Extract the data digest value from the data lineage record and determine the current data digest value of the current target data; The current data digest value is compared with the data digest value in the data lineage record, and a data integrity verification conclusion is generated based on the comparison result; The model configuration parameters and training timestamp are extracted from the training proof. The model configuration parameters are checked according to the preset compliance policy, and the training timestamp is verified to meet the time series requirements. Based on the check results and verification results, a model training compliance verification conclusion is generated. Based on the data integrity verification conclusion and the model training compliance verification conclusion, a verification report is generated.

[0104] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a data lineage recording and training verification method on the server side.

[0105] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the functions or steps of a data lineage recording and training verification method on the client side.

[0106] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Acquire business data, extract attribute metadata of the business data according to a preset data parsing strategy, and determine the data digest value of the business data; A data notarization transaction containing the attribute metadata and data digest value is constructed, the data notarization transaction is broadcast to the consensus nodes of the distributed ledger network for consensus verification, and a data lineage record is generated in the distributed ledger network; During the training of the target model, the batch identifier of the current training batch is obtained, the set of data summary values ​​corresponding to the training data read in the current training batch is obtained from the data lineage record, and the model configuration parameters corresponding to the target model and the training timestamp of the current training batch are obtained. Generate training proof based on the batch identifier, data digest value set, model configuration parameters, and training timestamp; The training proof is written into the distributed ledger network, and an association index between the data lineage record and the target model is established in the distributed ledger network; In response to a verification request, data lineage records and training proofs are retrieved from the distributed ledger network based on the associated index to verify data integrity and model training compliance.

[0107] In one embodiment, a non-volatile computer-readable storage medium is provided, which may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, it performs the following steps: Acquire business data, extract attribute metadata of the business data according to a preset data parsing strategy, and determine the data digest value of the business data; A data notarization transaction containing the attribute metadata and data digest value is constructed, the data notarization transaction is broadcast to the consensus nodes of the distributed ledger network for consensus verification, and a data lineage record is generated in the distributed ledger network; During the training of the target model, the batch identifier of the current training batch is obtained, the set of data summary values ​​corresponding to the training data read in the current training batch is obtained from the data lineage record, and the model configuration parameters corresponding to the target model and the training timestamp of the current training batch are obtained. Generate training proof based on the batch identifier, data digest value set, model configuration parameters, and training timestamp; The training proof is written into the distributed ledger network, and an association index between the data lineage record and the target model is established in the distributed ledger network; In response to a verification request, data lineage records and training proofs are retrieved from the distributed ledger network based on the associated index to verify data integrity and model training compliance.

[0108] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0109] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0110] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

[0111] The user personal information involved in this application embodiment is all authorized (knowing and consenting) by the relevant parties or fully authorized by all parties, and the executing entity can obtain it through various open, legal and compliant means. The collection, storage, use, processing, transmission, provision and disclosure of the information, data and signals involved all comply with the relevant laws and regulations of the relevant countries and regions, and do not violate public order and good morals.

Claims

1. A method for data lineage recording and training proof, characterized in that, Includes the following steps: Acquire business data, extract attribute metadata of the business data according to a preset data parsing strategy, and determine the data digest value of the business data; A data notarization transaction containing the attribute metadata and data digest value is constructed, the data notarization transaction is broadcast to the consensus nodes of the distributed ledger network for consensus verification, and a data lineage record is generated in the distributed ledger network; During the training of the target model, the batch identifier of the current training batch is obtained, the set of data summary values ​​corresponding to the training data read in the current training batch is obtained from the data lineage record, and the model configuration parameters corresponding to the target model and the training timestamp of the current training batch are obtained. Generate training proof based on the batch identifier, data digest value set, model configuration parameters, and training timestamp; The training proof is written into the distributed ledger network, and an association index between the data lineage record and the target model is established in the distributed ledger network; In response to a verification request, data lineage records and training proofs are retrieved from the distributed ledger network based on the associated index to verify data integrity and model training compliance.

2. The data lineage recording and training proof method as described in claim 1, characterized in that, Acquire business data, extract attribute metadata of the business data according to a preset data parsing strategy, and determine the data digest value of the business data, including: Receive business data items from multiple data systems; Each business data item is parsed according to a preset data parsing strategy, and the data source system identifier is extracted. Record the data generation timestamp, data type, and sensitivity level for each business data item; Assign data labels and annotation information to each business data item; Use a hash function to determine the data digest value for each business data item; The data source system identifier, data generation timestamp, data type, sensitivity level, data tag, annotation information, and data summary value are combined to form attribute metadata.

3. The data lineage recording and training proof method as described in claim 1, characterized in that, Construct a data notarization transaction containing the attribute metadata and data digest value, broadcast the data notarization transaction to the consensus nodes of the distributed ledger network for consensus verification, and generate a data lineage record in the distributed ledger network, including: The attribute metadata and data digest values ​​are encapsulated into a blockchain transaction structure; Choose a private blockchain or a consortium blockchain as the distributed ledger network; The data storage transaction is broadcast to the consensus nodes of the distributed ledger network, and consensus verification is performed between the consensus nodes through the consensus module. Add the data storage transactions that have passed consensus verification to the blockchain ledger; A unique transaction index identifier is generated for each of the data storage transactions; Based on all verified data storage transactions and their corresponding transaction index identifiers, a data lineage record is generated in the distributed ledger network.

4. The data lineage recording and training proof method as described in claim 1, characterized in that, During the training of the target model, the batch identifier of the current training batch is obtained, the set of data summary values ​​corresponding to the training data read in the current training batch is obtained from the data lineage record, and the model configuration parameters corresponding to the target model and the training timestamp of the current training batch are obtained, including: During the training of the target model, the training batch start event is monitored through an event listener; In response to the training batch start event, the identifier generator is invoked to generate the batch identifier for the current training batch; Parse the data identifier information of the training data read in the current training batch; Based on the data identification information, the corresponding data summary value is retrieved from the data lineage record; Verify the integrity and validity of the data digest values, and combine the data digest values ​​that have passed the integrity and validity verification in a preset order to form a data digest value set; Parse the model configuration parameters, including model type and hyperparameters, from the configuration file of the target model; Record the start and end times of the current training batch, and select one of the start and end times as the training timestamp.

5. The data lineage recording and training proof method as described in claim 1, characterized in that, Training proofs are generated based on the batch identifier, data digest value set, model configuration parameters, and training timestamps, including: The batch identifier, data digest value set, and model configuration parameters are integrated to generate a combined data block; The training timestamp is added to the combined data block to form a proof data block; The hash value of the proof data block is determined using a hash function; The hash value is digitally signed to generate a digital signature; The proof data block, hash value, and digital signature are structured and assembled to generate the training proof.

6. The data lineage recording and training proof method as described in claim 1, characterized in that, The training proof is written into the distributed ledger network, and an association index between the data lineage record and the target model is established in the distributed ledger network, including: The training proof is encapsulated into a blockchain and written into a transaction; The blockchain transaction is submitted to the nodes of the distributed ledger network for consensus verification. After consensus verification is passed, the training proof is recorded in the blockchain ledger of the distributed ledger network; Generate a storage location index for the training proofs recorded in the blockchain ledger; Extract the data transaction index corresponding to the training data read in the current training batch from the data lineage record; Obtain the model identification information of the target model; Based on the storage location index of the training proof, the data transaction index, and the model identification information, an associated index is generated; The associated index is stored in the distributed ledger network.

7. The data lineage recording and training proof method as described in claim 1, characterized in that, In response to a verification request, data lineage records and training proofs are retrieved from the distributed ledger network based on the association index to verify data integrity and model training compliance, including: Receive a verification request containing the identifier of the target to be verified; Based on the association index, retrieve the data lineage record and training proof corresponding to the target identifier from the distributed ledger network; Extract the data digest value from the data lineage record and determine the current data digest value of the current target data; The current data digest value is compared with the data digest value in the data lineage record, and a data integrity verification conclusion is generated based on the comparison result; The model configuration parameters and training timestamp are extracted from the training proof. The model configuration parameters are checked according to the preset compliance policy, and the training timestamp is verified to meet the time series requirements. Based on the check results and verification results, a model training compliance verification conclusion is generated. Based on the data integrity verification conclusion and the model training compliance verification conclusion, a verification report is generated.

8. A data lineage recording and training verification device, characterized in that, The data lineage recording and training verification device includes: The data parsing module is used to acquire business data, extract attribute metadata of the business data according to a preset data parsing strategy, and determine the data digest value of the business data. The evidence storage module is used to construct a data evidence storage transaction containing the attribute metadata and data digest value, broadcast the data evidence storage transaction to the consensus nodes of the distributed ledger network for consensus verification, and generate a data lineage record in the distributed ledger network. The training acquisition module is used to obtain the batch identifier of the current training batch during the training process of the target model, obtain the set of data summary values ​​corresponding to the training data read in the current training batch from the data lineage record, and obtain the model configuration parameters corresponding to the target model and the training timestamp of the current training batch. The proof generation module is used to generate training proofs based on the batch identifier, the set of data digest values, the model configuration parameters, and the training timestamp. An on-chain indexing module is used to write the training proof into the distributed ledger network and establish an association index between the data lineage record and the target model in the distributed ledger network; The verification module is used to respond to verification requests by retrieving data lineage records and training proofs from the distributed ledger network based on the associated index to verify data integrity and model training compliance.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a data lineage recording and training verification program stored in the memory and executable on the processor, wherein the data lineage recording and training verification program, when executed by the processor, implements the steps of the data lineage recording and training verification method as described in any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that, The storage medium stores a data lineage record and training proof program, which, when executed by a processor, implements the steps of the data lineage record and training proof method as described in any one of claims 1-7.

Images