Data-ancestry-based vehicle-mounted model training traceability method and device

Abstract

The present application relates to model training traceability technical field, especially vehicle-mounted model training traceability method and device based on data blood relationship, including the following steps: S1: in response to the training starting instruction of vehicle-mounted model, obtain training configuration information, the training configuration information includes training data set identification, model version number, training hyperparameter and training environment parameter.The present application has the advantages that: through the acquisition sensor identification, sensor model, acquisition geographical position and vehicle driving state parameter of each training data, cooperate with the whole process record of data cleaning, marking, enhancement and other preprocessing operation chain and its timestamp and execution parameter, and the integrity, accuracy, timeliness and consistency of the complete evaluation generated by the vehicle-mounted data quality management module, the whole link information unified acquisition from the data acquisition end to the preprocessing completion end is realized, the problems of dispersed record of each link data flow information, lack of unified acquisition mechanism and blood relationship information being easy to be tampered are solved.

Description

Technical Field

[0001] This invention relates to the field of model training tracing technology, and in particular to a method and apparatus for vehicle-mounted model training tracing based on data lineage. Background Technology

[0002] With the deepening application of artificial intelligence technology in the field of intelligent connected vehicles, the training of onboard models increasingly relies on massive amounts of multi-source heterogeneous data. The sources of training data include real road data collected by onboard sensors, synthetic data generated by simulation platforms, and labeled data transmitted from the cloud. In this context, the performance and reliability of onboard models are highly dependent on the quality and reliability of the training data.

[0003] However, existing vehicle-mounted model training processes have the following shortcomings in terms of data traceability: 1. Fragmentation and Tampering Risks in Training Data Lineage Tracking. Current vehicle model training involves multiple stages, including data acquisition, preprocessing, annotation, augmentation, and iterative training. Data flow information for each stage is recorded in different systems, lacking a unified mechanism for data lineage acquisition and correlation. Existing lineage tracking systems typically stop recording before model training, focusing only on how data is transformed without tracking how these transformations affect the model training process. Furthermore, traditional data recording methods lack tamper-proofing measures, making lineage information easily modified or lost, which fails to meet the requirements of vehicle model security auditing and compliance verification.

[0004] 2. Lack of a traceable link between model training results and training data. When in-vehicle models experience performance degradation, abnormal decision-making, or safety issues after deployment, developers struggle to quickly pinpoint the specific batch or source of training data causing the problem. Existing methods primarily rely on static model architecture similarity to infer the lineage between models, failing to capture the dynamic characteristics of knowledge evolution during training. Due to the lack of a complete lineage propagation path from data source to model output, attribution analysis of model behavior often remains at the level of empirical guesswork, making accurate root cause localization difficult.

[0005] 3. The impact of data quality on model performance is difficult to quantify. Vehicle training data comes from various sensors and different acquisition environments, resulting in inconsistent data quality. However, current technologies lack systematic collection and management of quality labels for each piece of training data, and even more so, lack methods for quantitatively correlating data quality with model training effectiveness. When model performance is substandard, it is impossible to quantify the contribution of each piece of training data to the model output, leading to a lack of clear direction for data optimization and model iteration.

[0006] 4. The model training tracing lacks dynamic traceability throughout its entire lifecycle. Existing tracing methods mainly cover the model training phase. After the model enters the online inference phase, there is a lack of continuous monitoring of model performance fluctuations and automatic tracing triggering mechanisms. When the performance of the vehicle-mounted model degrades in the actual operating environment, it cannot automatically link back to the data and process records from the training phase, resulting in a significant time lag between problem discovery and root cause localization. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and device for training and tracing vehicle-mounted models based on data lineage, which effectively solves the deficiencies of the prior art.

[0008] The objective of this invention is achieved through the following technical solution: The data lineage-based method for tracing the training origins of in-vehicle models includes the following steps: S1: In response to the training start command of the vehicle model, obtain training configuration information, which includes training dataset identifier, model version number, training hyperparameters and training environment parameters; S2: Based on the training dataset identifier, read the corresponding training data from the vehicle data lake, and collect the data source information, data collection timestamp, data preprocessing operation chain and data quality label for each piece of training data to generate the data lineage metadata of the training data; S3: Input the training data into the vehicle model to perform iterative training. During the training process, record the data batch identifier, model parameter update record and loss function value used in each iteration in real time. Associate and map the data batch identifier with the data lineage metadata in step S2 to generate a lineage association record for the training process. S4: After the vehicle-mounted model completes training, based on the data lineage metadata collected in step S2 and the training process lineage association records generated in step S3, a lineage tracing graph of the vehicle-mounted model is constructed. The lineage tracing graph includes data nodes, model nodes, training task nodes, and lineage relationship edges between each node. S5: Bind the model version number, training completion timestamp, and model performance evaluation index of the vehicle-mounted model to the lineage tracing map to generate a model-lineage association index, and store the model-lineage association index in the vehicle-mounted blockchain evidence storage module; S6: In response to the source tracing query request for the target vehicle model, parse the query conditions carried in the source tracing query request, wherein the query conditions include at least one of the following: model version number, training data identifier, training time period, and model performance threshold; S7: Based on the query conditions, perform a matching search in the model-bloodline association index to determine the target bloodline tracing map corresponding to the query conditions; S8: Execute a path search algorithm on the target kinship tracing graph to extract a complete kinship propagation path from the data source node to the model node from the target kinship tracing graph. The complete kinship propagation path includes intermediate data processing nodes, data transformation operation nodes, and training iteration nodes. S9: Based on the complete bloodline propagation path, generate a source tracing report, which includes a list of training data sources, a description of the data preprocessing link, key training iteration records, and a quantified value of the contribution of each data node to the model output. S10: Output the source tracing report to the human-machine interface of the vehicle terminal in the form of a visual map, and display the data nodes in the visual map in layers and colors according to the blood distance.

[0009] Preferably, in any of the above schemes, the data source information in step S2 includes the sensor identifier, sensor model, geographical location of data acquisition sensor, and vehicle driving status parameters at the time of acquisition. The data preprocessing operation chain includes data cleaning operation, data labeling operation, data augmentation operation, and timestamps and execution parameters for each operation. The data quality labels include data integrity score, data accuracy score, data timeliness score, and data consistency score. The data quality labels are automatically generated by the vehicle data quality management module based on preset quality rules when evaluating the training data.

[0010] Preferably, in any of the above schemes, the method for constructing a kinship tracing map in step S4 includes: S4.1: Using each piece of training data as a data node, record the attribute information of the data node, including data identifier, data source, data size, and data format; S4.2: Using the vehicle-mounted model as a model node, record the attribute information of the model node, including model identifier, model architecture, and model version number; S4.3: Using each training task as a training task node, record the attribute information of the training task node, including task identifier, training start time, training end time, and training hyperparameters; S4.4: Based on the lineage metadata and the lineage association records of the training process, establish input lineage relationship edges between data nodes and training task nodes, output lineage relationship edges between training task nodes and model nodes, and data transformation lineage relationship edges between data nodes.

[0011] Preferably, in any of the above schemes, the vehicle-mounted blockchain evidence storage module in S5 adopts a lightweight blockchain architecture, the model-lineage association index is stored in the vehicle-mounted blockchain evidence storage module in a Merkle tree structure, and the hash value of the lineage tracing graph corresponding to each model version is used as the leaf node of the Merkle tree.

[0012] Preferably, in any of the above schemes, the traceability query request in step S6 further includes a data node traceability mode or a model node traceability mode; The data node tracing mode is used to query which vehicle models were trained using specified training data. The model node traceability mode is used to query which training data was used to train a specified vehicle model. The path search algorithm in S8 adopts a bidirectional breadth-first search algorithm, which performs the search simultaneously from both the data source node and the model node. The search stops when the search paths of the two ends intersect, and the intersecting path is output as the complete lineage propagation path.

[0013] Preferably, in step S9, the contribution metric of each data node to the model output is calculated as follows: the gradient update amount of the training data corresponding to the data node in each iteration during model training is obtained, the contribution of the data node to the model parameters is determined based on the cumulative value of the gradient update amount, and the contribution is normalized and used as the contribution metric.

[0014] Preferably, any of the above schemes further includes step S11: after the vehicle-mounted model enters the online inference stage, continuously monitor the inference performance index of the vehicle-mounted model. When the inference performance index is lower than a preset threshold, automatically trigger a source tracing query request to locate the data node or training iteration node that caused the performance degradation and generate a performance attribution analysis report.

[0015] A data lineage-based vehicle-mounted model training tracing device, used to implement the above method, includes: The data lineage acquisition module is used to respond to the training start command of the vehicle model, obtain training configuration information, read the corresponding training data from the vehicle data lake according to the training dataset identifier, and collect the data source information, data acquisition timestamp, data preprocessing operation chain and data quality label for each piece of training data to generate the data lineage metadata of the training data. The training process tracking module is used to input the training data into the vehicle model to perform iterative training. During the training process, it records the data batch identifier, model parameter update record and loss function value used in each iteration in real time, and associates and maps the data batch identifier with the data lineage metadata to generate a training process lineage association record. The lineage graph construction module is used to construct the lineage tracing graph of the vehicle model after the vehicle model has completed training, based on the data lineage metadata and the lineage association records of the training process. The lineage tracing graph includes data nodes, model nodes, training task nodes, and lineage relationship edges between each node. The index storage module is used to bind the model version number, training completion timestamp, and model performance evaluation indicators of the vehicle-mounted model with the lineage tracing map, generate a model-lineage association index, and store the model-lineage association index in the vehicle-mounted blockchain storage module. The tracing query module is used to respond to the tracing query request for the target vehicle model, parse the query conditions carried in the tracing query request, perform matching and retrieval in the model-bloodline association index according to the query conditions, and determine the target bloodline tracing map corresponding to the query conditions; The path extraction module is used to perform a path search algorithm on the target kinship tracing graph to extract the complete kinship propagation path from the data source node to the model node from the target kinship tracing graph; The report generation module is used to generate a source tracing report based on the complete bloodline transmission path. The source tracing report includes a list of training data sources, a description of the data preprocessing link, key training iteration records, and a quantified value of the contribution of each data node to the model output. The visualization output module is used to output the source tracing report to the human-machine interface of the vehicle terminal in the form of a visualization map, and to display the data nodes in the visualization map in a layered and colored manner according to the blood distance.

[0016] Preferably, in any of the above schemes, the vehicle-mounted blockchain evidence storage module in the index evidence storage module adopts a lightweight blockchain architecture, the model-bloodline association index is stored in a Merkle tree structure, and the hash value of the bloodline tracing graph corresponding to each model version is used as the leaf node of the Merkle tree.

[0017] Preferably, in any of the above schemes, the path extraction module uses a bidirectional breadth-first search algorithm to extract the complete kinship propagation path from the target kinship tracing map; the source tracing query module supports data node tracing mode and model node tracing mode. The data node tracing mode is used to query which vehicle-mounted models were trained by specified training data, and the model node tracing mode is used to query which training data were used to train a specified vehicle-mounted model.

[0018] The present invention has the following advantages: 1. This method and device for tracing the origin of vehicle-mounted model training based on data lineage, by collecting sensor identifiers, sensor models, collection geographical locations, and vehicle driving status parameters for each piece of training data, along with the entire process of recording data cleaning, labeling, enhancement, and other preprocessing operations, their timestamps, and execution parameters, and by automatically evaluating and generating four-dimensional quality labels of integrity, accuracy, timeliness, and consistency by the vehicle-mounted data quality management module, achieves unified information collection across the entire chain from the data collection end to the preprocessing completion end. This effectively solves the problems of scattered recording of data flow information at each stage, lack of a unified collection mechanism, and easy tampering of lineage information in existing technologies, providing a complete and verifiable data lineage metadata foundation for tracing the origin of vehicle-mounted model training.

[0019] 2. This data lineage-based vehicle-mounted model training tracing method and device constructs three types of core nodes based on training data, vehicle-mounted models, and training tasks. It establishes input lineage relationship edges, output lineage relationship edges, and data transformation lineage relationship edges based on lineage metadata and lineage association records during the training process. This forms a unified structured lineage tracing graph covering the entire process of data acquisition, preprocessing, enhancement, and iterative training. It solves the problem in existing technologies where lineage tracing stops recording before model training and cannot track how data transformation affects the model training process. This enables the attribution analysis of model behavior to have the ability to represent the complete knowledge propagation path from the data source to the model output.

[0020] 3. This method and device for tracing the training source of vehicle-mounted models based on data lineage obtains the gradient update amount of the training data corresponding to each data node in each iteration round, determines the contribution of the data node to the model parameters by the cumulative value of the gradient update amount and performs normalization processing, transforms the qualitative correlation between data quality and model training effect into a quantifiable contribution value, solves the problem in the prior art that it is impossible to quantitatively evaluate the contribution of each training data to the model output, and provides clear quantitative data support and priority ranking basis for data optimization and model iteration when the model performance is not up to standard.

[0021] 4. This data lineage-based vehicle-mounted model training tracing method and device continuously monitors inference performance indicators after the vehicle-mounted model enters the online inference stage. When the indicators fall below a preset threshold, it automatically triggers a tracing query request, locates the data nodes or training iteration nodes related to the performance degradation, and generates an attribution analysis report. This extends the tracing capability from the training stage to the entire lifecycle after model deployment. It solves the problem in the existing technology that model training tracing only covers the training stage and that performance fluctuations in the online inference stage cannot be automatically linked back to the training stage data and process records. This effectively shortens the time lag between problem discovery and root cause localization and realizes dynamic tracing management of the entire lifecycle of the vehicle-mounted model. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the device frame of the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0024] Example 1: As Figure 1 As shown, a data lineage-based method for tracing the training source of in-vehicle models is proposed. This method is applied to the training scenario of in-vehicle models in intelligent connected vehicles, and is specifically applicable to the training source management of autonomous driving perception models.

[0025] In this embodiment, the vehicle-mounted model is a multimodal autonomous driving perception model deployed on the vehicle-mounted terminal. This model needs to simultaneously process image data from the vehicle-mounted camera, point cloud data from the LiDAR, and radar data from the millimeter-wave radar. The vehicle-mounted terminal is connected to the vehicle-mounted data lake via the vehicle-mounted communication network. The vehicle-mounted data lake stores historical data collected from the aforementioned multiple sensors, as well as simulation data and labeled data downloaded from the cloud via vehicle-cloud communication.

[0026] The method in this embodiment includes the following steps.

[0027] In step S1, the system responds to the training start command of the vehicle-mounted model and obtains the training configuration information. Specifically, when the vehicle-mounted terminal receives the model training trigger signal from the cloud, the vehicle-mounted model training system first parses the training configuration parameters carried in the trigger signal. The training configuration information includes the training dataset identifier, model version number, training hyperparameters, and training environment parameters. Among them, the training dataset identifier is used to uniquely identify the training dataset used in this training, for example, a dataset identified as "DS_CAM_LIDAR_RADAR_20260601"; the model version number is used to identify the version of the model to be trained, for example, version number "V3.2.1"; the training hyperparameters include a learning rate set to 0.001, a batch size set to 32, a training epoch set to 100 epochs, and an optimizer type selected as the Adam optimizer; the training environment parameters include the computing resource information of the vehicle-mounted terminal, such as the GPU model being NVIDIA DRIVEOrin and the available video memory being 16GB.

[0028] In step S2, the system reads the corresponding training data from the vehicle data lake according to the training dataset identifier, and collects the data source information, data collection timestamp, data preprocessing operation chain and data quality label for each piece of training data to generate data lineage metadata of the training data.

[0029] Specifically, the system reads the corresponding training data from the vehicle data lake based on the training dataset identifier "DS_CAM_LIDAR_RADAR_20260601". In this embodiment, the training data contains a total of 12,000 sample data, of which 4,000 are camera image data, 4,000 are LiDAR point cloud data, and 4,000 are millimeter-wave radar data. For each piece of training data, the system performs the following acquisition operations.

[0030] First, the system collects data source information. For camera image data, the data source information includes the camera sensor identifier "CAM_FRONT_001", the sensor model "SONYIMX728", the acquisition location as 121.5 degrees east longitude and 31.2 degrees north latitude, and the vehicle driving status parameters at the time of acquisition, including a vehicle speed of 60 kilometers per hour, a steering wheel angle of 5 degrees, and a brake pedal opening of zero. For LiDAR point cloud data, the data source information includes the LiDAR sensor identifier "LIDAR_TOP_002", the sensor model "HESAIPandarXT-32", the acquisition location, and the vehicle driving status parameters.

[0031] Secondly, the system collects data with timestamps. Each piece of training data records the specific time it was collected by the sensor, with timestamps accurate to the millisecond level, such as "2026-06-01 14:32:25.687".

[0032] Next, the system collects the data preprocessing operation chain. This chain records all preprocessing operations performed on the training data from the raw sensor data to its final inclusion in the training dataset. Taking a single camera image as an example, the preprocessing operation chain includes: data cleaning, which removes abnormal image frames with excessively high or low sensitivity, with parameters including a brightness threshold set between 30 and 220; data annotation, which adds bounding boxes to targets such as pedestrians, vehicles, and traffic signs in the image, with parameters including the annotation tool version "LabelTool_V2.3" and the annotator ID "ANNO_087"; and data augmentation, which performs random rotation, scaling, and color jittering on the image, with parameters including a rotation angle range of ±15 degrees, a scaling ratio range of 0.8 to 1.2, and a color jitter intensity of 10%. Each operation is accompanied by a corresponding timestamp and execution parameters.

[0033] Finally, the system collects data quality labels. These labels are automatically generated by the vehicle-mounted data quality management module based on preset quality rules when evaluating the training data. The data quality labels include scores across four dimensions: data integrity score, which assesses whether the data contains complete sensor data, such as whether a camera image contains a complete pixel matrix and corresponding metadata; data accuracy score, which assesses the accuracy of the annotation, such as whether the overlap between the bounding box and the target object is over 90%; data timeliness score, which assesses whether the data was collected within a valid time window, such as whether the collection time is more than three months from the current training time; and data consistency score, which assesses the consistency of the data with other data in the same batch in terms of format and content. Each training data point's four-dimensional scores are expressed as a percentage; for example, a certain image data point might have a quality label of 95 for integrity, 92 for accuracy, 88 for timeliness, and 96 for consistency.

[0034] The system organizes the collected data source information, data collection timestamps, data preprocessing operation chains, and data quality labels according to a preset data structure to generate the data lineage metadata for this training data. The data lineage metadata is stored in JSON format and contains four main parts: data identifier, source information block, timestamp block, preprocessing chain block, and quality label block.

[0035] In step S3, the system inputs training data into the vehicle-mounted model to perform iterative training. During the training process, it records the data batch identifier, model parameter update record, and loss function value used in each iteration in real time. The data batch identifier is associated with the data lineage metadata generated in step S2 to generate a lineage association record for the training process.

[0036] Specifically, the system divides the 12,000 training data points read in step S2 into 375 data batches according to a batch size of 32. Each data batch is assigned a unique batch identifier, such as "BATCH_0001" to "BATCH_0375". The system sequentially inputs each data batch into the vehicle-mounted model for iterative training, performing a total of 100 training iterations. In each iteration, the system records the following information in real time: a list of data batch identifiers used in that iteration, parameter update records for each layer of the model, including changes in the weight matrix and bias vector, and the loss function value at the end of that iteration. The system associates each data batch identifier with the data lineage metadata of each training data point contained in that batch in step S2, establishing a three-level association relationship of "batch identifier - data identifier - lineage metadata", generating a training process lineage association record. The training process lineage association record is stored in the form of a relational database table, containing a batch identifier field, a data identifier field, a lineage metadata pointer field, an iteration round field, a parameter update record field, and a loss function value field.

[0037] In step S4, after the vehicle-mounted model completes training, the system constructs a lineage tracing map of the vehicle-mounted model based on the data lineage metadata collected in step S2 and the training process lineage association records generated in step S3.

[0038] Specifically, the construction of a kinship tracing map includes the following sub-steps.

[0039] In sub-step S4.1, the system uses each piece of training data as a data node and records the attribute information of the data node. The attribute information includes data identifier, data source, data size, and data format. Taking a piece of camera image data as an example, its data node attribute information is: data identifier "DATA_CAM_00001", data source "CAM_FRONT_001", data size "4.2MB", and data format "JPEG".

[0040] In sub-step S4.2, the system uses the vehicle-mounted model as a model node and records the attribute information of the model node. The attribute information includes the model identifier, model architecture, and model version number. In this embodiment, the attribute information of the model node is: model identifier "MODEL_PERCEPTION_001", model architecture "multimodal fusion Transformer architecture", and model version number "V3.2.1".

[0041] In sub-step S4.3, the system uses each training task as a training task node and records the attribute information of the training task node. The attribute information includes task identifier, training start time, training end time, and training hyperparameters. In this embodiment, the attribute information of the training task node is: task identifier "TASK_20260601_001", training start time "2026-06-01 14:30:00", training end time "2026-06-02 02:15:30", and training hyperparameters including learning rate 0.001, batch size 32, number of training epochs 100, and optimizer type Adam.

[0042] In sub-step S4.4, the system establishes lineage relationship edges between nodes based on lineage metadata and lineage association records during the training process. Specifically, this includes: establishing input lineage relationship edges between data nodes and training task nodes, indicating that the training data corresponding to the data node is input into the training task corresponding to the training task node to participate in model training; establishing output lineage relationship edges between training task nodes and model nodes, indicating that the training task corresponding to the training task node produces the vehicle model corresponding to the model node; and establishing data transformation lineage relationship edges between data nodes, indicating the parent-child data relationship formed by the data undergoing transformation operations such as cleaning, labeling, and augmentation during preprocessing.

[0043] Through the above sub-steps, the system constructs a directed acyclic graph containing 12,000 data nodes, one model node, one training task node, and tens of thousands of bloodline relationship edges, namely the bloodline tracing graph.

[0044] In step S5, the system binds the model version number, training completion timestamp, model performance evaluation index and lineage tracing map of the vehicle model to generate a model-lineage association index and stores the model-lineage association index in the vehicle blockchain storage module.

[0045] Specifically, the system binds the model version number "V3.2.1", the training completion timestamp "2026-06-02 02:15:30", and model performance evaluation metrics, including an average precision of 87.3% on the validation set and an average precision of 72.1% when the intersection-union ratio threshold is 0.5, to the kinship tracing graph constructed in step S4. Based on the bound data, the system generates a model-kinship association index, which is stored in key-value pairs. The key is the model version number, and the value is a pointer to the storage address of the kinship tracing graph and its corresponding hash value. The system stores the model-kinship association index in the vehicle-mounted blockchain evidence storage module. The vehicle-mounted blockchain evidence storage module adopts a lightweight blockchain architecture, and the model-kinship association index is stored in the vehicle-mounted blockchain evidence storage module in a Merkle tree structure. Specifically, the system calculates the SHA-256 hash value for the kinship tracing graph corresponding to each model version, uses these hash values ​​as leaf nodes of the Merkle tree, and performs pairwise hash calculations layer by layer to generate upper-level nodes, ultimately forming the Merkle root. When it is necessary to verify whether the traceability information of a certain version has been tampered with, it is only necessary to recalculate the hash value corresponding to that version and compare it with the leaf node value stored in the Merkle tree to complete the quick verification.

[0046] In step S6, the system responds to the traceability query request for the target vehicle model by parsing the query conditions carried in the traceability query request.

[0047] Specifically, when the vehicle-mounted model experiences a decrease in perception accuracy after deployment, developers initiate a source tracing query request through the vehicle terminal's human-machine interface. Upon receiving the request, the system parses the query conditions carried within. These conditions include at least one of the following: model version number, training data identifier, training time period, and model performance threshold. In this embodiment, the developer inputs the model version number "V3.2.1" and the model performance threshold "average accuracy is less than 85%".

[0048] In step S7, the system performs a matching search in the model-bloodline association index based on the query conditions to determine the target bloodline tracing map corresponding to the query conditions.

[0049] Specifically, the system uses the model version number "V3.2.1" as the search key, performs an exact match query in the model-bloodline association index, locates the storage address of the bloodline tracing map corresponding to the model version number, and identifies the map as the target bloodline tracing map. In step S8, the system performs a path search algorithm on the target kinship tracing map to extract the complete kinship propagation path from the data source node to the model node.

[0050] Specifically, the system employs a bidirectional breadth-first search algorithm to perform pathfinding. This algorithm performs a breadth-first search simultaneously from both the data source node and the model node. At the start of the search, the system first expands outward layer by layer from the data source node along the direction of the lineage relationship edges, and simultaneously expands outward layer by layer from the model node along the opposite direction of the lineage relationship edges. When the search paths from both ends intersect for the first time at an intermediate node, the algorithm immediately stops searching, concatenates the path from the data source node to the intersection node with the path from the intersection node to the model node, and outputs the complete lineage propagation path. In this embodiment, the system starts from 12,000 data source nodes and simultaneously from the model node "MODEL_PERCEPTION_001," with the bidirectional search intersecting at the data processing node "DATA_PREPROCESS_AUG_005." The output complete lineage propagation path includes the data source node, data cleaning node, data labeling node, data augmentation node, training task node, and model node. The path also includes data transformation lineage relationship edges and input / output lineage relationship edges between the nodes.

[0051] In step S9, the system generates a source tracing report based on the complete bloodline transmission path.

[0052] Specifically, the source tracing report includes the following: First, a list of training data sources, which lists the identifiers, models, and number of data entries contributed by each sensor that participated in the model training. Second, a description of the data preprocessing chain, which records in chronological order all preprocessing operations and their execution parameters that the training data underwent from initial acquisition to model training. Third, a record of key training iterations, which lists the ten iterations in which the loss function value decreased most significantly during training and the data batch identifiers used in each iteration. Fourth, a metric for the contribution of each data node to the model output. The metric is calculated as follows: the system obtains the gradient update amount of the training data corresponding to each data node in each iteration during model training, determines the contribution of the data node to the model parameters based on the cumulative value of the gradient update amount, and normalizes this contribution as the metric. Specifically, for a certain camera image data, the system records the gradient update vector generated in each iteration, calculates the cumulative norm of this vector in each iteration, normalizes the cumulative norm of all data nodes, and obtains the metric for the contribution of that data node. In this embodiment, the contribution quantification value of the top 5% of data nodes with the highest contribution is above 0.8, and the contribution quantification value of the bottom 10% of data nodes with the lowest contribution is below 0.1.

[0053] In step S10, the system outputs the traceability report to the human-machine interface of the vehicle terminal in the form of a visual map, and displays the data nodes in layers and colors according to the blood distance in the visual map.

[0054] Specifically, the system renders the lineage propagation paths in the source tracing report as a visual graph. In this graph, data nodes are represented by circular icons, model nodes by square icons, training task nodes by diamond icons, and lineage relationships are represented by connecting lines with arrows. The system uses layered coloring based on the lineage distance of each data node from the model node: data nodes with a one-hop lineage distance are red, those with a two-hop lineage distance are orange, those with a three-hop lineage distance are yellow, and those with a four-hop or higher lineage distance are blue. Developers can click on any node in the graph to view its detailed attribute information and contribution metric, thereby quickly locating key data nodes that cause model performance degradation.

[0055] Example 2: This example provides another method for tracing the training origins of in-vehicle models based on data lineage. The difference between this example and Example 1 is that this example further includes dynamic tracing and monitoring steps during the online inference phase of the in-vehicle model, as well as the specific connections and interactions between the various modules of the device. This example is also applied to the in-vehicle model training tracing scenario of intelligent connected vehicles, and is suitable for application scenarios with higher requirements for the tracing and management of the entire model lifecycle.

[0056] In this embodiment, the vehicle-mounted model is an autonomous driving decision-making and planning model deployed on the vehicle-mounted terminal. This model needs to perform path planning and driving decisions based on the environmental information output by the perception model. The vehicle-mounted terminal is connected to the vehicle-mounted data lake and the vehicle-mounted blockchain evidence storage module through the vehicle-mounted communication network.

[0057] In addition to all the steps of Embodiment 1, the method of this embodiment also includes step S11. Step S11 is executed after the vehicle-mounted model enters the online inference stage.

[0058] In step S11, after the vehicle-mounted model enters the online inference stage, the system continuously monitors the inference performance indicators of the vehicle-mounted model. When the inference performance indicators are lower than the preset threshold, the system automatically triggers a source tracing query request to locate the data node or training iteration node that caused the performance degradation and generates a performance attribution analysis report.

[0059] Specifically, after the vehicle-mounted model completes training and is deployed to the vehicle terminal, it enters the online inference phase. The system initiates a background monitoring process that continuously collects inference performance metrics of the vehicle-mounted model in a real road environment. Inference performance metrics include decision accuracy, path planning success rate, and average decision latency. Preset thresholds are set as follows: decision accuracy below 80%, path planning success rate below 75%, or average decision latency greater than 200 milliseconds.

[0060] In a specific application scenario of this embodiment, a vehicle equipped with the decision-making and planning model experienced three consecutive decision-making anomalies while traveling on a specific road segment. Specifically, the model misjudged the presence of an obstacle ahead on an unobstructed straight road section and performed unnecessary emergency braking. The system's background monitoring process detected this anomaly in real time, finding that the decision accuracy had dropped to 72%, below the preset threshold of 80%. The system then automatically triggered a source tracing query request without manual intervention.

[0061] The automatically triggered source tracing query request uses the current model version number "V3.2.1" and the timestamp of the triggered exception "2026-07-01 10:23:45" as query conditions, and automatically executes the source tracing query process from steps S6 to S10. The system retrieves the kinship tracing map corresponding to the model version number through the model-kinship association index, extracts the complete kinship propagation path using a bidirectional breadth-first search algorithm, generates a source tracing report, and outputs a performance attribution analysis report.

[0062] The performance attribution analysis report indicates that the key training data causing the decision anomalies originated from a batch of image data collected in rainy conditions by a rear-view camera designated "CAM_REAR_003". The data integrity score for this batch of data was only 62 points, significantly lower than the average integrity score of 90 points for the model's training dataset. The system further pinpointed the training iteration nodes corresponding to this low-quality data to epochs 52 to 58, within which the model's loss function value exhibited abnormal fluctuations. The attribution analysis report recommends that developers re-label and augment this low-quality data, or remove it from the training dataset and retrain the model.

[0063] This embodiment also provides a data lineage-based vehicle model training tracing device, which is used to implement the above method. The device includes a data lineage acquisition module, a training process tracking module, a lineage map construction module, an index storage module, a tracing query module, a path extraction module, a report generation module, and a visualization output module. The connections and interactions between the modules are as follows.

[0064] The input of the data lineage acquisition module is connected to the onboard data lake, and its output is connected to the input of the training process tracking module and the first input of the lineage graph construction module. Responding to the training start command from the onboard model, the data lineage acquisition module obtains training configuration information, reads the corresponding training data from the onboard data lake based on the training dataset identifier, and collects the data source information, data acquisition timestamp, data preprocessing operation chain, and data quality label for each piece of training data, generating data lineage metadata. The data lineage acquisition module then sends the generated data lineage metadata to both the training process tracking module and the lineage graph construction module.

[0065] The input of the training process tracking module is connected to the output of the data lineage acquisition module, and the output of the training process tracking module is connected to the second input of the lineage graph construction module. The training process tracking module receives data lineage metadata and corresponding training data from the data lineage acquisition module, inputs the training data into the vehicle-mounted model for iterative training, and records the data batch identifier, model parameter update records, and loss function values ​​used in each iteration in real time during training. It then associates and maps the data batch identifier with the data lineage metadata to generate training process lineage association records. The training process tracking module then sends the generated training process lineage association records to the lineage graph construction module.

[0066] The first input of the pedigree graph construction module is connected to the output of the data pedigree acquisition module, the second input is connected to the output of the training process tracking module, and the output is connected to the input of the index storage module. After the vehicle-mounted model completes training, the pedigree graph construction module receives data pedigree metadata sent by the data pedigree acquisition module and training process pedigree association records sent by the training process tracking module, and constructs a pedigree tracing graph for the vehicle-mounted model based on this data. The pedigree tracing graph includes data nodes, model nodes, training task nodes, and pedigree relationship edges between each node. The pedigree graph construction module then sends the completed pedigree tracing graph to the index storage module.

[0067] The input of the index storage module is connected to the output of the pedigree graph construction module, and the output of the index storage module is connected to the first input of both the vehicle-mounted blockchain storage module and the traceability query module. The index storage module receives the pedigree tracing graph sent by the pedigree graph construction module, binds the vehicle model's model version number, training completion timestamp, and model performance evaluation metrics to the pedigree tracing graph, and generates a model-pedigree association index. The index storage module stores the model-pedigree association index in the vehicle-mounted blockchain storage module, and simultaneously sends a retrieved copy of the model-pedigree association index to the traceability query module.

[0068] The first input of the tracing query module is connected to the output of the index storage module, the second input is connected to the output of the vehicle terminal, and the output is connected to the input of the path extraction module. The tracing query module receives a copy of the model-lineage association index retrieved from the index storage module and receives a tracing query request for the target vehicle model from the vehicle terminal. The tracing query module parses the query conditions carried in the tracing query request, performs a matching search in the model-lineage association index based on the query conditions, and determines the target lineage tracing map corresponding to the query conditions. The tracing query module supports two query modes: data node tracing mode and model node tracing mode. The data node tracing mode is used to query which vehicle models were trained using specified training data, while the model node tracing mode is used to query which training data was used to train a specified vehicle model. The tracing query module sends the retrieved target lineage tracing map to the path extraction module.

[0069] The input of the path extraction module is connected to the output of the source tracing query module, and the output of the path extraction module is connected to the input of the report generation module. The path extraction module receives the target kinship tracing map sent by the source tracing query module and performs a path search algorithm on the target kinship tracing map. The path extraction module uses a bidirectional breadth-first search algorithm, performing searches simultaneously from both the data source node and the model node. The search stops when the search paths from both ends intersect, and the intersecting path is output as the complete kinship propagation path. The path extraction module then sends the extracted complete kinship propagation path to the report generation module.

[0070] The report generation module's input is connected to the path extraction module's output, and its output is connected to the visualization output module's input. The report generation module receives the complete lineage propagation path from the path extraction module and generates a source tracing report based on this path. The source tracing report includes a list of training data sources, a description of the data preprocessing chain, key training iteration records, and quantified values ​​of each data node's contribution to the model output. The report generation module then sends the generated source tracing report to the visualization output module.

[0071] The input end of the visualization output module is connected to the output end of the report generation module, and the output end of the visualization output module is connected to the human-machine interface of the vehicle terminal. The visualization output module receives the traceability report sent by the report generation module, outputs the traceability report to the human-machine interface of the vehicle terminal in the form of a visualization map, and displays the data nodes in the visualization map in layers and colors according to the kinship distance.

[0072] The device in this embodiment also includes an online monitoring module. The online monitoring module is connected to the vehicle-mounted model and is used to continuously monitor the inference performance indicators of the vehicle-mounted model after it enters the online inference stage. When the inference performance indicators fall below a preset threshold, a source tracing query request is automatically triggered. The automatically triggered query conditions are sent to the second input terminal of the source tracing query module, which then initiates the source tracing query process to ultimately locate the data node or training iteration node that caused the performance degradation and generate a performance attribution analysis report.

[0073] In summary, the present invention, through the two embodiments described above, fully demonstrates the implementation of the method and apparatus for tracing the origins of vehicle-mounted model training based on data lineage. Embodiment one focuses on the complete implementation process of the method steps, detailing the entire operation from training initiation to the generation of the tracing report; Embodiment two, based on Embodiment one, further adds dynamic tracing monitoring during the online inference stage and details the connections and interactions between the various modules of the apparatus. The two embodiments complement each other, together constituting the complete implementation scheme of the present invention.

[0074] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for tracing the training origins of vehicle-mounted models based on data lineage, characterized in that: Includes the following steps: S1: In response to the training start command of the vehicle model, obtain training configuration information, which includes training dataset identifier, model version number, training hyperparameters and training environment parameters; S2: Based on the training dataset identifier, read the corresponding training data from the vehicle data lake, and collect the data source information, data collection timestamp, data preprocessing operation chain and data quality label for each piece of training data to generate the data lineage metadata of the training data; S3: Input the training data into the vehicle model to perform iterative training. During the training process, record the data batch identifier, model parameter update record and loss function value used in each iteration in real time. Associate and map the data batch identifier with the data lineage metadata in step S2 to generate a lineage association record for the training process. S4: After the vehicle-mounted model completes training, based on the data lineage metadata collected in step S2 and the training process lineage association records generated in step S3, a lineage tracing graph of the vehicle-mounted model is constructed. The lineage tracing graph includes data nodes, model nodes, training task nodes, and lineage relationship edges between each node. S5: Bind the model version number, training completion timestamp, and model performance evaluation index of the vehicle-mounted model to the lineage tracing map to generate a model-lineage association index, and store the model-lineage association index in the vehicle-mounted blockchain evidence storage module; S6: In response to the source tracing query request for the target vehicle model, parse the query conditions carried in the source tracing query request, wherein the query conditions include at least one of the following: model version number, training data identifier, training time period, and model performance threshold; S7: Based on the query conditions, perform a matching search in the model-bloodline association index to determine the target bloodline tracing map corresponding to the query conditions; S8: Execute a path search algorithm on the target kinship tracing graph to extract a complete kinship propagation path from the data source node to the model node from the target kinship tracing graph. The complete kinship propagation path includes intermediate data processing nodes, data transformation operation nodes, and training iteration nodes. S9: Based on the complete bloodline propagation path, generate a source tracing report, which includes a list of training data sources, a description of the data preprocessing link, key training iteration records, and a quantified value of the contribution of each data node to the model output. S10: Output the source tracing report to the human-machine interface of the vehicle terminal in the form of a visual map, and display the data nodes in the visual map in layers and colors according to the blood distance.

2. The method for tracing the training source of vehicle-mounted models based on data lineage according to claim 1, characterized in that: The data source information in step S2 includes the sensor identifier, sensor model, geographical location of data acquisition, and vehicle driving status parameters at the time of acquisition. The data preprocessing operation chain includes data cleaning operation, data labeling operation, data augmentation operation, and timestamps and execution parameters for each operation. The data quality labels include data integrity score, data accuracy score, data timeliness score, and data consistency score. The data quality labels are automatically generated by the vehicle data quality management module based on preset quality rules when evaluating the training data.

3. The method for tracing the training source of an in-vehicle model based on data lineage as described in claim 1, characterized in that: The method for constructing the kinship tracing map in step S4 includes: S4.1: Using each piece of training data as a data node, record the attribute information of the data node, including data identifier, data source, data size, and data format; S4.2: Using the vehicle-mounted model as a model node, record the attribute information of the model node, including model identifier, model architecture, and model version number; S4.3: Using each training task as a training task node, record the attribute information of the training task node, including task identifier, training start time, training end time, and training hyperparameters; S4.4: Based on the lineage metadata and the lineage association records of the training process, establish input lineage relationship edges between data nodes and training task nodes, output lineage relationship edges between training task nodes and model nodes, and data transformation lineage relationship edges between data nodes.

4. The method for tracing the training source of vehicle-mounted models based on data lineage according to claim 1, characterized in that: The vehicle-mounted blockchain evidence storage module in S5 adopts a lightweight blockchain architecture. The model-bloodline association index is stored in the vehicle-mounted blockchain evidence storage module in a Merkle tree structure, and the hash value of the bloodline tracing graph corresponding to each model version is used as the leaf node of the Merkle tree.

5. The method for tracing the training source of an in-vehicle model based on data lineage as described in claim 1, characterized in that: The traceability query request in step S6 also includes data node traceability mode or model node traceability mode. The data node tracing mode is used to query which vehicle models were trained using specified training data. The model node traceability mode is used to query which training data was used to train a specified vehicle model. The path search algorithm in S8 adopts a bidirectional breadth-first search algorithm, which performs the search simultaneously from both the data source node and the model node. The search stops when the search paths of the two ends intersect, and the intersecting path is output as the complete lineage propagation path.

6. The method for tracing the training source of an in-vehicle model based on data lineage according to claim 1, characterized in that: In step S9, the contribution metric of each data node to the model output is calculated as follows: the gradient update amount of the training data corresponding to the data node in each iteration during the model training process is obtained, the contribution of the data node to the model parameters is determined based on the cumulative value of the gradient update amount, and the contribution is normalized and used as the contribution metric.

7. The method for tracing the training source of vehicle-mounted models based on data lineage according to claim 1, characterized in that: It also includes step S11: After the vehicle-mounted model enters the online inference stage, the inference performance index of the vehicle-mounted model is continuously monitored. When the inference performance index is lower than a preset threshold, a source tracing query request is automatically triggered to locate the data node or training iteration node that caused the performance degradation and generate a performance attribution analysis report.

8. A vehicle-mounted model training tracing device based on data lineage, used to implement the method as described in any one of claims 1-7, characterized in that: include: The data lineage acquisition module is used to respond to the training start command of the vehicle model, obtain training configuration information, read the corresponding training data from the vehicle data lake according to the training dataset identifier, and collect the data source information, data acquisition timestamp, data preprocessing operation chain and data quality label for each piece of training data to generate the data lineage metadata of the training data. The training process tracking module is used to input the training data into the vehicle model to perform iterative training. During the training process, it records the data batch identifier, model parameter update record and loss function value used in each iteration in real time, and associates and maps the data batch identifier with the data lineage metadata to generate a training process lineage association record. The lineage graph construction module is used to construct the lineage tracing graph of the vehicle model after the vehicle model has completed training, based on the data lineage metadata and the lineage association records of the training process. The lineage tracing graph includes data nodes, model nodes, training task nodes, and lineage relationship edges between each node. The index storage module is used to bind the model version number, training completion timestamp, and model performance evaluation indicators of the vehicle-mounted model with the lineage tracing map, generate a model-lineage association index, and store the model-lineage association index in the vehicle-mounted blockchain storage module. The tracing query module is used to respond to the tracing query request for the target vehicle model, parse the query conditions carried in the tracing query request, perform matching and retrieval in the model-bloodline association index according to the query conditions, and determine the target bloodline tracing map corresponding to the query conditions; The path extraction module is used to perform a path search algorithm on the target kinship tracing graph to extract the complete kinship propagation path from the data source node to the model node from the target kinship tracing graph; The report generation module is used to generate a source tracing report based on the complete bloodline transmission path. The source tracing report includes a list of training data sources, a description of the data preprocessing link, key training iteration records, and a quantified value of the contribution of each data node to the model output. The visualization output module is used to output the source tracing report to the human-machine interface of the vehicle terminal in the form of a visualization map, and to display the data nodes in the visualization map in a layered and colored manner according to the blood distance.

9. The vehicle-mounted model training tracing device based on data lineage according to claim 1, characterized in that: The vehicle-mounted blockchain evidence storage module in the index storage module adopts a lightweight blockchain architecture. The model-bloodline association index is stored in a Merkle tree structure, and the hash value of the bloodline tracing graph corresponding to each model version is used as the leaf node of the Merkle tree.

10. The vehicle-mounted model training tracing device based on data lineage according to claim 1, characterized in that: The path extraction module uses a bidirectional breadth-first search algorithm to extract the complete kinship propagation path from the target kinship tracing map; the source tracing query module supports data node tracing mode and model node tracing mode. The data node tracing mode is used to query which vehicle models were trained by specified training data, and the model node tracing mode is used to query which training data were used to train a specified vehicle model.

Images