Model behavior interpretable attribution method and device, equipment and medium

Abstract

The invention relates to the technical field of intelligent decision making, can be applied to business scenes such as financial science and technology and medical health, and discloses a model behavior interpretable attribution method, device and equipment and a medium. The method comprises the following steps: generating data source metadata comprising a data source, a timestamp and a processing step, constructing a training evidence chain based on the data source metadata, and generating a model behavior interpretable attribution result; and extracting input data from the prediction request, generating a prediction result by using the trained model, obtaining prediction explanation information, and generating an attribution report in combination with the training evidence chain, the prediction result and the prediction explanation information. According to the method, the traceable link penetrating through data acquisition, model training and model prediction is constructed, so that the model behaviors have clear source basis and decision basis, the transparency and auditing performance of the model are improved, and responsibility division and compliance management are facilitated.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making technology, and in particular to a method, apparatus, device, and medium for interpretable attribution of model behavior. Background Technology

[0002] In the fintech sector, AI models have been widely deployed in core business processes such as customer risk assessment, intelligent claims processing, fraud detection, and intelligent recommendations. With the influx of multi-source data, the models' coverage of business scenarios has continuously improved. However, existing systems generally lack a complete record mechanism for data sources and processing flows. This makes it difficult for business institutions to trace the specific raw data used and how it was processed during training when abnormal predictions or unreasonable decisions occur. Furthermore, most financial models rely on complex deep learning structures, whose internal decision-making mechanisms are difficult to explain. It's hard to clearly explain how the predicted output is formed from the perspective of input data and the model's internal path, leading to insufficient transparency in key business scenarios. When business personnel need to review, verify, or provide evidence to regulatory agencies, they often lack systematic evidence. Moreover, due to the lack of verifiable predictive explanations and their correspondence with historical training data, it's difficult to determine whether the problem stems from data quality, feature processing steps, or the model structure itself when the model exhibits abnormal behavior, thus hindering the implementation of accountability mechanisms.

[0003] In the healthcare sector, intelligent models are used for applications such as risk screening, assisted diagnosis, medical image recognition, and intelligent follow-up. Due to the complexity of medical data sources and frequent inter-institutional flows, existing systems often struggle to record the source, generation time, and processing history of each data point. This results in a lack of reliable data-level evidence when healthcare institutions face controversial diagnostic results or prediction errors. Regarding model behavior, healthcare institutions typically cannot know which input features the model relied on during diagnosis or prediction, or which key paths influenced the final judgment. This makes it difficult to explain model behavior to medical staff, support doctor-patient communication, and meet the transparency and traceability requirements of medical compliance reviews. When predictions differ from actual clinical judgments, existing systems also lack effective mechanisms to help healthcare institutions determine the source of abnormal behavior—for example, whether it's due to training sample bias, abnormal input data, or a disordered model decision-making path—significantly increasing the difficulty of holding and tracing model behavior accountable. Summary of the Invention

[0004] The main objective of this invention is to provide a method, apparatus, device, and storage medium for interpretable attribution of model behavior, aiming to solve the technical problem that the existing technology cannot simultaneously record the source links of multi-source data, the model training process, and the prediction behavior in a complete, correlated, and verifiable manner, resulting in a lack of interpretability and traceability of model behavior.

[0005] To achieve the above objectives, the present invention provides a method for interpretable attribution of model behavior, comprising: Collect data from multiple sources and generate data source metadata that includes data source, timestamp, and processing steps; During the process of training the model using the multi-source data, a training evidence chain containing the contribution value of training samples and preprocessing information is generated based on the metadata of the data sources. Receive a prediction request and extract input data from the prediction request; The trained model is used to predict the input data to obtain prediction results. Based on the metadata of the data source, prediction explanation information is generated by analyzing the contribution value of the input features and the decision path. Attribution analysis is performed based on the training evidence chain, the prediction results, and the prediction explanation information to generate an attribution report.

[0006] Furthermore, to achieve the above objectives, the present invention provides a model behavior interpretability attribution device, comprising: The data acquisition and tracing module is used to collect multi-source data and generate data source metadata that includes data source, timestamp, and processing steps; The training evidence chain construction module is used to generate a training evidence chain containing the contribution value of training samples and preprocessing information based on the metadata of the data sources during the process of training the model using the multi-source data. The prediction request parsing module is used to receive prediction requests and extract input data from the prediction requests; The prediction and explanation generation module is used to predict the input data using the trained model, obtain the prediction results, and generate prediction explanation information based on the metadata of the data source by analyzing the contribution value of the input features and the decision path. The attribution analysis and report generation module is used to perform attribution analysis based on the training evidence chain, the prediction results, and the prediction explanation information, and generate an attribution report.

[0007] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a model behavior interpretable attribution program stored in the memory and executable on the processor, wherein when the model behavior interpretable attribution program is executed by the processor, it implements the steps of the model behavior interpretable attribution method as described above.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a non-volatile computer-readable storage medium storing a model behavior interpretable attribution program, which, when executed by a processor, implements the steps of the model behavior interpretable attribution method as described above.

[0009] Beneficial Effects: This invention relates to the field of intelligent decision-making technology and can be applied to business scenarios such as fintech and healthcare. It discloses a method, apparatus, device, and medium for interpretable attribution of model behavior, comprising: collecting multi-source data and generating data source metadata including data source, timestamp, and processing steps; generating a training evidence chain containing training sample contribution values ​​and preprocessing information based on the data source metadata during model training; receiving prediction requests and extracting input data; using the trained model to predict the input data, obtaining prediction results, and generating prediction explanation information by analyzing input feature contribution values ​​and decision paths; and performing attribution analysis based on the training evidence chain, prediction results, and prediction explanation information to generate an attribution report. This invention establishes a traceable evidence chain for data flow, model training, and model prediction, forming a full-process explanation system from input data to prediction behavior. This makes the model prediction logic, input feature contributions, and training sample sources verifiable, achieving transparent management and responsibility allocation of model behavior, and improving the auditability and compliance of the system in complex business environments. Attached Figure Description

[0010] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of an application environment for the model behavior interpretability attribution method in one embodiment of the present invention; Figure 2 This is a flowchart illustrating an embodiment of the model behavior interpretability attribution method of the present invention; Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the model behavior interpretability attribution device of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0011] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0012] The model behavior interpretable attribution method provided in this embodiment of the invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can collect multi-source data from the client and generate data source metadata containing data source, timestamp, and processing steps; during model training, it generates a training evidence chain containing training sample contribution values ​​and preprocessing information based on the data source metadata; it receives prediction requests and extracts input data; it uses the trained model to predict the input data, obtains prediction results, and generates prediction explanation information by analyzing input feature contribution values ​​and decision paths; it performs attribution analysis based on the training evidence chain, prediction results, and prediction explanation information, and generates an attribution report. This invention establishes a traceable evidence chain for data flow, model training, and model prediction, forming a full-process explanation system from input data to prediction behavior. This makes the model prediction logic, input feature contribution, and training sample source verifiable, achieving transparent management and responsibility allocation of model behavior, and improving the auditability and compliance of the system in complex business environments. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0013] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the model behavior interpretability attribution method provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0014] like Figure 2 As shown, the model behavior interpretability attribution method proposed in this invention includes the following steps: S10: Collect multi-source data and generate data source metadata including data source, timestamp, and processing steps; In this embodiment, the process of collecting multi-source data requires establishing a unified data access capability. The multi-source data originates from different business nodes, external service interfaces, interactive terminals, or sensing devices. Due to inconsistencies in source structures and encoding standards, the data format, content structure, and transmission protocol need to be parsed and standardized by the access component to ensure that data from different sources can enter the system in a processable structure. Upon receiving data, the source information needs to be extracted to indicate the system, device, or business unit that generated the data. This source information can be identified from fixed fields in the data payload, source tags, or source mapping lists, or it can be determined by matching with source registration records.

[0015] When data enters the system, a timestamp needs to be generated to record the entry time and maintain the sortability of the time series. The timestamp can come from a local clock or a distributed time synchronization system to ensure a unified time base in a multi-node environment. The timestamp provides a time-series location basis for subsequent data tracking. Data source metadata also needs to record the processing history the data underwent before entering the system, including cleaning, extraction, transformation, or aggregation operations. These operations may come from processing tags in the data packets or from log records of external systems. By identifying this processing history, the original processing chain of the data can be reconstructed.

[0016] Once the source information, timestamp, and processing history are identified, they need to be encapsulated as data source metadata and indexed with the corresponding data entries. The metadata encapsulation process is completed using a unified structured format, allowing it to be directly read during training, prediction, and audit analysis. This entire process forms a continuous chain; after data access, identification, recording, and encapsulation, it constitutes a traceable data infrastructure, providing a clear data source chain for subsequent training evidence chain construction.

[0017] This embodiment generates metadata such as source information, timestamp, and processing history when data enters the system, forming a traceable data link. This enables subsequent training, prediction, and accountability analysis to be performed based on a transparent data structure, improving data credibility and interpretability, and reducing the problem of opaque decision-making caused by unclear sources or missing processing links.

[0018] S20, during the process of training the model using the multi-source data, a training evidence chain containing the contribution value of training samples and preprocessing information is generated based on the metadata of the data sources. In this embodiment, during the training of the model using multi-source data, it is necessary to combine the multi-source data with the metadata of the data sources to form training samples that can be received by the model. Training samples not only contain feature fields but also need to include source labels, time records, and processing trajectories. This is achieved through combination with the metadata of the data sources, giving each training sample a complete source chain. During actual training, the model processes the same training sample multiple times, continuously adjusting its internal parameters through forward inference and gradient updates. In this process, the influence of the training sample on parameter changes can be quantified using contribution calculation methods, thus forming a training sample contribution value. The contribution value reflects the actual role of the training sample during the training phase and can be calculated based on indicators such as gradient magnitude, loss reduction magnitude, or parameter change ratio to ensure that the influence of the training sample is quantifiable.

[0019] In addition to contribution value calculation, the training phase includes a series of data processing transformations, such as normalization, missing value handling, and feature engineering. Normalization maps data with different dimensions and distributions to a space more suitable for model training; missing value handling restores the usability of training samples through imputation or deletion mechanisms; feature engineering transformations include class encoding, numerical discretization, or structural reorganization to generate data formats suitable for model input. Each processing step requires recording the processing type, timing, and post-processing data form to create a traceable processing trajectory.

[0020] Once the contribution values ​​of the training samples and the preprocessing information are recorded, they need to be combined into a training evidence chain. The training evidence chain links key information experienced by the training samples during the training process in a structured format, creating a continuous link between sample source, sample role, and sample processing history. The generation of the evidence chain is automatically triggered by the training process; the contribution values ​​and preprocessing records generated in each training iteration are incrementally written into the evidence chain structure, thus forming a traceable training history.

[0021] This embodiment generates training sample contribution values ​​and preprocessing information during the training process and forms a training evidence chain, enabling the model training stage to have traceable sources, reproducible processing, and quantifiable sample effects. This provides verifiable training evidence in subsequent prediction interpretation and accountability analysis, improving the transparency and interpretability of the training process.

[0022] S30, Receive a prediction request and extract input data from the prediction request; In this embodiment, the process of receiving prediction requests requires building input channels facing both external and internal system interfaces to ensure that calls from different business systems can be processed in a standardized manner. Prediction requests typically include a request body, parameter fields, user-triggered actions, or system-triggered instructions. After entering the processing unit through the interface access layer, the request structure needs to be identified through a communication protocol parsing mechanism. This parsing process separates parameter fields, structured data fragments, and data payloads, allowing subsequent data extraction to focus on content related to the prediction behavior. A unique identifier is assigned to each prediction request upon entering the system. This unique identifier is constructed using a random sequence generation algorithm, hash transformation algorithm, or time series encoding algorithm, ensuring the uniqueness of each request globally. This identifier is ultimately used for tracing and associating input data.

[0023] After parsing, the data payload in the prediction request needs to undergo integrity and format checks. Integrity checks can be implemented through field coverage checks, required field checks, or data consistency checks. For example, string fields are checked for length, array fields for dimension checks, and structured fields for key-value structure checks. Format checks ensure that the input data meets model requirements; for example, field types must conform to the expected numeric, enumeration, or text types, and time fields must meet standardized formats. Only after passing the checks will the input data related to the prediction behavior be extracted from the data payload. The receiving time point needs to be recorded simultaneously with the input data extraction. The receiving time point is generated by the system's internal timestamp service and associated with the input data, making the data input process time-traceable.

[0024] After extraction and recording, the input data needs to be bound to its corresponding unique identifier. This binding can be achieved using key-value mapping, a relational structure, or a graph structure, as long as a sustainable query link is maintained between the input data, the request source, and the time record. Through this mapping, subsequent prediction results, prediction explanations, attribution analysis records, and audit information can all be traced back from the same prediction request, ensuring the integrity of the entire request chain.

[0025] This embodiment uses a multi-layered mechanism of parsing, detection, extraction, and association to receive prediction requests and extract input data. This ensures that the input data entering the model has traceable sources, structural consistency, and time verifiability, providing an accurate input basis for subsequent prediction execution, interpretation generation, and accountability, and reducing model bias or analysis errors caused by abnormal input data.

[0026] S40, use the trained model to predict the input data, obtain the prediction result, and generate prediction explanation information based on the metadata of the data source by analyzing the contribution value of the input features and the decision path; In this embodiment, a trained model typically refers to a model instance that has undergone a complete training process and whose parameters have been frozen. It contains an inference structure for generating prediction results, such as a neural network layer, a tree model structure, or a graph model structure. Input data, after being received and parsed, enters the model's inference structure. Through input layer encoding, feature mapping processing, and the action of internal computing units, the output result is formed. The prediction result is a quantifiable result of the model's inference calculations on the input data, which can be expressed as a numerical range, probability value, classification label, trend direction, etc. The generation process of the prediction result depends on the model's internal parameter weights and inference graph structure; therefore, it is necessary to ensure that the input data has undergone format and integrity verification before being input into the model.

[0027] After obtaining the prediction results, it is necessary to parse the input features involved in the model calculation from the input data. Input features refer to the fields, variables, or numerical expressions directly used by the model during inference. For example, in a text analysis model, input features can be vectorized word segments; in a numerical model, input features can be numerical fields of different dimensions. The data source corresponding to the input features can be traced through data source metadata. Data source metadata includes the source system name, time record, and processing history, used to determine the complete chain of input data in the collection, cleaning, and processing stages, making the source of the input data traceable.

[0028] To generate predictive explanations, the contribution value of each input feature needs to be analyzed. The contribution value represents the magnitude of the input feature's influence on the prediction result during model computation. For example, contribution values ​​can be obtained through methods such as gradient sensitivity analysis, perturbation sensitivity detection, or path-weighted analysis, reflecting the model's focus. Contribution values ​​can be calculated for a single feature or a combination of multiple features to reflect the interaction between different features.

[0029] A decision path is the internal logical chain that leads to the result during model inference, typically formed by computational nodes, branching relationships, or attention distribution within the model. In a tree structure, the decision path manifests as a sequence of branch choices; in a neural network, it can be represented by attention distribution, activation intensity, and intermediate layer channel weights. By analyzing the decision path, we can identify the data components that the model prioritizes during inference, thus explaining the reasons behind the model's output.

[0030] Predictive explanatory information is constructed from the contribution values ​​of input features, decision paths, and metadata about data sources. It provides the source chain of the input data, the model's inference logic, and the impact of input features on the prediction results, making the prediction process understandable. The structure of predictive explanatory information can include contribution value ranking information, a visual representation of the decision path, and a source chain index, ensuring the results are verifiable and reproducible.

[0031] This embodiment performs predictive processing on the input data and generates predictive explanation information, so that the model output not only includes the prediction results, but also the degree of influence of input features and the decision logic chain. Combined with the metadata of the data source, a complete explanation link is formed, thereby improving the transparency and traceability of the model behavior, reducing the compliance risks caused by black box reasoning, and providing accurate basic data for subsequent attribution analysis.

[0032] S50, Attribution analysis is performed based on the training evidence chain, the prediction results, and the prediction explanation information to generate an attribution report.

[0033] In this embodiment, the training evidence chain is a set of records used to represent the data sources, processing procedures, and influence structures during the training phase. It typically includes the contribution values ​​of training samples, preprocessing information, and the source chain, reflecting which data plays a crucial role in the model's final behavior during the training phase. The construction of the training evidence chain is synchronized with model training, thus possessing temporal characteristics and traceability directly related to model behavior.

[0034] Prediction results are the output of a trained model after it has completed the inference process upon receiving input data. They can include classification labels, ratings, trend directions, probability distributions, or other structured predictions. Prediction results play a crucial role in attribution analysis, determining whether the model's behavior deviates from expectations under current input conditions or whether abnormal patterns exist.

[0035] Predictive explanatory information is a set of explanations used to describe the model's decision-making process and the degree of influence of input features. It typically includes the contribution value of input features, the decision path, and information on data sources. Predictive explanatory information can reveal the model's reasoning logic, enabling attribution analysis to identify the causes of behavior based on the model's internal decision-making criteria.

[0036] Attribution analysis is the process of comprehensively judging the source and responsibility of output behavior based on the training evidence chain, prediction results, and prediction explanations. Attribution analysis requires examining model behavior from multiple perspectives, including influencing factors during training, the decision-making chain during inference, and the logical consistency of the results. To implement attribution analysis, it is necessary to identify key influencing samples in the training data, input features with significant weight contributions in the prediction explanations, and anomalous decision-making patterns exhibited in the prediction results.

[0037] Key impact samples are data units in the training samples that have a high impact on model parameters or model behavior. They can be identified based on the contribution value of the training samples, changes in parameter sensitivity, or the impact records of training epochs. Key impact samples can reveal the fundamental factors that cause model deviations during the training phase.

[0038] Key input features are the features in the input data that play a major role in the formation of the prediction results. These features can be identified by their contribution values ​​in the prediction explanation information, which further supports the traceability of the results.

[0039] Anomaly decision-making patterns are characteristics of prediction results that do not conform to normal reasoning logic, such as abnormally concentrated feature contributions, decision paths deviating from the expected model structure, abnormally drastic weight distribution, or inconsistencies with historical prediction behavior. Identification of anomaly decision-making patterns can be based on weight change analysis, indicator threshold determination, or behavioral trajectory comparison.

[0040] The source of responsibility is the root cause of behavior inferred from key impact samples, key input features, and anomalous decision-making patterns. This can be due to factors such as training data bias, abnormal input features, model parameter deviations, or errors in the source chain. Determining the source of responsibility requires establishing correlations between the evidence chains to ensure logical consistency in the attribution process.

[0041] Attribution reports are structured presentations of attribution analysis results, including information on the identification of key impact samples, the weighting of key input features, the determination of abnormal patterns, and a description of the source of responsibility. They are used to support audits, corrective actions, or compliance checks. Attribution reports must be structured, traceable, and verifiable to facilitate subsequent automated analysis or manual review.

[0042] This embodiment combines the training evidence chain, prediction results, and prediction explanations to perform attribution analysis. It comprehensively assesses the causes of prediction behavior from three dimensions: the training phase, the influence of input features, and reasoning behavior, achieving traceability of the result's source and identifiability of behavioral responsibility. The generation of attribution reports makes the model's behavior inspectable and verifiable, supporting subsequent auditing and governance efforts, and improving the model's transparency, reliability, and regulatory adaptability in real-world business environments.

[0043] In one embodiment, step S10 above includes: S101, Receive multi-source data from multiple data sources, and perform source identification processing on each piece of multi-source data to determine the data source corresponding to the multi-source data; S102, generate a timestamp at the time point when the multi-source data is received, and establish a binding relationship between the timestamp and the corresponding multi-source data; S103 records the history of data processing steps that each piece of multi-source data has undergone before collection; S104, combine the data source, the timestamp, and the data processing step history to form data source metadata, and establish an index mapping relationship between the data source metadata and the corresponding multi-source data.

[0044] In this embodiment, the acquisition of multi-source data first relies on a unified data access capability. Multiple data sources can be internal business systems, business databases, log collection platforms, external service provider interfaces, or terminal device reporting channels. Different data sources often use different transmission protocols, field structures, and encoding methods. The data acquisition component, by configuring data source identifiers, access addresses, authentication information, and field mapping relationships, aggregates data streams from multiple data sources into the same access channel. Each piece of multi-source data entering the access channel is treated as an independent record upon entry, and all subsequent identification, recording, and combination actions revolve around this single record.

[0045] Source identification processing is performed on each piece of multi-source data to determine the data's source attribute. Source identification can be based on source fields carried in the data packet, such as business system identifiers, device identifiers, and service identifiers, or it can deduce source information through the mapping relationship between the access channel and the data source configuration. In implementation, source identification processing parses the fields of the multi-source data, prioritizing the presence of explicit source fields; if no explicit field exists, the source identifier is determined based on the access channel marker, connection parameters, or authentication credentials. The result of source identification is appended to the multi-source data record in the form of a data source, ensuring that the origin of each piece of multi-source data can always be traced during subsequent training and analysis.

[0046] Timestamp generation and multi-source data binding are used to record the time information of data entering the acquisition terminal. The timestamp is generated by the time service the instant the multi-source data arrives at the access terminal. The time service can maintain time accuracy based on a unified time synchronization mechanism to avoid time offsets between different nodes affecting timing judgment. The generated timestamp is then bound to the corresponding multi-source data. This binding can be done through field embedding, adding a timestamp field to the multi-source data record, or through external association, recording the relationship between the timestamp and the multi-source data identifier in a key-value mapping within an independent metadata structure. This binding action allows for precise location of the time when multi-source data enters the system during the training or accountability phases, thereby supporting event sequence analysis and data freshness assessment.

[0047] The data processing step history records the processing steps each piece of multi-source data has undergone before collection. Before reaching the collection end, data typically undergoes some processing upstream, such as format conversion, field cleaning, missing value imputation, field aggregation, or encryption / masking. These operations alter the data content or structure; without recording, subsequent analysis would struggle to determine the differences between the current and original data forms. The data processing step history can be extracted from processing logs attached to upstream systems or embedded with processing tags in the data packets by the reporting end. When receiving multi-source data, the collection end parses these processing records, organizing them into an ordered list according to time or dependency order, forming the corresponding data processing step history for each multi-source data. Each processing record includes at least a processing type identifier, an execution location identifier, and a processing order location to ensure complete reconstruction of the data processing chain.

[0048] Data source, timestamp, and data processing history need to be structurally combined when forming data source metadata. During this combination, the data source serves as the generator identifier, the timestamp as the time dimension identifier, and the data processing history as the processing link identifier; together, these three constitute the complete data source metadata. Data source metadata can be expressed using key-value structures, document structures, or table structures, as long as it supports retrieval and association by source, time, and processing link dimensions. After combination, an index mapping relationship needs to be established between the data source metadata and the corresponding multi-source data. The index mapping can use the unique identifier of the multi-source data as the key and the data source metadata record as the value, allowing for quick retrieval of the corresponding data source metadata based on training samples or input data during the training and attribution phases. The index mapping relationship can also be used in reverse, i.e., retrieving all corresponding multi-source data starting from the data source metadata to support batch analysis by source or processing link.

[0049] This embodiment introduces a joint record of data source, timestamp, and data processing step history during the multi-source data acquisition stage, and performs structured combination and index management in the form of data source metadata. This allows for the explicit construction of the source and processing links before the data enters the training and prediction process. This enables precise tracking of data source, arrival time, and processing history during subsequent training evidence chain generation and attribution analysis, reducing explanation gaps caused by unclear data sources or lack of processing records, and improving the transparency and verifiability of the data-driven decision-making process.

[0050] In one embodiment, step S20 above includes: S201, combine the multi-source data with the corresponding data source metadata to form a training sample with source identifier; S202, the model is iteratively trained using the training samples, and during the training process, the degree of influence of each training sample on the model is determined through contribution analysis, and the contribution value of the training samples is generated. S203, during the iterative training process, the normalization processing step, the missing value imputation processing step, and the feature engineering transformation step performed on the training samples are recorded; S204, based on the training sample contribution value, normalization processing step, missing value filling processing step, and feature engineering transformation step, a training evidence chain containing preprocessed historical information is generated.

[0051] In this embodiment, before multi-source data participates in training, it is necessary to structurally combine the multi-source data with data source metadata to generate training samples with source identifiers. Multi-source data carries original business information or observation information, while data source metadata includes data source, time records, pre-processing procedures, etc. The combination of the two can be achieved by introducing a source identifier area, a time attribute area, and a pre-processing description area into the training sample structure. For example, in addition to feature vectors and label fields, the training sample also includes a source identifier field, a collection time field, and a pre-processing description field, so that each training sample is already bound to the corresponding data source metadata when entering the training process. This combination process is usually completed during the data loading stage. The sample construction module matches each piece of multi-source data and writes the corresponding data source metadata into a unified training sample structure to achieve a one-to-one correspondence.

[0052] During training, the model is iteratively updated using training samples with source identifiers. Iterative training can employ batch updates or streaming updates. In each round of parameter updates, the training engine reads a set of training samples and performs forward inference and error signal propagation. To characterize the impact of each training sample on the model, a contribution analysis process needs to be introduced into the training flow. Contribution analysis can quantify the impact of each training sample on the model parameters or model output in the current training stage as a training sample contribution value, based on metrics such as loss change, gradient magnitude, gradient direction consistency, or parameter update magnitude. In implementation, after a parameter update, the proportion attribution of the training samples participating in the update to the loss convergence or performance metric change can be calculated, and this proportion can be stored in the training sample contribution value record area. Alternatively, the contribution records of the same training sample can be accumulated over multiple iterations to obtain a cumulative contribution, thus reflecting the long-term impact during training.

[0053] During iterative training, various preprocessing operations are often performed on training samples, including normalization, missing value imputation, and feature engineering transformation. Normalization primarily targets numerical features, mapping values ​​of different units and ranges to a unified numerical interval. Common implementations include linear scaling, standardization mapping, or quantile mapping. Missing value imputation handles null values ​​caused by missing fields, failed data collection, or reported anomalies. This can be achieved through mean imputation, median imputation, interpolation, or imputation based on similar samples. Feature engineering transformation combines, bins, encodes, or derives original features. For example, it extracts periodic components from time fields, performs encoding mapping on text fields, and generates segmented features for continuous variables. Each normalization, missing value imputation, and feature engineering transformation changes the representation of training samples in the feature space; therefore, these operations need to be recorded during training to construct a complete preprocessing history.

[0054] The preprocessing history can be recorded in the form of an operation sequence. This sequence includes the normalization operation type, parameter configuration, and application order for each training sample before it enters the training engine; the missing value imputation strategy, imputation value source, target field range; and the rule expression, target field, and generated field of feature engineering transformation. These details are written into the preprocessing record structure in chronological order. In implementation, a preprocessing record list can be maintained for each training sample. As the sample transforms from its original form to a trainable form, a record is appended to the list after each operation. This allows for rapid reconstruction of the complete preprocessing path based on the training sample identifier after training.

[0055] The training evidence chain is constructed by comprehensively utilizing the contribution values ​​of training samples and preprocessing history information. The training evidence chain can be understood as an evidence path from training samples to model behavior, encompassing both the influence of the training sample during training and how it was processed before entering training. The training evidence chain can be formed by creating a structured record for each training sample, storing the sample identifier, data source metadata references, training sample contribution value, and preprocessing history information in the evidence record. Alternatively, a global evidence view can be constructed for the model, where training samples are nodes, contribution values ​​are weight markers, and preprocessing history information is an attribute on the edges, achieving a visual connection from training input to model behavior. Regardless of the structure used, as long as it supports retrieving training samples based on source information, filtering key training samples based on contribution values, and reconstructing the data processing flow based on preprocessing history, it constitutes an effective representation of the training evidence chain.

[0056] This embodiment generates a training evidence chain containing the contribution value of training samples and preprocessing history information based on the metadata of the data source during the training process. This enables each training sample to form a complete association in terms of data source, processing process, and training impact. Through this structured record, when the model behavior is abnormal or the prediction results are questioned, the potential causes can be accurately traced from the two dimensions of training data contribution and preprocessing process. This reduces the opacity caused by the "black box" of the training stage, improves the interpretability and auditability of the model update and application process, and provides reliable data basis for subsequent attribution analysis, accountability processing, and compliance review.

[0057] In one embodiment, step S30 above includes: S301, Receive a prediction request message from an external interface or an internal system interface, and assign a unique identifier to the prediction request message; S302, parse the prediction request message to obtain the data payload in the prediction request message; S303, verify the format and integrity of the data payload, extract input data from the verified data payload, and record the receiving time of the input data; S304, establish an association between the input data and the corresponding unique identifier.

[0058] In this embodiment, the process of receiving prediction requests uses either an external interface or an internal system interface as the entry point. External interfaces can be external service gateways, open interface platforms, third-party system call entry points, etc., while internal system interfaces can be internal business system call channels, task scheduling channels, or the consumer end of message middleware. After receiving a prediction request message, the interface access layer assigns a unique identifier to each prediction request message. This unique identifier can be generated based on a combination of timestamps, caller identifiers, sequence counters, or random entropy sources. It is used to distinguish different prediction requests in subsequent stages and to associate prediction request messages with their corresponding input data, prediction results, and explanatory information one-to-one. The generation of unique identifiers must satisfy the constraint that no conflicts occur within the same operating environment to ensure the stability of subsequent tracing links.

[0059] Prediction request messages are typically structured, consisting of a protocol header and a data payload. The parsing process parses the prediction request message to identify the calling protocol type, encoding method, and message structure layout. The parsing process extracts control information such as the sender identifier, business type marker, and version marker from the protocol header, and extracts the data payload from the message body. The data payload carries the actual business content and may contain multiple sets of fields, such as model input fields, context information fields, and additional description fields. The parsing process needs to restore the data payload to a unified internal data structure according to the preset message format description or interface agreement, providing a foundation for subsequent format validation and input data extraction.

[0060] The format and integrity validation phase performs structural and content checks on the parsed data payload. The structural check confirms whether the data payload conforms to the expected field layout, including whether fields exist, field types match, and nested structures are complete. The content check confirms whether required fields are not empty, whether enumerated field values ​​are within the agreed range, whether numeric fields are within a reasonable range, and whether text fields meet length constraints. The validation logic can be executed based on a set of field constraint rules or a data schema description language. Only data payloads that pass format and integrity validation are considered valid and proceed to the input data extraction phase. Data payloads that fail validation are recorded as exception requests and handled by the alarm or auditing module.

[0061] The input data extraction phase is executed after the data payload has passed validation. Input data is a set of fields used to drive the trained model to generate prediction results. It typically includes numerical features, categorical features, temporal features, text features, or image features, etc. These features need to be organized into a form that conforms to the model input structure requirements. The extraction process, according to the model input specifications, selects fields from the validated data payload that correspond one-to-one with the model input dimensions, and transforms these fields into an internally unified representation format, such as vectors, tensors, or key-value maps. Upon completion of extraction, the current time is requested from a time service. The received time is recorded before the input data officially enters the subsequent prediction phase, and this time information is written into the input data record for subsequent time-series analysis, latency evaluation, and audit trail.

[0062] The association between input data and unique identifiers is maintained through an index structure or mapping structure. Hash mapping, key-value stores, or relational tables can be used, with the unique identifier as the primary key and the storage location, receiving time, and source interface information of the input data written as association fields into the mapping record. Once the association is established, the prediction result generation stage can use the same unique identifier to align the results with the input data; the interpretation generation stage can use the unique identifier to trace back the input data and source information; and the attribution analysis and audit generation stages can simultaneously access the input data, prediction results, and interpretation content on the same path, thus constructing a complete event tracing path.

[0063] This embodiment performs unified access, structure parsing, format and integrity verification, input data extraction, and association management with unique identifiers on prediction request messages. This ensures that input data has clear source markings and time records from the moment it enters the system, establishing a stable one-to-one correspondence between input data and prediction requests. This approach prevents data with abnormal formats or missing fields from entering the model inference stage, improving the reliability of the prediction process. Furthermore, it provides a foundation for establishing a precise traceability link between prediction results and subsequent explanatory and attribution analysis results, making the entire request processing chain traceable and auditable. This facilitates behavioral accountability and compliance auditing in complex business environments.

[0064] In one embodiment, step S40 above includes: S401, Input the input data into the trained model to obtain the prediction result; S402, Extract the input features that form the basis of the model's decision from the input data; S403, based on the data source metadata, trace the data source information of the input data; S404, The feature contribution analysis module is used to determine the contribution value of each input feature to the prediction result; S405, Analyze the decision-making process of the model and extract the feature paths that play a key role in the prediction results during the decision-making process as decision paths; S406, Based on the input feature contribution value, the decision path, and the data source information, generate prediction explanation information.

[0065] In this embodiment, when input data enters the prediction stage, it first needs to establish a connection with the model that has completed parameter convergence during the training phase. The trained model can be a tree-structured model, a linear model, a deep neural network model, or a multi-model ensemble structure, all uniformly exposed as an inference interface. The process of inputting input data into the trained model includes two parts: first, organizing the input data into vectors, matrices, or tensors according to the model's expected format and performing type conversion and dimension alignment; second, triggering the model's forward inference operation to obtain the prediction result. The prediction result can be represented as a classification label, probability distribution, risk score, or continuous numerical value, the specific form depending on the business scenario and model structure, but in the interpretation stage, it is all abstracted into a set of output variables that can establish a mapping relationship with the input features.

[0066] Extracting the input features that form the basis of the model's decision-making from the input data is the process of establishing a clear correspondence between the original request data and the model's internal input space. Input data typically contains multiple fields, some of which serve as model input features, while others serve as contextual or audit information. The parsing process, based on the model input configuration or feature dictionary, selects the fields that participate in the prediction calculation and maps these fields to the various dimensions of the model input vector according to a preset order or naming convention. For example, in structured data scenarios, fields such as age, region, historical behavior frequency, and product category can be mapped to numerical features or category-coded features; in text or image scenarios, text-coded vectors or image feature vectors can be considered as input feature groups. The parsing results form a clear list of input features for subsequent contribution value analysis and decision path analysis.

[0067] The metadata for the data source originates from the preceding data acquisition stage and includes information such as data source, time information, and processing history. When tracing the data source information of input data based on this metadata, it's necessary to use the input data's identifier or index as a connection point to retrieve the corresponding record in the metadata storage structure, thereby reconstructing the source path and processing of the input data during the acquisition stage. In practice, this can be achieved by querying the index mapping relationship between input data and data source metadata, referencing the source system identifier, acquisition time record, and previous processing links into the current interpretation context. This way, when analyzing the prediction results, we can not only focus on the structure and values ​​of the input features themselves but also, in conjunction with the data source dimension, determine whether the prediction behavior is related to a specific source, a specific time period, or a specific processing path.

[0068] Feature contribution analysis measures the degree of influence of a single input feature on the prediction result. Feature contribution analysis can be implemented based on mechanisms such as perturbation analysis, gradient analysis, or decomposition analysis. For example, it involves constructing feature perturbations on the numerical input and observing the changes in the prediction result; using gradient information in a differentiable structure to estimate sensitivity; or decomposing the prediction output, breaking down the overall output into the sum of the contributions of each feature. In implementation, the target input feature needs to be varied while keeping other input features constant or subject to controlled changes, and the changes in the prediction result are recorded. The relationship between the amount of change and the baseline output forms the feature contribution value. This process is performed on all input features, generating a set of input feature contribution value vectors, establishing a one-to-many influence relationship with the prediction result, and serving as part of the explanatory information.

[0069] Analyzing the decision-making process of a prediction model and extracting key feature paths involves identifying influencing links within the model's internal structure or logical execution trajectory. For tree-structured models, each feature splitting condition and corresponding threshold can be recorded along the path from the root node to the leaf node, and these condition sequences can be considered as feature paths. In deep network structures, feature pathways highly correlated with the prediction results can be identified through activation intensity, attention weights, or intermediate layer feature selection strategies, and transformed into readable decision path expressions. For example, in attention structures, the distribution of attention weights that contribute significantly to the output can be collected, and the input feature combinations corresponding to high-weight regions can be considered as key paths; in rule-based structures, rule-triggered links can be directly recorded. Through this process, the computational trajectory within a complex model can be abstracted into one or more feature paths, providing structured support for interpreting information.

[0070] When generating predictive explanations, it's necessary to integrate input feature contribution values, decision paths, and data source information. After completing feature contribution analysis and decision path extraction, the explanation generation component associates the contribution value of each input feature with its position in the decision path, while also adding data source dimension information, such as the source system category, collection time interval, and references to previous processing links. Integration can employ a hierarchical structure: placing the prediction results at the top layer, key input features and their contribution values ​​at the second layer, corresponding decision path segments at the third layer, and data source identifiers and processing history references at the fourth layer. Alternatively, a tabular structure can be used, recording the input feature name, contribution value, path position, and source information in a single record. The resulting predictive explanations not only explain the reasons for the prediction results from a feature value perspective but also demonstrate the internal logic from a decision path perspective and reveal the business entities and temporal context behind the data from a source dimension perspective, providing complete and verifiable explanations for subsequent attribution analysis and audit verification.

[0071] This embodiment obtains prediction results by inputting input data into a trained model during the prediction phase. Simultaneously, it analyzes input features, traces data source information, calculates feature contribution values, and extracts decision paths. Ultimately, it generates prediction explanation information containing input feature contribution values, decision path structure, and data source information. This allows for simultaneous coverage of result, feature, and source dimensions within a single processing loop, enabling multi-faceted explanations of a single prediction. This ensures that the model output goes beyond simple numerical results, providing a complete causal explanation. Furthermore, it offers a directly usable explanatory foundation for subsequent attribution analysis and auditing, clearly revealing the relationships between data sources, feature influences, and internal decision logic. This enhances the transparency and accountability of model behavior in real-world business environments.

[0072] In one embodiment, step S50 above includes: S501, Analyze the training evidence chain, identify key influencing samples in the training samples, and obtain the key influencing sample identification results; S502, Analyze the prediction explanation information, identify the input features that make key contributions to the prediction results, and obtain the key input feature identification results; S503, Based on the prediction results, identify abnormal decision-making patterns and obtain abnormal decision-making pattern identification results; S504, Based on the key impact sample identification results, key input feature identification results, and abnormal decision-making pattern identification results, determine the source of responsibility and obtain the source of responsibility determination result; S505, Based on the key impact sample identification results, key input feature identification results, abnormal decision pattern identification results, and responsibility source determination results, an attribution report is generated.

[0073] In this embodiment, the attribution analysis stage takes the training evidence chain, prediction results, and prediction explanation information as inputs to trace the influence path from training data to a single prediction behavior from a unified attribution perspective. The training evidence chain records the source information of training samples, preprocessing history, and contribution values ​​of training samples. The prediction results reflect the model's output behavior on the current input data, and the prediction explanation information includes the input feature contribution values ​​and decision path, among other things. Combining these three elements allows for simultaneous analysis from the data source layer, feature influence layer, and decision behavior layer, identifying key factors affecting prediction behavior and providing a basis for determining subsequent responsibility.

[0074] The analysis of the training evidence chain begins with the training sample set, requiring the identification of key influencing samples. Key influencing samples refer to those that significantly impact model parameter adjustments during training, or whose contribution value in the training evidence chain is significantly higher than other samples. This can be achieved by ranking the contribution values ​​of the training samples and selecting the set of samples with high contribution values, or by combining data source dimensions to identify sample groups with high contribution concentration from certain sources or within certain time intervals. Simultaneously, it's necessary to consider preprocessing history information to determine whether these samples underwent special preprocessing procedures, such as imputation of extreme missing values ​​or transformation of anomalous features. In this way, a key influencing sample identification result can be generated, recording the key influencing sample's identifier, source information, preprocessing chain summary, and contribution index, providing a data foundation for subsequent attribution analysis.

[0075] The analysis of predictive explanatory information focuses on the influence relationship between input features and prediction results, requiring the identification of input features that make key contributions to the prediction outcome. Predictive explanatory information generally already includes the contribution value and decision path information for each input feature. Based on this, the contribution values ​​of input features can be ranked, and a group of features with contribution values ​​significantly higher than the background level can be selected as candidate key features. Simultaneously, features with high frequency of occurrence and high threshold sensitivity in the decision path can be combined to filter and cross-validate the candidate set, thereby obtaining the key input feature identification results. This identification result not only records the feature name and contribution value but also the position of these features in the decision path, their combination relationships with other features, and the corresponding data source information, facilitating subsequent retrieval of the performance of these features during the training phase within the training evidence chain.

[0076] When identifying anomalous decision-making patterns based on prediction results, it is necessary to analyze the model's performance on the current or a series of prediction requests from the perspective of output behavior. Anomalous decision-making patterns may include output values ​​significantly deviating from historical distributions, output categories concentrated in a few categories, and excessive rejection or approval for specific customer groups. The identification process can compare the prediction results with historical statistical baselines to observe whether the current prediction results show abnormal deviations within the same business type, region, or time interval. It can also combine external feedback annotation information to detect whether the deviation between the prediction results and the actual results is concentrated in certain patterns. For example, in risk scoring scenarios, there may be a phenomenon of consistently giving high scores to a certain type of input. This phenomenon can be classified as an anomalous decision-making pattern identification result, accompanied by pattern description information, the range of affected samples, and the corresponding time interval.

[0077] After obtaining the results of key impact sample identification, key input feature identification, and abnormal decision pattern identification, it is necessary to determine the source of responsibility based on these three results. The source of responsibility can fall at different levels, such as the data source at the data acquisition layer, the preprocessing process at the feature construction layer, the sample selection strategy at the model training stage, and the parameter configuration at the prediction service stage. When determining the source of responsibility, one can first trace the distribution of corresponding training samples in the training evidence chain based on the input data sets involved in the abnormal decision patterns, observing whether there is any overlap between the key impact samples and the abnormal prediction behavior. For example, whether the key impact samples and the abnormal decision inputs are concentrated in the same data source, the same time interval, or the same business type. Then, combined with the key input feature identification results, analyze whether these key input features exhibit outlier distributions, extreme frequencies, or abnormal preprocessing links in the key impact samples. If abnormal patterns that highly match the key input features can be observed in the key impact samples, and these patterns highly overlap with the abnormal decision patterns, then the source of responsibility can be pointed to a specific data source, a specific preprocessing process, or the sample selection strategy at the training stage. The results of determining the source of responsibility can be expressed in a structured form, recording the category of the source of responsibility, the data source involved, the set of features involved, the time interval involved, and the training process involved.

[0078] The attribution report generation process summarizes and structures the aforementioned analysis. During generation, the results of key impact sample identification, key input feature identification, abnormal decision-making pattern identification, and responsibility source determination are organized into readable and searchable attribution information units. It can generate concise attribution summaries for single prediction behaviors, directly linking a prediction result to the corresponding key input features and responsibility sources; it can also generate aggregated attribution views for batch prediction behaviors over a period of time, displaying the main influencing factors and potential responsibility concentration areas of model behavior in a specific business scenario. The attribution report can be layered according to data source, input feature, and decision-making pattern dimensions, retaining both refined sample-level tracking capabilities and supporting summary analysis from an audit or management perspective, forming a complete attribution chain from a single event to overall behavior.

[0079] This embodiment introduces a joint analysis mechanism of training evidence chain, prediction results, and prediction explanation information in the attribution process. It first identifies key influencing samples from the training evidence chain, then identifies key input features from the prediction explanation information, and simultaneously identifies abnormal decision-making patterns based on the prediction results. Based on this, it comprehensively determines the source of responsibility and generates an attribution report. This allows for a unified analysis of model behavior from three levels: data source, feature influence, and output performance. Individual prediction behaviors are no longer isolated but are embedded in the complete training and usage chain. Therefore, in the event of disputes or risk events, it can clearly indicate whether the problem may originate from training data, feature construction, or a deviation in decision-making patterns. This provides managers and auditors with directly usable evidence for assigning responsibility, significantly improving the traceability and accountability capabilities in the model application process.

[0080] In one embodiment, after step S50 above, the method further includes: S601, construct a knowledge graph that includes business entities, business relationships and business strategies; S602, Map the prediction explanation information in the attribution report to the corresponding business entity in the knowledge graph; S603, map the key impact samples in the training evidence chain to the corresponding business entities in the knowledge graph; S604, based on the business strategy in the knowledge graph, verify the consistency between the predicted explanation information and the knowledge graph, and obtain the first consistency verification result; S605, based on the business strategy in the knowledge graph, verifies the consistency between the training evidence chain and the knowledge graph, and obtains the second consistency verification result; S606, detect semantic conflicts between predictive explanation information and business strategies, and obtain semantic conflict detection results; S607, Based on the attribution report, the first consistency check result, the second consistency check result, and the semantic conflict detection result, an audit report is generated.

[0081] In this embodiment, after the attribution analysis is completed, when introducing a knowledge graph for constraint inspection and audit output, a graph-structured data space covering the business scenario needs to be constructed first. Business entities in the knowledge graph can represent objects with independent identifiers and attribute sets in the business process, such as customers, contracts, products, policies, service items, channels, and business nodes. Each type of business entity is described through a unique identifier, attribute fields, and a set of tags. Business relationships are used to depict the connections between business entities in business processes, cash flow, information flow, and responsibility chains, such as the insurance relationship between customers and policies, the compensation relationship between policies and claims records, and the correspondence between medical service records and treatment items. Business strategies are used to express the rule-based constraints existing in business management and compliance requirements. They can be stored in the graph structure as strategy nodes, strategy edges, or constraints attached to business relationships. Content can include quota restriction strategies, access strategies, risk control strategies, medication guidelines strategies, follow-up frequency strategies, etc. When constructing a knowledge graph, entity and relationship information can be extracted from business system databases, business process configuration libraries, and policy document parsing results. Through entity alignment, relationship extraction, and strategy modeling, this information is transformed into a unified graph structure and stored in a graph database or graph computing engine.

[0082] Attribution reports typically include predictive explanations of input features, feature contribution values, decision paths, and corresponding data source information. This information needs to be linked to business entities in the knowledge graph to create a visualized business context. During the mapping process, entity identifiers and business features, such as customer identifiers, policy numbers, medical record numbers, product codes, and channel codes, can be parsed from the predictive explanations. Then, business entity nodes matching these identifiers are retrieved from the knowledge graph. If the predictive explanations only contain indirect features, such as age groups, regional groups, or product category groups, these features can be mapped to multiple business entity sets using a reverse index from features to entity sets. After mapping, explanation tags or associated explanation records are added to the corresponding business entities in the knowledge graph. This allows the same business entity to be associated with both the original business data and the predictive explanations generated by the model, providing a unified view of the business behaviors and model behaviors experienced by a given entity from a graph perspective.

[0083] Key impact samples in the training evidence chain also need to be mapped to business entities in the knowledge graph to enable business-dimensional auditing of training-phase behavior. Key impact samples already possess information such as sample identifier, source system identifier, collection time, and preprocessing history within the training evidence chain. The sample identifier can often be traced back to original business records, such as customer events, claims records, and medical records. During mapping, foreign key relationships or associated fields between samples and business records can be used to map key impact samples to business entity nodes in the knowledge graph. For a key impact sample, if it is associated with multiple business entities, such as both customer and product entities, connections can be established between the sample entity and these business entities in the graph. This records the degree of influence of the sample on model parameter adjustments during the training phase, thus demonstrating the contribution of a business entity to model behavior during the training phase at the graph level.

[0084] When performing consistency checks between predictive explanations and the knowledge graph based on business strategies within the knowledge graph, the business strategies need to be treated as a set of constraints or logical expressions. The checks must then examine whether the explanations violate these constraints. Business strategy nodes or the policy constraints attached to relationships typically include conditions and expected behaviors, such as "a certain type of product should not be automatically approved for a specific population," "a certain type of drug combination requires manual review," and "a certain level of risk scoring must trigger manual confirmation." During consistency checks, the decision path, key input features, and prediction results in the predictive explanations can be matched against these policy conditions. If a discrepancy is found between the explanation and the expected behavior in the strategy, the inconsistency entry is recorded in the first consistency check result, including the associated business entity, the triggered policy identifier, and the inconsistent expression fragment. If the explanation fully complies with the policy constraints, the pass status is recorded in the result, providing support for subsequent audit reports.

[0085] For consistency verification between the training evidence chain and the knowledge graph, it is necessary to pay attention to whether the data usage behavior during the training phase complies with business policy requirements. For example, some business policies restrict the training data from containing records of specific groups, specific time periods, or specific business states, or require special processing of certain sensitive features. During verification, the mapping results of key impact samples in the knowledge graph can be used to query the corresponding business entities and business relationships of these samples, and check whether they fall within the scope of policy prohibitions, and whether necessary de-identification or normalization processing has been omitted. If it is found that key impact samples are concentrated in a certain type of data segment that should not be used as the main reference, or violate a certain business policy, the corresponding sample set, business entity set, and the violated policy item should be recorded in the second consistency verification result, thereby pointing out potential compliance risks in the training phase.

[0086] Semantic conflict detection between predictive explanatory information and business strategy focuses more on understanding and comparison at the natural language level. Predictive explanatory information, when presented to reviewers, is often in text form, explaining reasons for decisions such as "overestimating the risk level of a certain group of people" or "underestimating the risk of a certain treatment plan." Business strategy documents also contain a large amount of textual expression, such as "adverse decisions should not be made against a certain type of customer based on a single factor" or "claims should not be automatically rejected for treatment plans for specific disease types." Semantic conflict detection can employ keyword matching, pattern matching, or semantic vector matching mechanisms to map the predictive explanatory text and business strategy text into a unified semantic representation space. Similarity calculations and semantic direction judgments determine whether there is a tendency for conflict between the two. For example, if the explanatory text expresses "increasing risk assessment solely due to residential area," while the strategy text explicitly states "adverse decisions should not be made solely based on regional factors," this can be identified as a semantic conflict. The conflicting paragraphs and related entities are recorded in the semantic conflict detection results, providing auditors with intuitive review clues.

[0087] When generating audit reports, attribution reports, consistency verification results, and semantic conflict detection results need to be integrated. Attribution reports provide the impact chain from training data to predicted behavior; consistency verification results provide information on compliance with business strategies and knowledge graph structures; and semantic conflict detection results provide markers of potential violations at the textual level. During integration, predicted events or business entities can be used as index units to aggregate relevant attribution information, strategy verification information, and semantic conflict information into a single audit entry. Each entry provides a behavioral description, data source description, model explanation summary, strategy matching results, a list of inconsistencies, and suggested directions for manual review. Audit reports can be filtered and summarized by time, business line, and model version, allowing regulators and internal managers to quickly grasp the overall compliance status and delve into details at the specific event level, providing a basis for subsequent rectification, strategy adjustments, or model updates.

[0088] This embodiment introduces a knowledge graph containing business entities, business relationships, and business strategies after attribution analysis. It maps predictive explanation information and training evidence chains uniformly to a business graph space, and performs consistency checks and semantic conflict detection based on the business strategies in the graph. An audit report is then generated based on this, elevating explanatory information that previously only existed at the data and model layers to a unified perspective at the business and compliance levels. On the one hand, it clarifies the correspondence between model behavior and actual business objects and processes; on the other hand, it automatically identifies inconsistencies with existing business strategies or semantic conflicts. Thus, while maintaining the model's explanatory power, it introduces structured, visualized, and searchable audit support, placing model training and predictive behavior within an auditable business context. This facilitates the construction of stable responsibility chains and compliance supervision mechanisms in complex business environments.

[0089] In one embodiment, a model behavior interpretability attribution apparatus is provided, which corresponds one-to-one with the model behavior interpretability attribution method described in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the model behavior interpretability attribution device of the present invention. The modules include: data acquisition and tracing module 10, training evidence chain construction module 20, prediction request parsing module 30, prediction and interpretation generation module 40, and attribution analysis and report generation module 50. Detailed descriptions of each functional module are as follows: The data acquisition and traceability module 10 is used to acquire multi-source data and generate data source metadata that includes data source, timestamp, and processing steps; The training evidence chain construction module 20 is used to generate a training evidence chain containing the contribution value of training samples and preprocessing information based on the metadata of the data sources during the process of training the model using the multi-source data. The prediction request parsing module 30 is used to receive prediction requests and extract input data from the prediction requests. The prediction and explanation generation module 40 is used to predict the input data using the trained model, obtain the prediction result, and generate prediction explanation information based on the metadata of the data source by analyzing the contribution value of the input features and the decision path. The attribution analysis and report generation module 50 is used to perform attribution analysis based on the training evidence chain, the prediction results, and the prediction explanation information, and generate an attribution report.

[0090] In one embodiment, the data acquisition and tracing module 10 is specifically used for: Receive multi-source data from multiple data sources, and perform source identification processing on each piece of multi-source data to determine the data source corresponding to the multi-source data; A timestamp is generated at the time point when the multi-source data is received, and a binding relationship is established between the timestamp and the corresponding multi-source data. Record the history of data processing steps that each piece of multi-source data has undergone before collection; The data source, the timestamp, and the history of data processing steps are combined to form data source metadata, and an index mapping relationship is established between the data source metadata and the corresponding multi-source data.

[0091] In one embodiment, the training evidence chain construction module 20 is specifically used for: The multi-source data is combined with the corresponding data source metadata to form training samples with source identifiers; The model is iteratively trained using the training samples, and during the training process, the degree of influence of each training sample on the model is determined through contribution analysis to generate the contribution value of the training samples. During the iterative training process, the normalization processing steps, missing value imputation processing steps, and feature engineering transformation steps performed on the training samples are recorded. Based on the training sample contribution value, normalization processing step, missing value imputation processing step, and feature engineering transformation step, a training evidence chain containing preprocessed historical information is generated.

[0092] In one embodiment, the prediction request parsing module 30 is specifically used for: Receive prediction request messages from external interfaces or internal system interfaces, and assign a unique identifier to the prediction request message; The prediction request message is parsed to obtain the data payload in the prediction request message; Verify the format and integrity of the data payload, extract input data from the verified data payload, and record the time point of receiving the input data; The input data is associated with the corresponding unique identifier.

[0093] In one embodiment, the prediction and interpretation generation module 40 is specifically used for: The input data is fed into the trained model to obtain the prediction result; Extract the input features that form the basis of the model's decision-making from the input data; Based on the data source metadata, trace the data source information of the input data; The feature contribution analysis module is used to determine the contribution value of each input feature to the prediction result; Analyze the decision-making process of the model and extract the feature paths that play a key role in the prediction results as decision paths; Based on the input feature contribution value, the decision path, and the data source information, predictive explanation information is generated.

[0094] In one embodiment, the attribution analysis and report generation module 50 is specifically used for: Analyze the training evidence chain to identify key influencing samples in the training samples and obtain the key influencing sample identification results; Analyze the prediction explanation information, identify the input features that make key contributions to the prediction results, and obtain the key input feature identification results; Based on the prediction results, abnormal decision-making patterns are identified, and abnormal decision-making pattern identification results are obtained. Based on the identification results of key impact samples, key input feature identification results, and abnormal decision-making pattern identification results, the source of responsibility is determined, and the source of responsibility determination result is obtained. Based on the results of the identification of key impact samples, key input features, abnormal decision-making patterns, and determination of the source of responsibility, an attribution report is generated.

[0095] In one embodiment, the attribution analysis and report generation module 50 is specifically used for: Construct a knowledge graph that includes business entities, business relationships, and business strategies; Map the predictive explanation information in the attribution report to the corresponding business entities in the knowledge graph; Map the key impact samples in the training evidence chain to the corresponding business entities in the knowledge graph; Based on the business strategies in the knowledge graph, the consistency between the predicted explanation information and the knowledge graph is verified to obtain the first consistency verification result. Based on the business strategies in the knowledge graph, the consistency between the training evidence chain and the knowledge graph is verified to obtain the second consistency verification result. Detect semantic conflicts between predictive explanation information and business strategies, and obtain semantic conflict detection results; An audit report is generated based on the attribution report, the first consistency check result, the second consistency check result, and the semantic conflict detection result.

[0096] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a model behavior interpretability attribution method server-side.

[0097] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements functions or steps on the client side of a model behavior interpretability attribution method.

[0098] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Collect data from multiple sources and generate data source metadata that includes data source, timestamp, and processing steps; During the process of training the model using the multi-source data, a training evidence chain containing the contribution value of training samples and preprocessing information is generated based on the metadata of the data sources. Receive a prediction request and extract input data from the prediction request; The trained model is used to predict the input data to obtain prediction results. Based on the metadata of the data source, prediction explanation information is generated by analyzing the contribution value of the input features and the decision path. Attribution analysis is performed based on the training evidence chain, the prediction results, and the prediction explanation information to generate an attribution report.

[0099] In one embodiment, a non-volatile computer-readable storage medium is provided, which may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, it performs the following steps: Collect data from multiple sources and generate data source metadata that includes data source, timestamp, and processing steps; During the process of training the model using the multi-source data, a training evidence chain containing the contribution value of training samples and preprocessing information is generated based on the metadata of the data sources. Receive a prediction request and extract input data from the prediction request; The trained model is used to predict the input data to obtain prediction results. Based on the metadata of the data source, prediction explanation information is generated by analyzing the contribution value of the input features and the decision path. Attribution analysis is performed based on the training evidence chain, the prediction results, and the prediction explanation information to generate an attribution report.

[0100] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0101] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0102] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

[0103] The user personal information involved in this application embodiment is all authorized (knowing and consenting) by the relevant parties or fully authorized by all parties, and the executing entity can obtain it through various open, legal and compliant means. The collection, storage, use, processing, transmission, provision and disclosure of the information, data and signals involved all comply with the relevant laws and regulations of the relevant countries and regions, and do not violate public order and good morals.

Claims

1. A method for interpretable attribution of model behavior, characterized in that, Includes the following steps: Collect data from multiple sources and generate data source metadata that includes data source, timestamp, and processing steps; During the process of training the model using the multi-source data, a training evidence chain containing the contribution value of training samples and preprocessing information is generated based on the metadata of the data sources. Receive a prediction request and extract input data from the prediction request; The trained model is used to predict the input data to obtain prediction results. Based on the metadata of the data source, prediction explanation information is generated by analyzing the contribution value of the input features and the decision path. Attribution analysis is performed based on the training evidence chain, the prediction results, and the prediction explanation information to generate an attribution report.

2. The model behavior interpretability attribution method as described in claim 1, characterized in that, Collect data from multiple sources and generate data source metadata that includes data source, timestamp, and processing steps, including: Receive multi-source data from multiple data sources, and perform source identification processing on each piece of multi-source data to determine the data source corresponding to the multi-source data; A timestamp is generated at the time point when the multi-source data is received, and a binding relationship is established between the timestamp and the corresponding multi-source data. Record the history of data processing steps that each piece of multi-source data has undergone before collection; The data source, the timestamp, and the history of data processing steps are combined to form data source metadata, and an index mapping relationship is established between the data source metadata and the corresponding multi-source data.

3. The model behavior interpretability attribution method as described in claim 1, characterized in that, During the process of training the model using the multi-source data, a training evidence chain containing the contribution values ​​of training samples and preprocessing information is generated based on the metadata of the data sources, including: The multi-source data is combined with the corresponding data source metadata to form training samples with source identifiers; The model is iteratively trained using the training samples, and during the training process, the degree of influence of each training sample on the model is determined through contribution analysis to generate the contribution value of the training samples. During the iterative training process, the normalization processing steps, missing value imputation processing steps, and feature engineering transformation steps performed on the training samples are recorded. Based on the training sample contribution value, normalization processing step, missing value imputation processing step, and feature engineering transformation step, a training evidence chain containing preprocessed historical information is generated.

4. The model behavior interpretability attribution method as described in claim 1, characterized in that, Receive a prediction request and extract input data from the prediction request, including: Receive prediction request messages from external interfaces or internal system interfaces, and assign a unique identifier to the prediction request message; The prediction request message is parsed to obtain the data payload in the prediction request message; Verify the format and integrity of the data payload, extract input data from the verified data payload, and record the time point of receiving the input data; The input data is associated with the corresponding unique identifier.

5. The model behavior interpretability attribution method as described in claim 1, characterized in that, The trained model is used to predict the input data to obtain prediction results. Based on the metadata of the data source, prediction explanation information is generated by analyzing the contribution values ​​of input features and decision paths, including: The input data is fed into the trained model to obtain the prediction result; Extract the input features that form the basis of the model's decision-making from the input data; Based on the data source metadata, trace the data source information of the input data; The feature contribution analysis module is used to determine the contribution value of each input feature to the prediction result; Analyze the decision-making process of the model and extract the feature paths that play a key role in the prediction results as decision paths; Based on the input feature contribution value, the decision path, and the data source information, predictive explanation information is generated.

6. The model behavior interpretability attribution method as described in claim 1, characterized in that, Attribution analysis is performed based on the training evidence chain, the prediction results, and the prediction explanation information to generate an attribution report, including: Analyze the training evidence chain to identify key influencing samples in the training samples and obtain the key influencing sample identification results; Analyze the prediction explanation information, identify the input features that make key contributions to the prediction results, and obtain the key input feature identification results; Based on the prediction results, abnormal decision-making patterns are identified, and abnormal decision-making pattern identification results are obtained. Based on the identification results of key impact samples, key input feature identification results, and abnormal decision-making pattern identification results, the source of responsibility is determined, and the source of responsibility determination result is obtained. Based on the results of the identification of key impact samples, key input features, abnormal decision-making patterns, and determination of the source of responsibility, an attribution report is generated.

7. The model behavior interpretability attribution method as described in claim 1, characterized in that, After performing attribution analysis based on the training evidence chain, the prediction results, and the prediction explanation information, and generating an attribution report, the process further includes: Construct a knowledge graph that includes business entities, business relationships, and business strategies; Map the predictive explanation information in the attribution report to the corresponding business entities in the knowledge graph; Map the key impact samples in the training evidence chain to the corresponding business entities in the knowledge graph; Based on the business strategies in the knowledge graph, the consistency between the predicted explanation information and the knowledge graph is verified to obtain the first consistency verification result. Based on the business strategies in the knowledge graph, the consistency between the training evidence chain and the knowledge graph is verified to obtain the second consistency verification result. Detect semantic conflicts between predictive explanation information and business strategies, and obtain semantic conflict detection results; An audit report is generated based on the attribution report, the first consistency check result, the second consistency check result, and the semantic conflict detection result.

8. A model behavior interpretability attribution device, characterized in that, The model behavior interpretability attribution device includes: The data acquisition and tracing module is used to collect multi-source data and generate data source metadata that includes data source, timestamp, and processing steps; The training evidence chain construction module is used to generate a training evidence chain containing the contribution value of training samples and preprocessing information based on the metadata of the data sources during the process of training the model using the multi-source data. The prediction request parsing module is used to receive prediction requests and extract input data from the prediction requests; The prediction and explanation generation module is used to predict the input data using the trained model, obtain the prediction results, and generate prediction explanation information based on the metadata of the data source by analyzing the contribution value of the input features and the decision path. The attribution analysis and report generation module is used to perform attribution analysis based on the training evidence chain, the prediction results, and the prediction explanation information, and generate an attribution report.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a model behavior interpretability attribution program stored in the memory and executable on the processor, wherein the model behavior interpretability attribution program, when executed by the processor, implements the steps of the model behavior interpretability attribution method as described in any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that, The storage medium stores a model behavior interpretable attribution program, which, when executed by a processor, implements the steps of the model behavior interpretable attribution method as described in any one of claims 1-7.

Images