An AI model development whole-process management method based on a trusted execution environment

Abstract

This invention relates to the fields of Trusted Execution Environment (TEE) and Federated Learning, and discloses a method for managing the entire AI model development process based on a TEE. This method is executed within a central server enclave. By creating sliding window traceability management instances and dynamic data distribution feature profiles for each participant, incremental traceability processing is performed on streaming training data to generate window-level aggregate fingerprints. When the window is closed, the window integrity summary and window-level distribution statistics are bound together in an atomic operation to form a traceability-certified distribution snapshot. Distribution drift is detected based on the distribution snapshot sequence, and a differential anomaly detection threshold is set. After calibrating the gradient direction, an updated model result containing traceability audit records is generated according to adaptive aggregation weights. This method solves the technical problems of infinite growth of the traceability chain, distribution estimation contamination by poisoned data, and gradient misjudgment caused by distribution drift in streaming data scenarios.

Description

Technical Field

[0001] This invention relates to the fields of trusted execution environment and federated learning technology, and more specifically, to a method for managing the entire process of AI model development based on a trusted execution environment. Background Technology

[0002] In the end-to-end management scenario of AI model development based on a Trusted Execution Environment (TEE), multiple federated participant nodes continuously generate real-time streaming training data. The central server (TEE) enclave assumes full-process control responsibilities, from data source tracing and verification, data quality assessment, to gradient aggregation and model updates. In existing technical solutions, during the data tracing phase, a chain node is established for each batch of data and added to the tracing chain. During training, a complete reverse path verification from the current state to the original data is periodically performed to confirm data integrity. During the gradient aggregation phase, the central TEE enclave receives encrypted gradients uploaded by each participant, obtains a credibility score based on statistical distribution anomaly detection, and performs weighted aggregation to complete the model update.

[0003] However, under the condition that streaming data continues to arrive and the data distribution of each participant exhibits non-independent and identically distributed characteristics, the existing technology has the following problems: First, the traceability chain grows infinitely with the data stream, and the latency of complete reverse traversal verification is positively correlated with the chain length, which cannot meet the low latency requirements of online training for data verification, making the data traceability management stage a bottleneck in the entire process; Second, data traceability verification and training quality control operate as independent links, and the data distribution statistics used for gradient aggregation decisions may be calculated based on data that has not been fully verified, posing a risk of distribution estimation being contaminated by poisoned data; Third, the data distribution characteristics of each participant remain static after being collected once during the initialization phase, failing to capture distribution drift in streaming data, causing normal gradient changes caused by distribution drift to be misjudged as abnormal by a fixed anomaly detection threshold, affecting the quality of model updates. Summary of the Invention

[0004] This invention provides a method for managing the entire process of AI model development based on a trusted execution environment, which solves the technical problems in related technologies such as insufficient data traceability in federated learning scenarios, inability to effectively cope with dynamic data distribution drift, gradient aggregation weight solidification, and lack of a trusted audit mechanism for the entire process.

[0005] This invention discloses a method for managing the entire AI model development process based on a Trusted Execution Environment (TEE), executed within a central server enclave. The method includes the following steps: initializing the AI ​​model development process management controller; creating a sliding window traceability management instance and dynamic data distribution feature profile for each federated participant; allocating window size and window sliding step size to each sliding window traceability management instance; initializing the initial value of the window-level aggregated fingerprint and the active window traceability sub-chain; receiving encrypted streaming training data records from each participant and decrypting them within the TEE enclave; accumulating and updating the window-level aggregated fingerprint for data records within the active window using an incremental hash algorithm, while simultaneously generating a micro-credential and appending it to the active window traceability sub-chain; when the sliding window triggers sliding, calculating the window integrity digest for the window about to close; extracting window-level distribution statistics from the data records within the window that have passed traceability verification; and combining the window-level distribution statistics with... The window integrity summary is bound in an atomic manner to form a traceable and certified distribution snapshot; window-level distribution statistics are extracted from the traceable and certified distribution snapshot sequence of each participant, the temporal change trend of distribution statistics between adjacent windows is calculated to detect distribution drift events, the distribution distance metric between each participant and the global estimated distribution is calculated, and the dynamic data distribution feature profile is updated; encrypted gradients from each participant are received and decrypted within the TEE enclave, and differentiated anomaly detection thresholds are set for each participant based on the dynamic data distribution feature profile to filter abnormal gradients; directional calibration is performed on gradients that pass anomaly detection, the calibration intensity parameter is adjusted based on drift direction information, and a calibrated gradient is generated; adaptive aggregation weights are calculated based on the data volume, distribution distance metric, and directional calibration magnitude of each participant, and weighted aggregation is performed on the calibrated gradients to generate a global model update gradient, outputting the model update result containing traceability audit records.

[0006] Furthermore, the calculation process of the incremental hash algorithm is as follows: For the k-th data record within the active window, the feature value of the k-th data record is concatenated with the current window-level aggregated fingerprint to calculate the hash value, thereby obtaining the updated window-level aggregated fingerprint; when k equals 1, the window-level aggregated fingerprint is obtained by taking the hash seed value derived from the TEE enclave internal sealed key during initialization; the micro-credential includes the record sequence number, timestamp hash, and incremental contribution value, wherein the incremental contribution value is the independent hash value of the feature value of a single data record; while comparing the timestamp of the data record with the window sequence counter, the temporal monotonicity between the timestamp and the timestamp of the preceding data record of the same participant is verified; when timestamp disorder or timestamp jump is detected, a temporal anomaly marker containing an anomaly type identifier and an anomaly timestamp difference is generated and associated with the corresponding micro-credential.

[0007] Furthermore, the window integrity summary includes a window-level aggregate fingerprint, the number of data records within the window, time-series anomaly statistics, and a sub-chain root hash. The sub-chain root hash is the root node hash value obtained by constructing a Merkle tree using all micro-credentials in the traceability sub-chain of the soon-to-be-closed window arranged in order of record number as leaf nodes. The window-level distribution statistics include the mean vector and covariance matrix of each feature dimension. The extraction objects are data records with complete micro-credentials that have not been marked as time-series anomalies, as well as data records that have been marked as time-series anomalies but whose anomaly type is a timestamp difference deviation within a preset tolerance range. When the traceability-certified distribution snapshot is appended to the historical window summary chain, if the length of the historical window summary chain exceeds the preset historical window retention quantity K, the earliest window summary node is compressed into a simplified summary containing only the window-level aggregate fingerprint and the sub-chain root hash, and the corresponding window-level distribution statistics storage space is released.

[0008] Furthermore, the method for detecting distribution drift events is as follows: For the i-th participant, extract the window-level distribution statistics sequence of the most recent L consecutive windows, and use the KL divergence algorithm to calculate the distribution distance sequence between adjacent windows. When the mean of the most recent m values ​​in the distribution distance sequence exceeds the preset distribution drift detection sensitivity parameter, the corresponding participant is determined to be in a distribution drift state. The drift amplitude is the mean of the most recent m distribution distance values, and the drift direction is the unit vector of the difference between the mean vector of the most recent window and the mean vector of the previous window. Where L is the historical window backtracking length, m is the number of short-term windows used for drift determination, and m is less than L. The global estimated distribution is the aggregated distribution feature obtained by weighting the window-level distribution statistics of the latest windows of all participants according to the number of data records. The distribution distance metric is calculated using the KL divergence algorithm to calculate the degree of deviation between the window-level distribution statistics of the latest window of each participant and the global estimated distribution.

[0009] Furthermore, the differential anomaly detection threshold is set as follows: based on the baseline anomaly detection threshold, for participants in a distribution drift state, the differential anomaly detection threshold is expanded to the sum of the products of the baseline anomaly detection threshold, the drift amplitude, and the threshold expansion coefficient; for participants in a distribution stable state, the differential anomaly detection threshold is kept as the baseline anomaly detection threshold; the anomaly detection is performed by calculating the L2 norm of the gradient of each participant, comparing the L2 norm value with the corresponding differential anomaly detection threshold, and determining gradients with L2 norm values ​​exceeding the differential anomaly detection threshold as abnormal gradients and filtering them out.

[0010] Furthermore, for participants in a stable distribution state, a gradient history consistency check is introduced: the Euclidean distance algorithm is used to calculate the distance between the current round gradient and the average gradient of the corresponding participant in the most recent R rounds. When the distance exceeds a preset multiple of the standard deviation of the historical gradient distance of the corresponding participant, the differential anomaly detection threshold of the corresponding participant is tightened to the product of the benchmark anomaly detection threshold and the tightening coefficient. The tightening coefficient is a positive number less than 1, and R is the number of historical gradient backtracking rounds.

[0011] Furthermore, the direction calibration process is as follows: A cosine similarity algorithm is used to calculate the cosine of the angle between the gradient direction of each participant and the global gradient estimation direction. For gradients with a cosine value lower than a preset calibration threshold, they are decomposed into parallel and perpendicular components consistent with the global gradient estimation direction. The perpendicular component is attenuated according to the calibration intensity parameter and then recombined with the parallel component to obtain the calibrated gradient. For gradients with a cosine value not lower than the calibration threshold, direction calibration is not performed. The calibration intensity parameter is adjusted as follows: When a participant is in a distribution drift state, the similarity between the drift direction vector and the gradient perpendicular component direction is calculated. When the similarity is positive, the calibration intensity parameter is set to the increasing function value of the baseline calibration intensity parameter and the similarity. When the similarity is non-positive or the participant is in a stable distribution state, the calibration intensity parameter is taken as the baseline calibration intensity parameter.

[0012] Furthermore, the adaptive aggregation weights are calculated as follows: data preprocessing is performed on the data volume, distribution distance metric, and orientation calibration magnitude of each participant to obtain normalized data volume, standardized distribution distance, and standardized orientation calibration magnitude; the original weight factor of each participant is equal to the normalized data volume multiplied by the negative exponential decay factor of the distribution distance and then multiplied by the linear penalty factor of the orientation calibration magnitude, wherein the negative exponential decay factor is controlled by the distribution distance decay coefficient to control the decay rate of the standardized distribution distance, and the linear penalty factor is controlled by the orientation deviation penalty coefficient to control the penalty degree of the standardized orientation calibration magnitude; normalization processing is performed on the original weight factors to obtain adaptive aggregation weights; the orientation calibration magnitude is defined as the difference between the cosine similarity of the gradient before and after calibration and 1; the global model update gradient is the result of weighted summation of the post-calibration gradients of all participants that passed anomaly detection according to the adaptive aggregation weights.

[0013] Furthermore, the normalized adaptive aggregation weights are truncated: a maximum weight upper limit and a minimum weight lower limit are set for each participant. For participants whose adaptive aggregation weights exceed the maximum weight upper limit, their weights are truncated to the maximum weight upper limit. For participants whose adaptive aggregation weights are below the minimum weight lower limit, their weights are truncated to the minimum weight lower limit. After truncation, normalization is re-executed to ensure that the sum of all adaptive aggregation weights is 1. The source audit record includes the adaptive aggregation weights of each participant, the window identifier corresponding to the contribution data of each participant, the drift status flag, and the anomaly detection result. The source audit record is appended to the model version history chain. The model version history chain is a chain data structure stored in the TEE enclave. Each chain node includes a hash digest of the model update gradient, a source audit record, and a hash reference of the previous node.

[0014] This invention provides a full-process management system for AI model development based on a Trusted Execution Environment (TEE), comprising: an initialization module for initializing the AI ​​model development full-process management controller within a central server TEE enclave, creating a sliding window traceability management instance and dynamic data distribution feature profile for each federated participant, allocating window size and window sliding step size to each sliding window traceability management instance, and initializing the initial value of the window-level aggregate fingerprint and the active window traceability sub-chain; an incremental traceability processing module for receiving encrypted streaming training data records from each participant and decrypting them within the TEE enclave, accumulating and updating the window-level aggregate fingerprint for data records within the active window using an incremental hash algorithm, generating a micro-credential and appending it to the active window traceability sub-chain; and a distribution snapshot generation module for calculating the window integrity digest for the window about to close when the sliding window triggers sliding, extracting window-level distribution statistics from data records within the window that have passed traceability verification, and combining the window-level distribution statistics with the window integrity digest using atomic operations. The system consists of several modules: a distribution snapshot with traceable and certified binding; a distribution drift detection module, which extracts window-level distribution statistics from the traceable and certified distribution snapshot sequence of each participant, calculates the temporal change trend of distribution statistics between adjacent windows to detect distribution drift events, calculates the distribution distance metric between each participant and the globally estimated distribution, and updates the dynamic data distribution feature profile; a gradient anomaly detection module, which receives encrypted gradients from each participant and decrypts them within the TEE enclave, sets differentiated anomaly detection thresholds for each participant based on the dynamic data distribution feature profile, and filters out anomalous gradients; a gradient direction calibration module, which performs direction calibration on gradients that pass anomaly detection, adjusts the calibration intensity parameter based on drift direction information, and generates calibrated gradients; and a weighted aggregation and auditing module, which calculates adaptive aggregation weights based on the data volume, distribution distance metric, and direction calibration magnitude of each participant, performs weighted aggregation on the calibrated gradients to generate global model update gradients, and outputs model update results containing traceable audit records.

[0015] The AI ​​model development end-to-end management method based on a trusted execution environment provided by this invention solves the technical problems of excessive verification latency caused by the infinite growth of the traceability chain in streaming data scenarios, the estimation of the distribution of poisoned data contamination, and gradient misjudgment caused by distribution drift. It achieves the following technical effects: First, by replacing the full-chain appending traceability method with sliding window segmented management, the effective verification length and storage usage of the traceability chain are determined by the window size and the number of historical windows retained, and kept within a controlled range. The computational latency of the traceability verification operation is no longer positively correlated with the total amount of all historical data records, eliminating the latency bottleneck of the entire process in the data traceability management stage; Second, by executing synchronously in an atomic manner when the window is closed. The integrity summary calculation and window-level distribution statistics extraction and binding form a traceable and certified distribution snapshot, ensuring that the distribution information used in the training control phase comes only from data whose integrity has been verified. This blocks the path of poisoned data contaminating the distribution estimation without integrity verification from the entire process level. Third, by performing time series analysis on the traceable and certified distribution snapshot sequence, the data distribution change trend is continuously tracked and distribution drift events are detected. This enables the differential anomaly detection threshold and gradient direction calibration intensity parameters to be adaptively adjusted according to the distribution drift state, avoiding the misjudgment of normal gradient offsets caused by data distribution changes during distribution drift as anomalies by fixed thresholds, and maintaining the aggregation quality of model update gradients. Attached Figure Description

[0016] Figure 1 This is a flowchart of the AI ​​model development end-to-end management method based on a trusted execution environment provided in an embodiment of the present invention; Figure 2 This is a schematic diagram comparing the initialization window configuration parameters of various hospitals provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the time-series distribution of the first batch of data records for Hospital A provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the hospital D-series anomaly detection results provided in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the trend of distribution distance changes before and after the replacement of hospital equipment C, as provided in an embodiment of the present invention. Figure 6 This is a schematic diagram of the distribution distance measurement values ​​and drift states of each participant provided in the embodiments of the present invention; Figure 7 This is a schematic diagram showing the comparison of gradient anomaly detection results in the 58th round provided in an embodiment of the present invention; Figure 8 This is a schematic diagram illustrating the gradient direction calibration parameters and effect analysis provided in an embodiment of the present invention. Detailed Implementation

[0017] In the end-to-end management scenario of AI model development based on Trusted Execution Environment (TEE), multiple federated participant nodes continuously generate real-time streaming training data. The central TEE enclave server assumes the responsibility for end-to-end control from data source verification and data quality assessment to gradient aggregation and model updates. In existing technical solutions, during the data source verification phase, a chain node is established for each batch of data and appended to the source chain. During training, a complete reverse path verification from the current state to the original data is periodically performed to confirm data integrity. During the gradient aggregation phase, the central TEE enclave receives encrypted gradients uploaded by each participant, obtains a credibility score based on statistical distribution anomaly detection, and performs weighted aggregation to complete the model update. Data source management and training process management are executed separately as two independent control links in the end-to-end process.

[0018] However, under the condition that streaming data continues to arrive and the data distribution of each participant exhibits a non-independent and identically distributed characteristic, the above-mentioned end-to-end management method has the following problems: First, the traceability chain grows infinitely with the data flow, and the latency of complete reverse traversal verification is positively correlated with the chain length, which cannot meet the low latency requirements of online training for data verification, making the data traceability management stage a bottleneck in the entire process.

[0019] Second, data source tracing and verification and training quality control are operated as independent processes. The data distribution statistics used for gradient aggregation decisions may be calculated based on data that has not been verified for integrity, which poses a risk of the distribution estimation being contaminated by poisoned data.

[0020] Third, the data distribution characteristics of each participant are collected once during the initialization phase and remain static, which fails to capture the distribution drift in the streaming data. This causes normal gradient changes caused by distribution drift to be misjudged as abnormal by the fixed anomaly detection threshold, affecting the quality of model updates.

[0021] According to an embodiment of this implementation, a method for managing the entire AI model development process based on a Trusted Execution Environment (TEE) is provided. This method is executed within a central server TEE enclave. It should be understood that the hardware environment for executing this TEE-based AI model development management method includes: at least one central server equipped with TEE functionality, the TEE enclave possessing encrypted memory isolation capabilities; multiple federated participant nodes, each node having an encrypted data upload channel and establishing a remote authentication connection with the central server TEE enclave; and the central server possessing a network interface that meets the bandwidth requirements for streaming data processing and enclave memory space that meets the requirements for window-level data caching.

[0022] At least one embodiment of the present invention discloses a method for managing the entire development process of AI models based on a trusted execution environment, such as... Figure 1 As shown, it includes the following steps: Step 1: Initialize the AI ​​model to develop a full-process management controller, and generate sliding window traceability management instances and dynamic data distribution feature profiles for each participant; Within the central server TEE enclave, an AI model development end-to-end management controller is initialized. This controller retrieves a list of registered federated participants, creates a sliding window traceability management instance and a dynamic data distribution feature profile for each participant, and assigns a window size to each sliding window traceability management instance. and window sliding step Initialize the window sequence counter, the initial value of the window-level aggregated fingerprint, and the active window tracing subchain. At the same time, initialize the distribution statistics storage area, the distribution distance metric, and the drift status flag for each dynamic data distribution feature profile.

[0023] Furthermore, the aforementioned window size The sliding window step size is the maximum number of data records that can be accommodated within it. This is the data record offset relative to the starting position of the current window when the window is slid. and The value is determined based on the data upload frequency of each participant and the online training cycle. There are overlapping areas of data between adjacent windows to ensure the continuity of distribution statistics. There is no overlap between adjacent windows.

[0024] Furthermore, the initial value of the aforementioned window-level aggregated fingerprint is a preset fixed hash seed value. The fixed hash seed value is generated by derivation from the sealed key inside the enclave during the initialization of the TEE enclave, and the seed values ​​of each participant are different.

[0025] Furthermore, the aforementioned active window tracing subchain refers to the tracing record linked list structure corresponding to the window currently receiving data. The tracing record linked list structure is stored in the encrypted memory area of ​​the TEE enclave and is used to record the micro credentials of each data record within the window. When the window is closed, the contents of the active window tracing subchain will be used to calculate the window integrity digest.

[0026] Furthermore, the aforementioned distribution statistics storage area is used to store the distribution statistics of each participant's historical windows, including the mean vector and covariance matrix of each feature dimension. The capacity of the distribution statistics storage area is determined by the number of historical windows retained. The distribution distance metric is used to record the degree of deviation between the participant's current distribution and the globally estimated distribution, and the drift status flag is used to identify whether the participant is currently in a distribution drift state.

[0027] Step 2: Receive encrypted streaming training data records from each participant, perform incremental tracing processing within the active window, and generate window-level aggregate fingerprints and micro credentials; The system receives encrypted streaming training data records uploaded in real time by each participant and decrypts them within the TEE enclave. It extracts the timestamp of each data record and compares it with the window sequence counter of the corresponding participant to determine the window interval to which the data record belongs. For data records within the currently active window, it uses an incremental hash algorithm to accumulate and update the feature values ​​of new data records to the window-level aggregate fingerprint. Simultaneously, it generates a micro-credential containing the record sequence number, timestamp hash, and incremental contribution value, and appends the micro-credential to the traceability sub-chain of the corresponding participant's active window.

[0028] Furthermore, the aforementioned incremental hash algorithm is a hash calculation method that supports cumulative updates. The calculation process of the incremental hash algorithm is as follows: for the active window... The first data record will be the first Feature values ​​of data records With the current window-level aggregated fingerprint After concatenation, the hash value is calculated to obtain the updated window-level aggregated fingerprint. ,Right now:

[0029] in, The sequence number of the data record within the active window. To process the first Window-level aggregated fingerprints after each data record For the first The characteristic values ​​of each data record This indicates a data concatenation operation. Pre-configured cryptographic hash functions within the TEE enclave. When hour, Take the initial value of the window-level aggregated fingerprint initialized in step 1 (i.e., the hash seed value corresponding to the participant).

[0030] Furthermore, the incremental contribution value in the aforementioned micro-vouchers is This refers to the independent hash value of a single data record's feature value, used to individually verify specific data records within a window when necessary without tracing back the entire traceability subchain.

[0031] Furthermore, the feature values ​​of the aforementioned data records This is a combination of key fields extracted from the original data record for traceability verification, including a hash digest of the data content, a data source identifier, and a data generation timestamp. This combination of fields can uniquely identify a data record and support subsequent integrity verification.

[0032] In this embodiment of the application, in order to identify timing anomalies as early as possible during the data reception stage, the timing monotonicity between the timestamp of the data record and the timestamp of the preceding data record of the same participant is also verified while comparing the timestamp of the data record with the window sequence counter. Specifically, when a timestamp out-of-order or timestamp jump is detected (i.e., the difference between the timestamps of two adjacent data records exceeds a preset time interval threshold), a timing anomaly marker is generated and associated with the micro voucher corresponding to the data record.

[0033] Furthermore, the aforementioned time-series anomaly marker includes an anomaly type identifier and an anomaly timestamp difference, which are included in the statistical scope of the window integrity summary when the window is closed. The anomaly type identifier is used to distinguish between two types of time-series anomalies: timestamp out-of-order and timestamp jumps. The anomaly timestamp difference records the degree of deviation between the actual timestamp difference and the normal time interval threshold.

[0034] Step 3: When the sliding window triggers the sliding, calculate the window integrity summary and extract the window-level distribution statistics to generate a source-verified distribution snapshot; When the number of data records accumulated in the sliding windows of all participants reaches the window size When a window slides, an integrity summary calculation is performed on the window that is about to close, obtaining a window integrity summary. Simultaneously, window-level distribution statistics are extracted from the source-verified data records within the window that is about to close. These window-level distribution statistics are bound to the window integrity summary to form a source-verified distribution snapshot, which is then appended to the historical window summary chain of the corresponding participant. After appending, the detailed sub-chain data and original data record cache of the window about to close are released.

[0035] Furthermore, the aforementioned window integrity summary includes the following fields: window-level aggregated fingerprint. The data includes the number of data records within the window, time-series anomaly statistics, and subchain root hashes. The subchain root hash is the Merkle tree root hash value calculated sequentially by record number for all micro-credentials in the traceability subchain of the soon-to-be-closed window. The time-series anomaly statistics include the number of time-series anomaly data records and the distribution statistics of each anomaly type.

[0036] Furthermore, the above Merkel tree root hash value is calculated as follows: all micro vouchers in the window are arranged in order of record number as leaf nodes, and the hash value of the parent node is calculated by pairing them up layer by layer until the hash value of a single root node is obtained. The hash value of a single root node is the root hash of the child chain.

[0037] Furthermore, the aforementioned window-level distribution statistics are statistical features extracted from all data records within the window that have passed source verification, including the mean vector and covariance matrix of each feature dimension. Here, "passed source verification" means that the data record has complete micro-credentials and has not been marked as a time-series anomaly, or although marked as a time-series anomaly, the anomaly type is tolerable. A tolerable type refers to timestamp jumps where the deviation of the timestamp difference is within a preset tolerance range.

[0038] Furthermore, the aforementioned traceable and certified distribution snapshots are the product of binding window-level distribution statistics and window integrity summaries as a single data structure. The binding operation is performed atomically within the TEE enclave, ensuring that the window-level distribution statistics and window integrity summaries in any traceable and certified distribution snapshot strictly correspond to the same batch of data records within the same window.

[0039] Furthermore, the aforementioned atomic operation method refers to the binding process being executed under the protection of the TEE enclave's transaction mechanism. The binding process either completes fully and generates a valid, traceable, and certified distribution snapshot, or it rolls back completely without generating any intermediate states in the event of an anomaly, thereby ensuring the internal consistency of the traceable and certified distribution snapshot.

[0040] In this embodiment of the application, in order to control the storage overhead of the historical window summary chain, when appending the traceable and authenticated distributed snapshot to the historical window summary chain, it is also checked whether the length of the historical window summary chain exceeds the preset number of historical windows to be retained. When this limit is exceeded, the window integrity digest of the earliest window digest node is further compressed into a simplified digest containing only the window-level aggregate fingerprint and the subchain root hash, and the storage space for the window-level distribution statistics of the window digest node is released. Through this process, the number of nodes in the historical window digest chain that retain the complete window-level distribution statistics always does not exceed [a certain limit]. This ensures that the effective verification length and storage usage of the traceability chain remain within a controlled range. Reserve a number of historical windows.

[0041] Furthermore, the number of historical windows retained above The number of historical windows to retain is determined based on the historical window backtracking length required for distribution drift detection. The value should not be less than the historical window backtracking length used for distribution drift detection. This ensures that the historical window-level distribution statistics required for distribution drift detection are always available.

[0042] Step 4: Based on the traceable and certified distribution snapshot sequence, calculate the distribution drift detection results and distribution distance metric, and update the dynamic data distribution feature profile; For each participant, a sequence of traceable and verified distribution snapshots of the most recent consecutive windows is obtained from the historical window summary chain. Window-level distribution statistics for each window are extracted, and the temporal trend of the window-level distribution statistics between adjacent windows is calculated. Distribution drift events are detected based on the temporal trend, and the drift magnitude and direction are labeled for each detected event. The distribution distance metric between the window-level distribution statistics of each participant's latest window and the globally estimated distribution is calculated. The distribution drift detection results, drift direction information, and distribution distance metric are updated to the dynamic data distribution feature profile of the corresponding participant.

[0043] Furthermore, the aforementioned global estimated distribution is an aggregated distribution feature obtained by weighting the window-level distribution statistics of the latest windows of all current participants according to the number of data records, including the global mean vector. and global covariance matrix The specific calculation method is as follows: Let the first... The mean vector of the latest window for each participant is The covariance matrix is The number of data records is Then the global mean vector is The global covariance matrix is ,in The total number of participants. Index for participating parties.

[0044] Furthermore, the aforementioned distribution distance metric is calculated using the KL divergence algorithm, with the input being the first... Window-level distribution statistics (mean vector) of the latest window of each participant Covariance Matrix ) and the statistics of the global estimated distribution (global mean vector) and global covariance matrix The output is a distribution distance metric. .

[0045] Furthermore, the detection method for the aforementioned distribution drift event is as follows: for the first... Each participant extracts the most recent consecutive... Calculate the distribution distance sequence between adjacent windows from the window-level distribution statistics sequence of the nth window. Specifically, for the nth window... The first window and the first For each window, the distribution distance between the two is calculated using the KL divergence algorithm. ,in The window number is the index, and its value range is the [number]. The first to the second Each window corresponds to a pair of adjacent windows. When the distribution distance is closest to the sequence... The mean of the values Exceeding the preset distribution drift detection sensitivity parameters At that time, the corresponding participant is determined to be in a distribution drift state. The drift amplitude is the most recent The mean of the distribution distance values The drift direction is the difference between the mean vectors of the most recent window and the previous window. unit vector .in The historical window backtracking length used for distribution drift detection. The number of short-term windows used for drift determination. For the distribution drift detection sensitivity parameter, For the summation index, For the first The mean vector of each window, For the first The mean vector of each window.

[0046] Furthermore, the above-mentioned distribution drift detection sensitivity parameters The tolerance level for distribution changes is set according to the application scenario. The smaller the value, the more sensitive it is to changes in distribution, and the easier it is to identify a distribution drift state. Number of short-term windows The value should be less than the history window backtracking length. It is used to focus on the most recent distribution change trend in a historical window sequence.

[0047] Step 5: Receive the encrypted gradients from each participant, set differentiated anomaly detection thresholds based on the dynamic data distribution feature profile, perform anomaly detection on the gradients, and filter out abnormal gradients. Receive encrypted gradients uploaded by each participant in the current training round and decrypt them within the TEE enclave. Obtain distribution distance metrics and drift status flags from the dynamic data distribution feature profiles of each participant, and set differential anomaly detection thresholds for each participant based on these metrics. Perform anomaly detection on the gradients of each participant based on the differential anomaly detection thresholds, filtering out gradients that fail the anomaly detection and retaining those that pass.

[0048] Furthermore, the aforementioned differential anomaly detection threshold The setting method is as follows: based on a preset baseline anomaly detection threshold. Based on this, for participants in a distribution drift state, the differential anomaly detection threshold for the corresponding participants is expanded to... ,in For the first The drift amplitude of each participant, This is the threshold expansion coefficient; for participants in a stable distribution state, it remains... .

[0049] Furthermore, the aforementioned benchmark anomaly detection threshold Based on the statistical distribution of historical gradient norms during global model training, the threshold expansion coefficient is typically set to the mean of historical gradient norms plus a certain multiple of the standard deviation. Controlling the tolerance for gradient anomalies under distributed drift conditions, The larger the value, the greater the expansion of the differential anomaly detection threshold for participants in a state of distribution drift.

[0050] In this embodiment, to further identify anomalous gradient mutations during a stable distribution period, a gradient history consistency check is introduced into the differential anomaly detection threshold setting for participants in a stable distribution state. Specifically, for participants in a stable distribution state, the Euclidean distance algorithm is used to calculate the gradient of the current round and the closest gradient of the corresponding participant. The distance between the mean gradients of each round, when exceeding a preset multiple of the standard deviation of the historical gradient distance of the corresponding participant, will tighten the differential anomaly detection threshold for that participant. ,in A tightening factor less than 1 This represents the number of historical gradient backtracking rounds.

[0051] Furthermore, the aforementioned preset multiple is set based on the sensitivity requirements to gradient abrupt changes, typically taking 2 to 3 times the standard deviation. Tightening coefficient. Control the stringency of gradient abrupt changes in a stable distribution state. The smaller the value, the greater the tightening of the differential anomaly detection threshold, and the more stringent the detection of gradient mutations.

[0052] Furthermore, the above anomaly detection is performed as follows: the gradients of each participant are calculated. Norm, will Norm value and corresponding differential anomaly detection threshold Comparison, Norm value exceeds The gradient is identified as an abnormal gradient and filtered out.

[0053] Step 6: Perform orientation calibration on the gradients that pass the anomaly detection, adjust the calibration intensity based on the drift direction information, and generate the calibrated gradient; For the gradients of each participant that pass anomaly detection, a cosine similarity algorithm is used to calculate the cosine value of the angle between each gradient direction and the global gradient estimation direction. Gradients with a cosine value lower than a preset calibration threshold undergo direction decomposition, decomposing them into parallel and perpendicular components consistent with the global gradient estimation direction. Based on the drift direction information in the dynamic data distribution feature archive, the calibration intensity parameter is adjusted, and the perpendicular component is attenuated according to the calibration intensity parameter. The attenuated perpendicular component is then recombined with the parallel component to obtain the calibrated gradient.

[0054] Furthermore, the above-mentioned directional decomposition and recombination process is as follows: when the cosine value of the included angle is lower than the calibration threshold... When, the gradient Decomposed into parallel components and vertical components :

[0055]

[0056] Obtain the calibrated gradient :

[0057] in, For the first Each participant uses the gradient vector of anomaly detection. The unit direction vector of the gradient is updated for the global model in the previous round. Gradient With unit direction vector The inner product, gradient Parallel components in the global gradient estimation direction, gradient The vertical component relative to the global gradient estimation direction, For the first The calibration intensity parameters for each participant have a range of values. , For the first The post-calibration gradient of each participant. When the cosine of the included angle is not lower than the calibration threshold. At that time, direction calibration is not performed, causing .

[0058] Furthermore, the aforementioned calibration threshold The tolerance level for gradient direction deviation is set accordingly. The closer the value is to 1, the lower the tolerance for gradient direction deviation, and the easier it is to trigger direction calibration. Calibration intensity parameter Control the degree to which the vertical component is retained. The smaller the value, the stronger the attenuation of the vertical component, and the closer the calibrated gradient is to the global gradient estimation direction.

[0059] Furthermore, the above calibration intensity parameters The adjustment method is as follows: when the first When a participant is in a distribution drift state, the cosine similarity algorithm is used to calculate the similarity between the drift direction vector of the corresponding participant and the gradient vertical component direction of the corresponding participant. .when When the value is positive, it indicates that the vertical component direction is consistent with the drift direction. In this case, increasing the calibration intensity parameter is appropriate. To retain more vertical component information, i.e. ;when When the value is negative or zero, keep .in These are the preset benchmark calibration strength parameters. When the... When all participants are in a stable distribution state .

[0060] Furthermore, the aforementioned benchmark calibration intensity parameters The value is typically set between 0.3 and 0.7, based on the requirement to preserve gradient components in non-mainstream directions. Similarity The range of values ​​is ,when When the value is positive, it indicates that the angle between the vertical component direction and the drift direction is less than 90 degrees. In this case, it is believed that the vertical component may carry effective gradient information caused by the distribution drift.

[0061] Step 7: Calculate the adaptive aggregation weights, perform weighted aggregation on the calibrated gradients, and generate model update results containing traceability audit records; Adaptive aggregation weights are calculated based on the data volume, distribution distance metric, and orientation calibration magnitude of each participant. The calibrated gradients of each participant are weighted and aggregated according to these adaptive aggregation weights to generate a global model update gradient. The global gradient estimation direction is updated for use in the next round. The aggregation results of this round are associated with the window identifiers of each participant's contribution data, and the model update results, including source audit records, are output.

[0062] Furthermore, due to the amount of data from each participating party... Distribution distance metric and orientation calibration amplitude Since they have different dimensions and numerical ranges, these parameters need to be preprocessed before calculating the adaptive aggregation weights.

[0063] Furthermore, the aforementioned directional calibration amplitude Defined as the degree of change in gradient direction before and after calibration, calculated as follows: ,in To calibrate the gradient, This is the calibrated gradient. The range of values ​​is A larger value indicates a greater change in orientation caused by calibration. When orientation calibration is not triggered in step 6 (i.e., let...), ), .

[0064] Furthermore, the aforementioned adaptive aggregation weights The calculation method is as follows: for the first Each participant, after completing data preprocessing, uses the processed normalized data volume. Standardized distribution distance and standardized directional calibration amplitude Calculate the original weighting factors for the corresponding participants:

[0065] in, For the first The original weighting factors of each participant, For the first The normalized data volume of each participant For the first Standardized distribution distance of each participant For the first Standardized directional calibration amplitude for each participating party The attenuation coefficient is the distance distribution factor. The penalty coefficient for directional deviation. This represents an exponential function with the natural constant as its base. The set of participants that passed the anomaly detection in the current round is the target of the above data preprocessing and normalization operations. The corresponding parameters for all participants are then determined. The original weight factors are normalized to obtain the final adaptive aggregation weights.

[0066] in, For the first Adaptive aggregation weights for each participant For summation index.

[0067] Furthermore, the aforementioned distribution distance attenuation coefficient Controlling the degree of influence of distribution distance on adaptive aggregation weights, A larger value indicates that the weights of participants with larger distribution distances decay more rapidly. (Direction deviation penalty coefficient) The degree to which the control direction calibration magnitude penalizes the adaptive aggregation weights. The larger the value, the heavier the weight penalty for participants with larger calibration amplitudes. This is to ensure the original weight factors are properly configured. Non-negative, The value of should satisfy .

[0068] Furthermore, the aforementioned global model updates the gradient. The calculation method is as follows:

[0069] in, Update gradients for the global model. For the first The calibrated gradient of each participant. For summation index. The global gradient estimation direction is updated as follows: .

[0070] Furthermore, the aforementioned source tracing audit records indicate that the gradient for updating the global model in this round was... The audit data structure is formed by associating the following information: adaptive aggregation weights of each participant. The system includes the window identifier (i.e., window sequence counter value) corresponding to the data contributed by each participant, the drift status flag of each participant, and the anomaly detection results. Source tracing audit records are appended to the model version history chain to support subsequent model behavior auditing and data contribution tracing.

[0071] Furthermore, the aforementioned model version history chain is a chain-like data structure stored within the TEE enclave. Each chain node corresponds to a model update event in a training round. The chain node content includes a hash digest of the model update gradient, a traceability audit record, and a hash reference of the previous node. The complete historical path of model evolution can be traced through the chain-like data structure.

[0072] In this embodiment, to avoid the impact of extreme skewness in single-round adaptive aggregation weights on model update stability, a truncation process is performed on the normalized adaptive aggregation weights when calculating them. Specifically, a maximum weight cap for a single participant is set. and minimum weight lower bound The normalized adaptive aggregation weight values ​​exceed The participants will truncate the adaptive aggregation weights of the corresponding participants to For adaptive aggregation weight values ​​lower than The participants will truncate the adaptive aggregation weights of the corresponding participants to After truncation, normalization is re-executed to ensure that the sum of all adaptive aggregate weights is 1.

[0073] Furthermore, the aforementioned maximum weight cap and minimum weight lower bound Based on the need to constrain the influence of a single participant. Usually taken Several times the weight of the adaptive aggregation weight of a single participant is used to prevent the adaptive aggregation weight of a single participant from becoming too large and dominating the model update. Usually taken A fraction of that is used to guarantee a minimum level of participation from each party, of which The total number of participants.

[0074] This implementation provides a comprehensive management method for AI model development based on a trusted execution environment. Addressing the disconnect between data tracing and training control in streaming data scenarios, it achieves a targeted solution through the following technical approaches: First, this implementation adopts a sliding window segmented management approach instead of the traditional full-chain append-based tracing method. When each window closes, a window integrity summary is calculated and detailed sub-chain data within the window is released. This ensures that the effective verification length and storage usage of the tracing chain no longer grow indefinitely with the continuous arrival of data, but are instead determined by the window size and the number of historical windows retained, remaining within a controlled range. Therefore, the computational delay of the tracing verification operation is no longer positively correlated with the total amount of all historical data records, eliminating the bottleneck of delay in the entire process during the data tracing management phase.

[0075] Second, this implementation method simultaneously performs integrity summary calculation and window-level distribution statistic extraction using atomic operations when the window is closed, and binds the two to form a traceable and verified distribution snapshot. Since the extraction of window-level distribution statistics is strictly limited to data records within the window that have passed traceability verification, and the binding operation ensures that the window-level distribution statistics in the traceable and verified distribution snapshot always correspond to the same batch of data within the same window as the window integrity summary, the distribution information used in the subsequent training and control phases only comes from data whose integrity has been verified. This process blocks the path of poisoned data contaminating the distribution estimation without integrity verification at the entire process level.

[0076] Third, this implementation method continuously tracks the changing trends of data distribution for each participant and detects distribution drift events by performing time-series analysis on the traceable and verified distribution snapshot sequences of multiple consecutive windows for each participant. Based on the distribution drift detection results, the differentiated anomaly detection threshold in the gradient aggregation stage can expand the tolerance range for participants in the distribution drift period and tighten the detection standard for participants with stable distributions but abrupt gradient changes. At the same time, the calibration intensity parameter of gradient direction calibration can also be adaptively adjusted according to the drift direction information. Therefore, normal gradient shifts caused by changes in data distribution during distribution drift are no longer misjudged as anomalies by fixed thresholds, and the aggregation quality of model update gradients is maintained.

[0077] A regional medical consortium comprises five hospitals (Hospital A, Hospital B, Hospital C, Hospital D, and Hospital E) forming a federated learning network to jointly train a nodule recognition model for lung CT images. Each hospital continuously generates patient CT scan data and extracts image features as training data. A central server's TEE (Transmission over Environment) coordinating and managing the entire training process is handled by a dedicated network. Due to differences in equipment models, patient demographics, and scanning protocols among the hospitals, the data distribution exhibits a non-independent, identically distributed pattern. Between March and June 20XX, Hospital C replaced its CT equipment, causing a significant shift in its data distribution. Hospital D experienced an anomaly in its data acquisition timestamps in May 20XX due to equipment malfunction. The consortium needs to achieve adaptive management of distribution drift and abnormal data, ensuring data traceability and verifiability, to guarantee the quality of model training.

[0078] The central server (TEE) enclave launches the AI ​​model development full-process management controller and retrieves a list of 5 registered hospital participants. A window size is then allocated to each hospital. Image feature records, window sliding step size To maintain the continuity of distribution statistics, each adjacent window retains an overlapping area of ​​100 records. The window sequence counter for each hospital's sliding window traceability management instance is initialized to 0. A hash seed value specific to each hospital is derived using the TEE enclave sealing key as the initial value for the window-level aggregate fingerprint. Simultaneously, a dynamic data distribution feature profile is created for each hospital, and the number of historical windows to retain is set. Distribution drift detection backtracking length Number of short-term windows Distribution drift detection sensitivity parameters .

[0079] Table 1 Initialization configuration parameters for each participant:

[0080] Table 2 Initial State of Dynamic Data Distribution Feature Archives:

[0081] On March 15, 20XX, Hospital A uploaded its first batch of CT image feature data. The TEE enclave received and decrypted the encrypted data stream, extracted the timestamp of each record, compared it with the window sequence counter, and confirmed that it belonged to window sequence 0 (the first active window). Incremental hash updates were performed on records 1 through 5 within the window. The feature value of each record contained a hash digest of the 512-dimensional image feature vector, the hospital identifier, and the acquisition timestamp.

[0082] Table 3 Example of first batch data records for Hospital A:

[0083] Perform incremental hash calculation on the third record of Hospital A, and aggregate fingerprint at the current window level. The third record's feature value The hash is The updated window-level aggregated fingerprint is:

[0084] The generated micro-credential includes: record number = 3, timestamp hash = Incremental contribution value = Added to the traceability sub-chain of the active window of Hospital A.

[0085] On May 12, 20XX, Hospital D experienced an equipment malfunction that caused a jump in data timestamps. The TEE enclave detected the timestamp anomaly when receiving records 327 to 329 from Hospital D.

[0086] Table 4. Time-series abnormal data records for Hospital D:

[0087] Record 329's timestamp difference of 7649 seconds exceeds the normal interval threshold of 1200 seconds, generating a time-series anomaly marker (anomaly type = timestamp jump, anomaly timestamp difference = 6449 seconds) and associating it with the record's micro-credential.

[0088] On March 22, 20XX, Hospital A's active window accumulated 500 records, triggering a window slide. The TEE enclave performs closure processing on Hospital A's window sequence 0 and calculates the window integrity digest. First, the window-level aggregate fingerprint is obtained. The number of data records in the statistics window is 500, and the number of time-series anomaly records is 0. A Merkle tree is constructed from the 500 micro-certificates in the window according to their record numbers, and the root hash of the sub-chains is calculated. .

[0089] Table 5 Summary of the Completeness of Hospital A Window Sequence 0:

[0090] Window-level distribution statistics of 512-dimensional image features are extracted from 500 verified data records within the window, and the mean vector of each dimension is calculated. Covariance Matrix The window-level distribution statistics are bound to the window integrity summary using atomic operations to form a traceable and authenticated distribution snapshot, which is then appended to the historical window summary chain of Hospital A.

[0091] Table 6. Distribution snapshots of hospital A window sequence 0 after traceability verification (partial feature dimensions):

[0092] After the window is closed, the detailed traceability subchain data and original data record cache of window sequence 0 are released, and only the traceability-authenticated distribution snapshot is retained in the historical window summary chain. Hospital A's window sequence counter is updated to 1, and new active windows begin receiving data from record 401 (due to the sliding step size). ).

[0093] On April 10, 20XX, Hospital A's historical window summary chain had accumulated 11 window snapshots, exceeding the limit for retaining historical windows. The TEE enclave performs compression on the earliest window sequence snapshot 0.

[0094] Table 7. Hospital A Historical Window Summary Chain Compression Processing:

[0095] After compression, window sequence 0 retains only the window-level aggregated fingerprint. and subchain root hash Release storage space for window-level distribution statistics. The number of nodes in the historical window summary chain that retain complete window-level distribution statistics remains at 10.

[0096] On April 15, 20XX, Hospital C completed data collection for window sequences 0 to 7, generating traceable and authenticated distribution snapshots for each window. TEE enclave extraction of Hospital C's most recent... Calculate the distribution distance between adjacent windows by using the window-level distribution statistics sequence of each window.

[0097] Table 8. Distance sequence of adjacent windows in Hospital C (window sequence 0 to 7):

[0098] Calculate the most recent Mean distance of distribution between window pairs:

[0099] because Hospital C is determined to be in a stable distribution state, and the drift state flag remains "stable".

[0100] On May 8, 20XX, Hospital C completed the data acquisition for window sequences 8 to 10 after the CT equipment was replaced. The adjustment of the scanning parameters of the new equipment caused changes in the distribution of image features.

[0101] Table 9. Distance sequence of adjacent windows after replacement of equipment C in hospital (window sequences 5 to 10):

[0102] Calculate the most recent Mean distance of distribution between window pairs:

[0103] because Hospital C was determined to have entered a distribution drift state. The drift amplitude was recorded. The drift direction is .

[0104] Simultaneously, the distribution distance metric between the latest window for each hospital and the globally estimated distribution is calculated. The global mean vector and global covariance matrix of the globally estimated distribution are calculated as a weighted average of the window-level distribution statistics of the latest windows for the five hospitals.

[0105] Table 10: Distance metrics and drift status updates for each participant (May 8, 20XX):

[0106] Updated dynamic data distribution profile of Hospital C: Distribution distance metric = Drift status flag = "Drift", drift amplitude = .

[0107] On May 10, 20XX, all hospitals completed the 58th round of model training and uploaded the encrypted gradients to the TEE enclave. After decrypting the gradients, the TEE enclave retrieved the distribution distance metric and drift status flags from each hospital's dynamic data distribution feature archives, and set differentiated anomaly detection thresholds. The baseline anomaly detection threshold was then established. (Determined based on historical gradient norm statistics), threshold expansion coefficient .

[0108] For hospital C, which is in a state of distribution drift, the differential anomaly detection threshold is expanded:

[0109] For hospitals A, B, D, and E, which are in a stable distribution state, maintain differentiated anomaly detection thresholds. .

[0110] Table 11 Gradient anomaly detection results for each participant in round 58:

[0111] Hospital C's gradient norm of 2.68, under distribution drift conditions, was detected using an expanded threshold of 2.815, preventing normal gradient changes caused by equipment replacement from being misjudged as abnormal. Hospital D's gradient norm of 3.12 exceeded the threshold of 2.5, was identified as an abnormal gradient, and filtered out, not participating in subsequent aggregation.

[0112] A gradient historical consistency check was performed on hospital A (number of historical gradient backtracking rounds). The Euclidean distance between the gradient of round 58 and the mean gradient of the last 5 rounds is calculated to be 0.38. The standard deviation of the historical gradient distance for Hospital A is 0.42, which is less than twice the standard deviation. This does not trigger threshold tightening.

[0113] For the gradients of hospitals A, B, C, and E that passed the anomaly detection, calculate the direction of each gradient and its relation to the global gradient estimation direction. The cosine of the angle between (from round 57). Calibration threshold. Reference calibration strength parameters .

[0114] Table 12 Gradient Direction Calibration Processing in Round 58

[0115] The cosine of the angle between Hospital A and Hospital E is higher than the calibration threshold, so orientation calibration is not triggered. , .

[0116] The cosine of the included angle at hospital B is 0.78, which is below the calibration threshold, triggering direction calibration. The gradient of hospital B is decomposed into parallel and perpendicular components:

[0117]

[0118] Since Hospital B is in a stable distribution state, the calibration intensity parameter Generate the calibrated gradient:

[0119] The cosine of the included angle of hospital C is 0.73, which is lower than the calibration threshold, triggering direction calibration. Since hospital C is in a distribution drift state, the similarity between the drift direction vector of hospital C and the direction of the gradient's vertical component is calculated. This indicates that the vertical component direction is consistent with the drift direction. Adjust the calibration intensity parameter:

[0120] Generate the hospital's C-calibrated gradient:

[0121] Since Hospital C is in a state of distribution drift and the drift direction is consistent with the vertical component of the gradient, the calibration intensity parameter is increased from 0.5 to 0.68 to retain more vertical component information in order to capture the effective gradient changes caused by distribution drift.

[0122] Step 7 Implementation: Calculate adaptive aggregation weights and generate model update results containing source audit records. For the set of participants that passed the anomaly detection Hospital A, Hospital B, Hospital C, Hospital E The adaptive aggregation weights are calculated. First, data preprocessing is performed on the data volume, distribution distance metric, and orientation calibration magnitude of each participant.

[0123] Calculation of directional calibration amplitude for Hospital C:

[0124] Table 13: Data Preprocessing for Adaptive Aggregation Weight Calculation in Round 58:

[0125] Set the distribution distance attenuation coefficient Direction deviation penalty coefficient Calculate the original weighting factors for each participant:

[0126]

[0127]

[0128]

[0129] Normalization yields adaptive aggregation weights:

[0130] Similarly, the calculation yields , , .

[0131] Table 14: Adaptive Aggregation Weights and Model Update Results for Round 58

[0132] Calculate the global model update gradient:

[0133] Update global gradient estimation direction For use in round 59. Generate source tracing audit logs, including adaptive aggregation weights for each participant, contribution window identifiers (hospitals A through E are all window sequences of 10), drift status flags, and anomaly detection results (hospital D is filtered). Update the source tracing audit logs with the global model's gradient hash digest. After binding, it is appended to the model version history chain.

[0134] Throughout the implementation process, data starts from the original CT image feature records of each hospital. First, in step 2, the data enters the active windows of each participant and undergoes incremental source tracing processing, generating micro-credentials and window-level aggregated fingerprints. Once a window accumulates 500 records, step 3 closes the window and calculates a window integrity summary. Simultaneously, window-level distribution statistics are extracted from the verified data, and these two are bound together to form a source-authenticated distribution snapshot, which is then appended to the historical window summary chain. Step 4 performs time-series analysis based on multiple consecutive source-authenticated distribution snapshot sequences in the historical window summary chain, detecting the distribution drift event of hospital C after equipment replacement and updating the dynamic data distribution feature profile. Step 5, in the gradient aggregation stage, expands the anomaly detection threshold for hospital C to 2.815 based on the drift status in the dynamic data distribution feature profile, ensuring that the gradient change (norm 2.68) caused by equipment replacement is detected instead of being misjudged as anomaly by the fixed threshold of 2.5; simultaneously, the abnormal gradient (norm 3.12) of hospital D is successfully filtered out. In step 6, when performing directional calibration on the detected gradients, Hospital C, being in a state of distribution drift with its drift direction aligned with the vertical component of the gradient, has its calibration intensity parameter increased from the baseline value of 0.5 to 0.68, preserving more effective gradient information caused by distribution drift. In step 7, when calculating the adaptive aggregation weights, Hospital C's original weight factor drops to 0.047 due to its larger distribution distance metric (0.34) and larger directional calibration amplitude (0.27), resulting in a final adaptive aggregation weight of only 0.094, thus preventing its drift-period data from having an excessive impact on the global model. Hospitals A and E, with stable distributions, receive higher weights (0.343 and 0.297, respectively). The final generated global model update gradient is bound to the source audit record and appended to the model version history chain, achieving integrated management of the entire process from data source tracing and distribution tracking to gradient aggregation, ensuring the quality of model updates while maintaining the traceability of data contributions.

[0135] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A method for managing the entire development process of AI models based on a Trusted Execution Environment (TEE), executed within a central server enclave, characterized in that... Includes the following steps: The system initializes the AI ​​model development process and manages the entire process controller. It creates a sliding window traceability management instance and dynamic data distribution feature profile for each federated participant, assigns a window size and sliding step size to each instance, and initializes the initial value of the window-level aggregated fingerprint and the active window traceability sub-chain. It receives encrypted streaming training data records from each participant and decrypts them within the TEE enclave. For data records within the active window, it accumulates and updates the window-level aggregated fingerprint using an incremental hash algorithm, while simultaneously generating micro-credentials and appending them to the active window traceability sub-chain. When the sliding window triggers a slide, it calculates the window integrity digest for the window about to close, extracts window-level distribution statistics from the traceability-verified data records within the window, and binds the window-level distribution statistics and the window integrity digest using atomic operations to form a traceability-verified distribution block. The system extracts window-level distribution statistics from the traceable and verified distribution snapshot sequences of each participant, calculates the temporal change trend of distribution statistics between adjacent windows to detect distribution drift events, calculates the distribution distance metric between each participant and the globally estimated distribution, and updates the dynamic data distribution feature profile. It receives encrypted gradients from each participant and decrypts them within the TEE enclave, sets differentiated anomaly detection thresholds for each participant based on the dynamic data distribution feature profile, and filters out anomalous gradients. It performs direction calibration on gradients that pass anomaly detection, adjusts the calibration intensity parameter based on drift direction information, and generates calibrated gradients. Based on the data volume, distribution distance metric, and direction calibration magnitude of each participant, it calculates adaptive aggregation weights, performs weighted aggregation on the calibrated gradients to generate global model update gradients, and outputs model update results containing traceability audit records.

2. The AI ​​model development end-to-end management method based on a trusted execution environment according to claim 1, characterized in that, The incremental hash algorithm is calculated as follows: For the k-th data record within the active window, the feature value of the k-th data record is concatenated with the current window-level aggregated fingerprint to calculate the hash value, thus obtaining the updated window-level aggregated fingerprint; when k equals 1, the window-level aggregated fingerprint is obtained by deriving the hash seed value generated by the TEE enclave internal sealing key during initialization; the micro-credential includes the record sequence number, timestamp hash, and incremental contribution value, wherein the incremental contribution value is the independent hash value of the feature value of a single data record; While comparing the timestamp of the data record with the window sequence counter, the temporal monotonicity between the timestamp and the timestamp of the preceding data record of the same participant is verified. When timestamp disorder or timestamp jump is detected, a temporal anomaly marker containing an anomaly type identifier and an anomaly timestamp difference is generated and associated with the corresponding micro voucher.

3. The AI ​​model development end-to-end management method based on a trusted execution environment according to claim 1, characterized in that, The window integrity summary includes the window-level aggregate fingerprint, the number of data records within the window, time-series anomaly statistics, and the sub-chain root hash. The sub-chain root hash is the root node hash value obtained by constructing a Merkle tree using all micro-credentials in the traceability sub-chain of the soon-to-be-closed window arranged in order of record number as leaf nodes. The window-level distribution statistics include the mean vector and covariance matrix of each feature dimension. The extraction objects are data records with complete micro-credentials that have not been marked as time-series anomalies, as well as data records that have been marked as time-series anomalies but whose anomaly type is that the deviation of timestamp difference is within a preset tolerance range. When the traceability-certified distribution snapshot is appended to the historical window summary chain, if the length of the historical window summary chain exceeds the preset number K of historical windows to be retained, the earliest window summary node is compressed into a simplified summary containing only the window-level aggregate fingerprint and the sub-chain root hash, and the corresponding window-level distribution statistics storage space is released.

4. The method for managing the entire AI model development process based on a trusted execution environment according to claim 1, characterized in that, The method for detecting distribution drift events is as follows: For the i-th participant, extract the window-level distribution statistics sequence of the most recent L consecutive windows, and use the KL divergence algorithm to calculate the distribution distance sequence between adjacent windows. When the mean of the most recent m values ​​in the distribution distance sequence exceeds the preset distribution drift detection sensitivity parameter, the corresponding participant is determined to be in a distribution drift state. The drift amplitude is the mean of the most recent m distribution distance values, and the drift direction is the unit vector of the difference between the mean vector of the most recent window and the mean vector of the previous window. Where L is the historical window backtracking length, m is the number of short-term windows used for drift determination, and m is less than L. The global estimated distribution is the aggregated distribution feature obtained by weighting the window-level distribution statistics of the latest windows of all participants according to the number of data records. The distribution distance metric is calculated using the KL divergence algorithm to calculate the degree of deviation between the window-level distribution statistics of the latest window of each participant and the global estimated distribution.

5. The AI ​​model development end-to-end management method based on a trusted execution environment according to claim 1, characterized in that, The differential anomaly detection threshold is set as follows: based on the baseline anomaly detection threshold, for participants in a state of distribution drift, the differential anomaly detection threshold is expanded to the sum of the products of the baseline anomaly detection threshold, the drift amplitude, and the threshold expansion coefficient; for participants in a state of stable distribution, the differential anomaly detection threshold is kept as the baseline anomaly detection threshold; the anomaly detection is performed by calculating the L2 norm of the gradient of each participant, comparing the L2 norm value with the corresponding differential anomaly detection threshold, and determining gradients with L2 norm values ​​exceeding the differential anomaly detection threshold as abnormal gradients and filtering them out.

6. The method for managing the entire development process of AI models based on a trusted execution environment according to claim 5, characterized in that, For participants in a stable distribution state, a gradient history consistency check is also introduced: the Euclidean distance algorithm is used to calculate the distance between the current round gradient and the average gradient of the corresponding participant in the most recent R rounds. When the distance exceeds a preset multiple of the standard deviation of the historical gradient distance of the corresponding participant, the differential anomaly detection threshold of the corresponding participant is tightened to the product of the benchmark anomaly detection threshold and the tightening coefficient. The tightening coefficient is a positive number less than 1, and R is the number of historical gradient backtracking rounds.

7. The method for managing the entire development process of AI models based on a trusted execution environment according to claim 1, characterized in that, The orientation calibration process is as follows: the cosine similarity algorithm is used to calculate the cosine value of the angle between the gradient direction of each participant and the global gradient estimation direction. For gradients whose cosine value of the angle is lower than the preset calibration threshold, they are decomposed into parallel components and vertical components that are consistent with the global gradient estimation direction. The vertical component is attenuated according to the calibration intensity parameter and then recombined with the parallel component to obtain the calibrated gradient. Gradients with an included angle cosine value not lower than the calibration threshold are not subject to directional calibration. The calibration intensity parameter is adjusted as follows: when the participants are in a distribution drift state, the similarity between the drift direction vector and the gradient perpendicular component direction is calculated. When the similarity is positive, the calibration intensity parameter is set to the increasing function value of the reference calibration intensity parameter and the similarity. When the similarity is non-positive or the participants are in a distribution stable state, the calibration intensity parameter is taken as the reference calibration intensity parameter.

8. The method for managing the entire development process of AI models based on a trusted execution environment according to claim 1, characterized in that, The adaptive aggregation weights are calculated as follows: Data preprocessing is performed on the data volume, distribution distance metric, and orientation calibration magnitude of each participant to obtain normalized data volume, standardized distribution distance, and standardized orientation calibration magnitude. The original weight factor for each participant is equal to the normalized data volume multiplied by the negative exponential decay factor of the distribution distance, and then multiplied by the linear penalty factor of the orientation calibration magnitude. The negative exponential decay factor controls the decay rate of the standardized distribution distance by the distribution distance decay coefficient, and the linear penalty factor controls the penalty degree of the standardized orientation calibration magnitude by the orientation deviation penalty coefficient. Normalization processing is performed on the original weight factors to obtain the adaptive aggregation weights. The orientation calibration magnitude is defined as the difference between the cosine similarity of the gradient before and after calibration and 1. The global model update gradient is the result of a weighted sum of the calibrated gradients of all participants that passed the anomaly detection, calculated using adaptive aggregation weights.

9. The method for managing the entire development process of AI models based on a trusted execution environment according to claim 8, characterized in that, The normalized adaptive aggregation weights are truncated: a maximum weight upper limit and a minimum weight lower limit are set for each participant. For participants whose adaptive aggregation weights exceed the maximum weight upper limit, their weights are truncated to the maximum weight upper limit. For participants whose adaptive aggregation weights are below the minimum weight lower limit, their weights are truncated to the minimum weight lower limit. After truncation, normalization is re-executed to ensure that the sum of all adaptive aggregation weights is 1. The source audit record includes the adaptive aggregation weights of each participant, the window identifier corresponding to the contribution data of each participant, the drift status flag, and the anomaly detection result. The source audit record is appended to the model version history chain. The model version history chain is a chain data structure stored in the TEE enclave. Each chain node includes a hash digest of the model update gradient, a source audit record, and a hash reference of the previous node.

10. A full-process management system for AI model development based on a trusted execution environment, used to execute the full-process management method for AI model development based on a trusted execution environment as described in any one of claims 1 to 9, characterized in that, include: The initialization module is used to initialize the AI ​​model development full-process management controller within the central server TEE enclave, create a sliding window traceability management instance and dynamic data distribution feature profile for each federation participant, allocate window size and window sliding step size to each sliding window traceability management instance, and initialize the initial value of the window-level aggregate fingerprint and the active window traceability sub-chain. The incremental tracing module receives encrypted streaming training data records from each participant and decrypts them within the TEE enclave. It accumulates and updates the window-level aggregate fingerprint for data records within the active window using an incremental hash algorithm, generating micro-credentials and appending them to the active window's tracing sub-chain. The distribution snapshot generation module calculates the window integrity summary for the window about to close when the sliding window triggers a slide. It extracts window-level distribution statistics from data records within the window that have passed tracing verification and binds the window-level distribution statistics and the window integrity summary using atomic operations to form a traceable and certified distribution snapshot. The distribution drift detection module extracts window-level distribution statistics from the traceable and certified distribution snapshot sequence from each participant, calculates the temporal change trend of distribution statistics between adjacent windows to detect distribution drift events, calculates the distribution distance metric between each participant and the globally estimated distribution, and updates the dynamic data distribution feature profile. The gradient anomaly detection module receives encrypted gradients from each participant and decrypts them within the TEE enclave. Based on the dynamic data distribution feature profile, it sets differentiated anomaly detection thresholds for each participant and filters out anomalous gradients. The gradient orientation calibration module is used to perform orientation calibration on gradients that have passed anomaly detection, adjust the calibration intensity parameters based on drift direction information, and generate a calibrated gradient. The weighted aggregation and auditing module is used to calculate adaptive aggregation weights based on the data volume, distribution distance metric, and orientation calibration magnitude of each participant, perform weighted aggregation on the calibrated gradient to generate a global model update gradient, and output the model update result containing traceability audit records.

Images