Federated learning based internet of things privacy data collaborative modeling system

Abstract

The application discloses a federated learning-based Internet of Things privacy data collaborative modeling system, which comprises the following steps: issuing a collaborative modeling task parameter, an initial model identifier and a public distillation sample identifier set to a plurality of Internet of Things terminals, updating a local model based on local data by each terminal, and performing forward inference on the public distillation sample to generate a model output vector; an interface layer maps the output vector into a winner position discrete index set according to a WTA rule and uploads the winner position discrete index set. A collaborative distillation module performs sample-level alignment and aggregation on the index set, constructs a distillation supervision signal, performs federated distillation update, and outputs a target model result. The application relates to the technical fields of Internet of Things data processing and machine learning.

Description

Technical Field

[0001] This invention relates to the fields of IoT data processing and machine learning technology, and in particular to an IoT privacy data collaborative modeling system based on federated learning. Background Technology

[0002] With the advancement of smart cities and the Industrial Internet of Things (IIoT), a large number of IoT terminals in scenarios such as environmental monitoring, intelligent manufacturing, vehicle networking, and intelligent security are continuously generating multi-source heterogeneous data. Collaborative machine learning modeling based on multi-terminal data has become an important technical path to improve the generalization and adaptability of models. Currently, multi-terminal collaborative modeling generally adopts centralized training or traditional federated learning frameworks, achieving cross-terminal knowledge fusion by aggregating data or sharing model parameters.

[0003] Existing technologies still have significant limitations in collaborative modeling scenarios for IoT privacy data. On the one hand, centralized training requires uploading raw data, which is difficult to meet the requirements of dispersed IoT terminal data and the prohibition of external transmission of sensitive data. Moreover, the complexity of data sources leads to high costs for unified cleaning and labeling. On the other hand, traditional federated learning typically shares gradients or model parameters, resulting in high communication overhead on the terminal side and difficulty in continuous iteration under conditions of limited terminal computing power and unstable network status. At the same time, sharing methods based on model parameters or continuous prediction output may still expose model discrimination details, posing a risk of reverse inference. Existing federated distillation schemes mostly rely on continuous probability outputs as distillation information. The output information is large and difficult to stably align among heterogeneous terminals, resulting in complex sample-level aggregation and distillation update processes, making it difficult to form a reusable output interface layer mechanism.

[0004] Therefore, how to provide a collaborative modeling system for IoT privacy data based on federated learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a collaborative modeling system for IoT privacy data based on federated learning. This invention completes local model training and public sample inference on multiple IoT terminals, and then discretizes the model output according to the winner position index before performing sample-level alignment aggregation and federated distillation updates. This achieves collaborative modeling without sharing the original data and has the advantages of low privacy leakage risk, low communication overhead, and adaptability to resource-constrained terminals.

[0006] The IoT privacy data collaborative modeling system based on federated learning according to an embodiment of the present invention includes the following steps:

[0007] The collaborative task and model configuration module configures the collaborative modeling task parameters, initial model identifier, and common distillation sample identifier set, and sends the collaborative modeling task parameters, initial model identifier, and common distillation sample identifier set to multiple IoT terminals;

[0008] The model initialization module is deployed on each IoT terminal. It constructs the initial model based on the initial model identifier and generates a local model instance on the terminal.

[0009] The terminal local data management module is deployed in each IoT terminal to store and manage the local data set generated by the corresponding terminal.

[0010] The terminal local model training module, based on the parameters of the collaborative modeling task, uses the local dataset to train and update the terminal local model instance, generating the updated terminal local model instance.

[0011] The common distillation sample inference module performs forward inference on the common distillation samples corresponding to the common distillation sample identifier set based on the updated local model instance on the terminal, and generates the model output vector corresponding to the common distillation sample identifier.

[0012] The interface layer WTA index mapping module maps the model output vector to a discrete discriminant index set containing only the winner position index according to the WTA rules;

[0013] The discriminant index collaborative distillation module receives discrete discriminant index sets from multiple IoT terminals, performs sample-level alignment and aggregation based on a common distillation sample identifier set, and performs federated distillation updates on the target model based on the aggregated discrete discriminant index set, outputting the target model result after collaborative modeling.

[0014] Optional, the collaborative task and model configuration module includes:

[0015] The parameters for collaborative modeling tasks include task identifier, set of participating terminal identifiers, number of local training steps, local training batch size, local learning rate, and number of WTA winner indexes.

[0016] The initial model identifier includes the model structure identifier, the model version identifier, and the model parameter index. The model structure identifier is used to indicate the network structure type and output dimension of the local model instance on the terminal. The model version identifier is used to indicate the model version number. The model parameter index is used to locate the initial model parameter file.

[0017] The public distillation sample identifier set consists of multiple public distillation sample identifiers, each of which includes a sample source identifier, a sample index number, and a sample version number.

[0018] Optionally, the generation of local model instances on the terminal includes:

[0019] The initial model identifier parsing unit is used to parse the initial model identifier to obtain the model structure identifier, model version identifier, and model parameter index.

[0020] The structure template acquisition unit is used to locate the target network structure template in the model structure template library based on the model structure identifier, and output the network layer sequence definition, input feature definition and output dimension definition corresponding to the target network structure template;

[0021] Output node construction unit, used to define and construct output node set based on output dimension, and to solidify the indexing rules of output node set into output index space;

[0022] A computation graph construction unit is used to construct network layers sequentially based on the network layer sequence definition and connect the network layers to generate a computation graph, wherein the computation graph includes a set of output nodes and trainable variables;

[0023] The parameter location and loading unit is used to locate the initial model parameter file based on the model version identifier and model parameter index, and load the set of parameter tensors corresponding to the trainable variables in the computation graph.

[0024] Parameter binding is used to bind a set of parameter tensors to trainable variables in the computation graph to obtain an initial model;

[0025] The model instantiation unit generates a local model instance for a terminal by binding the initial model with the terminal identifier in the set of participating terminal identifiers of the collaborative modeling task parameters.

[0026] Optionally, the local data set stored and managed by the terminal local data management module includes:

[0027] A subset of sample feature vectors, used to store sample feature vectors consistent with the input feature definition of the collaborative modeling task parameters;

[0028] A subset of sample labels is used to store the sample labels corresponding to the sample feature vectors;

[0029] A subset of the sample index is used to store the correspondence between sample identifiers, sample feature vectors, and sample labels, as well as the sample collection timestamps;

[0030] The data is partitioned into subsets to store the training set index, validation set index, and test set index obtained from the parameter partitioning of the collaborative modeling task.

[0031] A subset of data versions is used to store the data version identifier and incremental update records of the local data set.

[0032] Optionally, the generation of the updated terminal-local model instance includes:

[0033] The collaborative modeling task parameter parsing unit is used to parse the task identifier, local training steps, local training batch size, local learning rate and input feature definition from the collaborative modeling task parameters, and generate the training configuration.

[0034] The local dataset localization unit is used to locate the training set index in the local dataset based on the task identifier in the training configuration, and to define the sample feature vector and sample label corresponding to the training set index based on the input features.

[0035] The training sample pair building unit is used to align the sample feature vector with the sample label according to the sample identifier to generate a set of training sample pairs;

[0036] The training batch generation unit is used to select training sample pairs from the training sample pair set based on the local training batch size and generate training batch data, and generate batch identifiers for the training batch data.

[0037] The forward computation unit is used to input the sample feature vectors from the training batch data into the local model instance on the terminal and generate a prediction output corresponding to the batch identifier.

[0038] The loss calculation unit is used to calculate the training loss based on the predicted output and the sample labels in the training batch data, and to bind the training loss to the batch identifier.

[0039] The backpropagation unit is used to perform backpropagation calculation on the training loss bound to the batch identifier, and generate a set of parameter gradients corresponding to the trainable variables in the computation graph of the local model instance on the terminal.

[0040] The parameter update unit is used to update the parameter gradient set based on the local learning rate in the training configuration, and generate an updated terminal local model instance.

[0041] The training iteration control unit is used to perform training batch construction, forward inference calculation, training loss calculation, backpropagation calculation, and parameter update in a loop according to the local training step number.

[0042] Optionally, performing forward inference on the public distillation samples corresponding to the public distillation sample identifier set and generating the model output vector corresponding to the public distillation sample identifier includes:

[0043] The public distillation sample identifier parsing unit is used to obtain public distillation sample identifiers from the public distillation sample identifier set and parse them to obtain the sample source identifier, sample index number and sample version number;

[0044] The common distillation sample reading unit is used to locate common distillation samples based on the sample source identifier, sample index number and sample version number, and to read the common distillation sample data.

[0045] The input feature construction unit is used to extract features and align dimensions of common distillation sample data according to the input feature definition, and generate the model input vector.

[0046] The inference batch generation unit is used to assemble multiple model input vectors into inference batch data and generate inference batch identifiers for the inference batch data.

[0047] The forward inference computation unit is used to input the inference batch data into the updated terminal local model instance, perform layer-by-layer forward computation according to the network layer sequence definition of the terminal local model instance, and obtain the prediction output corresponding to the inference batch identifier.

[0048] The output vector generation unit is used to organize the predicted output into a model output vector according to the indexing rules of the output node set of the local model instance on the terminal, and to establish a correspondence between the model output vector and the public distillation sample identifier.

[0049] Optional, WTA rules include:

[0050] The model output vector receiving unit is used to receive the model output vector that establishes a correspondence with the public distillation sample identifier, and to read the output index space that is fixed in the local model instance on the terminal.

[0051] The candidate index sequence generation unit is used to generate sorted pairs for each output component of the model output vector in the output index space, and output the candidate index sequence in descending order.

[0052] The consistency interval generation unit is used to extract the first sorting output value and the second sorting output value from the candidate index sequence and calculate the winner consistency interval.

[0053] The winner pattern determination unit is used to input the winner consistency interval into the consistency determination rule and output the pattern identifier, which includes a single winner pattern identifier and a multi-winner pattern identifier.

[0054] The candidate winner index set construction unit is used to select the first sorted output value index from the candidate index sequence to generate the candidate winner index set under the single winner mode identifier, and to read the number of WTA winner indexes in the collaborative modeling task parameters under the multi-winner mode identifier, and extract them from the candidate index sequence to generate the candidate winner index set.

[0055] The parallel conflict set extraction unit is used to identify parallel sets related to the candidate winner index set based on the candidate index sequence. The parallel sets include the parallel sets with the highest ranking values ​​and the first... Given a set of boundary sorted values, output the set of corresponding indexes for that set.

[0056] The deterministic resolution and pruning unit is used to deterministically sort the set of parallel indices based on the numbering rules of the output index space. Under the single-win mode, it selects the first index after sorting from the set of parallel indices with the highest sort value as the unique winner position index. Under the multi-win mode, it selects the first index after sorting from the set of parallel indices with the highest sort value as the unique winner position index. The set of boundary sorted values ​​is padded or trimmed according to the deterministic sorting result. By using a set of indexes, we can obtain the unique winner index set.

[0057] The Discrete Discriminant Index Set Encapsulation Unit is used to encapsulate the unique winner position index and the unique winner index set into a discrete discriminant index set, and output it while maintaining the correspondence with the common distillation sample identifier.

[0058] Optionally, the output of the target model results for collaborative modeling includes:

[0059] The sample-level alignment unit is used to receive discrete discriminant index sets from multiple IoT terminals and perform sample-level grouping of the discrete discriminant index sets based on the common distillation sample identifier set, merging discrete discriminant index sets with the same common distillation sample identifier into sample index groups;

[0060] The sample-level aggregation unit is used to perform index counting statistics on the discrete discriminant index set in each sample index group, generate an index count table, and select the top K index positions in descending order of the count value based on the index count table to generate an aggregated discrete discriminant index set, where K is the number of WTA winner indexes in the collaborative modeling task parameters;

[0061] The parallel counting and resolution unit performs deterministic sorting of the parallel indices based on the numbering rules of the output index space, and completes or trims the aggregated discrete discriminant index set to K index positions according to the deterministic sorting results;

[0062] The distillation supervision signal construction unit is used to convert the aggregated discrete discriminant index set into a distillation supervision signal vector, and to establish a correspondence between the distillation supervision signal vector and the public distillation sample identifier;

[0063] The target model federated distillation update unit is used to perform forward inference on the target model with common distillation samples as input to obtain the target model output. Based on the target model output and the distillation supervision signal vector, the distillation loss is calculated and backpropagation and parameter update are performed to obtain the updated target model.

[0064] The target model result output unit is used to output the updated target model as the target model result for completing the collaborative modeling.

[0065] The beneficial effects of this invention are:

[0066] (1) This invention completes local model training and public distillation sample inference on multiple IoT terminal sides, and only uploads the discrete discriminant index set obtained by mapping the model output vector, so as to realize that the original data is not shared during the cross-terminal collaborative modeling process, thereby reducing the privacy exposure risk caused by the transmission of terminal data.

[0067] (2) This invention uses a sample-level alignment and aggregation mechanism to merge and count the discrete discriminant index sets from multiple IoT terminals according to the common distillation sample identifier, forming a unified aggregated discrete discriminant index set, and constructing a distillation supervision signal vector accordingly to complete the federated distillation update of the target model, ensuring the executability and consistency of the collaborative modeling process.

[0068] (3) This invention introduces a winner pattern determination and a deterministic resolution mechanism for parallel conflicts into the WTA rules, so that the discrete discriminant index set can maintain an alignable index expression under different terminal output conditions, and forms a fixed number of winner index sets through deterministic sorting and completion pruning mechanisms, thereby improving the stability and reproducibility of the sample-level aggregation and distillation update process. Attached Figure Description

[0069] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0070] Figure 1 This is a flowchart of the IoT privacy data collaborative modeling system based on federated learning proposed in this invention;

[0071] Figure 2 This is a schematic diagram of the WTA index mapping rule process proposed in this invention. Detailed Implementation

[0072] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0073] refer to Figures 1-2 A federated learning-based collaborative modeling system for IoT privacy data includes the following steps:

[0074] The collaborative task and model configuration module configures the collaborative modeling task parameters, initial model identifier, and common distillation sample identifier set, and sends the collaborative modeling task parameters, initial model identifier, and common distillation sample identifier set to multiple IoT terminals;

[0075] The model initialization module is deployed on each IoT terminal. It builds the initial model based on the initial model identifier and generates a local model instance on the terminal.

[0076] The terminal local data management module is deployed in each IoT terminal to store and manage the local data set generated by the corresponding terminal.

[0077] The terminal local model training module, based on the parameters of the collaborative modeling task, uses the local dataset to train and update the terminal local model instance, generating the updated terminal local model instance.

[0078] The common distillation sample inference module performs forward inference on the common distillation samples corresponding to the common distillation sample identifier set based on the updated local model instance on the terminal, and generates the model output vector corresponding to the common distillation sample identifier.

[0079] The interface layer WTA index mapping module maps the model output vector to a discrete discriminant index set containing only the winner position index according to the WTA rules;

[0080] The discriminant index collaborative distillation module receives discrete discriminant index sets from multiple IoT terminals, performs sample-level alignment and aggregation based on a common distillation sample identifier set, and performs federated distillation updates on the target model based on the aggregated discrete discriminant index set, outputting the target model result after collaborative modeling.

[0081] In this embodiment, the collaborative task and model configuration module includes:

[0082] The parameters for collaborative modeling tasks include task identifier, set of participating terminal identifiers, number of local training steps, local training batch size, local learning rate, and number of WTA winner indexes.

[0083] The initial model identifier includes the model structure identifier, the model version identifier, and the model parameter index. The model structure identifier is used to indicate the network structure type and output dimension of the local model instance on the terminal. The model version identifier is used to indicate the model version number. The model parameter index is used to locate the initial model parameter file.

[0084] The public distillation sample identifier set consists of multiple public distillation sample identifiers, each of which includes a sample source identifier, a sample index number, and a sample version number.

[0085] In this embodiment, the generation of the terminal local model instance includes:

[0086] The initial model identifier parsing unit is used to parse the initial model identifier to obtain the model structure identifier, model version identifier, and model parameter index.

[0087] The structure template acquisition unit is used to locate the target network structure template in the model structure template library based on the model structure identifier, and output the network layer sequence definition, input feature definition and output dimension definition corresponding to the target network structure template;

[0088] Output node construction unit, used to define and construct output node set based on output dimension, and to solidify the indexing rules of output node set into output index space;

[0089] A computation graph construction unit is used to construct network layers sequentially based on the network layer sequence definition and connect the network layers to generate a computation graph, wherein the computation graph includes a set of output nodes and trainable variables;

[0090] The parameter location and loading unit is used to locate the initial model parameter file based on the model version identifier and model parameter index, and load the set of parameter tensors corresponding to the trainable variables in the computation graph.

[0091] Parameter binding is used to bind a set of parameter tensors to trainable variables in the computation graph to obtain an initial model;

[0092] The model instantiation unit generates a local model instance for a terminal by binding the initial model with the terminal identifier in the set of participating terminal identifiers of the collaborative modeling task parameters.

[0093] In this embodiment, the local data set stored and managed by the terminal local data management module includes:

[0094] A subset of sample feature vectors, used to store sample feature vectors consistent with the input feature definition of the collaborative modeling task parameters;

[0095] A subset of sample labels is used to store the sample labels corresponding to the sample feature vectors;

[0096] A subset of the sample index is used to store the correspondence between sample identifiers, sample feature vectors, and sample labels, as well as the sample collection timestamps;

[0097] The sample identifier is a unique number generated by the local data management module of the terminal for each sample feature vector in the local data set. It is composed of the terminal identifier, the collection timestamp and the sample sequence number, and is used to establish the correspondence between the sample feature vector and the sample label.

[0098] The data is partitioned into subsets to store the training set index, validation set index, and test set index obtained from the parameter partitioning of the collaborative modeling task.

[0099] A subset of data versions is used to store the data version identifier and incremental update records of the local data set.

[0100] In this embodiment, the generation of the updated terminal local model instance includes:

[0101] The collaborative modeling task parameter parsing unit is used to parse the task identifier, local training steps, local training batch size, local learning rate and input feature definition from the collaborative modeling task parameters, and generate the training configuration.

[0102] The local dataset localization unit is used to locate the training set index in the local dataset based on the task identifier in the training configuration, and to define the sample feature vector and sample label corresponding to the training set index based on the input features.

[0103] The training sample pair building unit is used to align the sample feature vector with the sample label according to the sample identifier to generate a set of training sample pairs;

[0104] Alignment by sample identifier refers to using the sample identifier generated by the terminal's local data management module as the key, retrieving data corresponding to the same key value in the subset of sample feature vectors and the subset of sample labels, and forming training sample pairs by combining the retrieved sample feature vectors and sample labels;

[0105] The training batch generation unit is used to select training sample pairs from the training sample pair set based on the local training batch size and generate training batch data, and generate batch identifiers for the training batch data.

[0106] The forward computation unit is used to input the sample feature vectors from the training batch data into the local model instance on the terminal and generate a prediction output corresponding to the batch identifier.

[0107] The loss calculation unit is used to calculate the training loss based on the predicted output and the sample labels in the training batch data, and to bind the training loss to the batch identifier.

[0108] Training loss calculation refers to inputting the predicted output and the sample labels in the training batch data into a preset loss function. Cross-entropy loss is used for classification tasks and mean squared error loss is used for regression tasks. The training loss value is obtained and bound to the batch identifier.

[0109] The backpropagation unit is used to perform backpropagation calculation on the training loss bound to the batch identifier, and generate a set of parameter gradients corresponding to the trainable variables in the computation graph of the local model instance on the terminal.

[0110] Backpropagation computation refers to calculating the gradient of the computation graph of the local model instance on the terminal from the output node to the input direction with the training loss as the target, and obtaining the partial derivatives of each trainable variable according to the chain rule to form a set of parameter gradients.

[0111] The parameter update unit is used to update the parameter gradient set based on the local learning rate in the training configuration, and generate an updated terminal local model instance.

[0112] Parameter update refers to updating the parameter tensors corresponding to the parameter gradient set one by one according to the local learning rate. The gradient descent update rule is used to subtract the product of the local learning rate and the corresponding gradient from each parameter tensor to obtain the updated parameter tensor set.

[0113] The training iteration control unit is used to perform training batch construction, forward inference calculation, training loss calculation, backpropagation calculation, and parameter update in a loop according to the local training step number.

[0114] In this embodiment, performing forward inference on the public distillation samples corresponding to the public distillation sample identifier set and generating the model output vector corresponding to the public distillation sample identifier includes:

[0115] The public distillation sample identifier parsing unit is used to obtain public distillation sample identifiers from the public distillation sample identifier set and parse them to obtain the sample source identifier, sample index number and sample version number;

[0116] The common distillation sample reading unit is used to locate common distillation samples based on the sample source identifier, sample index number and sample version number, and to read the common distillation sample data.

[0117] The input feature construction unit is used to extract features and align dimensions of common distillation sample data according to the input feature definition, and generate the model input vector.

[0118] Feature extraction and dimension alignment refers to extracting corresponding fields from common distillation sample data according to the input feature definition and concatenating them into a feature vector in a preset order. Missing fields are filled with default values, and numerical fields are normalized according to a preset range to obtain a model input vector that is consistent with the input dimension.

[0119] The inference batch generation unit is used to assemble multiple model input vectors into inference batch data and generate inference batch identifiers for the inference batch data.

[0120] The forward inference computation unit is used to input the inference batch data into the updated terminal local model instance, perform layer-by-layer forward computation according to the network layer sequence definition of the terminal local model instance, and obtain the prediction output corresponding to the inference batch identifier.

[0121] Layer-by-layer forward computation refers to using the model input vector as the input layer output, executing the operator computation of each network layer sequentially according to the network layer sequence definition, using the output of the previous layer as the input of the next layer, until the output layer generates the predicted output;

[0122] The output vector generation unit is used to organize the predicted output into a model output vector according to the indexing rules of the output node set of the local model instance on the terminal, and to establish a correspondence between the model output vector and the public distillation sample identifier.

[0123] In this implementation, the WTA rules include:

[0124] The model output vector receiving unit is used to receive the model output vector that establishes a correspondence with the public distillation sample identifier, and to read the output index space that is fixed in the local model instance on the terminal.

[0125] The candidate index sequence generation unit is used to generate sorted pairs for each output component of the model output vector in the output index space, and output the candidate index sequence in descending order.

[0126] The consistency interval generation unit is used to extract the first sorting output value and the second sorting output value from the candidate index sequence and calculate the winner consistency interval.

[0127] The winner pattern determination unit is used to input the winner consistency interval into the consistency determination rule and output the pattern identifier, which includes a single winner pattern identifier and a multi-winner pattern identifier.

[0128] The consistency determination rule refers to comparing the winner consistency interval with a preset interval threshold. When the winner consistency interval is greater than or equal to the interval threshold, a single-win mode identifier is output, and when the winner consistency interval is less than the interval threshold, a multi-win mode identifier is output.

[0129] The candidate winner index set construction unit is used to select the first sorted output value index from the candidate index sequence to generate the candidate winner index set under the single winner mode identifier, and to read the number of WTA winner indexes in the collaborative modeling task parameters under the multi-winner mode identifier, and extract them from the candidate index sequence to generate the candidate winner index set.

[0130] The parallel conflict set extraction unit is used to identify parallel sets related to the candidate winner index set based on the candidate index sequence. The parallel sets include the parallel sets with the highest ranking values ​​and the first... Given a set of boundary sorted values, output the set of corresponding indexes for that set.

[0131] The deterministic resolution and pruning unit is used to deterministically sort the set of parallel indices based on the numbering rules of the output index space. Under the single-win mode, it selects the first index after sorting from the set of parallel indices with the highest sort value as the unique winner position index. Under the multi-win mode, it selects the first index after sorting from the set of parallel indices with the highest sort value as the unique winner position index. The set of boundary sorted values ​​is padded or trimmed according to the deterministic sorting result. By using a set of indexes, we can obtain the unique winner index set.

[0132] The numbering rule of the output index space refers to the sequential numbering of the output node set according to the output dimension definition. The numbering starts from 0 or 1 and increments. The number corresponds one-to-one with the position of each output component of the model output vector and serves as the value range of the winner position index.

[0133] Deterministic sorting refers to arranging a set of parallel indices in ascending order according to the numbering rules of the output index space. The sorting result is fixed and reproducible, and the sorted index order is used as the basis for index selection and pruning in the case of parallel indices.

[0134] "Completing or pruning to K indexes" means, in a multi-winner model, sequentially selecting parallel indexes according to the deterministic sorting results and adding them to the winner index set until the number reaches K, or when the number of winner index sets exceeds K, retaining the first K indexes according to the sorting results.

[0135] The Discrete Discriminant Index Set Encapsulation Unit is used to encapsulate the unique winner position index and the unique winner index set into a discrete discriminant index set, and output it while maintaining the correspondence with the common distillation sample identifier.

[0136] In this embodiment, the output of the target model result for collaborative modeling includes:

[0137] The sample-level alignment unit is used to receive discrete discriminant index sets from multiple IoT terminals and perform sample-level grouping of the discrete discriminant index sets based on the common distillation sample identifier set, merging discrete discriminant index sets with the same common distillation sample identifier into sample index groups;

[0138] The sample-level aggregation unit is used to perform index counting statistics on the discrete discriminant index set in each sample index group, generate an index count table, and select the top K index positions in descending order of the count value based on the index count table to generate an aggregated discrete discriminant index set, where K is the number of WTA winner indexes in the collaborative modeling task parameters;

[0139] Index counting statistics refer to traversing and accumulating the number of times each index position appears in the sample index group by each discrete discriminant index set, thus forming an index count table corresponding to the index position and the number of occurrences.

[0140] The parallel counting and resolution unit performs deterministic sorting of the parallel indices based on the numbering rules of the output index space, and completes or trims the aggregated discrete discriminant index set to K index positions according to the deterministic sorting results;

[0141] The distillation supervision signal construction unit is used to convert the aggregated discrete discriminant index set into a distillation supervision signal vector, and to establish a correspondence between the distillation supervision signal vector and the public distillation sample identifier;

[0142] The distillation supervision signal vector is a vector with the same dimension as the output index space. It is assigned a value of 1 at the index position corresponding to the aggregated discrete discriminant index set, and a value of 0 at the other index positions. It is also associated with the common distillation sample identifier.

[0143] The target model federated distillation update unit is used to perform forward inference on the target model with common distillation samples as input to obtain the target model output. Based on the target model output and the distillation supervision signal vector, the distillation loss is calculated and backpropagation and parameter update are performed to obtain the updated target model.

[0144] The target model is the model instance to be updated in the discriminant index co-distillation module, and has the same output index space as the local model instance on the terminal.

[0145] Calculating distillation loss and performing backpropagation and parameter update refers to inputting the target model output and the distillation supervision signal vector into the distillation loss function to obtain the distillation loss value, performing backpropagation calculation on the target model computation graph to obtain the parameter gradient set, and updating the target model parameter tensor one by one according to the learning rate to generate the updated target model.

[0146] The target model result output unit is used to output the updated target model as the target model result for completing the collaborative modeling.

[0147] Example 1:

[0148] To verify the feasibility of this invention in practice, it was applied to the energy and equipment operation and maintenance system of a large industrial park. The park contains multiple factories and independent tenants, encompassing different production units such as injection molding, stamping, surface mount technology (SMT), air compressor stations, central air conditioning rooms, and warehousing and logistics. Each unit deploys numerous IoT terminals to collect data on electricity, water, gas, temperature, humidity, vibration, motor current, power factor, equipment start / stop status, and production cycle time. The park's operation and maintenance department aims to use this terminal data to build a unified prediction and anomaly detection model to identify energy consumption anomalies, early equipment failures, and operational status drift. However, because the factories belong to different tenants, and the production data involves process parameters, output, and electricity consumption curves, it is highly commercially sensitive, and tenants typically do not allow raw data to be uploaded to the park's central server or the cloud. Furthermore, some terminals within the park are low-power gateways or embedded data collectors, resulting in intermittent packet loss and bandwidth fluctuations in the network links. Traditional centralized modeling requires long data upload times, making it difficult to meet stability requirements; while traditional federated learning requires frequent uploads of gradients or model parameters, incurring high communication overhead and prone to training interruptions or synchronization failures on some terminals. Therefore, during the pilot phase, the park introduced the IoT privacy data collaborative modeling system based on federated learning of this invention, which completes local training and updates on multiple IoT terminals and adopts a shared approach of public distillation sample inference and discrete discriminant index to complete cross-terminal collaborative distillation modeling.

[0149] In this scenario, the system is deployed on multiple factory gateway terminals, each connected to different types of sensors, resulting in a distinct heterogeneous data structure. The collaborative task and model configuration module runs on the park's edge server, configuring collaborative modeling task parameters, initial model identifiers, and a common distillation sample identifier set, and sending this information to participating terminals. Collaborative modeling task parameters include task identifiers, input feature definitions, output dimension definitions, local training steps, local training batch size, local learning rate, and the number of WTA winner indices. The input feature definition constrains each terminal to construct sample feature vectors with consistent dimensions, while the output dimension definition limits the output index space, ensuring a unified numbering system for subsequent winner position indices. The initial model identifier points to an initial model structure and parameter version that can be loaded on each terminal. The common distillation sample identifier set indicates a batch of common samples derived from the park's historical publicly available operational data and synthetic data. These samples do not contain any tenant process-sensitive fields and have been formatted according to the input feature definition, enabling collaborative inference by all terminals.

[0150] The model initialization module is deployed on each IoT terminal. After receiving the initial model identifier, the terminal parses it to obtain the model structure identifier, model version identifier, and model parameter index, and locates the corresponding network structure template from the model structure template library. The network structure template library stores network structure templates that correspond one-to-one with the model structure identifier on the terminal side. Each network structure template includes network layer sequence definitions, input feature definitions, and output dimension definitions. The terminal constructs a computation graph based on the network layer sequence definitions and builds an output node set based on the output dimension definitions, while simultaneously fixing the indexing rules of the output node set into the output index space. The terminal then loads the parameter tensor set according to the model version identifier and model parameter index and binds it to the trainable variables of the computation graph to obtain the initial model. Furthermore, it combines the terminal identifiers in the participating terminal identifier set of the collaborative modeling task parameters to generate a local model instance and a model instance identifier, providing a unified calling object for subsequent local training and inference.

[0151] The local data management module is deployed on each IoT terminal side to store and manage the local data set. In this embodiment, the local data set includes a sample feature vector storage subset, a sample label storage subset, a sample identifier association index subset, a dataset partitioning index subset, and a local data set version management subset. After collecting sensor data, the terminal constructs sample feature vectors according to the input feature definition. Sample labels are generated locally by the terminal based on device status or alarm records and correspond one-to-one with the sample feature vectors. Sample identifiers are generated by the terminal's local data management module, using a unique key formed by concatenating the terminal identifier, collection timestamp, and sample sequence number, used to establish a searchable correspondence between sample feature vectors and sample labels. The dataset partitioning index is generated locally by the terminal as a training set index, validation set index, and test set index, ensuring that the terminal's local model training module can locate the training sample range according to the task identifier and stably reproduce the training batch construction process during iteration.

[0152] The terminal local model training module trains and updates the terminal local model instance based on the collaborative modeling task parameters. The collaborative modeling task parameter parsing unit parses the task identifier, local training steps, local training batch size, local learning rate, and input feature definition to generate a training configuration. The local dataset location unit locates the training set index in the local dataset based on the task identifier in the training configuration, and locates the sample feature vector and sample label corresponding to the training set index based on the input feature definition. The training sample pair construction unit uses the sample identifier as the key to retrieve data corresponding to the same key value in the sample feature vector storage subset and the sample label storage subset, and combines the retrieved sample feature vectors and sample labels into a training sample pair set. The training batch generation unit selects training sample pairs from the training sample pair set according to the local training batch size and generates training batch data, while also generating a batch identifier. The forward computation unit inputs the sample feature vectors from the training batch data into the terminal local model instance to generate a prediction output corresponding to the batch identifier. The loss calculation unit inputs the prediction output and the sample labels from the training batch data into a preset loss function to calculate the training loss and binds the training loss to the batch identifier. The backpropagation unit performs gradient calculation on the computation graph of the local model instance on the terminal with the training loss as the target to generate a set of parameter gradients. The parameter update unit updates the parameter tensor one by one according to the local learning rate to obtain the updated local model instance on the terminal. The training iteration control unit executes the above process in a loop according to the local training steps.

[0153] After the terminal completes one round of local training, the common distillation sample inference module performs forward inference on the common distillation samples corresponding to the common distillation sample identifier set. The common distillation sample identifier parsing unit acquires each common distillation sample identifier and parses it to obtain the sample source identifier, sample index number, and sample version number. The common distillation sample reading unit uses this information to locate the common distillation sample and read the common distillation sample data. The input feature construction unit performs feature extraction and dimension alignment on the common distillation sample data according to the input feature definition. Specifically, it extracts corresponding fields from the common distillation sample data and concatenates them into feature vectors in a preset order, fills missing fields with default values, and normalizes numerical fields according to a preset range to obtain the model input vector. The inference batch generation unit assembles multiple model input vectors into inference batch data and generates inference batch identifiers. The forward inference calculation unit inputs the inference batch data into the updated terminal local model instance and performs forward calculation layer by layer according to the network layer sequence definition to obtain the predicted output. The output vector generation unit organizes the predicted output into model output vectors according to the index rules of the output node set and establishes a correspondence between the model output vectors and the common distillation sample identifiers.

[0154] The interface layer WTA index mapping module receives the model output vector and maps it to a discrete discriminative index set according to the improved WTA rules. The model output vector receiving unit receives the model output vector that corresponds to the common distillation sample identifier and reads the output index space fixed in the local model instance on the terminal. The candidate index sequence generation unit generates sorted pairs for each output component of the model output vector within the output index space and outputs the candidate index sequence in descending order. The consistency interval generation unit extracts the first and second sorted output values ​​from the candidate index sequence to calculate the winner consistency interval. The winner mode determination unit inputs the winner consistency interval into the consistency determination rules and outputs mode identifiers, including single-winner mode identifiers and multi-winner mode identifiers. The candidate winner index set construction unit selects the first sorted output value index from the candidate index sequence under the single-winner mode identifier to generate a candidate winner index set. Under the multi-winner mode identifier, it reads the number of WTA winner indexes from the collaborative modeling task parameters and extracts them from the candidate index sequence to generate a candidate winner index set. The parallel conflict set extraction unit identifies parallel sets related to the candidate winner index set based on the candidate index sequence. These parallel sets include the parallel set of maximum ranking values ​​and the parallel set of ranking values ​​at the Kth boundary, and output the corresponding parallel index set. The deterministic resolution and pruning unit performs deterministic sorting on the parallel index set based on the numbering rules of the output index space. Under the single-winner mode identifier, it selects the first index after sorting from the parallel set of maximum ranking values ​​as the unique winner position index. Under the multi-winner mode identifier, it completes or prunes the parallel set of ranking values ​​at the Kth boundary according to the deterministic sorting result to K indices, obtaining the unique winner index set. The discrete discriminant index set encapsulation unit encapsulates the unique winner position index and the unique winner index set into a discrete discriminant index set, maintaining its correspondence with the common distillation sample identifier before outputting it.

[0155] The discriminant index collaborative distillation module runs on the campus edge server. It receives discrete discriminant index sets from multiple IoT terminals, performs sample-level alignment and aggregation, and then performs federated distillation updates on the target model based on the aggregation results, outputting the completed collaborative modeling target model. The sample-level alignment unit groups the discrete discriminant index sets based on a common distillation sample identifier set, merging discrete discriminant index sets with the same common distillation sample identifier into sample index groups. The sample-level aggregation unit iterates through and accumulates the index positions in each discrete discriminant index set within the sample index group, recording the frequency of each index position to form an index count table. Based on the index count table, it selects the top K index positions in descending order of count values ​​to generate the aggregated discrete discriminant index set. The parallel counting resolution unit performs deterministic sorting based on the numbering rules of the output index space when parallel count values ​​exist in the index count table, and completes or trims the aggregated discrete discriminant index set to K index positions according to the deterministic sorting result. The distillation supervision signal construction unit converts the aggregated discrete discriminant index set into a distillation supervision signal vector. This vector has the same dimension as the output index space. It assigns a value of 1 to the index position corresponding to the aggregated discrete discriminant index set and 0 to the remaining index positions, establishing a correspondence with the common distillation sample identifier. The target model federated distillation update unit uses the common distillation samples corresponding to the common distillation sample identifier as input to perform forward inference on the target model to obtain the target model output. It calculates the distillation loss and performs backpropagation and parameter updates. Specifically, it inputs the target model output and the distillation supervision signal vector into the distillation loss function to obtain the distillation loss value. Backpropagation is then performed on the distillation loss value on the target model computation graph to obtain the parameter gradient set. The target model parameter tensor is then updated one by one according to the learning rate to generate the updated target model. The target model result output unit outputs the updated target model as the result of the collaborative modeling.

[0156] To verify the effectiveness of this invention, a pilot study selected multiple factory terminals to participate in collaborative modeling. The comparison schemes were a centralized training scheme and a traditional federated learning scheme. The centralized training scheme uploaded the raw data from the terminals to an edge server for unified training, while the traditional federated learning scheme used parameter aggregation for synchronization. The present invention employs a federated distillation method with shared discrete discriminant indexes. Evaluation metrics included model prediction accuracy, anomaly detection recall rate, uplink traffic on the terminal side, single-round collaborative processing time, training failure rate, and average CPU usage on the terminal side. Anomaly identification was based on park maintenance work orders and equipment alarm records; energy consumption anomalies were confirmed based on historical energy consumption distribution of similar equipment and manual verification.

[0157] Table 1 Comparison of Collaborative Modeling Results

[0158]

[0159] As shown in Table 1, without uploading the original data, the prediction accuracy of the proposed solution is close to that of centralized training, and its anomaly recognition recall rate is higher than that of traditional federated learning solutions. More importantly, the daily uplink communication volume on the terminal side of the proposed solution is significantly lower than that of traditional federated learning solutions, and the single-round collaboration time is shorter, making it adaptable to network fluctuation scenarios. Traditional federated learning solutions require uploading parameters or gradients, resulting in a significantly higher terminal training failure rate during network congestion. In contrast, the proposed solution uses a discrete discriminant index set as the collaborative information, resulting in a smaller message body and a more stable training process.

[0160] To further demonstrate the compatibility with different types of terminals, fine-grained statistics were performed on selected terminals, including the number of common sample inferences, the number of uploaded messages, the upload volume, and the number of inference batch failures for each terminal. The number of common sample inferences per day was consistent across all terminals. Inference was performed based on the common distilled sample identifier set, and a discrete discriminant index set was generated and uploaded. The number of uploaded messages corresponded to the number of inferences. Due to differences in terminal hardware and network conditions, the upload volume varied slightly, but remained within a low range overall. The number of inference batch failures was generally low, meeting the requirements for continuous operation.

[0161] Table 2. Data on Terminal Collaborative Information Upload Volume and Inference Stability

[0162]

[0163] During implementation, park maintenance personnel verified the effectiveness of anomaly event identification. Taking the "frequent load jitter" event at the air compressor station as an example, the terminal collected features of increased power fluctuations, exhaust pressure fluctuations, and increased start-stop frequency. During the inference phase, the collaborative model of this invention output a set of discrete discriminant indexes corresponding to the common distillation samples, which underwent concentrated changes. After sample-level aggregation, the discriminant index collaborative distillation module formed a stable distillation supervision signal vector. After the target model was updated with federated distillation, the consistency of its identification of similar patterns improved. Maintenance personnel discovered during on-site inspection that the intake filter blockage caused the load anomaly and addressed it promptly, preventing the anomaly from escalating further. This process demonstrates the feasibility of this invention in multi-terminal collaborative modeling scenarios, achieving cross-terminal knowledge alignment through common sample inference and discrete index aggregation.

[0164] In summary, this embodiment successfully deployed and verified the system collaborative modeling in a multi-tenant, heterogeneous terminal, and network fluctuation environment in an industrial park. Through local training and updating on the terminal side, common distillation sample inference, improved WTA discrete discriminant index mapping, sample-level alignment aggregation, and federated distillation updates, this invention achieves collaborative machine learning modeling of multiple distributed data sources without sharing the original data. It also obtains quantifiable data support in terms of model performance, communication load, and collaborative stability, and can meet the practical application needs of energy consumption prediction and equipment anomaly identification in industrial parks.

[0165] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A collaborative modeling system for privacy-preserving IoT data based on federated learning, characterized in that, Includes the following modules: The collaborative task and model configuration module configures the collaborative modeling task parameters, initial model identifier, and common distillation sample identifier set, and sends the collaborative modeling task parameters, initial model identifier, and common distillation sample identifier set to multiple IoT terminals; The model initialization module is deployed on each IoT terminal. It constructs the initial model based on the initial model identifier and generates a local model instance on the terminal. The terminal local data management module is deployed on each IoT terminal to store and manage the local data set generated by the corresponding terminal. The terminal local model training module, based on the parameters of the collaborative modeling task, uses the local dataset to train and update the terminal local model instance, generating the updated terminal local model instance. The common distillation sample inference module performs forward inference on the common distillation samples corresponding to the common distillation sample identifier set based on the updated local model instance on the terminal, and generates the model output vector corresponding to the common distillation sample identifier. The interface layer WTA index mapping module maps the model output vector to a discrete discriminant index set containing only the winner position index according to the WTA rules; The WTA rules include: The model output vector receiving unit is used to receive the model output vector that establishes a correspondence with the public distillation sample identifier, and to read the output index space that is fixed in the local model instance on the terminal. The candidate index sequence generation unit is used to generate sorted pairs for each output component of the model output vector in the output index space, and output the candidate index sequence in descending order. The consistency interval generation unit is used to extract the first sorting output value and the second sorting output value from the candidate index sequence and calculate the winner consistency interval. The winner pattern determination unit is used to input the winner consistency interval into the consistency determination rule and output the pattern identifier, which includes a single winner pattern identifier and a multi-winner pattern identifier. The candidate winner index set construction unit is used to select the first sorted output value index from the candidate index sequence to generate the candidate winner index set under the single winner mode identifier, and to read the number of WTA winner indexes in the collaborative modeling task parameters under the multi-winner mode identifier, and extract them from the candidate index sequence to generate the candidate winner index set. The parallel conflict set extraction unit is used to identify parallel sets related to the candidate winner index set based on the candidate index sequence. The parallel sets include the parallel sets with the highest ranking values ​​and the first... Given a set of boundary sorted values, output the set of corresponding indexes for that set. The deterministic resolution and pruning unit is used to deterministically sort the set of parallel indices based on the numbering rules of the output index space. Under the single-win mode, it selects the first index after sorting from the set of parallel indices with the highest sort value as the unique winner position index. Under the multi-win mode, it selects the first index after sorting from the set of parallel indices with the highest sort value as the unique winner position index. The set of boundary sorted values ​​is padded or trimmed according to the deterministic sorting result. By using a set of indexes, we can obtain the unique winner index set. The Discrete Discriminant Index Set Encapsulation Unit is used to encapsulate the unique winner position index and the unique winner index set into a discrete discriminant index set, and output it while maintaining the correspondence with the common distillation sample identifier; The discriminant index collaborative distillation module receives discrete discriminant index sets from multiple IoT terminals, performs sample-level alignment and aggregation based on a common distillation sample identifier set, and performs federated distillation update on the target model based on the aggregated discrete discriminant index set, outputting the target model result after collaborative modeling is completed. The output of the target model result after completing the collaborative modeling includes: The sample-level alignment unit is used to receive discrete discriminant index sets from multiple IoT terminals and perform sample-level grouping of the discrete discriminant index sets based on the common distillation sample identifier set, merging discrete discriminant index sets with the same common distillation sample identifier into sample index groups; The sample-level aggregation unit is used to perform index counting statistics on the discrete discriminant index set in each sample index group, generate an index count table, and select the top K index positions in descending order of the count value based on the index count table to generate an aggregated discrete discriminant index set, where K is the number of WTA winner indexes in the collaborative modeling task parameters; The parallel counting and resolution unit performs deterministic sorting of the parallel indices based on the numbering rules of the output index space, and completes or trims the aggregated discrete discriminant index set to K index positions according to the deterministic sorting results; The distillation supervision signal construction unit is used to convert the aggregated discrete discriminant index set into a distillation supervision signal vector, and to establish a correspondence between the distillation supervision signal vector and the public distillation sample identifier; The target model federated distillation update unit is used to perform forward inference on the target model with common distillation samples as input to obtain the target model output. Based on the target model output and the distillation supervision signal vector, the distillation loss is calculated and backpropagation and parameter update are performed to obtain the updated target model. The target model result output unit is used to output the updated target model as the target model result for completing the collaborative modeling.

2. The IoT privacy data collaborative modeling system based on federated learning according to claim 1, characterized in that, The collaborative task and model configuration module includes: The parameters for collaborative modeling tasks include task identifier, set of participating terminal identifiers, number of local training steps, local training batch size, local learning rate, and number of WTA winner indexes. The initial model identifier includes the model structure identifier, the model version identifier, and the model parameter index. The model structure identifier is used to indicate the network structure type and output dimension of the local model instance on the terminal. The model version identifier is used to indicate the model version number. The model parameter index is used to locate the initial model parameter file. The public distillation sample identifier set consists of multiple public distillation sample identifiers, each of which includes a sample source identifier, a sample index number, and a sample version number.

3. The IoT privacy data collaborative modeling system based on federated learning according to claim 1, characterized in that, The generation of local model instances on the terminal includes: The initial model identifier parsing unit is used to parse the initial model identifier to obtain the model structure identifier, model version identifier, and model parameter index. The structure template acquisition unit is used to locate the target network structure template in the model structure template library based on the model structure identifier, and output the network layer sequence definition, input feature definition and output dimension definition corresponding to the target network structure template; Output node construction unit, used to define and construct output node set based on output dimension, and to solidify the indexing rules of output node set into output index space; A computation graph construction unit is used to construct network layers sequentially based on the network layer sequence definition and connect the network layers to generate a computation graph, wherein the computation graph includes a set of output nodes and trainable variables; The parameter location and loading unit is used to locate the initial model parameter file based on the model version identifier and model parameter index, and load the set of parameter tensors corresponding to the trainable variables in the computation graph. Parameter binding is used to bind a set of parameter tensors to trainable variables in the computation graph to obtain an initial model; The model instantiation unit generates a local model instance for a terminal by binding the initial model with the terminal identifier in the set of participating terminal identifiers of the collaborative modeling task parameters.

4. The IoT privacy data collaborative modeling system based on federated learning according to claim 1, characterized in that, The local data set stored and managed by the terminal local data management module includes: A subset of sample feature vectors, used to store sample feature vectors consistent with the input feature definition of the collaborative modeling task parameters; A subset of sample labels is used to store the sample labels corresponding to the sample feature vectors; A subset of the sample index is used to store the correspondence between sample identifiers, sample feature vectors, and sample labels, as well as the sample collection timestamps; The data is partitioned into subsets to store the training set index, validation set index, and test set index obtained from the parameter partitioning of the collaborative modeling task. A subset of data versions is used to store the data version identifier and incremental update records of the local data set.

5. The IoT privacy data collaborative modeling system based on federated learning according to claim 1, characterized in that, The updated generation of local terminal model instances includes: The collaborative modeling task parameter parsing unit is used to parse the task identifier, local training steps, local training batch size, local learning rate and input feature definition from the collaborative modeling task parameters, and generate the training configuration. The local dataset localization unit is used to locate the training set index in the local dataset based on the task identifier in the training configuration, and to define the sample feature vector and sample label corresponding to the training set index based on the input features. The training sample pair building unit is used to align the sample feature vector with the sample label according to the sample identifier to generate a set of training sample pairs; The training batch generation unit is used to select training sample pairs from the training sample pair set based on the local training batch size and generate training batch data, and generate batch identifiers for the training batch data. The forward computation unit is used to input the sample feature vectors from the training batch data into the local model instance on the terminal and generate a prediction output corresponding to the batch identifier. The loss calculation unit is used to calculate the training loss based on the predicted output and the sample labels in the training batch data, and to bind the training loss to the batch identifier. The backpropagation unit is used to perform backpropagation calculation on the training loss bound to the batch identifier, and generate a set of parameter gradients corresponding to the trainable variables in the computation graph of the local model instance on the terminal. The parameter update unit is used to update the parameter gradient set based on the local learning rate in the training configuration, and generate an updated terminal local model instance. The training iteration control unit is used to perform training batch construction, forward inference calculation, training loss calculation, backpropagation calculation, and parameter update in a loop according to the local training step number.

6. The IoT privacy data collaborative modeling system based on federated learning according to claim 1, characterized in that, Perform forward inference on the common distillation samples corresponding to the common distillation sample identifier set and generate the model output vector corresponding to the common distillation sample identifier, including: The public distillation sample identifier parsing unit is used to obtain public distillation sample identifiers from the public distillation sample identifier set and parse them to obtain the sample source identifier, sample index number and sample version number; The common distillation sample reading unit is used to locate common distillation samples based on the sample source identifier, sample index number and sample version number, and to read the common distillation sample data. The input feature construction unit is used to extract features and align dimensions of common distillation sample data according to the input feature definition, and generate the model input vector. The inference batch generation unit is used to assemble multiple model input vectors into inference batch data and generate inference batch identifiers for the inference batch data. The forward inference computation unit is used to input the inference batch data into the updated terminal local model instance, perform layer-by-layer forward computation according to the network layer sequence definition of the terminal local model instance, and obtain the prediction output corresponding to the inference batch identifier. The output vector generation unit is used to organize the predicted output into a model output vector according to the indexing rules of the output node set of the local model instance on the terminal, and to establish a correspondence between the model output vector and the public distillation sample identifier.

Images