Data isomerism-oriented steady-state prototype federal learning training method

Abstract

The invention provides a data isomerism-oriented steady-state prototype federated learning training method. The method comprises the steps of 1, initializing a steady-state semantic anchor federated learning system; 2, randomly selecting a client, a global model and a cross-end semantic anchor library for synchronous broadcasting; step 3, only inactivation reasoning Dropout-only uncertainty estimation is carried out, and a sample level coupling amplitude modulator is generated; step 4, performing combined training on the dual-reference steady-state drift suppression constraint and the semantic anchor alignment constraint under unified coupling amplitude modulation; the client constructs and performs joint optimization in local training; the client is trained to obtain the updated local model parameters of the round; step 5, intra-class streaming convergence statistics and end-side semantic abstract uploading are carried out; and 6, updating and returning unit norm inertial fusion of the server aggregation and the semantic anchor library. According to the method, client drift and training oscillation can be effectively relieved in a heterogeneous Internet of Things data scene, and the generalization performance and convergence stability of a global model are improved.

Description

Technical Field

[0001] This invention belongs to the field of IoT security and edge intelligence technology, specifically relating to a steady-state prototype federated learning training method for heterogeneous data. Background Technology

[0002] In IoT security scenarios, security data is typically distributed across numerous terminals and edge nodes to support tasks such as abnormal behavior identification, intrusion detection, attack type classification, and security alerts. Due to significant differences in terminal types and deployment environments, the data collection conditions, network topology, service load, device firmware, and protocol stack implementations of different nodes often vary, leading to significant differences in security data category distribution, feature patterns, noise levels, and sample quality. Furthermore, IoT security data often contains sensitive information such as critical business traffic, device operation logs, or on-site images. Constrained by privacy protection, industry compliance, and security management regulations, it is difficult to centrally aggregate this data for unified training, thus limiting the training and iteration efficiency of centralized AI security models.

[0003] Federated learning, as a distributed collaborative training technique, achieves joint modeling with "data remaining on each end" by distributing global model parameters from the server, training on local data and uploading model updates from clients, and then aggregating the data on the server. Therefore, it is considered suitable for privacy-sensitive and multi-terminal collaborative scenarios such as IoT security. However, in real-world IoT environments, client data typically exhibits non-independent identically distributed (Non-IID) characteristics, manifesting as class skew, missing classes, scene differences, and domain shifts. This data heterogeneity can cause client drift and training process oscillations, slowing down the convergence speed of the global model, increasing performance fluctuations, and making it difficult to maintain consistent semantic representations of the same attack / anomaly category across different terminals, thus affecting cross-terminal generalization and transfer capabilities.

[0004] While existing technologies have been optimized through improved aggregation strategies, the introduction of regularization or consistency constraints, and personalized modeling, they still have shortcomings in IoT security scenarios. On the one hand, fixed-strength constraints often fail to distinguish the contributions of high-confidence samples from high-uncertainty samples, easily leading to overly strong constraints on noisy or domain-biased samples and instability. On the other hand, the lack of shared anchor points capable of stably carrying cross-client category semantics limits the effectiveness of cross-terminal "same-type semantic alignment." Furthermore, there is a lack of effective steady-state suppression mechanisms for inter-round drift and oscillation during federated round iterations, making it difficult to simultaneously ensure convergence stability and final accuracy. Therefore, it is necessary to propose a federated learning training method more suitable for heterogeneous data conditions across multiple terminals in IoT security, to improve the stability of the training process and enhance cross-terminal generalization capabilities. Summary of the Invention

[0005] Purpose of the Invention: This invention addresses the problems of client drift, training oscillation, and difficulty in aligning similar security semantics caused by the non-independent and identically distributed (Non-IID) nature of multi-terminal data during federated learning training in IoT security scenarios. It provides a steady-state prototype federated learning training method for heterogeneous IoT security data. Without uploading the original client security data, this invention achieves steady-state convergence and improves the generalization ability of the global model during federated training by constructing and continuously calibrating a cross-terminal semantic anchor library, introducing an uncertainty-driven sample-level coupling amplitude modulator, and employing a collaborative training mechanism of dual-reference steady-state drift suppression constraints and semantic anchor alignment constraints.

[0006] The method includes the following steps:

[0007] Step 1: Initialize the steady-state semantic anchor federated learning system and configure the parameters of the sample-level coupled amplitude modulator driven by unified uncertainty.

[0008] Step 2: Randomly select clients to synchronously broadcast with the global model and cross-platform semantic anchor library;

[0009] Step 3: Dropout-only uncertainty estimation is generated with a sample-level coupled amplitude modulator;

[0010] Step 4: Jointly train the dual-reference steady-state drift suppression constraint and semantic anchor alignment constraint under unified coupled amplitude modulation. The client constructs and jointly optimizes the supervised learning term, the dual-reference steady-state drift suppression constraint DRDD, and the semantic anchor alignment constraint SAA during local training. The client obtains the updated local model parameters after training. ;

[0011] Step 5: Intra-class streaming aggregation statistics and end-side semantic summary upload;

[0012] Step 6, Unit Norm Inertial Fusion Update and Backhaul of Server Aggregation and Semantic Anchor Library: The model parameters uploaded by the client are obtained from the server aggregation. Simultaneously, unit norm inertial fusion updates are performed on semantic anchors of each category, and the semantic summaries received in this round from the end-side are integrated into the cross-end semantic anchor library to obtain the updated cross-end semantic anchor library. The server will distribute the data in the next round of synchronization. and Repeat steps 2 through 6 until the termination condition is met.

[0013] Step 1 includes: initializing global model parameters on the server side. And establish a cross-platform semantic anchor library Configure hyperparameters, including server-side configuration of sample-level coupling amplitude modulation parameters, steady-state drift suppression constraint weights, semantic anchor alignment constraint weights, and anchor library rolling update parameters; complete local training initialization on the client side and establish a backtracking reference model cache. .

[0014] Step 2 includes: In each round of federated training, the server determines the set of participating clients. And broadcast global model parameters synchronously to participating clients. and cross-platform semantic anchor libraries ;in This refers to the cross-platform semantic anchor library maintained and distributed by the server in round t. Used as a shared semantic anchor point across clients to support subsequent semantic alignment constraints.

[0015] Step 3 includes: Client Receive the server's output and Then, Monte Carlo random inactivation MC-Dropout uncertainty estimation is performed on the local sample x to obtain the sample uncertainty measure. And map to generate sample-level amplitude modulation coefficients. ; It will serve as a unified modulation factor and will take effect simultaneously in subsequent steady-state drift suppression constraints and semantic anchor alignment constraints. This represents the global model parameters on the server side in round t;

[0016] The client employs a Dropout-only inference control method, placing only the layers containing Dropout in the training state while keeping the remaining layers in the inference state. It performs inference on samples more than twice to estimate uncertainty and generate sample-level amplitude modulation coefficients; this can be expressed by the formula:

[0017] ,

[0018] ,

[0019] Where x represents the local training sample on the client; The uncertainty measure for sample x is obtained from the fluctuation statistics of multiple random inactivation inference outputs of Monte Carlo random inactivation MC-Dropout, and then normalized. ; The confidence amplitude factor; This is the amplitude modulation reference coefficient; This is the uncertainty adjustment coefficient; and These are the lower and upper limits of amplitude modulation, respectively, satisfying... ; The interval cutoff function is defined as follows: ;in represents the intermediate parameter, and e represents the natural constant.

[0020] Step 4 includes:

[0021] Step 4-1, Dual-reference steady-state drift suppression constraint DRDD uses the broadcast reference model issued by the server. Client-side backtracking reference model As a dual reference source, it suppresses abrupt shifts in the relative reference during this round of updates; whereby... This represents the backtracking reference model parameters cached on the client side;

[0022] The dual-reference steady-state drift suppression constraint (DRDD) includes:

[0023] Broadcast reference: Global model parameters issued by the server. This is used to guide the current update to align the global direction.

[0024] Backtracking reference: Local model parameters saved by the client in the previous round This is used to suppress excessively rapid shifts in the current update cycle;

[0025] For the same sample x, the client extracts the current representation. Broadcast reference representation Backtracking reference representation After calculating the similarity, a steady-state loss is constructed, which is determined by the sample-level coupling amplitude modulation coefficient. Adaptive scaling loss strength:

[0026] ,

[0027] ,

[0028] ,

[0029] in, This represents the current model's representation of sample x; Let x represent the broadcast reference model's representation of sample x, with parameters as follows. ; This represents the backtracking reference model's representation of sample x, with parameters as follows: ; Cosine similarity or unit vector dot product. This indicates the similarity to the broadcast reference. This indicates the similarity term with the backreferenced reference. This is the two-reference steady-state drift suppression loss; exp is the natural exponential function; Softplus is the smoothing activation function;

[0030] Step 4-2, Semantic Anchor Alignment Constraints (SAA) for Cross-Platform Semantic Anchor Libraries The category semantic anchor vector in the data serves as an alignment benchmark, promoting the convergence of similar semantics across clients;

[0031] Step 4-3, Construct Anchor-Ready Gating: Only when Semantic Anchor Alignment (SAA) constraint is only applied to samples when a semantic anchor corresponding to the true category already exists in the sample data. Step 4-4, Unified Coupling Amplitude Modulation: Both the Dual Reference Steady-State Drift Suppression (DRDD) constraint and the Semantic Anchor Alignment (SAA) constraint are generated from the same sample-level coupling amplitude modulation coefficients in step 3. Uniform scaling;

[0032] In step 4-2, the semantic anchor alignment constraint (SAA) includes:

[0033] Maintain cross-platform semantic anchor library on the server side As shared semantic anchors, the client constructs a contrastive alignment constraint based on the similarity between sample representations and semantic anchor vectors during local training. This constraint brings the true class semantic anchors closer and separates the negative class semantic anchors. Anchor-ready gating is employed, enabling the constraint only for samples where true class anchors already exist in the semantic anchor library. The semantic anchor alignment constraint (SAA) is determined by a sample-level coupling amplitude modulation coefficient. Uniform scaling;

[0034] The anchor-ready gating is represented as follows:

[0035] ,

[0036] The similarity between the representation of a real category sample and a semantic anchor of the same category is calculated using the following formula. :

[0037] ,

[0038] The negative anchor aggregation term is calculated using the following formula. :

[0039] ,

[0040] The semantic anchor alignment loss is calculated using the following formula. :

[0041] ,

[0042] in, This represents the anchor-ready gate value for sample x, taking a value of 0 or 1. When semantic anchor alignment constraints are enabled, This indicates that it is not enabled; This indicates an indicator function that takes the value 1 if the condition within the parentheses is true, and 0 otherwise. Let y represent the semantic anchor vector of category c maintained by the server in round t; y is the true label of sample x. This represents the semantic anchor vector of category y in the cross-platform semantic anchor library maintained by the server in round t; when the number of negative class samples K is set, This is the set of negative classes after sampling.

[0043] Step 4-4 includes:

[0044] The client jointly optimizes the supervised learning term, the dual-reference steady-state drift suppression loss, and the semantic anchor alignment loss during local training to obtain the updated local model parameters. Joint optimization function The format is:

[0045] ,

[0046] in, Indicates the loss in supervised learning; To suppress drift loss in dual-reference steady state; This represents the semantic anchor alignment loss; The weighting hyperparameters for steady-state loss; The weight hyperparameters for semantic anchor alignment loss; To calculate the average of mini-batch samples, the client minimizes... For local model parameters Iterative updates will be performed.

[0047] Step 5 includes: the client maintains a cumulative vector sum and sample count for each category c; during training, the normalized representations of samples of the same category are accumulated; at the end of the round, based on the cumulative vector sum and sample count by category, the mean representation of the category is obtained and normalized to obtain the set of semantic summary vectors for the client-side categories. The client uploads the updated local model parameters to the server. and local class category semantic summary vector set Simultaneously, update the backtracking reference model cache for the next round of dual-reference steady-state drift suppression constraints.

[0048] Step 6 includes:

[0049] The server performs inertial fusion and normalization on the semantic anchor vectors of each category to maintain the unit vector scale; when a category semantic anchor has not yet been initialized, it is initialized using the category-side semantic summary obtained from this round of aggregation; the updated cross-end semantic anchor library is broadcast to participating clients in the next round along with the global model parameters.

[0050] The aggregation formula for category-level end-side summaries is:

[0051] ,

[0052] The formula for unit norm inertial fusion update is:

[0053] ,

[0054] in, This represents the set of clients participating in round t. This represents the semantic summary vector of category c generated and uploaded by client i in round t. This represents the aggregation weight assigned by the server to client i when performing semantic summary on the client-side of aggregation category c in round t. This represents the semantic summary of the category obtained by the server in the t-th round of aggregation. , These represent the category semantic anchor vectors before and after the update, respectively. It is a parameter for the anchor library to be updated on a rolling basis. It is a normalized function of the unit norm. It is a very small constant.

[0055] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0056] Beneficial Effects: The method of this invention achieves secure semantic alignment and steady-state convergence during training by constructing and continuously calibrating a cross-terminal semantic anchor library, introducing an uncertainty-driven sample-level coupled amplitude modulator, and employing a collaborative training mechanism combining dual-reference steady-state drift suppression constraints and semantic anchor alignment constraints. This makes it suitable for tasks such as edge computing, IoT, and image classification in privacy-sensitive scenarios. In heterogeneous IoT data scenarios, this method effectively mitigates client drift and training oscillations, improving the generalization performance and convergence stability of the global model. Compared to classical federated learning methods, the proposed method not only achieves privacy and security protection in IoT but also offers higher accuracy. Attached Figure Description

[0057] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0058] Figure 1 This is a schematic diagram of the steady-state prototype federated learning training method framework for heterogeneous data proposed in this invention.

[0059] Figure 2 This is an experimental comparison chart of the method of the present invention (data was divided using pathological settings on the CIFAR-10 dataset, with 20 clients).

[0060] Figure 3 This is an experimental comparison chart of the method of the present invention (on the iCubWorld28 dataset, under actual settings, the Dirichlet distribution is used to simulate the actual heterogeneous data distribution, and the number of clients is 20). Detailed Implementation

[0061] like Figure 1 As shown, this embodiment of the invention provides a steady-state prototype federated learning training method for heterogeneous data, including the following steps:

[0062] Step 1: Initialization of the steady-state semantic anchor federated learning system and configuration of sample-level coupled amplitude modulator parameters driven by unified uncertainty;

[0063] Server-side initialization of global model parameters And establish a cross-platform semantic anchor library Configure hyperparameters, including sample-level coupling amplitude modulation parameters, steady-state drift suppression constraint weights, semantic anchor alignment constraint weights, and anchor library rolling update parameters on the server. The client completes local training initialization and establishes a backtracking reference model. .

[0064] Step 2: Randomly select clients to broadcast synchronously with the "global model + cross-platform semantic anchor library". In each round of federated training, the server determines the set of participating clients. And broadcast global model parameters synchronously to participating clients. and cross-platform semantic anchor libraries .in, Used as a shared semantic anchor point across clients to support subsequent semantic alignment constraints.

[0065] Step 3: Dropout-only uncertainty estimation is generated and coupled with a sample-level amplitude modulator. (Client) Receive the server's output and Then, Monte Carlo random inactivation MC-Dropout uncertainty estimation is performed on the local sample x to obtain the sample uncertainty measure. And map to generate sample-level amplitude modulation coefficients. The aforementioned It will serve as a unified modulation factor, taking effect simultaneously in subsequent steady-state drift suppression constraints and semantic anchor alignment constraints.

[0066] To avoid perturbing the training statistics during the uncertainty estimation process, the client employs a dropout-only inference control mode. This means that only layers containing random dropout layers are set to the training state, while the remaining layers remain in the inference state. In this mode, multiple inferences are performed on the samples to estimate the uncertainty and generate sample-level amplitude modulation coefficients.

[0067] Step 4: Joint training of dual-reference steady-state drift suppression constraints and semantic anchor alignment constraints under unified coupled amplitude modulation. The client constructs and jointly optimizes the supervised learning term, dual-reference steady-state drift suppression constraints (DRDD), and semantic anchor alignment constraints (SAA) during local training. Specifically, this includes:

[0068] (1) Dual-reference steady-state drift suppression constraint DRDD uses a broadcast reference model issued by the server. Client-side backtracking reference model As a dual reference source, it suppresses the mutation drift relative to the reference in this round of updates;

[0069] (2) Semantic anchor alignment constraint SAA with cross-end semantic anchor library The category semantic anchor vector in the middle is used as the alignment benchmark to promote the convergence of the same semantics across clients; (3) Construct anchor point ready gating: only when SAA is only enabled for a sample when a semantic anchor corresponding to the true category already exists in the sample, in order to avoid incorrect alignment caused by missing categories or uninitialized prototypes; (4) Unified coupling amplitude modulation: The above dual-reference steady-state drift suppression constraint DRDD and semantic anchor alignment constraint SAA are both generated by the same sample-level coupling amplitude modulation coefficient in step 3. Uniform scaling reduces the constraint strength of high-uncertainty samples in the dual-reference steady-state drift suppression constraint and semantic anchor alignment constraint path simultaneously, avoiding the problem of reducing the weight of only one factor while still introducing gradient noise into the other.

[0070] The client obtains the updated local model parameters through this training step. .

[0071] Step 5: Intra-class streaming aggregation statistics and client-side semantic summary upload. During this training round, the client performs intra-class streaming aggregation statistics on sample feature representations by category. At the end of the round, a client-side category semantic summary vector set is generated based on the "cumulative vector sum by category and sample count". The client uploads the updated local model parameters to the server. and the set of local class semantic summary vectors Simultaneously, the backtracking reference model cache is updated for use in the next round of dual-reference steady-state drift suppression constraints.

[0072] Step 6: Server aggregation and semantic anchor library unit norm inertial fusion update and backhaul. The model parameters uploaded by the client are obtained from the server aggregation. Simultaneously, unit norm inertial fusion updates are performed on semantic anchors of each category, and the semantic summaries received from the end-side in this round are integrated into the cross-end semantic anchor library to obtain the updated cross-end semantic anchor library. The server will distribute the data in the next round of synchronization. and Repeat steps 2 through 6 until the termination condition is met.

[0073] Step 3, the generation of the sample-level coupled amplitude modulator specifically includes:

[0074] To differentiate the different impacts of high-confidence samples and high-uncertainty samples on consistency constraints, the client uses the Monte Carlo random deactivation (MC-Dropout) method to estimate the uncertainty for each sample x. And map it to the sample-level amplitude modulation coefficient. This is used to scale the constraint strength of subsequent steady-state drift suppression constraints and semantic anchor alignment constraints. To ensure that uncertainty estimation does not disturb the training process, a dropout-only control is preferred: only the randomly deactivated dropout layer is set to the training state, while the remaining layers remain in the inference state. In this mode, multiple inferences are performed on the samples to estimate the uncertainty and generate sample-level coupling amplitude modulation coefficients. This amplitude modulation is calculated in the implementation as follows:

[0075] ,

[0076] ,

[0077] Where x represents the local training sample on the client; The uncertainty measure for sample x is obtained from the fluctuation statistics of multiple random inactivation inference outputs of Monte Carlo random inactivation MC-Dropout, and then normalized. ; The confidence amplitude factor. The larger the value, the more reliable the sample. The sample-level coupling amplitude modulation coefficient of sample x is used to uniformly scale the strength of the dual-reference steady-state drift suppression constraint DRDD and the semantic anchor alignment constraint SAA. This is the amplitude modulation reference coefficient; This is the uncertainty adjustment coefficient; and These are the lower and upper limits of amplitude modulation, respectively, satisfying... ; This represents a truncation function used to cut off the input to a range of... Within the interval, it is defined as The key limitation of this invention is that: the same This will be discussed later. and This reduces the gradient contribution of highly uncertain samples on both constrained paths simultaneously.

[0078] The specific details of the "Dual Reference Steady-State Drift Suppression Constraint (DRDD)" in step 4 are as follows:

[0079] To suppress client drift and training oscillations between federated training rounds, the client introduces dual reference sources in this training round:

[0080] (1) Broadcast reference: Global model parameters issued by the server. This is used to guide the current update to align the global direction.

[0081] (2) Backtracking reference: Local model parameters saved by the client in the previous round This is used to suppress excessively rapid shifts in the current update cycle.

[0082] For the same sample x, the client extracts the current representation. Broadcast reference representation Backtracking reference representation After calculating the similarity, a steady-state loss is constructed, which is determined by the sample-level coupling amplitude modulation coefficient. Adaptively scale the loss intensity:

[0083] ,

[0084] ,

[0085] ,

[0086] in, This represents the current model's representation of sample x; Let x represent the broadcast reference model's representation of sample x, with parameters as follows. ; This represents the backtracking reference model's representation of sample x, with parameters as follows: ; Cosine similarity or unit vector dot product. This indicates the similarity to the broadcast reference. This indicates the similarity term with the backreferenced reference. This invention employs a dual-reference steady-state drift suppression loss mechanism to reduce inter-round drift and training oscillations. Its key difference lies in that it couples this loss to sample-level amplitude modulation. The coupling enables the steady-state strength to adapt to the sample uncertainty, making it more suitable for noisy and domain-offset sample scenarios in IoT security data, and sharing the same amplitude scale with the semantic anchor alignment term described later.

[0087] The semantic anchor alignment constraint (SAA) in step 4 is as follows:

[0088] To align the representation of similar security events across clients at the semantic level, this invention maintains a cross-client semantic anchor library on the server side. As shared semantic anchors, the client constructs a contrastive alignment constraint during local training based on the similarity between sample representations and semantic anchor vectors. This constraint brings true-class semantic anchors closer and separates negative-class semantic anchors. An anchor-ready gating is employed, enabling this feature only for samples where a true-class anchor already exists in the semantic anchor library, thus preventing misalignment due to missing anchors. This constraint is also determined by sample-level coupling amplitude modulation coefficients. Uniform scaling.

[0089] (1) Anchor-ready gating:

[0090] ,

[0091] (2) Similarity between real category sample representations and semantic anchors of the same category:

[0092] ,

[0093] (3) Negative anchor aggregate term:

[0094] ,

[0095] (4) Semantic anchor alignment loss:

[0096] ,

[0097] in, This represents the anchor-ready gate value for sample x, taking a value of 0 or 1. When semantic anchor alignment constraints are enabled, This indicates that it is not enabled; This indicates an indicator function that returns 1 if the condition within the parentheses is true, and 0 otherwise. This refers to the cross-platform semantic anchor library maintained and distributed by the server in round t. Let y represent the semantic anchor vector of category c maintained by the server in round t, and y be the true label of sample x. This represents the semantic anchor vector of category y in the cross-end semantic anchor library maintained by the server in round t; This indicates the similarity between the sample representation and similar semantic anchors. Indicates a negative anchor aggregate term; Cosine similarity or unit vector inner product; when setting the number of negative class samples hour, The set of negative classes after sampling can be all classes except y or the K classes obtained from sampling; This is the semantic anchor alignment loss.

[0098] The joint optimization of the above-mentioned constraints and supervision losses in step 4 is as follows:

[0099] The client jointly optimizes the supervised learning term, the dual-reference steady-state drift suppression loss, and the semantic anchor alignment loss during local training to obtain the updated local model parameters. The joint optimization form is as follows:

[0100] ,

[0101] in, This represents the supervised learning loss, used to ensure the model's basic ability to distinguish IoT security events or categories. Used to suppress wheel drift and improve training stability; Used for semantic anchor alignment of similar semantics across clients; This is the steady-state loss weight hyperparameter, used to balance the contributions of the steady-state term and the monitoring term; This is a hyperparameter for the semantic anchor alignment loss weights, used to balance the contributions of the alignment and supervision terms; To calculate the average of mini-batch samples, the client minimizes... For local model parameters Iterative updates are performed. The key is: Simultaneously enter and This enables the gradient contribution of highly uncertain samples to be reduced synchronously on both constraint paths, achieving unified coupled amplitude modulation, thereby obtaining stronger steady-state convergence and semantic alignment.

[0102] The specific method for generating the end-side category semantic summary vector set by class in step 5 is as follows:

[0103] The client maintains a cumulative vector sum and sample count for each category c. During training, the normalized representations of samples within the same category are accumulated. At the end of each round, the mean representation of the category is obtained by dividing the cumulative vector sum by the sample count, and then normalized to serve as the edge-side category semantic summary vector. A category summary is generated and uploaded only when a category appears in the local data in this round (sample count greater than 0) to reduce invalid transmissions and noisy updates.

[0104] In step 6, the unit norm inertial fusion update of the server-side cross-platform semantic anchor library is as follows:

[0105] The server performs inertial fusion and normalization on the semantic anchor vectors of each category to maintain a unit vector scale. When the semantic anchors of a certain category have not yet been initialized, they can be initialized using the semantic summary of that category obtained in this round of aggregation. The updated cross-end semantic anchor library is broadcast to participating clients in the next round along with the global model parameters.

[0106] Aggregation of category-level end-side summaries:

[0107] ,

[0108] Unit norm inertial fusion update:

[0109] ,

[0110] in, This represents the set of clients participating in round t. This represents the semantic summary vector of category c generated and uploaded by client i in round t. This represents the aggregation weight assigned by the server to client i when performing semantic summary on the client-side of aggregation category c in round t. This represents the semantic summary of the category obtained by the server in the t-th round of aggregation. , These represent the category semantic anchor vectors before and after the update, respectively. It is a parameter for the anchor library to be updated on a rolling basis. It is a normalized function of the unit norm. It is a minimal constant to avoid the denominator being zero. Wherein This can be taken as the sample count of client i in category c. .

[0111] In a preferred embodiment, to balance training stability and alignment performance, the hyperparameters involved in this invention can be selected as follows: Minimal constant. Generally taken Sample-level coupling amplitude modulation coefficient amplitude modulation reference coefficient The uncertainty adjustment coefficient is typically set to 0.5. The amplitude modulation (AM) is typically set between 0.5 and 5.0, with 2.0 being the preferred value; the lower limit of AM is... 0.2 is acceptable; upper limit of amplitude adjustment A value of 0.8 is acceptable. Steady-state loss weight hyperparameter. Typically, the value is taken as 0.1 to 5.0; the semantic anchor alignment loss weight hyperparameter The value is typically between 0.1 and 10.0, with 2.0 being a preferred choice. In the semantic anchor alignment constraint, the number of negative class samples K is preferably selected as 16 or 32; when K is -1, it is set to "all negative class mode," indicating that all available negative class semantic anchors, excluding the true class, participate in the alignment calculation. In the unit norm inertial fusion update of the cross-end semantic anchor library, the anchor library parameters are updated on a rolling basis. A value of 0.98 is acceptable. If the Dirichlet distribution is used to partition the data into non-independent and identically distributed (Non-IID) partitions, the concentration parameter... Generally, the value is taken as 0.05 to 1.0, with 0.1 or 0.3 being preferred; when The degree of heterogeneity is higher when the size is smaller, when When the size is larger, the data distribution is closer to uniform. The above parameters can be adjusted according to the degree of data heterogeneity, model size, and number of clients.

[0112] To comprehensively evaluate the superior performance of the innovative method proposed in this invention in practical applications, the invention selected the specialized experimental datasets CIFAR-10 and iCubWorld28, and compared the accuracy of the proposed method with two traditional federated learning algorithms. The training run consisted of 200 epochs. The experimental configuration included an Intel Core i5-14600KF 14-core processor, 32GB of DDR5 6000MHz memory, and an NVIDIA GeForce RTX 5060 Ti 16GB GPU. In the specific implementation, the CIFAR-10 and iCubWorld28 datasets were divided into training and testing sets, with 80% allocated for training and 20% for testing.

[0113] In a specific implementation, the method of this invention is experimentally compared with the classic federated learning algorithms FedNTD and FedLC. Regarding the experimental setup, the CIFAR-10 dataset was used with a pathological setting to partition the data, extracting two categories from 10 categories for each client, with each category containing disjoint data of different quantities; the iCubWorld28 dataset was used with parameters... Dir distribution ( Generate client data partitions to simulate the actual heterogeneous data distribution of multiple IoT terminals. Set it to 0.1. For example... Figure 2 As shown, under the CIFAR-10 dataset conditions, the method of this invention exhibits faster convergence speed and higher final accuracy during global training, with a maximum accuracy of 86.99%, representing an 8.12% improvement over the best result in FedNTD and a 9.93% improvement over the best result in FedLC. Figure 3 As shown, under the iCubWorld28 dataset conditions, the method of this invention also outperforms the comparative algorithms in terms of training stability and final accuracy, with a final accuracy of approximately 81.42%, which is an improvement of 5.16% compared to the best result in FedNTD and 4.37% compared to the best result in FedLC. The above experimental results demonstrate that the method of this invention can effectively alleviate client drift and training oscillations in heterogeneous IoT data scenarios, and improve the generalization performance and convergence stability of the global model.

[0114] In summary, compared with classical federated learning methods, the method proposed in this invention not only achieves privacy and security protection in the Internet of Things, but also has higher accuracy.

[0115] This invention provides a steady-state prototype federated learning training method for heterogeneous data. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A steady-state prototype federated learning training method for heterogeneous data, characterized in that, Includes the following steps: Step 1: Initialize the steady-state semantic anchor federated learning system and configure the parameters of the sample-level coupled amplitude modulator driven by unified uncertainty. Step 2: Randomly select clients to synchronously broadcast with the global model and cross-platform semantic anchor library; Step 3: Dropout-only uncertainty estimation is generated with a sample-level coupled amplitude modulator; Step 4: Jointly train the dual-reference steady-state drift suppression constraint and semantic anchor alignment constraint under unified coupled amplitude modulation. The client constructs and jointly optimizes the supervised learning term, the dual-reference steady-state drift suppression constraint DRDD, and the semantic anchor alignment constraint SAA during local training. The client obtains the updated local model parameters after training. ; Step 5: Intra-class streaming aggregation statistics and end-side semantic summary upload; Step 6, Unit Norm Inertial Fusion Update and Backhaul of Server Aggregation and Semantic Anchor Library: The model parameters uploaded by the client are obtained from the server aggregation. Simultaneously, unit norm inertial fusion updates are performed on semantic anchors of each category, and the semantic summaries received in this round from the end-side are integrated into the cross-end semantic anchor library to obtain the updated cross-end semantic anchor library. ; The server will distribute the data in the next round of synchronization. and Repeat steps 2 through 6 until the termination condition is met.

2. The method according to claim 1, characterized in that, Step 1 includes: initializing global model parameters on the server side. And establish a cross-platform semantic anchor library Configure hyperparameters, including server-side configuration of sample-level coupling amplitude modulation parameters, steady-state drift suppression constraint weights, semantic anchor alignment constraint weights, and anchor library rolling update parameters; complete local training initialization on the client side and establish a backtracking reference model cache. .

3. The method according to claim 2, characterized in that, Step 2 includes: In each round of federated training, the server determines the set of participating clients. And broadcast global model parameters synchronously to participating clients. and cross-platform semantic anchor libraries ;in This refers to the cross-platform semantic anchor library maintained and distributed by the server in round t. Used as a shared semantic anchor point across clients to support subsequent semantic alignment constraints.

4. The method according to claim 3, characterized in that, Step 3 includes: Client Receive the server's output and Then, Monte Carlo random inactivation MC-Dropout uncertainty estimation is performed on the local sample x to obtain the sample uncertainty measure. And map to generate sample-level amplitude modulation coefficients. ; It will serve as a unified modulation factor and will take effect simultaneously in subsequent steady-state drift suppression constraints and semantic anchor alignment constraints. This represents the global model parameters on the server side in round t; The client employs a Dropout-only inference control method, placing only the layers containing Dropout in the training state while keeping the remaining layers in the inference state. It performs inference on samples more than twice to estimate uncertainty and generate sample-level amplitude modulation coefficients; this can be expressed by the formula: ， ， Where x represents the local training sample on the client; A measure of uncertainty for sample x; The confidence amplitude factor; This is the amplitude modulation reference coefficient; This is the uncertainty adjustment coefficient; and These are the lower and upper limits of amplitude modulation, respectively, satisfying... ; The interval cutoff function is defined as follows: ;in represents the intermediate parameter, and e represents the natural constant.

5. The method according to claim 4, characterized in that, Step 4 includes: Step 4-1, Dual-reference steady-state drift suppression constraint DRDD uses the broadcast reference model issued by the server. Client-side backtracking reference model As a dual reference source, it suppresses abrupt shifts in the relative reference during this round of updates; whereby... This represents the backtracking reference model parameters cached on the client side; The dual-reference steady-state drift suppression constraint (DRDD) includes: Broadcast reference: Global model parameters issued by the server. This is used to guide the current update to align the global direction. Backtracking reference: Local model parameters saved by the client in the previous round This is used to suppress excessively rapid shifts in the current update cycle; For the same sample x, the client extracts the current representation. Broadcast reference representation Backtracking reference representation After calculating the similarity, a steady-state loss is constructed, which is determined by the sample-level coupling amplitude modulation coefficient. Adaptive scaling loss strength: ， ， ， in, This represents the current model's representation of sample x; Let x represent the broadcast reference model's representation of sample x, with parameters as follows. ; This represents the backtracking reference model's representation of sample x, with parameters as follows: ; Cosine similarity or unit vector dot product. This indicates the similarity to the broadcast reference. This indicates the similarity term with the backreferenced reference. This is the two-reference steady-state drift suppression loss; exp is the natural exponential function; Softplus is the smoothing activation function; Step 4-2, Semantic Anchor Alignment Constraints (SAA) for Cross-Platform Semantic Anchor Libraries The category semantic anchor vector in the data serves as an alignment benchmark, promoting the convergence of similar semantics across clients; Step 4-3, Construct Anchor-Ready Gating: Only when Semantic Anchor Alignment (SAA) constraint is only applied to samples when a semantic anchor corresponding to the true category already exists in the sample data. Step 4-4, Unified Coupling Amplitude Modulation: Both the Dual Reference Steady-State Drift Suppression (DRDD) constraint and the Semantic Anchor Alignment (SAA) constraint are generated from the same sample-level coupling amplitude modulation coefficients in step 3. Uniform scaling.

6. The method according to claim 5, characterized in that, In step 4-2, the semantic anchor alignment constraint (SAA) includes: Maintaining cross-platform semantic anchor libraries on the server side As shared semantic anchors, the client constructs a contrastive alignment constraint based on the similarity between sample representations and semantic anchor vectors during local training. This constraint brings the true class semantic anchors closer and separates the negative class semantic anchors. An anchor-ready gating is employed, enabling the constraint only for samples where true class anchors already exist in the semantic anchor library. The semantic anchor alignment constraint (SAA) is determined by a sample-level coupling amplitude modulation coefficient. Uniform scaling; The anchor-ready gating is represented as follows: ， The similarity between the representation of a real category sample and a semantic anchor of the same category is calculated using the following formula. : ， The negative anchor aggregation term is calculated using the following formula. : ， The semantic anchor alignment loss is calculated using the following formula. : ， in, This represents the anchor-ready gate value for sample x, taking a value of 0 or 1. When semantic anchor alignment constraints are enabled, This indicates that it is not enabled; This indicates an indicator function that takes the value 1 if the condition within the parentheses is true, and 0 otherwise. Let y represent the semantic anchor vector of category c maintained by the server in round t; y is the true label of sample x. This represents the semantic anchor vector of category y in the cross-platform semantic anchor library maintained by the server in round t; when the number of negative class samples K is set, This is the set of negative classes after sampling.

7. The method according to claim 6, characterized in that, Step 4-4 includes: The client jointly optimizes the supervised learning term, the dual-reference steady-state drift suppression loss, and the semantic anchor alignment loss during local training to obtain the updated local model parameters. Joint optimization function The format is: ， in, Indicates the loss in supervised learning; To suppress drift loss in dual-reference steady state; This represents the semantic anchor alignment loss; The weighting hyperparameters for steady-state loss; The weight hyperparameters for semantic anchor alignment loss; To calculate the average of mini-batch samples, the client minimizes... For local model parameters Iterative updates will be performed.

8. The method according to claim 7, characterized in that, Step 5 includes: the client maintains a cumulative vector sum and sample count for each category c; during training, the normalized representations of samples of the same category are accumulated; at the end of the round, based on the cumulative vector sum and sample count by category, the mean representation of the category is obtained and normalized to obtain the set of semantic summary vectors for the client-side categories. The client uploads the updated local model parameters to the server. and local class category semantic summary vector set Simultaneously, update the backtracking reference model cache for the next round of dual-reference steady-state drift suppression constraints.

9. The method according to claim 8, characterized in that, Step 6 includes: The server performs inertial fusion and normalization on the semantic anchor vectors of each category to maintain the unit vector scale; when a category semantic anchor has not yet been initialized, it is initialized using the category-side semantic summary obtained from this round of aggregation; the updated cross-end semantic anchor library is broadcast to participating clients in the next round along with the global model parameters. The aggregation formula for category-level end-side summaries is: ， The unit norm inertial fusion update formula is: ， in, This represents the set of clients participating in round t. This represents the semantic summary vector of category c generated and uploaded by client i in round t. This represents the aggregation weight assigned by the server to client i when performing semantic summary on the client-side of aggregation category c in round t. This represents the semantic summary of the category obtained by the server in the t-th round of aggregation. , These represent the category semantic anchor vectors before and after the update, respectively. It is a parameter for the anchor library to be updated on a rolling basis. It is a normalized function of the unit norm. It is a very small constant.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 9.

Images