A federated learning method, device and equipment based on semantic resonance and expert selection

Abstract

This invention provides a federated learning method, apparatus, and device based on semantic resonance and expert selection, relating to the field of distributed machine learning technology. The invention receives a semantic anchor matrix from a server, extracts semantic features to obtain semantic feature vectors, and normalizes them. Next, a temperature coefficient is introduced to calculate a semantic resonance score and map it to obtain a semantic resonance probability. The semantic resonance probabilities of all local samples are statistically analyzed to obtain a client-level semantic relevance index. A comprehensive semantic matching score is calculated to obtain an expert selection set. The model parameter information from the expert selection set transmitted from the server is received, and gating weights are introduced to perform local forward computation to obtain the expert model output. Finally, local training optimization is performed, and the updated parameter information is uploaded to the server for selective aggregation and updating on the server side. This invention can significantly reduce communication overhead and improve the semantic interpretability and adaptability of expert selection to Non-IID data.

Description

Technical Field

[0001] This invention relates to the field of distributed machine learning technology, and more specifically, to a federated learning method, apparatus, and device based on semantic resonance and expert selection. Background Technology

[0002] With the rapid proliferation of IoT and edge computing devices, massive amounts of data are continuously generated at the network edge, driving the widespread application of machine learning technology in fields such as natural language processing and intelligent recommendation. Traditional centralized learning paradigms typically upload data from terminal devices to a central server for unified model training. However, this approach not only carries a high risk of privacy breaches but also incurs significant communication overhead due to frequent transmission of raw data, making it unsuitable for the practical needs of large-scale distributed scenarios. To address these challenges, federated learning has emerged. Its core idea is to complete model training locally on the client side and only upload model parameters or gradients to the server for aggregation, thereby achieving collaborative modeling while protecting data privacy.

[0003] However, with the continuous growth in the scale of deep learning models, especially the introduction of models with large numbers of parameters, traditional federated learning frameworks face new bottlenecks: on the one hand, resource-constrained edge devices struggle to support the local training of complete models; on the other hand, frequent transmission of large-scale model parameters or gradients significantly increases communication costs, limiting their deployment capabilities in low-bandwidth or high-latency network environments.

[0004] To alleviate communication and computational burdens, researchers have proposed various optimization strategies, including model parameter compression, sparse updates, and the introduction of Mixture of Experts (MoE) models. MoE, by activating only a subset of "expert" sub-models in each forward propagation, effectively reduces the number of effective parameters involved in a single iteration. In federated learning, the MoE structure is further used to reduce the model size that the client needs to process. However, existing MoE-based federated learning methods generally rely on learnable black-box gating networks for expert routing, lacking explicit modeling of the semantic level. This results in an uninterpretable expert selection process that is difficult to adapt to the semantic characteristics of the client's local data. Furthermore, since the gating mechanism cannot predict which experts are highly relevant to the local data before training, the client often still needs to download a large number of irrelevant expert parameters, and the communication burden is not fundamentally alleviated. More importantly, in the case of non-independent identically distributed (Non-IID) data, the semantic distribution of data differs significantly among different clients. Existing methods use a unified expert selection and aggregation strategy, which makes it difficult to dynamically adapt to the semantic needs of each client. This can easily lead to some experts not being activated for a long time or having overlapping functions, resulting in problems such as uneven expert utilization, decreased model performance, and even unstable training.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] The present invention aims to provide a federated learning method, apparatus and device based on semantic resonance and expert selection, so as to solve the problems of high communication overhead, lack of semantic interpretability of expert routing, weak adaptability to non-independent and identically distributed data and redundancy of expert functions in existing federated learning frameworks after the introduction of hybrid expert models.

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0008] A federated learning method based on semantic resonance and expert selection, applied to the client side, includes:

[0009] S1, Receive the semantic anchor matrix sent by the server; wherein, the semantic anchor matrix is ​​an orthogonal matrix obtained by the server through orthogonal decomposition and normalization, which is composed of unique semantic anchors pre-configured for each expert sub-model according to the task requirements.

[0010] S2, randomly select local samples from the local dataset to extract semantic features, obtain semantic feature vectors, and normalize them;

[0011] S3, Based on the normalized semantic feature vector and the semantic anchor matrix, a temperature coefficient is introduced to calculate the semantic resonance score for each local sample, and the semantic resonance probability is mapped to obtain the semantic resonance probability.

[0012] S4, Calculate the semantic resonance probability of all local samples to obtain a client-level semantic relevance index that includes average semantic relevance, expert coverage and resonance uncertainty;

[0013] S5. Based on the client-level semantic relevance index, calculate the comprehensive semantic matching score to evaluate the matching degree between each expert sub-model and the local sample data, and obtain the expert selection set.

[0014] S6, Receive the expert sub-model parameter information in the expert selection set transmitted by the server, and introduce the gating weights based on the semantic resonance score to perform local forward calculation to obtain the expert model output result;

[0015] S7. Based on the output results of the expert model, perform local training optimization, and upload the updated parameter information of the expert selection set after training to the server to complete the selective aggregation and update of the expert sub-model parameters on the server.

[0016] Preferably, the process of generating the semantic anchor matrix is ​​as follows:

[0017] The server initializes a global hybrid expert model containing N expert sub-models and assigns a semantic anchor to each expert sub-model. All semantic anchors form a semantic anchor matrix, represented as follows:

[0018] ;

[0019] in, This is a semantic anchor matrix; Let be the semantic anchor vector corresponding to the i-th expert sub-model; The number of expert sub-models; Dimension of the semantic feature space;

[0020] The semantic anchor matrix is ​​generated using an orthogonal initialization method, and L2 normalization is performed on each semantic anchor vector, as follows:

[0021] ;

[0022] ;

[0023] in, This represents the initial state of the semantic anchor matrix; This indicates that the matrix has been orthogonalized. This indicates the generation of a random matrix that follows a standard normal distribution; express The i-th vector; This represents a very small constant to prevent the denominator from being zero; This represents the L2 norm of a vector.

[0024] Preferably, the formula for calculating the semantic resonance score is:

[0025] ;

[0026] in, This represents the semantic resonance score between the sample and the i-th expert sub-model; The normalized semantic feature vector; T is the transpose symbol; This is a temperature coefficient used to adjust the concentration level of expert selection; Let be the semantic anchor vector corresponding to the i-th expert sub-model;

[0027] The formula for calculating the semantic resonance probability is:

[0028] ;

[0029] in, This represents the semantic resonance probability of a sample with respect to the i-th expert sub-model; The number of expert sub-models; It is an exponential function.

[0030] Preferably, the formula for calculating the average semantic relevance is:

[0031] ;

[0032] in, This represents the average semantic relevance between the i-th expert sub-model and the overall client data; This represents the semantic resonance probability corresponding to the m-th sample of the i-th expert sub-model; This represents the total number of local samples;

[0033] An exponential moving average is introduced to reduce inter-cycle volatility; the formula is as follows:

[0034] ;

[0035] in, This represents the average semantic relevance after exponential moving average. For smoothing coefficients; Historical moving average semantic relevance;

[0036] The formula for calculating the expert coverage rate is as follows:

[0037] ;

[0038] in, This represents the frequency with which the i-th expert sub-model enters the candidate set at the sample level, i.e., the expert coverage rate of the i-th expert sub-model. For indicator functions; Indicates the number of candidate experts at the sample level; This represents the semantic resonance probability corresponding to the m-th sample; This indicates that the first m-th sample is selected. An expert sub-model;

[0039] The formula for calculating the resonance uncertainty is:

[0040] ;

[0041] in, Let represent the resonance uncertainty of the i-th expert sub-model.

[0042] Preferably, the expression for the comprehensive semantic matching score is:

[0043] ;

[0044] in, This represents the comprehensive semantic matching score of the i-th expert sub-model for the client; , , These are the weighting coefficients; This represents the average semantic relevance after exponential moving average. This represents the expert coverage of the i-th expert sub-model; Let be the resonance uncertainty of the i-th expert sub-model;

[0045] The expert selection set is obtained by ranking all expert sub-models based on the comprehensive semantic matching score and combining it with threshold constraints, as expressed in the following expression:

[0046] ;

[0047] in, This represents the set of expert sub-models selected by client c in the current federated training round; This is the minimum selection threshold used to avoid selecting experts with low semantic matching. The number of experts selected for the client-side; A comprehensive semantic matching score is given to all expert sub-models.

[0048] Preferably, the expression for the gating weight is:

[0049] ;

[0050] in, This represents the contribution weight of the i-th expert sub-model output; , This represents the semantic resonance score between the sample and the i-th and j-th expert sub-models; Let represent the set of expert sub-models selected by client c in the current federated training round.

[0051] Preferably, when performing local training optimization based on the output results of the expert model, a local total loss function is used for training; the expression of the local total loss function is:

[0052] ;

[0053] ;

[0054] ;

[0055] in, This is the local total loss function; , These are the loss weighting coefficients; Loss to the main task; For discrete constraint loss of semantic anchor points; For gating sparsity loss; , These represent the semantic anchor vectors corresponding to different expert sub-models; T is the transpose symbol. Let the contribution weight of the i-th expert sub-model output be used to represent the contribution weight. This represents the set of expert sub-models selected by client c in the current federated training round; This represents a minimal constant to prevent the denominator from being zero.

[0056] Preferably, when the server performs selective aggregation updates on the expert sub-model parameters, the calculation is based on a weighted average method, expressed as:

[0057] ;

[0058] ;

[0059] ;

[0060] in, Update the aggregation parameters for the i-th expert sub-model; For client c, the weighting coefficients; This represents the set of clients that select and train the i-th expert sub-model in the current round; Update the expert parameters uploaded by client c; , These represent the number of samples used by clients c and e in this round to train the i-th expert sub-model, respectively.

[0061] Based on the aggregation parameter update results, the server updates the parameters of the i-th expert sub-model using the following formula:

[0062] ;

[0063] in, These are the current parameters of the i-th expert sub-model.

[0064] The present invention also provides a federated learning device based on semantic resonance and expert selection, comprising:

[0065] The semantic anchor matrix unit is used to receive the semantic anchor matrix sent by the server; wherein, the semantic anchor matrix is ​​an orthogonal matrix obtained by the server through orthogonal decomposition and normalization, which is composed of unique semantic anchors pre-configured for each expert sub-model according to task requirements.

[0066] The semantic feature extraction unit is used to randomly select local samples from the local dataset to extract semantic features, obtain semantic feature vectors, and normalize them.

[0067] The semantic resonance score calculation unit is used to calculate the semantic resonance score for each local sample based on the normalized semantic feature vector and the semantic anchor matrix, by introducing a temperature coefficient, and to map the semantic resonance probability.

[0068] The semantic relevance index calculation unit is used to calculate the semantic resonance probability of all local samples and obtain a client-level semantic relevance index that includes average semantic relevance, expert coverage and resonance uncertainty.

[0069] The expert selection unit is used to calculate a comprehensive semantic matching score based on the client-level semantic relevance index to evaluate the degree of matching between each expert sub-model and the local sample data, and to obtain the expert selection set.

[0070] The local forward computation unit is used to receive the expert sub-model parameter information in the expert selection set transmitted by the server, and to perform local forward computation by introducing the gating weights based on the semantic resonance score to obtain the expert model output result.

[0071] The selective aggregation update unit is used to perform local training optimization based on the output results of the expert model, and upload the updated parameter information of the expert selection set after training to the server to complete the selective aggregation update of the expert sub-model parameters on the server.

[0072] The present invention also provides a federated learning device based on semantic resonance and expert selection, including a processor and a memory, wherein the memory stores a computer program that can be executed by the processor to implement a federated learning method based on semantic resonance and expert selection as described above.

[0073] The present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device in which the computer-readable storage medium resides, implement a federated learning method based on semantic resonance and expert selection as described above.

[0074] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0075] This invention introduces learnable and normalized semantic anchors on the server side and constructs an explicit routing mechanism on the client side based on the dot product similarity between semantic features and anchors, replacing the traditional black-box gating network, thus enabling expert selection to have clear semantic basis and interpretability.

[0076] The client only needs to download and upload a subset of expert parameters that are highly semantically related to its local data, which significantly reduces the amount of uplink and downlink communication data; different clients dynamically select experts based on their own data distribution, effectively adapting to the semantic heterogeneity in Non-IID scenarios and avoiding performance degradation caused by unified model updates;

[0077] This invention forces experts to disperse in the semantic space by using semantic anchor point discrete constraint loss, thereby preventing functional redundancy and improving the overall utilization of experts.

[0078] The selective parameter update and expert-level aggregation mechanism employed in this invention ensures that unselected expert parameters are not invalidally disturbed, thus maintaining the stability of the global model.

[0079] This invention achieves coordinated optimization of communication efficiency, computational efficiency, and model performance without sacrificing model convergence and generalization ability. Attached Figure Description

[0080] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0081] Figure 1 This is a schematic diagram of a federated learning method based on semantic resonance and expert selection, provided in Example 1.

[0082] Figure 2 The flowchart illustrates a federated learning method based on semantic resonance and expert selection, as provided in Example 1.

[0083] Figure 3 This is a schematic diagram of a federated learning device based on semantic resonance and expert selection, provided in Embodiment 2.

[0084] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0086] Example 1

[0087] Embodiment 1 of the present invention provides a federated learning method based on semantic resonance and expert selection, which can be implemented by a federated learning device based on semantic resonance and expert selection (hereinafter referred to as federated learning training device), specifically, executed by one or more processors within the federated learning training device.

[0088] In this embodiment, the federated learning training device may be an electronic device equipped with a processor, which carries a computer program for the federated learning method based on semantic resonance and expert selection, and the computer program can be executed, such as a computer, smartphone, smart tablet, workstation, etc., without limitation.

[0089] This embodiment proposes a federated learning method based on semantic resonance and expert selection. The server first establishes a global semantic reference for the expert sub-model through semantic anchors. The client then maps its local data to this semantic space and determines the expert sub-model that best matches its data based on the semantic resonance calculation results. The client only performs local training on the selected experts and uploads the updated results. The server then selectively aggregates the client's updates based on the expert dimension and feeds back the updated semantic anchors and expert parameters to the next round of training, thus forming a stable federated training closed loop.

[0090] like Figures 1-2 As shown, a federated learning method based on semantic resonance and expert selection includes steps S1 to S7.

[0091] S1, receive the semantic anchor matrix sent by the server; wherein, the semantic anchor matrix is ​​an orthogonal matrix composed of unique semantic anchors pre-configured by the server for each expert sub-model according to task requirements and obtained by orthogonal decomposition and normalization.

[0092] The client receives the semantic anchor matrix from the server through the federated learning communication link.

[0093] The specific process for generating the semantic anchor matrix is ​​as follows:

[0094] The server initializes a global hybrid expert model containing N expert sub-models. To enable subsequent clients to select appropriate expert sub-models based on semantic features, the server assigns a semantic anchor point to each expert sub-model. All semantic anchor points form a semantic anchor point matrix, represented as follows:

[0095] ;

[0096] in, This is a semantic anchor matrix; Let be the semantic anchor vector corresponding to the i-th expert sub-model; The number of expert sub-models; is the dimension of the semantic feature space.

[0097] The semantic anchor matrix is ​​generated using an orthogonal initialization method, and L2 normalization is performed on each semantic anchor vector, as follows:

[0098] ;

[0099] ;

[0100] in, This represents the initial state of the semantic anchor matrix; This indicates that the matrix has been orthogonalized. This indicates the generation of a random matrix that follows a standard normal distribution; express The i-th vector; This represents a very small constant to prevent the denominator from being zero; This represents the L2 norm of a vector.

[0101] This step involves analyzing the semantic anchor matrix to identify the expert sub-model index corresponding to each column vector, confirming that the matrix satisfies the requirements of orthogonality (the inner product of any two columns is 0) and normalization (the magnitude of each semantic anchor vector is 1), thus ensuring the effectiveness of subsequent semantic resonance calculations.

[0102] This step ensures that all clients use consistent semantic anchors for matching, avoiding expert selection bias caused by inconsistent reference standards; the orthogonally normalized anchor matrix guarantees the uniqueness and distinguishability of each expert's semantic features, laying the foundation for accurate matching.

[0103] S2: Select local samples randomly from the local dataset to extract semantic features, obtain semantic feature vectors, and normalize them.

[0104] After receiving the semantic anchor matrix from the server, the client selects samples from its local dataset. The sample is then processed using an encoding network to extract features, resulting in a semantic feature vector. Simultaneously, to improve the stability of the semantic resonance computation, the client performs normalization on the semantic feature vector, expressed as:

[0105] ;

[0106] ;

[0107] in, It is a semantic feature vector; This is the normalized semantic feature vector; This represents the input sample from the client's local machine. For feature encoding functions; These are the parameters of the encoding network; This represents a very small constant to prevent the denominator from being zero; This represents the L2 norm of a vector.

[0108] This step transforms local samples into semantic feature vectors of uniform dimension. By normalizing, the influence of feature magnitude differences on subsequent semantic matching is eliminated, realizing the semantic quantitative representation of local samples and making the similarity calculation between samples and semantic anchors more comparable. Randomly selecting samples can reduce the impact of local data bias on matching results and improve the generalization of expert selection.

[0109] S3. Based on the normalized semantic feature vector and the semantic anchor matrix, a temperature coefficient is introduced to calculate the semantic resonance score for each local sample, and the semantic resonance probability is mapped to obtain the semantic resonance probability.

[0110] Specifically, the formula for calculating the semantic resonance score is as follows:

[0111] ;

[0112] in, This represents the semantic resonance score between the sample and the i-th expert sub-model; The normalized semantic feature vector; T is the transpose symbol; This is a temperature coefficient used to adjust the concentration level of expert selection; Let be the semantic anchor vector corresponding to the i-th expert sub-model;

[0113] The formula for calculating the semantic resonance probability is:

[0114] ;

[0115] in, This represents the semantic resonance probability of a sample with respect to the i-th expert sub-model; The number of expert sub-models; It is an exponential function.

[0116] This step quantifies the semantic matching degree between a single sample and each expert sub-model, and controls the discriminative power of the matching results using a temperature coefficient. The semantic similarity between samples and experts is converted into probability values ​​that can be directly used for statistics. The introduction of the temperature coefficient makes the matching results more aligned with actual task requirements (e.g., in complex data distribution scenarios, the temperature coefficient can be increased to improve matching robustness); providing foundational data for subsequent client-level semantic relevance statistics.

[0117] S4. Calculate the semantic resonance probability of all local samples to obtain a client-level semantic relevance index that includes average semantic relevance, expert coverage, and resonance uncertainty.

[0118] Since expert selection in federated learning is usually carried out on a "client" basis, this embodiment statistically summarizes the resonance scores of multiple local samples on the client to obtain a client-level semantic relevance score.

[0119] The client performs statistical analysis on the semantic resonance probabilities of M local samples to obtain the correlation between the expert sub-model and the overall data distribution of the client.

[0120] The formula for calculating the average semantic relevance is as follows:

[0121] ;

[0122] in, This represents the average semantic relevance between the i-th expert sub-model and the overall client data; This represents the semantic resonance probability corresponding to the m-th sample of the i-th expert sub-model; This represents the total number of local samples;

[0123] An exponential moving average is introduced to reduce inter-cycle volatility; the formula is as follows:

[0124] ;

[0125] in, This represents the average semantic relevance after exponential moving average. For smoothing coefficients; Historical moving average semantic relevance;

[0126] The formula for calculating the expert coverage rate is as follows:

[0127] ;

[0128] in, This represents the frequency with which the i-th expert sub-model enters the candidate set at the sample level, i.e., the expert coverage rate of the i-th expert sub-model. For indicator functions; Indicates the number of candidate experts at the sample level; This represents the semantic resonance probability corresponding to the m-th sample; This indicates that the first m-th sample is selected. An expert sub-model;

[0129] The formula for calculating the resonance uncertainty is:

[0130] ;

[0131] in, Let represent the resonance uncertainty of the i-th expert sub-model.

[0132] This step comprehensively evaluates the compatibility of each expert with the client's local data distribution from three dimensions: "average matching degree," "coverage," and "matching consistency." This avoids the bias in expert selection caused by a single indicator (such as only looking at the average matching degree). By using multi-dimensional indicators, high-quality experts with "high matching degree, wide coverage, and stable matching" can be screened out, providing a comprehensive decision-making basis for subsequent expert selection.

[0133] S5. Based on the client-level semantic relevance index, calculate the comprehensive semantic matching score to evaluate the degree of matching between each expert sub-model and the local sample data, and obtain the expert selection set.

[0134] Based on the client-level semantic statistics obtained in the aforementioned steps, the client constructs a comprehensive semantic matching scoring function to evaluate the degree of matching between each expert sub-model and the local data.

[0135] The expression for the comprehensive semantic matching score is:

[0136] ;

[0137] in, This represents the comprehensive semantic matching score of the i-th expert sub-model for the client; , , These are the weighting coefficients; This represents the average semantic relevance after exponential moving average. This represents the expert coverage of the i-th expert sub-model; Let be the resonance uncertainty of the i-th expert sub-model;

[0138] The expert selection set is obtained by ranking all expert sub-models based on the comprehensive semantic matching score and combining it with threshold constraints, as expressed in the following expression:

[0139] ;

[0140] in, This represents the set of expert sub-models selected by client c in the current federated training round; This is the minimum selection threshold used to avoid selecting experts with low semantic matching. The number of experts selected for the client-side; A comprehensive semantic matching score is given to all expert sub-models.

[0141] This step, based on a comprehensive evaluation of multi-dimensional adaptability metrics, selects the most suitable subset of expert sub-models for the client's local data. This ensures that the selected experts are highly adapted to the local data, avoiding resource waste caused by training irrelevant or poorly adapted experts on the client side. At the same time, by controlling the number of experts, the computational cost of local training and model performance are balanced.

[0142] S6, Receive the expert sub-model parameter information in the expert selection set transmitted by the server, and perform local forward computation by introducing the gating weights based on the semantic resonance score to obtain the expert model output result.

[0143] Determining the expert selection set Subsequently, the client only requests and receives parameter information for the corresponding expert sub-model from the expert selection set from the server. Therefore, the client's downlink communication is limited to the parameters of the expert sub-model corresponding to the expert selection set, thus achieving communication constraints.

[0144] After downloading the expert parameters, the client performs local forward computation based only on the downloaded expert sub-model. For the input sample... Its semantic feature vector The input is given to the selected expert sub-model, and the output of the i-th expert sub-model is given. Defined as:

[0145] ;

[0146] in, Represents the forward computation function of the i-th expert sub-model; Let represent the set of expert sub-models selected by client c in the current federated training round.

[0147] To fuse the outputs of multiple experts, a gating weight based on semantic resonance scores is introduced to perform weighted fusion on the selected experts. The expression for the gating weight is:

[0148] ;

[0149] in, This represents the contribution weight of the i-th expert sub-model output; , This represents the semantic resonance score between the sample and the i-th and j-th expert sub-models.

[0150] The final model output is:

[0151] ;

[0152] in, This indicates the model output result on the current sample by the client.

[0153] This step utilizes semantic resonance information to construct a gating mechanism, allowing more adaptable experts to play a greater role in local forward computation and improving the accuracy of the output results. Through gated weighted fusion, the output contribution of highly adaptable experts is strengthened while suppressing the influence of poorly adaptable experts, making the forward computation results more closely match the semantic features of local samples; at the same time, only expert parameters from the expert selection set are loaded, reducing the consumption of local storage and computing resources.

[0154] S7. Based on the output results of the expert model, perform local training optimization, and upload the updated parameter information of the expert selection set after training to the server to complete the selective aggregation and update of the expert sub-model parameters on the server.

[0155] In this embodiment, a local total loss function is used for training during local training optimization. The client outputs the final model. Calculate the main task loss using the real labels. (For example, cross-entropy loss or MSE loss can be used). To maintain the differences between different expert sub-models in the semantic space and promote the differentiation of expert functions, a semantic anchor discrete constraint loss is introduced. .

[0156] The expression for the local total loss function is:

[0157] ;

[0158] ;

[0159] ;

[0160] in, This is the local total loss function; , These are the loss weighting coefficients; Loss to the main task; For discrete constraint loss of semantic anchor points; For gating sparsity loss; , These represent the semantic anchor vectors corresponding to different expert sub-models; T is the transpose symbol. Let the contribution weight of the i-th expert sub-model output be used to represent the contribution weight. This represents the set of expert sub-models selected by client c in the current federated training round; This represents a minimal constant to prevent the denominator from being zero.

[0161] The client only updates the parameters of the downloaded expert sub-models and uploads only the corresponding expert selection set after local training is complete. The parameter updates or gradient information of the expert sub-model are represented by the uploaded set as follows:

[0162] ;

[0163] in, This represents the set of expert parameter updates uploaded by client c in the current round; This represents the parameter update result of the i-th expert sub-model.

[0164] After receiving the updated expert parameter sets uploaded by each client, the server selectively aggregates the expert sub-model parameters. Unlike traditional federated learning, which aggregates the entire model uniformly, this invention uses expert sub-models as the basic unit and only performs aggregation operations on the expert sub-models that actually participate in the local training on the client.

[0165] When the server performs selective aggregation updates on the expert sub-model parameters, it calculates the values ​​using a weighted average method, expressed as follows:

[0166] ;

[0167] ;

[0168] ;

[0169] in, Update the aggregation parameters for the i-th expert sub-model; For client c, the weighting coefficients; This represents the set of clients that select and train the i-th expert sub-model in the current round; Update the expert parameters uploaded by client c; , These represent the number of samples used by clients c and e in this round to train the i-th expert sub-model, respectively.

[0170] Based on the aggregation parameter update results, the parameters of the i-th expert sub-model are updated using the following formula:

[0171] ;

[0172] in, These are the current parameters of the i-th expert sub-model.

[0173] For expert sub-models that are not selected by any client in the current federated training round, their parameters remain unchanged and they do not participate in this round of updates.

[0174] This step optimizes local training based on expert fit, improving the expert model's ability to fit local data while simultaneously feeding back targeted parameter updates to the server. Local training focuses on highly fit experts, improving training efficiency and model performance on local data; only parameter updates from selected experts are uploaded, reducing data transmission volume in the federated communication link and lowering communication overhead; the server can selectively aggregate updates based on targeted updates from each client, improving the overall adaptability of the global model.

[0175] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0176] This invention introduces an expert selection mechanism based on semantic resonance, an explicit routing strategy driven by semantic anchors, and a selective parameter communication and aggregation method. This significantly reduces the communication and computational overhead in the federated learning process while ensuring model training performance. Compared with existing technologies, this invention has at least the following advantages:

[0177] (1) The experts’ selection has clear semantic basis, which significantly enhances interpretability.

[0178] This invention constructs semantic anchors that correspond one-to-one with expert sub-models and selects experts based on the semantic resonance strength between input data and semantic anchors. This makes the expert routing process driven by explicit semantic similarity calculation, avoiding the uninterpretability problem caused by traditional black-box gated networks. As a result, there is a clear correspondence between the reasons why the client selects experts and the semantic features of its local data.

[0179] (2) Communication overhead is significantly reduced and resource utilization efficiency is effectively improved.

[0180] By selecting only a small number of expert sub-models that are highly semantically relevant to the local data for training and parameter interaction at the client level, the client does not need to download and upload all model parameters, fundamentally reducing the amount of data that needs to be transmitted during federated learning. While ensuring model performance, this effectively reduces the communication burden and improves the system's adaptability in low-bandwidth or resource-constrained environments.

[0181] (3) The ability to adapt to non-independent and identically distributed data is significantly enhanced.

[0182] Since different clients can select different expert sub-models to participate in training based on the semantic features of their own data, this invention avoids the "one-size-fits-all" update method of the existing technology that uses a unified model structure for all clients. This allows the model to better adapt to the differences in semantic distribution among clients and alleviates the impact of non-independent and identically distributed data on federated learning performance and convergence stability.

[0183] (4) Experts are sufficiently differentiated in their functions to avoid redundancy and ineffectiveness.

[0184] By introducing a semantic anchor discrete constraint mechanism during training, different expert sub-models are kept dispersed and differentiated in the semantic space, preventing multiple experts from learning the same or highly similar semantic functions, thereby improving the overall utilization of expert sub-models and reducing the probability of expert redundancy or long-term non-use.

[0185] (5) Training process stability and global model performance improvement

[0186] The semantic resonance-driven expert selection, selective communication, and parameter aggregation mechanism based on expert dimensions work together to form a stable federated training loop. This reduces communication and computational burden while maintaining the stability of the model training process and the convergence performance of the global model, enabling the method of this invention to maintain good training results in complex, multi-client, and heterogeneous data environments.

[0187] Example 2

[0188] like Figure 3 As shown, the second embodiment of the present invention also provides a federated learning device based on semantic resonance and expert selection, comprising:

[0189] The semantic anchor matrix unit is used to receive the semantic anchor matrix sent by the server; wherein, the semantic anchor matrix is ​​an orthogonal matrix obtained by the server through orthogonal decomposition and normalization, which is composed of unique semantic anchors pre-configured for each expert sub-model according to task requirements.

[0190] The semantic feature extraction unit is used to randomly select local samples from the local dataset to extract semantic features, obtain semantic feature vectors, and normalize them.

[0191] The semantic resonance score calculation unit is used to calculate the semantic resonance score for each local sample based on the normalized semantic feature vector and the semantic anchor matrix, by introducing a temperature coefficient, and to map the semantic resonance probability.

[0192] The semantic relevance index calculation unit is used to calculate the semantic resonance probability of all local samples and obtain a client-level semantic relevance index that includes average semantic relevance, expert coverage and resonance uncertainty.

[0193] The expert selection unit is used to calculate a comprehensive semantic matching score based on the client-level semantic relevance index to evaluate the degree of matching between each expert sub-model and the local sample data, and to obtain the expert selection set.

[0194] The local forward computation unit is used to receive the expert sub-model parameter information in the expert selection set transmitted by the server, and to perform local forward computation by introducing the gating weights based on the semantic resonance score to obtain the expert model output result.

[0195] The selective aggregation update unit is used to perform local training optimization based on the output results of the expert model, and upload the updated parameter information of the expert selection set after training to the server to complete the selective aggregation update of the expert sub-model parameters on the server.

[0196] Example 3

[0197] The third embodiment of the present invention also provides a federated learning device based on semantic resonance and expert selection, which includes a memory and a processor. The memory stores a computer program that can be executed by the processor to implement the federated learning method based on semantic resonance and expert selection as described above.

[0198] Example 4

[0199] The fourth embodiment of the present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device where the computer-readable storage medium is located, implement the federated learning method based on semantic resonance and expert selection as described above.

[0200] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0201] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0202] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0203] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0204] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0205] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0206] The use of "first" and "second" in the embodiments is merely to distinguish similar objects and does not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0207] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A federated learning method based on semantic resonance and expert selection, applied to a client, characterized in that, include: S1, Receive the semantic anchor matrix sent by the server; wherein, the semantic anchor matrix is ​​an orthogonal matrix composed of unique semantic anchors pre-configured by the server for each expert sub-model according to task requirements and obtained by orthogonal decomposition and normalization. S2, randomly select local samples from the local dataset to extract semantic features, obtain semantic feature vectors, and normalize them; S3, Based on the normalized semantic feature vector and the semantic anchor matrix, a temperature coefficient is introduced to calculate the semantic resonance score for each local sample, and the semantic resonance probability is mapped to obtain the semantic resonance probability. S4, Calculate the semantic resonance probability of all local samples to obtain a client-level semantic relevance index that includes average semantic relevance, expert coverage and resonance uncertainty; S5. Based on the client-level semantic relevance index, calculate the comprehensive semantic matching score to evaluate the matching degree between each expert sub-model and the local sample data, and obtain the expert selection set. S6, Receive the expert sub-model parameter information in the expert selection set transmitted by the server, and introduce the gating weights based on the semantic resonance score to perform local forward calculation to obtain the expert model output result; S7. Based on the output results of the expert model, perform local training optimization, and upload the updated parameter information of the expert selection set after training to the server to complete the selective aggregation and update of the expert sub-model parameters on the server. The formula for calculating the semantic resonance score is as follows: ； in, This represents the semantic resonance score between the local sample and the i-th expert sub-model; The normalized semantic feature vector; T is the transpose symbol; This is a temperature coefficient used to adjust the concentration level of expert selection; Let be the semantic anchor vector corresponding to the i-th expert sub-model; The formula for calculating the semantic resonance probability is: ； in, This represents the semantic resonance probability of a local sample with respect to the i-th expert sub-model; The number of expert sub-models; It is an exponential function.

2. The federated learning method based on semantic resonance and expert selection according to claim 1, characterized in that... The specific process for generating the semantic anchor matrix is ​​as follows: The server initializes a global hybrid expert model containing N expert sub-models and assigns a semantic anchor to each expert sub-model. All semantic anchors form a semantic anchor matrix, represented as follows: ； in, This is a semantic anchor matrix; Let be the semantic anchor vector corresponding to the i-th expert sub-model; The number of expert sub-models; Dimension of the semantic feature space; The semantic anchor matrix is ​​generated using an orthogonal initialization method, and L2 normalization is performed on each semantic anchor vector, as follows: ； ； in, This represents the initial state of the semantic anchor matrix; This indicates that the matrix has been orthogonalized. This indicates the generation of a random matrix that follows a standard normal distribution; express The i-th vector; This represents a very small constant to prevent the denominator from being zero; This represents the L2 norm of a vector.

3. The federated learning method based on semantic resonance and expert selection according to claim 1, characterized in that... The formula for calculating the average semantic relevance is as follows: ； in, This represents the average semantic relevance between the i-th expert sub-model and the overall client data; This represents the semantic resonance probability corresponding to the m-th sample of the i-th expert sub-model; This represents the total number of local samples; An exponential moving average is introduced to reduce inter-cycle volatility; the formula is as follows: ； in, This represents the average semantic relevance after exponential moving average. For smoothing coefficients; Historical moving average semantic relevance; The formula for calculating the expert coverage rate is as follows: ； in, This represents the frequency with which the i-th expert sub-model enters the candidate set at the sample level, i.e., the expert coverage rate of the i-th expert sub-model. For indicator functions; Indicates the number of candidate experts at the sample level; This represents the semantic resonance probability corresponding to the m-th sample; This indicates that the first m-th sample is selected. An expert sub-model; The formula for calculating the resonance uncertainty is: ； in, Let represent the resonance uncertainty of the i-th expert sub-model.

4. The federated learning method based on semantic resonance and expert selection according to claim 3, characterized in that... The expression for the comprehensive semantic matching score is: ； in, This represents the comprehensive semantic matching score of the i-th expert sub-model for the client; , , These are the weighting coefficients; This represents the average semantic relevance after exponential moving average. This represents the expert coverage of the i-th expert sub-model; Let be the resonance uncertainty of the i-th expert sub-model; The expert selection set is obtained by ranking all expert sub-models based on the comprehensive semantic matching score and combining it with threshold constraints, as expressed in the following expression: ； in, This represents the set of expert sub-models selected by client c in the current federated training round; This is the minimum selection threshold used to avoid selecting experts with low semantic matching. The number of experts selected for the client-side; A comprehensive semantic matching score is given to all expert sub-models.

5. The federated learning method based on semantic resonance and expert selection according to claim 4, characterized in that... The expression for the gating weight is: ； in, This represents the contribution weight of the i-th expert sub-model output; , This represents the semantic resonance score between the sample and the i-th and j-th expert sub-models; Let represent the set of expert sub-models selected by client c in the current federated training round.

6. The federated learning method based on semantic resonance and expert selection according to claim 5, characterized in that... When performing local training optimization based on the output results of the expert model, a local total loss function is used for training; the expression of the local total loss function is: ； ； ； in, This is the local total loss function; , These are the loss weighting coefficients; Loss to the main task; For discrete constraint loss of semantic anchor points; For gating sparsity loss; , These represent the semantic anchor vectors corresponding to different expert sub-models; T is the transpose symbol. Let the contribution weight of the i-th expert sub-model output be used to represent the contribution weight. This represents the set of expert sub-models selected by client c in the current federated training round; This represents a minimal constant to prevent the denominator from being zero.

7. The federated learning method based on semantic resonance and expert selection according to claim 5, characterized in that... When the server performs selective aggregation updates on the expert sub-model parameters, it calculates the values ​​using a weighted average method, expressed as follows: ； ； ； in, Update the aggregation parameters for the i-th expert sub-model; For client c, the weighting coefficients; This represents the set of clients that select and train the i-th expert sub-model in the current round; Update the expert parameters uploaded by client c; , These represent the number of samples used by clients c and e in this round to train the i-th expert sub-model, respectively. Based on the aggregation parameter update results, the server updates the parameters of the i-th expert sub-model using the following formula: ； in, These are the current parameters of the i-th expert sub-model.

8. A federated learning device based on semantic resonance and expert selection, used to implement a federated learning method based on semantic resonance and expert selection as described in any one of claims 1-7, characterized in that, include: The semantic anchor matrix unit is used to receive the semantic anchor matrix sent by the server; wherein, the semantic anchor matrix is ​​an orthogonal matrix obtained by the server through orthogonal decomposition and normalization, which is composed of unique semantic anchors pre-configured for each expert sub-model according to task requirements. The semantic feature extraction unit is used to randomly select local samples from the local dataset to extract semantic features, obtain semantic feature vectors, and normalize them. The semantic resonance score calculation unit is used to calculate the semantic resonance score for each local sample based on the normalized semantic feature vector and the semantic anchor matrix, by introducing a temperature coefficient, and to map the semantic resonance probability. The semantic relevance index calculation unit is used to calculate the semantic resonance probability of all local samples and obtain a client-level semantic relevance index that includes average semantic relevance, expert coverage and resonance uncertainty. The expert selection unit is used to calculate a comprehensive semantic matching score based on the client-level semantic relevance index to evaluate the degree of matching between each expert sub-model and the local sample data, and to obtain the expert selection set. The local forward computation unit is used to receive the expert sub-model parameter information in the expert selection set transmitted by the server, and to perform local forward computation by introducing the gating weights based on the semantic resonance score to obtain the expert model output result. The selective aggregation update unit is used to perform local training optimization based on the output results of the expert model, and upload the updated parameter information of the expert selection set after training to the server to complete the selective aggregation update of the expert sub-model parameters on the server.

9. A federated learning device based on semantic resonance and expert selection, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that can be executed by the processor to implement a federated learning method based on semantic resonance and expert selection as described in any one of claims 1-7.

Images