Federation method and device based on soft parameter sharing
By employing a soft parameter sharing mechanism, shallow consistency and deep adaptability of the generator and predictor are ensured in federated learning, which solves the problem of model performance degradation caused by non-IID data and improves the interpretability and classification accuracy of the model.
Patent Information
- Application Number
- CN202511488935.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-17
- Publication Date
- 2026-03-06
Smart Images
Figure CN121614920A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of natural language processing technology, and more specifically, to a federated rationalization method and apparatus based on soft parameter sharing. Background Technology
[0002] When training rationalization models in a federated learning environment, data heterogeneity (non-IID data) among clients leads to a significant performance degradation, particularly in generating consistent fundamental principles. Specifically, on the one hand, non-IID data in federated learning causes inconsistent learning directions among clients, reducing overall model performance and potentially causing model drift. On the other hand, for rationalization models, differences in text styles, feature distributions, or annotation standards among clients make it difficult for the model to generate stable and coherent reasons, weakening its interpretability and generalization ability. Furthermore, while existing federated learning methods (such as parameter regularization, personalized FL, and local model alignment) alleviate the non-IID problem to some extent, optimizing performance while maintaining the interpretability of rationalization models remains an unresolved issue. Summary of the Invention
[0003] To address at least one deficiency or improvement requirement of the prior art, this invention provides a federated rationalization method and apparatus based on soft parameter sharing. It employs a mechanism between the soft-shared generator and predictor to ensure the consistency of shallow networks and achieve better generalization capabilities.
[0004] To achieve the above objectives, according to a first aspect of the present invention, a federated rationalization method based on soft parameter sharing is provided, the method comprising: S1, the client uses the local dataset to update the local generator encoder parameters and the local predictor encoder parameters, and obtains the updated local generator encoder parameters and the local predictor encoder parameters. S2, the server receives the local generator encoder parameters and local predictor encoder parameters uploaded by all clients, and aggregates the generator global parameters and predictor global parameters respectively according to the amount of client data. The generator global parameters are obtained only by aggregating the local generator encoder parameters, and the predictor global parameters are obtained only by aggregating the local predictor encoder parameters. S3, repeat S1-S2 until the preset number of communication rounds is reached, so that the final global generator and global predictor maintain feature consistency in the shallow encoder, while their respective deep networks fully adapt to non-independent and identically distributed data, thereby improving the model classification accuracy and reason interpretability in a non-IID federated environment.
[0005] Furthermore, before the client updates the local generator encoder parameters and the local predictor encoder parameters using the local dataset, the following steps are also included: On the server side, the generator global parameters and the predictor global parameters are initialized respectively, and the generator global parameters and the predictor global parameters are distributed to N clients. The generator global parameters include at least the generator encoder parameters, and the predictor global parameters include at least the predictor encoder parameters.
[0006] Furthermore, the client updates the local generator encoder parameters and the local predictor encoder parameters using the local dataset to obtain updated local generator encoder parameters and local predictor encoder parameters, including: The client uses the sum of the prediction loss and the encoder loss as the joint loss. The encoder loss is the square of the Euclidean distance between the local generator encoder parameters and the local predictor encoder parameters. The joint loss is achieved by backpropagation to synchronize the updated local generator encoder parameters with the local predictor encoder parameters, allowing their respective deep parameters to independently adapt to the local data distribution. Furthermore, the generator also includes generator linear layer parameters, and the predictor also includes predictor linear layer parameters. The generator linear layer parameters and predictor linear layer parameters are updated locally in sync with the corresponding encoder parameters, and are aggregated with the corresponding encoder parameters on the server side.
[0007] Furthermore, the encoder loss uses the Frobenius norm to calculate the difference between the local generator encoder parameters and the local predictor encoder parameters, in order to further improve the accuracy and generalization balance in non-IID environments.
[0008] Furthermore, the local update step further includes: mapping the input text into a semantic feature vector using a GRU-based sequence encoder, and then generating a reason mask through a linear layer. The semantic feature vector and the reason mask are used together for subsequent prediction, wherein the GRU parameters are updated with the parameters of the local generator encoder.
[0009] Furthermore, the local dataset is a beer review dataset or a hotel review dataset, and the data for each client comes from only a single review aspect, in order to verify the effect in non-IID scenarios.
[0010] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the federated rationalization method based on soft parameter sharing as described above.
[0011] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: This invention presents a federated rationalization method based on soft parameter sharing. The method involves clients updating their local generator and predictor encoder parameters using local datasets, resulting in updated local generator and predictor encoder parameters. A server then receives these updated parameters from all clients and aggregates them weighted according to client data volume, achieving hierarchical aggregation. This aggregation is iteratively converged until a preset number of communication rounds are reached. This ensures that the generator and predictor global parameters maintain feature consistency in the shallow encoder weights while their deep network weights adapt to non-independent and identically distributed data, thus achieving federated rationalization. In this invention, the shallow encoders of the global generator and global predictor maintain feature consistency in each communication, while the deep networks evolve independently using local data. This mechanism enables the model to output stable and coherent rationales in non-IID federated environments, while also fitting the data distribution of each client, ultimately improving both classification accuracy and rationale interpretability. Attached Figure Description
[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0013] Figure 1 A schematic diagram of the structure of a proposed framework for an optional federal rationalization method based on soft parameter sharing, provided for embodiments of this application; Figure 2 A flowchart illustrating an optional federal rationalization method based on soft parameter sharing, provided for an embodiment of this application; Figure 3 This is a schematic diagram of an optional electronic device provided in an embodiment of this application. Detailed Implementation
[0014] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0015] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.
[0016] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.
[0017] Explainability is crucial in Natural Language Processing (NLP) because it enhances model transparency, fosters user trust, and supports decision-making. However, deep learning models are often black-box systems, making their reasoning processes difficult to understand, thus necessitating better interpretability in NLP models. Rationalization models address this need by extracting key input fragments (i.e., reasons) that explain the model's decisions while maintaining predictive performance. They consist of a generator and a predictor: the generator selects the most informative subset of text as reasons, which is then passed to the predictor for the final classification or decision. However, existing research has primarily focused on centralized implementations of these models, leaving their adaptability to distributed learning scenarios largely unexplored. With rising concerns about data privacy and security, extending rationalization models to distributed environments such as Federated Learning (FL) has become a key direction. However, the challenges posed by non-independent and identically distributed (non-IID) data further complicate this transition.
[0018] Rationalization models aim to enhance the interpretability of NLP by selecting key input fragments (reasons) that justify the model's predictions while maintaining predictive performance. These models typically consist of a generator that extracts the most informative subset of text and a predictor that makes the final decision based on the selected reason. As research into rationalization models has deepened, they can be further categorized into abstraction and extraction methods. Extraction rationalization models identify and extract keywords or sentences from the input text, capturing the most prominent features to explain the predictions. Some work has investigated extraction methods using an encoder-decoder framework. The encoder assigns a binary label to each word in the input sequence to indicate whether it is part of a fundamental principle. The decoder then processes only the highlighted fundamental principle words and maps them to the target category. Other work uses attention mechanisms to extract reasons. Abstract rationalization models generate reasons by constructing explanations using new words and sentences reconstructed from the input text. Some work has investigated text-to-text methods that leverage sequence-to-sequence translation models, combining labels and explanations. Others use generative methods to generate free-form explanations and then make predictions based on the generated abstract theories. Despite significant progress, most rationalization models are designed for centralized training, assuming access to a single dataset. The challenge of adapting them to foreign languages remains largely unexplored.
[0019] Non-IID in Federated Learning. Many existing studies have proposed solutions to mitigate client drift caused by non-IID data. Some methods rely on personalization techniques to tailor model optimization to the unique data distribution of each client. For example, FedROD integrates a globally shared general model with personalized client models through a joint training mechanism. FedProto utilizes prototypes—central representations of classes—to provide personalized model training for each client. There is also work on improving the generalization ability of the global model, such as parameter regularization and local model alignment methods. For example, FedProx introduces a proximal term in local subproblems to adjust local updates to account for differences between the global and local models. FedPer combines each client with globally shared model parameters. FedNH mitigates the impact of data heterogeneity on federated learning by integrating class prototypes into the global model. However, these methods are not suitable for rationalizing models.
[0020] In federated learning (FL), data heterogeneity arises when different clients collect data from different sources, leading to significant distributional variations. This non-IID nature causes inconsistent learning directions among clients, degrading overall model performance and potentially causing model drift. This problem is particularly pronounced for rationalization models, as text styles, feature distributions, or annotation standards differ between clients. This inconsistency makes it difficult for the model to generate stable and coherent reasoning, weakening its interpretability and generalization ability. Many methods have been proposed to address the non-IID problem, including parameter regularization, personalized FL, and local model alignment. While these methods alleviate the non-IID problem to some extent, optimizing federated learning performance to rationalize the model while maintaining its interpretability remains an open question.
[0021] To address the aforementioned issues, this invention proposes a federated rationalization method and apparatus based on soft parameter sharing to mitigate model drift. Specifically, it employs a mechanism that softly shares the generator and predictor to ensure consistency in shallow networks while allowing deep networks to adapt to non-IID data distributions (i.e., shallow consistency and deep generalization). This invention enables the rationalized model to maintain its interpretability while improving classification performance in non-IID scenarios, thereby achieving better generalization capabilities.
[0022] Figure 1 The illustration shows a flowchart of an optional federated rationalization method based on soft parameter sharing according to an embodiment of this application; as shown... Figure 1 As shown, the federated rationalization method based on soft parameter sharing according to embodiments of this application includes: S1, The server initializes the global parameters of the generator and the global parameters of the predictor, and sends the global parameters of the generator and the global parameters of the predictor to multiple clients; S2, the client updates the generator encoder parameters and predictor encoder parameters using a local database based on the joint loss, and obtains the updated local generator encoder parameters and updated local predictor encoder parameters; the joint loss is the sum of the prediction loss and the encoder loss, and the encoder loss is the square of the Euclidean distance between the generator encoder parameters and the predictor encoder parameters. S3, the server receives the updated local generator encoder parameters and the updated local predictor encoder parameters uploaded by the client, and aggregates the generator global parameters and predictor global parameters respectively according to the client data volume. The generator global parameters are obtained by aggregating the updated local generator encoder parameters, and the predictor global parameters are obtained by aggregating the updated local predictor encoder parameters. S4, repeat S2-S3 until the preset number of communication rounds is reached, so that the global generator and global predictor in the server maintain feature consistency in the shallow encoder, while their respective deep networks adapt to non-independent and identically distributed data, thus achieving federated rationalization.
[0023] The following is a specific example to illustrate this.
[0024] Figure 2 The illustration shows a schematic diagram of an optional federated rationalization architecture based on soft parameter sharing according to an embodiment of this application; as shown Figure 2 As shown, the server first generates initialized generator global parameters and initialized predictor global parameters containing encoder weights and linear layer weights in round t=0, and broadcasts them to all N clients. After receiving the initialized generator global parameters and initialized predictor global parameters, each client i executes K=N epochs on the local non-independent and identically distributed dataset Di: for each batch of samples, the generator encoder first extracts features, then the generator linear layer generates a mask, and the mask is used by the predictor encoder and predictor linear layer to obtain predictions; the cross-entropy prediction loss and encoder loss are calculated, and a joint loss is constructed, for example, by updating the local generator encoder parameters and local predictor encoder parameters with the Adam optimizer to obtain the updated local generator encoder parameters and updated local predictor encoder parameters.
[0025] After this round is completed, the server collects the updated local generator encoder parameters and the updated local predictor encoder parameters from all clients, calculates the weight wi=mi / Σmj based on the client data volume mi, aggregates the global parameters of the generator and the global parameters of the predictor respectively, and distributes them again. The above local update and global aggregation are repeated for a preset number of rounds, such as 500 rounds. Finally, the global generator and the global predictor maintain a CKA similarity of up to a preset threshold, such as 0.87 feature consistency, in the shallow encoder. At the same time, they achieve a suitable classification accuracy and a suitable Golden Ratio F1 score in the client scenario with the most severe non-IID, thus realizing federated rationalization.
[0026] Optionally, in the process of constructing the rationalization model, the rationalization model is represented as It consists of two parts: a generator and a predictor. The generator is the input paragraph. Basic principles of generation and extraction Then, the basic principles of generation. For Class prediction. The prediction process can be represented as... The entire process, namely the class prediction based on fundamental principles, can be integrated into a single model, represented as follows: FR
[18] will and The structure is divided into encoder layers. With linear layer The generator and predictor share the same... This ensures the consistency of shallow networks.
[0027] In the collaborative learning process, One client, each client Each has a local dataset ,in This refers to the number of data points in the dataset. FL's non-IID representation... The distribution differs among clients. In this embodiment, using... Represents the global model. Indicates the client Local model. In each communication round The server then aggregates updates from each client using a weighted average to update the global model. The standard FL process typically follows this: each client i on its own dataset... The local model is trained and its update is calculated. The local update process of the local model is as follows: , (1) in It is the learning rate. The server aggregates local model updates from all clients using a weighted average, where... yes Dataset size: (2) in It represents the total number of samples from all clients.
[0028] In the FedRNL framework, each client i uses the dataset it can access. Train its local model This approach aims to maintain privacy and avoid directly sharing raw data. It should be noted that, for simplicity, this embodiment will be omitted from the subsequent sections of this section. Labels. Local model It consists of two parts: the generator and predictor We further decompose these two components into an encoder layer and a linear layer. Specifically, the generator... include and Predictor Depend on and composition.
[0029] The generator is responsible for processing the input data. Basic principles of generation ,in This represents the subset or transformation of input considered most relevant to the prediction. First, the encoder layer of the generator processes the input text... Extract semantic features and generate feature representations for each label of x. .Then, The linear layer is passed to the generator and generates a mask. The generator selects the most important parts of the text for classification. Through this process, the generator creates the basic principle z, providing the necessary information for subsequent predictions. Then, z is passed to the encoder layer of the predictor, which is responsible for extracting the feature representation of z. . The data is then processed through a linear layer of the predictor, followed by a classification layer that performs the classification task and outputs the final prediction. This process enables the predictor to make accurate predictions based on the fundamental principles provided by the generator.
[0030] The entire process, namely the class prediction based on fundamental principles, can be integrated into a single model and defined as follows: It is worth noting that, in order to train and improve the performance of the rationalization model, the predicted output of the predictor can be calculated. and ground truth labels The cross-entropy loss between them. This loss is called prediction loss. The optimization constraints used during training are defined as follows: (3) in, The cross-entropy loss of the rationalization model is given.
[0031] Regarding the encoder loss, to enhance feature alignment between the generator and predictor, this invention introduces a soft-sharing mechanism that aligns the encoder layers learned by the generator and predictor across clients. Specifically, this invention designs an encoder loss... To control the encoder layer and The parameter similarity between the two encoders aims to optimize their parameters and ensure that their feature representations remain consistent. The optimization constraints are defined as follows: (4) in This represents the squared L1 norm of the Euclidean distance between two parameters. By minimizing this loss, and The parameters are guided to optimize in a more similar direction, improving the coordination between the generator and the predictor, and ultimately improving the model's performance in non-IID problems.
[0032] Regarding the joint loss, the total loss for each client is the sum of the prediction loss and the encoder loss: (5) in, It is a hyperparameter that controls the balance between prediction loss and encoder loss.
[0033] Regarding global model aggregation, when aggregating the global model, algorithms such as FedAvg can be used to aggregate local model parameters from all clients. In the... In each round, the server receives local model parameters from each client and updates the global model parameters. Specifically, the generator and predictor parameters are aggregated separately to update the global model. The global aggregation formula is as follows: , (6) in and This represents the global model parameters for the generator and predictor. This represents the local model parameters of the generator, derived from the encoder layer parameters. and linear layer parameters composition. This represents the local model parameters of the predictor, derived from the encoder layer parameters. and linear layer parameters composition.
[0034] In summary, the federated rationalization method based on soft parameter sharing of this invention utilizes the local dataset to update the local generator encoder parameters and local predictor encoder parameters on the client side, obtaining the updated local generator encoder parameters and local predictor encoder parameters, thus completing the local update. Then, the server receives the updated local generator encoder parameters and local predictor encoder parameters uploaded by all clients, and aggregates the generator global parameters and predictor global parameters separately according to the amount of client data, achieving hierarchical aggregation. Furthermore, through iterative convergence until a preset number of communication rounds are reached, the generator global parameters and predictor global parameters maintain feature consistency in the shallow encoder weights, while their respective deep network weights adapt to non-independent and identically distributed data, thereby achieving federated rationalization.
[0035] In this invention, the shallow encoders of the global generator and global predictor maintain feature consistency in every communication, while the deep network evolves independently using local data. This mechanism enables the model to output stable and coherent reasons in a non-IID federated environment, while also fitting the data distribution of each client, ultimately improving both classification accuracy and reason interpretability.
[0036] Exemplary electronic devices Figure 3 This is a schematic diagram of an optional electronic target device according to an embodiment of this application, such as... Figure 3 As shown, it includes a processor 302, a communication interface 304, a memory 306, and a communication bus 308. The processor 302, communication interface 304, and memory 306 communicate with each other via the communication bus 308. Memory 306 is used to store computer programs; When processor 302 executes a computer program stored in memory 306, it performs the following steps: S1, the server generates initialization generator global parameters and initialization predictor global parameters, and distributes the initialization generator global parameters and initialization predictor global parameters to multiple clients; the initialization generator global parameters include at least generator encoder parameters, and the initialization predictor global parameters include at least predictor encoder parameters.
[0037] S2, the client updates the generator encoder parameters and predictor encoder parameters using a local database based on the joint loss, and obtains the updated local generator encoder parameters and updated local predictor encoder parameters; the joint loss is the sum of the prediction loss and the encoder loss, and the encoder loss is the square of the Euclidean distance between the generator encoder parameters and the predictor encoder parameters. S3, the server receives the updated local generator encoder parameters and the updated local predictor encoder parameters uploaded by the client, and aggregates the generator global parameters and predictor global parameters separately according to the client data volume. The generator global parameters are obtained by aggregating the updated local generator encoder parameters, and the predictor global parameters are obtained by aggregating the updated local predictor encoder parameters. S4, repeat S2-S3 until the preset number of communication rounds is reached, so that the global parameters of the generator and the global parameters of the predictor maintain feature consistency in the shallow encoder weights, while their respective deep network weights adapt to the non-independent and identically distributed data, thereby achieving federated rationalization.
[0038] Optionally, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic target device and other devices.
[0039] The memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage target device located remotely from the aforementioned processor.
[0040] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0041] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this application may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the federal rationalization method based on soft parameter sharing according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.
[0042] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of this application. The programming languages include object-oriented programming languages such as Java and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0043] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the moving object tracking methods according to various embodiments of this application described in the "Exemplary Methods" section of this specification.
[0044] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, target device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.
[0045] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.
[0046] The block diagrams of devices, target apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, target apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.
[0047] It should also be noted that in the target apparatus, equipment, and method of this application, each component or step can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.
[0048] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.
[0049] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.
Claims
1. A federated rationalization method based on soft parameter sharing, characterized in that, The method comprises the following steps: S1, the server generates initialization generator global parameters and initialization predictor global parameters, and distributes the initialization generator global parameters and the initialization predictor global parameters to a plurality of clients; the initialization generator global parameters at least include generator encoder parameters, and the initialization predictor global parameters at least include predictor encoder parameters; S2, the client updates the generator encoder parameters and the predictor encoder parameters based on a joint loss by using a local database to obtain updated local generator encoder parameters and updated local predictor encoder parameters; the joint loss is the sum of a prediction loss and an encoder loss, and the encoder loss is the square of the Euclidean distance between the generator encoder parameters and the predictor encoder parameters; S3, the server receives the updated local generator encoder parameters and the updated local predictor encoder parameters uploaded by the client, and respectively aggregates the generator global parameters and the predictor global parameters according to the data quantity of the client, wherein the generator global parameters are obtained by aggregating the updated local generator encoder parameters, and the predictor global parameters are obtained by aggregating the updated local predictor encoder parameters; S4, repeating S2-S3 until a preset communication round is reached, so that the generator global parameters and the predictor global parameters maintain the feature consistency of the shallow encoder weights while the respective deep network weights adapt to the non-independent and identically distributed data, thereby realizing federated rationalization.
2. The federated rationalization method based on soft parameter sharing according to claim 1, wherein the joint loss synchronously constrains the updated local generator encoder parameters and the local predictor encoder parameters to be consistent and allows the respective deep parameters to independently adapt to the local data distribution through back propagation.
3. The federated rationalization method based on soft parameter sharing according to claim 1, wherein the generator global parameters further include generator linear layer parameters, and the predictor global parameters further include predictor linear layer parameters; the generator linear layer parameters and the predictor linear layer parameters are updated locally at the client side synchronously with the corresponding encoder parameters, and are aggregated at the server side with the corresponding encoder parameters, respectively.
4. The federated rationalization method based on soft parameter sharing according to claim 1, wherein the encoder loss calculates the difference between the local generator encoder parameters and the local predictor encoder parameters by using the Frobenius norm.
5. The federated rationalization method based on soft parameter sharing according to claim 2, wherein the parameter updating step comprises: mapping the input text into a semantic feature vector by using a GRU-based sequence encoder, and then generating a reason mask through a linear layer, wherein the semantic feature vector and the reason mask are used together for subsequent prediction, and the GRU parameters in the GRU-based sequence encoder are updated with the local generator and the local encoder of the client.
6. The federated rationalization method based on soft parameter sharing according to claim 1, wherein The server aggregates the generator global parameters and the predictor global parameters respectively by the FedAvg algorithm according to the client data amount.
7. The soft parameter sharing based federated rationalization method of any one of claims 1 to 6, wherein, The server receives the updated local generator encoder parameters and the updated local predictor encoder parameters uploaded by the clients, and aggregates the generator global parameters and the predictor global parameters respectively according to the client data amount, and further comprises: The server performs additional adaptive update on the generator global parameters and the predictor global parameters by using an Adam optimizer.
8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the federated rationalization method based on soft parameter sharing when executing the computer program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the federated rationalization method based on soft parameter sharing as claimed in any one of claims 1 to 7.
10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the federated rationalization method based on soft parameter sharing as claimed in any one of claims 1 to 7. The computer program is executed by the processor to implement the federated rationalization method based on soft parameter sharing as claimed in any one of claims 1 to 7.