Longitudinal federated learning method and device based on feature importance and electronic equipment

Abstract

The present application relates to a longitudinal federated learning method and device based on feature importance, and an electronic device, which is driven by local feature importance, and realizes a private budget generator at the client end through a lightweight neural network, and further realizes a collaborative closed loop from "feature-level privacy budget" to "embedding dimension perturbation" to "recommended privacy budget upload" to "gradient noise addition at the server end", and introduces a discrete level division and a recommended privacy budget perturbation mechanism to avoid the server end from deducing privacy content, thereby improving the overall differential privacy security and practicability of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of privacy computing, and in particular to a vertical federated learning method and device based on feature importance and an electronic device. BACKGROUND

[0002] Vertical federated learning (VFL) is a collaborative learning paradigm that protects data privacy, and is suitable for scenarios where the samples of each participant coincide but the feature spaces are different (such as joint modeling by a bank and an e-commerce company). In VFL, the client usually only has its own private feature dimension, and the server is responsible for aligning the samples and coordinating the training process of the joint model. In order to prevent feature leakage, mainstream VFL frameworks will apply differential privacy protection to the embedding vectors or their gradients transmitted by the client, such as adding noise, introducing random masks, etc.

[0003] Differential privacy (DP) is a mainstream mechanism for implementing privacy protection, which limits the upper bound of information leakage by adding noise to sensitive data or its statistics. Among them, the privacy budget (ε) is a key parameter for controlling the strength of the noise, which usually needs to be manually set. However, a fixed or uniform privacy budget in the feature-heterogeneous VFL scenario can easily cause two problems: one is that the low-sensitive features are over-noised, which reduces the model performance; the other is that the high-sensitive features are not noisy enough, which poses a risk of privacy leakage. Therefore, how to design a privacy budget allocation and noise adding mechanism that can automatically adapt to feature sensitivity and end-to-end optimization has become one of the key challenges in current VFL.

[0004] Existing technologies attempt to introduce feature-aware or adaptive perturbation control, mainly focusing on the following two directions:

[0005] Noise adding methods based on feature importance or gradient sensitivity (mainly applied to HFL): Some horizontal federated learning (HFL) schemes attempt to introduce feature weights, model attention or gradient amplitude to guide the allocation of privacy budget, such as adjusting the noise intensity by using feature importance scores. This kind of method can improve the privacy-performance trade-off effect in some scenarios, but they usually rely on global label information or global feature perspective, so it is difficult to migrate to the feature-heterogeneous and sample-aligned scenario of VFL;

[0006] Fixed group noise or uniform budget perturbation (applied to VFL): the mainstream differential privacy solution in existing VFL, such as the framework based on SecureBoost, SplitNN, etc., often uniformly applies the same strength of Laplace or Gaussian noise to all embedding or gradient dimensions. This method is simple to implement and easy to analyze, but lacks modeling of feature difference and embedding dimension structure, and cannot fine-tune the noise distribution according to the actual sensitivity of the features and the contribution of the model, resulting in a large performance loss. SUMMARY

[0007] The application provides a longitudinal federated learning method based on feature importance, device and electronic equipment, driven by local feature importance, a privacy budget generator is realized at the client end through a lightweight neural network, and a collaborative closed loop of "feature-level privacy budget" to "embedding dimension perturbation" to "recommended privacy budget upload" to "server-side gradient noise addition" is realized, and a discrete level division and recommended privacy budget perturbation mechanism are introduced to avoid the server-side backpropagation of privacy content, thereby improving the overall differential privacy security and practicability of the system.

[0008] Specifically, the application is implemented by the following technical solutions:

[0009] The longitudinal federated learning method based on feature importance comprises the following steps:

[0010] The client evaluates the local original features to generate a feature importance vector;

[0011] The client constructs a lightweight neural network as a privacy budget generator with the feature importance vector as input, and outputs a feature-level privacy budget vector corresponding to the feature importance vector;

[0012] The client generates an embedding-level privacy budget vector using a pseudo-model copy consistent with the local embedding model structure and sharing parameters, and adds noise to the local embedding vector according to the embedding-level privacy budget vector to generate a privacy-protected noise-added embedding vector;

[0013] The client generates a recommended privacy budget vector after discretizing the embedding-level privacy budget vector;

[0014] The server receives the noise-added embedding vector and the recommended privacy budget vector, and calculates the loss function by forward propagation;

[0015] The server performs gradient calculation and parameter update;

[0016] The server adds noise to the gradient based on the recommended privacy budget vector to generate a noise-added gradient;

[0017] The client receives the noise-added gradient and performs local embedding model update, privacy budget generator update and feature importance update.

[0018] Further, the Gini impurity is used to evaluate the local original features.

[0019] Further, the lightweight neural network is specifically used to train a local loss function comprising a gradient guiding term, a total budget constraint term and a distribution extremity penalty term.

[0020] Further, the local original features are sent to a local embedding model to generate embedding vectors after being filtered by bits, and the noise addition to the local embedding vectors is specifically Laplace noise disturbance.

[0021] Further, the specific method of discretization is as follows:

[0022]

[0023] wherein γ m is a recommended privacy budget vector, T1-T4 are four levels, τ1-τ3 are embedding level privacy budget vectors corresponding to 25%, 50%, 75% of the distribution setting points of all embedding level privacy budget vectors, is an embedding level privacy budget vector, d' m is an embedding dimension.

[0024] Further, the generation of the noisy gradient is specifically as follows: the server divides the total privacy budget that can be allocated to each client according to the number of clients, combines the four levels of the recommended privacy budget vector, calculates the final budget of the gradient corresponding to each client, and generates a noisy gradient by adding noise to the gradient of each dimension.

[0025] The application also provides a longitudinal federated learning device based on feature importance, which comprises:

[0026] a feature importance evaluation module, configured to evaluate local original features by the client and generate a feature importance vector;

[0027] a privacy budget generation module, configured to take the feature importance vector as input, and use the lightweight neural network constructed by the client as a privacy budget generator to output a feature level privacy budget vector corresponding to the feature importance vector;

[0028] an embedding disturbance and recommended budget generation module, configured to use a pseudo model copy consistent with the structure of the local embedding model and sharing parameters to generate an embedding level privacy budget vector by the client, add noise to the local embedding vector according to the embedding level privacy budget vector, and generate a privacy-protected noisy embedding vector;

[0029] and the client generates a recommended privacy budget vector after discretizing the embedding level privacy budget vector;

[0030] The gradient perturbation module is used by the server to receive the noisy embedding vector and the proposed privacy budget vector, and to calculate the loss function during forward propagation.

[0031] The server performs gradient calculations and updates parameters.

[0032] The server adds noise to the gradient based on the proposed privacy budget vector to generate a noisy gradient;

[0033] The client update module is used by the client to receive the noisy gradient and perform local embedding model updates, privacy budget generator updates, and feature importance updates.

[0034] The end-to-end joint optimization module runs through both the client and server sides and is used to embed the above processes into the end-to-end training framework of vertical federated learning.

[0035] The present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described longitudinal federated learning method based on feature importance.

[0036] Beneficial effects:

[0037] This invention enables more granular privacy protection strategies that are more aligned with model importance through feature-level budget modeling and learning, thereby improving the coordination between privacy protection and model performance. Secondly, through features selection and embedding with noise during uploading, and budget discretization feedback, the communication and noise costs in federated learning are effectively reduced. Finally, the constructed feedback loop supports online optimization and adaptive adjustment of privacy strategies, breaking through the static limitations of the privacy-accuracy trade-off and demonstrating greater practicality and promotional value. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the working principle and process of an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0040] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0041] The technical terms used in the following embodiments are explained as follows:

[0042] 1) Vertical Federated Learning (VFL): refers to a joint learning method in which data feature dimensions are distributed among different institutions or clients, and each party has the same sample but different features. In this setting, only part of the participants have label information (such as the server side), and the rest are passive participants (such as the client side).

[0043] 2) Differential Privacy (DP): a mathematical framework for measuring the strength of privacy protection. The basic idea is that under the premise of adding or deleting a certain sample, the distribution difference of the algorithm output is small enough, so it is difficult to infer whether a certain individual is in the data set.

[0044] 3) Privacy Budget: a key parameter for controlling the strength of noise addition, used to measure the trade-off between privacy protection and model accuracy. It is usually used to control the scale factor of Laplace or Gaussian noise.

[0045] 4) Feature Importance: a measure of the contribution of original features to the model's prediction results. In this invention, feature importance is used as input to the privacy budget generator to guide the personalized configuration of privacy budget.

[0046] 5) Embedding Vector: a representation method that maps high-dimensional sparse features to low-dimensional continuous vector space, facilitating model processing. Embedding is usually an intermediate representation form of the model, used to capture the semantic relationship between features. Participate in the joint training process and upload to the server for further modeling.

[0047] 6) Privacy Budget Allocator: in this invention, it refers to a lightweight neural network structure deployed locally on the client, used to generate feature-level privacy budget vectors based on feature importance, achieving dynamic and fine-grained noise control.

[0048] 7) Suggested Privacy Budget Vector: the embedding-level privacy budget vector generated by the client is discretized and uploaded as reference guidance information for server gradient noise addition, avoiding the reverse inference of original features.

[0049] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] Figure 1 For the working principle and flowchart of the present embodiment, as shown in Figure 1 the feature importance-based vertical federated learning method of the present embodiment specifically includes:

[0051] S1, the client evaluates the local original features to generate a feature importance vector.

[0052] Specifically, the Gini impurity is used to evaluate the local original features. According to this, feature screening is performed to eliminate redundant or low-contribution features, reduce communication overhead and model redundancy calculation, and improve overall efficiency. Compared with information gain, variance, mutual information and other methods, the Gini impurity method has higher requirements for label access, or has complex calculation and insufficient privacy protection. The Gini impurity method does not require complete model training and is suitable for privacy protection calculation in longitudinal federated settings. It can realize effective feature preliminary screening while ensuring security, and provide a good starting point for subsequent learnable privacy allocation strategies.

[0053] More specifically, this step is responsible for the feature importance evaluation module, and specifically includes the following steps:

[0054] S1.1 The server generates a label indication matrix and encrypts it to obtain Send to the client;

[0055] S1.2 The client divides the local sample set U m into b sub-clusters, and for each sub-cluster U m，i calculates the proportion of the kth label it belongs to Where I(U m，i ) represents the set of sample indexes;

[0056] S1.3 Calculate the Gini impurity of each sub-cluster Aggregate to get the global importance score of each feature Where j∈[d m ];

[0057] S1.4 The server decrypts After sending it back to the client, the client initializes the feature importance vector After normalization, the standardized feature importance vector is obtained:

[0058] In S1.2, for the client m, if its jth feature is a feature with b discrete values, divide its sample set U m into U m,1 ∪...∪U m,i . Where U m,i represents the sample set with the i th value of the feature. is the true class of the a th sample, if equal to 1, it means the k th class.

[0059] In S1.3, d m is the feature dimension, and c is the c-class label. fm，j the jth feature of the client m.

[0060] S2, taking the feature importance vector as input, the client builds a lightweight neural network as a privacy budget generator, outputting a feature-level privacy budget vector corresponding to the feature importance vector.

[0061] Specifically, the lightweight neural network is specifically trained on a local loss function containing a gradient guiding term, a total budget constraint term, and a distribution extremity penalty term. More specifically, the gradient guiding term: encourages the budget vector to match the gradient sensitivity; the total budget constraint term: controls the total consumption of the budget, satisfying the overall privacy constraint of the system; the distribution extremity penalty term: prevents the budget from being highly concentrated or excessively sparse. More specifically, compared to the fixed rule or static allocation method based on gradient size, which lacks understanding of feature sensitivity and task relevance, it cannot achieve "protection on demand", and performs significantly worse when the budget is limited. The present embodiment uses a learnable privacy budget generator to automatically align the budget allocation with the feature importance, achieving a dynamic trade-off between accuracy and privacy.

[0062] More specifically, this step is responsible for the privacy budget generation module, specifically including the following steps:

[0063] S2.1: receiving the feature importance vector μ from S1 as input into the privacy budget generator PBA (using a lightweight neural network); m φ ;

[0064] S2.2: generating a feature-level privacy budget vector where ∈ m is the feature-level privacy budget vector, PBA φ is the neural network mapping function, and φ is its parameter;

[0065] S2.3: the local loss function of the privacy budget generator consists of the following three parts:

[0066] S2.3.1: gradient reverse guiding term: guide PBA returned by the server to improve perturbation behavior;

[0067] S2.3.2: total budget control term, to prevent the budget from being too large: encourage budget sparsity or total budget limit;

[0068] S2.3.3: distribution balance term, to prevent distribution extremity:

[0069] The total loss function expression is

[0070] ​In S2.3.1, L is the loss function of the server, g m The noisy embedding vector uploaded by the client. The gradient of the loss function with respect to the embedding vector is calculated and returned to the server.

[0071] In S2.3.2, λ1 is the weight coefficient, ||∈ m ||1 is the L1 norm of the feature-level privacy budget vector.

[0072] In S2.3.3, λ2 is the weighting coefficient, Var(∈ m ) represents the variance of the feature-level privacy budget vector.

[0073] S3. The client generates an embedding-level privacy budget vector using a pseudo-model copy that has the same structure and shares parameters as the local embedding model. Based on the embedding-level privacy budget vector, the local embedding vector is denoised to generate a privacy-preserving denoised embedding vector.

[0074] Specifically, the original local features are filtered bit-by-bit and then fed into the local embedding model to generate embedding vectors. Noise is added to these local embedding vectors by applying Laplacian noise perturbation. More specifically, compared to most designs that only implement one-way privacy injection without client-side feedback and budget suggestion expression, resulting in the server's inability to add noise as needed and low budget utilization efficiency, the privacy budget suggestion feedback mechanism designed in this embodiment can express the client's intention regarding noise "level" to the server. The server then allocates the actual budget according to the level, enhancing the client's privacy control and the server's collaborative capabilities.

[0075] More specifically, this step is handled by the embedded perturbation and suggested budget generation module, and includes the following steps:

[0076] S3.1: Calculation This regularization term is used during backpropagation to control the sparsity of feature selection. It generates a feature sampling mask to suppress unimportant features.

[0077] S3.2: After filtering the original features bit by bit, they are fed into the embedding network to generate local embedding vectors. in

[0078] S3.3: Scale the local embedding vector to ensure that the L2 norm after scaling does not exceed a preset threshold c, thus protecting the sensitivity definition when adding differential privacy noise to the local embedding vector.

[0079] S3.4: Generate an embedding-level privacy budget vector using a pseudo-model copy that has the same structure and shares parameters as the local embedded model:

[0080] S3.5: Laplace noise perturbation according to the embedding-level privacy budget vector:

[0081] Φ(·) in S3.1 is the Gaussian cumulative distribution function (CDF), R m as a regularization term to control the sparsity of feature selection. m，i is the perturbed feature sampling weight, which controls whether the original feature is activated or not, and is used for soft mask selection of the original feature, where the σ1 hyperparameter is used to control the constraint strength of the regularization term on the sparsity of feature selection, and the σ2 hyperparameter is used to control the amplitude of random perturbation when generating the feature sampling mask.

[0082] x in S3.2 is the original feature, n，m is the original feature, is the filtered original feature, θ m is the parameter of the embedding network of the client m.

[0083] h in S3.3 is the embedding vector, is the clipped embedding vector, c is the upper limit of the norm set to limit the embedding sensitivity, ||h n，m ||2 is the L2 norm of the embedding vector.

[0084] The pseudo model structure used in S3.4 is exactly the same as the embedding model structure, and adopts a shared parameter method to facilitate gradient consistent update and functional decoupling, and is used for processing x n，m and ∈ m .

[0085] Δh in S3.5 is the embedding sensitivity. g n，m is the final noisy embedding vector uploaded to the server, which has been added with noise to satisfy differential privacy.

[0086] S4, the client discretizes the embedding-level privacy budget vector to generate a recommended privacy budget vector.

[0087] More specifically, this step is responsible for the embedding perturbation and recommended budget generation module, and specifically includes the following steps:

[0088] S4.1: Set the embedding-level privacy budget vectors corresponding to the 25%, 50%, and 75% quantile points to τ1-τ3 by statistically analyzing the distribution of all embedding-level privacy budget vectors.

[0089] S4.2: Discretize the embedding-level privacy budget vector generated in S3, and discretize it into four levels T1-T4, as follows: to obtain the recommended privacy budget vector γ m ;

[0090] S4.3: The client uploads the suggested privacy budget vector and the noisy embedding vector to the server.

[0091] S5: The server receives the noisy embedding vector and the suggested privacy budget vector, and forward-propagates to calculate the loss function.

[0092] Specifically, this step is responsible for the gradient perturbation module, and specifically includes the following steps:

[0093] S5.1: The server receives the suggested privacy budget vector and the embedding vector from the client;

[0094] S5.2: The server aggregates the embedding vectors of all clients to calculate the loss function Where y n is the label.

[0095] S6: The server performs gradient calculation and updates the parameters.

[0096] Specifically, this step is responsible for the gradient perturbation module, and specifically includes the following steps:

[0097] S6.1: The server calculates the gradient

[0098] S6.2: The server updates θ0={w1, w2,..., w m , α0}, and calculates and θ0 is the initial parameter set of the server-side model. η0 is the server-side learning rate.

[0099] S7: The server adds noise to the gradient based on the suggested privacy budget vector to generate a noisy gradient.

[0100] Specifically, generating a noisy gradient is specifically: the server divides the total privacy budget that can be allocated to each client by the number of clients, combines the four levels of the suggested privacy budget vector, calculates the final budget of the corresponding gradient of each client, and adds noise to each dimension of the gradient to generate a noisy gradient.

[0101] More specifically, this step is responsible for the gradient perturbation module, and specifically includes the following steps:

[0102] S7.1: The server divides the total privacy budget by the number of clients, and the total privacy budget that the server can allocate to each client is Map the four levels to the proportions r(T1)=0.1, r(T2)=0.2, t(T3)=0.3, r(T4)=0.4;

[0103] S7.2: Calculate the final budget of the embedding dimension: Here

[0104] S7.3: The gradient of each dimension is returned to the client after adding noise

[0105] S8, the client receives the noisy gradient and performs local embedding model update, privacy budget generator update and feature importance update.

[0106] Specifically, this step is responsible for the client update module, which specifically includes the following steps:

[0107] S8.1: The client receives the noisy gradient from the server, and uses the chain rule to calculate and

[0108] S8.2: Update and Where η m is the client learning rate; λ is the regularization parameter.

[0109] S8.3: Calculate and The local loss function gradient is as follows Update the privacy budget generator parameters

[0110] In addition, the embodiment also includes an end-to-end joint optimization module, which runs through the client and the server, and is used to embed the above-mentioned processes into the end-to-end training framework of the longitudinal federated learning. This module can integrate embedding learning, feature perturbation, gradient noise and main task loss optimization and other sub-modules, ensure that the privacy control mechanism and the model training process are consistent, and form a closed-loop optimization process.

[0111] Figure 2 The structure schematic diagram of the electronic device provided by the embodiment of the application. The electronic device can include:

[0112] The memory 201, the processor 202, and the computer program stored in the memory 201 and executable on the processor 202.

[0113] The processor 202 implements the feature importance-based longitudinal federated learning method provided in the above-mentioned embodiments when executing the program.

[0114] Further, the electronic device further includes:

[0115] The communication interface 203 is used for communication between the memory 201 and the processor 202.

[0116] The memory 201 is used to store the computer program executable on the processor 202.

[0117] The memory 201 can include a high-speed RAM (Random Access Memory) memory, and can also include a nonvolatile memory such as at least one disk memory.

[0118] If the memory 201, the processor 202 and the communication interface 203 are implemented independently, the communication interface 203, the memory 201 and the processor 202 can be connected to each other through a bus and complete communication between each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 2 In the figure, only one thick line is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0119] Optionally, in a specific implementation, if the memory 201, the processor 202 and the communication interface 203 are integrated on a chip, the memory 201, the processor 202 and the communication interface 203 can complete communication between each other through an internal interface.

[0120] The processor 202 can be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present application.

[0121] In this embodiment, each module collaborates according to the vertical federated learning framework. First, in the system initialization stage, the client first analyzes the contribution degree of the local original features through the feature importance evaluation module, generates a feature importance vector as input, and sends it to the privacy budget generator module. The generator outputs a feature-level privacy budget vector, which is processed by the embedding disturbance and recommended budget generation module to generate an embedding-level privacy budget vector, and guides the embedding vector to perform Laplace noise addition, thereby generating a privacy-protected noisy embedding vector; at the same time, the embedding-level privacy budget vector is also mapped to a discrete level to further form a recommended privacy budget vector, which is sent to the server. After receiving the recommended privacy budget vector, the gradient disturbance module on the server side assigns a gradient disturbance intensity to each embedding dimension according to the level information, realizes fine noise addition to the gradient, and updates the server-side model parameters; the client update module receives the noisy gradient sent by the server and updates the local embedding model, the privacy budget generator, and the feature importance. During the training process, the end-to-end joint optimization module is throughout the client and the server to uniformly model and solve the task loss function, the privacy budget control target, and the disturbance strategy, realize the dynamic balance of accuracy and privacy, and continuously optimize the generator parameters and the embedding representation quality, finally achieve the purpose of collaborative improvement of performance and privacy.

[0122] In summary, the embodiment innovatively realizes the adaptive privacy control strategy of client-server collaboration by constructing the "feature importance evaluation-differentiated privacy budget generation-embedding noise upload-budget recommendation feedback-server-side gradient noise-local update" closed-loop mechanism. Compared with the traditional uniform privacy budget mechanism, the embodiment has three significant advantages: first, through feature-level budget modeling and learning, the privacy protection strategy is more fine-grained and adapted to the model importance, improving the coordination of privacy protection and model performance; second, through feature screening, embedding noise upload, and budget discretization feedback design, the communication and noise addition cost in federated learning is effectively reduced; finally, the feedback loop constructed supports online optimization and adaptive adjustment of the privacy strategy, breaking the static limit of privacy-accuracy trade-off, and having stronger practicality and promotion value.

[0123] It should be understood that parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. As in another embodiment, if realized by hardware, any one or a combination of the following technologies known in the art can be used: discrete logic circuit with logic gate circuit for implementing logic function on data signal, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array, field programmable gate array, etc.

[0124] Those skilled in the art can understand that all or part of the steps carried out by the method of the above-mentioned embodiments can be instructed by a program to complete the relevant hardware, and the above-mentioned program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0125] Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A federated learning method based on feature importance, characterized in that, The method comprises the following steps: The client evaluates the local original features to generate a feature importance vector; Specifically, the server generates a label indication matrix and encrypts to obtain Send to the client; the client divides the local sample set Into b sub-clusters, for each sub-cluster Calculate the proportion of its belonging to the kth label Wherein Indicates a set of sample indexes, Is the true class of the ath sample, if equal to 1, it indicates that it is the kth class; calculate the Gini impurity of each sub-cluster Aggregated to get the global importance score of each feature Wherein , Is the feature dimension, Is The class label, Is the jth feature of the client m; The server sends the decrypted feature importance vector to the client The client normalizes the feature importance vector to obtain a standardized feature importance vector ;​ The client constructs a lightweight neural network as a privacy budget generator with the feature importance vector as input, and outputs a feature-level privacy budget vector corresponding to the feature importance vector; The client generates an embedding-level privacy budget vector using a pseudo model copy consistent with the structure of the local embedding model and sharing parameters, and generates a privacy-protected noisy embedding vector by adding noise to the local embedding vector according to the embedding-level privacy budget vector; The client generates a recommended privacy budget vector after discretizing the embedding-level privacy budget vector; The server receives the noisy embedding vector and the recommended privacy budget vector, and calculates the loss function by forward propagation; The server calculates the gradient and updates the parameters; The server adds noise to the gradient based on the recommended privacy budget vector to generate a noisy gradient; The client receives the noisy gradient and updates the local embedding model, the privacy budget generator, and the feature importance.

2. The feature importance based federated learning method of claim 1, wherein, The lightweight neural network is specifically trained on a local loss function including a gradient guide term, a total budget constraint term, and a distribution extremity penalty term.

3. The feature importance based federated learning method of claim 1, wherein, The local original features are sent to the local embedding model to generate an embedding vector after bit filtering, and the noise perturbation of Laplace is added to the local embedding vector.

4. The feature importance based federated learning method of claim 1, wherein, The specific method of discretization is: , wherein, is the suggested privacy budget vector, T1-T4 are four levels, is the embedding level privacy budget vector corresponding to the 25%, 50%, 75% quantile of the distribution of all embedding level privacy budget vectors, is the embedding level privacy budget vector, is the embedding dimension.

5. The feature importance based federated learning method of claim 1, wherein, The generation of the noisy gradient is specifically: the server divides the total privacy budget that can be allocated to each client according to the number of clients, combines the four levels of the recommended privacy budget vector, calculates the final budget of the gradient corresponding to each client, and adds noise to each dimension of the gradient to generate a noisy gradient.

6. The federated learning device based on feature importance, characterized in that, It comprises: a feature importance evaluation module for the client to evaluate the local original features using Gini impurity to generate a feature importance vector; a privacy budget generation module for the client to construct a lightweight neural network as a privacy budget generator with the feature importance vector as input, and output a feature-level privacy budget vector corresponding to the feature importance vector; an embedding perturbation and recommended budget generation module for the client to generate an embedding-level privacy budget vector using a pseudo model copy consistent with the structure of the local embedding model and sharing parameters, and generate a privacy-protected noisy embedding vector by adding noise to the local embedding vector according to the embedding-level privacy budget vector; and the client generates a recommended privacy budget vector after discretizing the embedding-level privacy budget vector; a gradient perturbation module for the server to receive the noisy embedding vector and the recommended privacy budget vector, and calculate the loss function by forward propagation; and the server calculates the gradient and updates the parameters; and the server adds noise to the gradient based on the recommended privacy budget vector to generate a noisy gradient; a client update module for the client to receive the noisy gradient and update the local embedding model, the privacy budget generator, and the feature importance; an end-to-end joint optimization module running through the client and the server, for embedding the above processes into an end-to-end training framework of vertical federated learning.

7. An electronic device, comprising: It comprises: A memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the federated learning method based on feature importance of any one of claims 1-5.

Images