Longitudinal federal learning method and device based on feature importance and electronic equipment

Abstract

The invention relates to a longitudinal federal learning method and device based on feature importance and electronic equipment, and aims to realize a local privacy budget generator of a client through a lightweight neural network by taking local feature importance as drive. Therefore, a cooperative closed loop from'feature-level privacy budget 'to'embedded dimension disturbance' to'suggested privacy budget uploading 'to'server-side gradient noise addition' is realized, and meanwhile, discrete grading and suggested privacy budget disturbance mechanisms are introduced to prevent the server-side from reversely deducing privacy contents, so that the differential privacy security and practicability of the whole system are improved.

Description

Technical Field

[0001] This invention relates to the field of privacy computing, and more particularly to a longitudinal federated learning method, apparatus, and electronic device based on feature importance. Background Technology

[0002] Vertical Federated Learning (VFL) is a data privacy-preserving collaborative learning paradigm suitable for scenarios where participants share similar samples but have different feature spaces (e.g., joint modeling by banks and e-commerce companies). In VFL, clients typically possess only their own private feature dimensions, while the server is responsible for aligning samples and coordinating the training process of the joint model. To prevent feature leakage, mainstream VFL frameworks apply differential privacy protection to the embedding vectors or gradients transmitted by the clients, such as adding noise or introducing random masks.

[0003] Differential privacy (DP) is a mainstream mechanism for achieving privacy protection. It limits the upper bound of the probability of information leakage by adding noise to sensitive data or its statistics. The privacy budget (ε), as a key parameter controlling the noise intensity, usually needs to be set manually. However, a fixed or uniform privacy budget can easily lead to two problems in the feature-heterogeneous VFL scenario: first, excessive noise addition to low-sensitivity features, reducing model performance; and second, insufficient noise addition to high-sensitivity features, posing a risk of privacy leakage. Therefore, designing a privacy budget allocation and noise addition mechanism that can automatically adapt to feature sensitivity and optimize end-to-end has become one of the key challenges in current VFL implementations.

[0004] Existing technologies attempt to introduce feature-aware or adaptive disturbance control, mainly focusing on the following two directions:

[0005] Noise-adding methods based on feature importance or gradient sensitivity (mainly applied to HFL): Some lateral federated learning (HFL) schemes attempt to introduce feature weights, model attention, or gradient magnitude to guide the allocation of privacy budgets, for example, by adjusting noise intensity using feature importance scores. These methods can improve the privacy-performance tradeoff in some scenarios, but they usually rely on global label information or a global feature perspective, making them difficult to transfer to VFL scenarios with feature heterogeneity and sample alignment.

[0006] Fixed-group noise addition or uniform budget perturbation (applied to VFL): Current mainstream differential privacy schemes in VFL, such as those based on SecureBoost and SplitNN frameworks, often apply the same intensity of Laplacian or Gaussian noise to all embeddings or gradient dimensions. This method is simple to implement and easy to analyze, but it lacks modeling of feature differences and embedding dimensional structure, and cannot finely control the noise distribution according to the actual sensitivity of features and model contribution, resulting in significant performance loss. Summary of the Invention

[0007] This invention provides a longitudinal federated learning method, apparatus, and electronic device based on feature importance. Driven by local feature importance, it implements a client-side local privacy budget generator through a lightweight neural network, thereby achieving a collaborative closed loop from "feature-level privacy budget" to "embedded dimension perturbation" to "suggested privacy budget upload" to "server-side gradient noise addition". At the same time, it introduces discrete level partitioning and suggested privacy budget perturbation mechanisms to avoid the server-side back-deriving of privacy content, thereby improving the overall differential privacy security and practicality of the system.

[0008] Specifically, the present invention is achieved through the following technical solution:

[0009] The longitudinal federated learning method based on feature importance includes the following steps:

[0010] The client evaluates the original local features and generates a feature importance vector;

[0011] Using the feature importance vector as input, the client constructs a lightweight neural network as a privacy budget generator, and outputs a feature-level privacy budget vector corresponding to the feature importance vector;

[0012] The client generates an embedding-level privacy budget vector using a pseudo-model copy that has the same structure and shares parameters as the local embedded model. Based on the embedding-level privacy budget vector, the local embedded vector is noise-added to generate a privacy-preserving noisy embedding vector.

[0013] The client generates a suggested privacy budget vector by discretizing the embedded-level privacy budget vector;

[0014] The server receives a noisy embedding vector and a proposed privacy budget vector, and calculates the loss function through forward propagation.

[0015] The server performs gradient calculations and updates the parameters;

[0016] The server adds noise to the gradient based on the proposed privacy budget vector, generating a noisy gradient.

[0017] The client receives the noisy gradient and performs local embedding model updates, privacy budget generator updates, and feature importance updates.

[0018] Furthermore, Gini impurity was used to evaluate the native original characteristics.

[0019] Furthermore, the lightweight neural network is specifically trained on a local loss function that includes a gradient guidance term, a total budget constraint term, and an allocation extremum penalty term.

[0020] Furthermore, the original local features are filtered bit by bit and then fed into the local embedding model to generate embedding vectors. The local embedding vectors are then noise-added by applying Laplacian noise perturbation.

[0021] Furthermore, the specific method of discretization is as follows:

[0022]

[0023] Where, γ m To suggest privacy budget vectors, T1-T4 represent four levels, and τ1-τ3 represent the embedding-level privacy budget vectors corresponding to the quantiles of 25%, 50%, and 75% for the distribution of all embedding-level privacy budget vectors. For the embedded-level privacy budget vector, d' m For the embedded dimension.

[0024] Furthermore, the noisy gradient generation process involves the server dividing the total privacy budget available to each client equally according to the number of clients, calculating the final budget for the gradient corresponding to each client based on the four levels of the proposed privacy budget vector, and generating a noisy gradient by adding noise to the gradient of each dimension.

[0025] This invention also provides a longitudinal federated learning apparatus based on feature importance, comprising:

[0026] The feature importance evaluation module is used by the client to evaluate the original local features and generate feature importance vectors;

[0027] The privacy budget generation module is used to take the feature importance vector as input, and the client builds a lightweight neural network as a privacy budget generator to output the feature-level privacy budget vector corresponding to the feature importance vector.

[0028] The embedding perturbation and suggestion budget generation module is used by the client to generate an embedding-level privacy budget vector using a pseudo-model copy with the same structure and shared parameters as the local embedding model. Based on the embedding-level privacy budget vector, the local embedding vector is noise-added to generate a privacy-preserving noisy embedding vector.

[0029] The client generates a suggested privacy budget vector by discretizing the embedded-level privacy budget vector.

[0030] The gradient perturbation module is used by the server to receive the noisy embedding vector and the proposed privacy budget vector, and to calculate the loss function during forward propagation.

[0031] The server performs gradient calculations and updates parameters.

[0032] The server adds noise to the gradient based on the proposed privacy budget vector to generate a noisy gradient;

[0033] The client update module is used by the client to receive the noisy gradient and perform local embedding model updates, privacy budget generator updates, and feature importance updates.

[0034] The end-to-end joint optimization module runs through both the client and server sides and is used to embed the above processes into the end-to-end training framework of vertical federated learning.

[0035] The present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described longitudinal federated learning method based on feature importance.

[0036] Beneficial effects:

[0037] This invention enables more granular privacy protection strategies that are more aligned with model importance through feature-level budget modeling and learning, thereby improving the coordination between privacy protection and model performance. Secondly, through features selection and embedding with noise during uploading, and budget discretization feedback, the communication and noise costs in federated learning are effectively reduced. Finally, the constructed feedback loop supports online optimization and adaptive adjustment of privacy strategies, breaking through the static limitations of the privacy-accuracy trade-off and demonstrating greater practicality and promotional value. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the working principle and process of an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0040] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0041] The technical terms used in the following embodiments are explained as follows:

[0042] 1) Vertical Federated Learning (VFL): This refers to a joint learning approach where data features are distributed across different institutions or clients, with each party possessing the same samples but different features. In this setup, only some participants have label information (e.g., the server), while the rest are passive participants (e.g., clients).

[0043] 2) Differential Privacy (DP): This is a mathematical framework for measuring the strength of privacy protection. Its basic idea is that, given the addition or removal of a sample, the distribution of the algorithm's output should have a sufficiently small difference, making it difficult to infer whether a specific individual is in the dataset.

[0044] 3) Privacy Budget: A key parameter controlling the intensity of noise addition, used to measure the trade-off between privacy protection and model accuracy. It is typically used to control the scaling factor of Laplacian or Gaussian noise.

[0045] 4) Feature Importance: A measure of how much the original features contribute to the model's prediction results. In this invention, feature importance is used as input to the privacy budget generator to guide the personalized configuration of the privacy budget.

[0046] 5) Embedding Vector: A way to map high-dimensional sparse features to a low-dimensional continuous vector space, facilitating model processing. Embeddings are often an intermediate representation of the model, used to capture semantic relationships between features. They participate in the joint training process and are uploaded to the server for further modeling.

[0047] 6) Privacy Budget Allocator: In this invention, it refers to a lightweight neural network structure deployed locally on the client side, used to adaptively generate feature-level privacy budget vectors based on feature importance, thereby achieving dynamic and refined noise control.

[0048] 7) Suggested Privacy Budget Vector: The client discretizes the generated embedding-level privacy budget vector and uploads it as a reference guide for the server to add noise to the gradient, avoiding the need to back-infer the original features.

[0049] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0050] Figure 1 This is a schematic diagram illustrating the working principle and process of this embodiment, as follows: Figure 1 As shown, the longitudinal federated learning method based on feature importance in this embodiment specifically includes:

[0051] S1. The client evaluates the original local features and generates a feature importance vector.

[0052] Specifically, Gini impurity is used to evaluate the original local features. Based on this, feature filtering is performed to remove redundant or low-contribution features, reducing communication overhead and redundant model computation, and improving overall efficiency. Compared to methods such as information gain, variance, and mutual information, which either have higher requirements for label access or are computationally complex and lack sufficient privacy protection, the Gini impurity method does not require full model training, is suitable for privacy-preserving computation in a vertical federated setting, can achieve effective initial feature screening while ensuring security, and provides a good starting point for subsequent learnable privacy allocation strategies.

[0053] More specifically, this step is handled by the feature importance assessment module, and includes the following steps:

[0054] The S1.1 server generates a tag indication matrix and encrypts it to obtain... Send to the client;

[0055] The S1.2 client will use the local sample set U m Divide into b subclusters, for each subcluster U m，i Calculate the proportion of its tags belonging to the k-th class. Among them I(U m，i ) represents the set of sample indices;

[0056] S1.3 Calculate the Gini impurity for each subcluster. The aggregation yields a global importance score for each feature. where j∈[d m ];

[0057] S1.4 server After decryption, the data is sent back to the client, which then initializes the feature importance vector. After normalization, the standardized feature importance vector is obtained:

[0058] In S1.2, for client m, if its j-th feature is a feature with b discrete values, its sample set U m Divided into U m,1 ∪...∪U m,i U m,i Let represent the set of samples whose feature value is the i-th value. This indicates whether the true class of the a-th sample is the k-th class. If it equals 1, it means that it is the k-th class.

[0059] S1.3 in d m is the feature dimension, and c is the label of class c.m，j Let j be the j-th feature of client m.

[0060] S2. Using the feature importance vector as input, the client constructs a lightweight neural network as a privacy budget generator, and outputs the feature-level privacy budget vector corresponding to the feature importance vector.

[0061] Specifically, the lightweight neural network is trained on a local loss function that includes a gradient guidance term, a total budget constraint term, and an allocation extreme penalty term. More specifically, the gradient guidance term encourages the budget vector to match gradient sensitivity; the total budget constraint term controls the total budget consumption to meet the overall system privacy constraints; and the allocation extreme penalty term prevents the budget from becoming highly concentrated or excessively sparse. More specifically, compared to static allocation methods based on fixed rules or gradient magnitude that lack understanding of feature sensitivity and task relevance and cannot achieve "on-demand protection," resulting in a significant performance drop when the budget is limited, this embodiment uses a learnable privacy budget generator to automatically align budget allocation with feature importance, achieving a dynamic trade-off between accuracy and privacy.

[0062] More specifically, this step is handled by the privacy budget generation module and includes the following steps:

[0063] S2.1: Receive the feature importance vector μ from S1 m The privacy budget generator PBA is passed in as input. φ Medium (using lightweight neural networks);

[0064] S2.2: Generate feature-level privacy budget vector Where ∈ m For feature-level privacy budget vectors, PBA φ Let φ be the neural network mapping function, and φ be its parameter.

[0065] S2.3: The local loss function of the privacy budget generator consists of the following three parts:

[0066] S2.3.1: Gradient backward guiding term: The gradient returned by the server guides PBA to improve perturbation behavior;

[0067] S2.3.2: Total budget control item to prevent excessive budget: Encourage budget sparsity or total budget constraints;

[0068] S2.3.3: Allocate balancing terms to prevent extreme allocation:

[0069] The total loss function expression is as follows:

[0070] In S2.3.1, L is the loss function of the server, g m The noisy embedding vector uploaded by the client. The gradient of the loss function with respect to the embedding vector is calculated and returned to the server.

[0071] In S2.3.2, λ1 is the weight coefficient, ||∈ m ||1 is the L1 norm of the feature-level privacy budget vector.

[0072] In S2.3.3, λ2 is the weighting coefficient, Var(∈ m ) represents the variance of the feature-level privacy budget vector.

[0073] S3. The client generates an embedding-level privacy budget vector using a pseudo-model copy that has the same structure and shares parameters as the local embedding model. Based on the embedding-level privacy budget vector, the local embedding vector is denoised to generate a privacy-preserving denoised embedding vector.

[0074] Specifically, the original local features are filtered bit-by-bit and then fed into the local embedding model to generate embedding vectors. Noise is added to these local embedding vectors by applying Laplacian noise perturbation. More specifically, compared to most designs that only implement one-way privacy injection without client-side feedback and budget suggestion expression, resulting in the server's inability to add noise as needed and low budget utilization efficiency, the privacy budget suggestion feedback mechanism designed in this embodiment can express the client's intention regarding noise "level" to the server. The server then allocates the actual budget according to the level, enhancing the client's privacy control and the server's collaborative capabilities.

[0075] More specifically, this step is handled by the embedded perturbation and suggested budget generation module, and includes the following steps:

[0076] S3.1: Calculation This regularization term is used during backpropagation to control the sparsity of feature selection. It generates a feature sampling mask to suppress unimportant features.

[0077] S3.2: After filtering the original features bit by bit, they are fed into the embedding network to generate local embedding vectors. in

[0078] S3.3: Scale the local embedding vector to ensure that the L2 norm after scaling does not exceed a preset threshold c, thus protecting the sensitivity definition when adding differential privacy noise to the local embedding vector.

[0079] S3.4: Generate an embedding-level privacy budget vector using a pseudo-model copy that has the same structure and shares parameters as the local embedded model:

[0080] S3.5: Apply Laplace noise perturbation based on the embedded privacy budget vector:

[0081] In S3.1, Φ(·) is the Gaussian cumulative distribution function (CDF), and R m As a regularization term, it controls the sparsity of feature selection. m，i The perturbation feature sampling weights control whether the original features are activated and are used for soft mask selection of the original features. The σ1 hyperparameter controls the constraint strength of the regularization term on the sparsity of feature selection, and the σ2 hyperparameter controls the magnitude of random perturbation when generating the feature sampling mask.

[0082] S3.2 x n，m Original features For the original features after filtering, θ m The parameters for the embedded network of client m.

[0083] S3.3 Here is the clipped embedding vector, c is the upper norm setting used to limit embedding sensitivity, and ||h n，m ||2 is the L2 norm of the embedding vector.

[0084] The pseudo-model structure used in S3.4 is completely consistent with the embedded model structure, employing a shared parameter approach to facilitate consistent gradient updates and functional decoupling, with each parameter used separately for x. n，m and ∈ m The processing.

[0085] In S3.5, Δh represents the embedding sensitivity. n，m The noisy embedding vector that is finally uploaded to the server has been noised to satisfy differential privacy.

[0086] S4. The client discretizes the embedded-level privacy budget vector and generates a suggested privacy budget vector.

[0087] More specifically, this step is handled by the embedded perturbation and suggested budget generation module, and includes the following steps:

[0088] S4.1: Statistically analyze the distribution of all embedded privacy budget vectors. Set the embedded privacy budget vectors corresponding to the quantiles of 25%, 50%, and 75% as τ1 to τ3, respectively.

[0089] S4.2: Discretize the embedded-level privacy budget vector generated in S3 into four levels, T1-T4, as follows: The suggested privacy budget vector γ is obtained. m ;

[0090] S4.3: The client uploads the suggested privacy budget vector and the noisy embedding vector to the server.

[0091] S5. The server receives the noisy embedding vector and the proposed privacy budget vector, and calculates the loss function through forward propagation.

[0092] Specifically, this step is handled by the gradient perturbation module and includes the following steps:

[0093] S5.1: The server receives the suggested privacy budget vector and embedding vector from the client;

[0094] S5.2: The server aggregates the embedding vectors of all clients to calculate the loss function. where y n For tags.

[0095] S6. The server performs gradient calculations and updates the parameters.

[0096] Specifically, this step is handled by the gradient perturbation module and includes the following steps:

[0097] S6.1: Server calculates gradient

[0098] S6.2: Server updates θ0 = {w1, w2, ..., w} m , α0}, calculate and θ0 is the initial parameter set of the server-side model. η0 is the server-side learning rate.

[0099] S7. The server adds noise to the gradient based on the proposed privacy budget vector to generate a noisy gradient.

[0100] Specifically, generating a noisy gradient involves the server dividing the total privacy budget available to each client equally according to the number of clients, calculating the final budget for the gradient corresponding to each client based on the four levels of the proposed privacy budget vector, and generating a noisy gradient by adding noise to the gradient of each dimension.

[0101] More specifically, this step is handled by the gradient perturbation module and includes the following steps:

[0102] S7.1: The server distributes the total privacy budget equally among the number of clients. The total privacy budget that the server can allocate to each client is... The four levels are mapped to proportions r(T1) = 0.1, r(T2) = 0.2, t(T3) = 0.3, and r(T4) = 0.4.

[0103] S7.2: Calculate the final budget for the embedding dimension: Here

[0104] S7.3: Add noise to the gradient of each dimension and then send it back to the client.

[0105] S8. The client receives the noisy gradient and performs local embedding model updates, privacy budget generator updates, and feature importance updates.

[0106] Specifically, this step is handled by the client update module and includes the following steps:

[0107] S8.1: The client receives the noisy gradient from the server and calculates it using the chain rule. and

[0108] S8.2: Update and Where η m λ is the client learning rate; λ is the regularization parameter.

[0109] S8.3: Calculation and The gradient of the local loss function is as follows Update privacy budget generator parameters

[0110] Furthermore, this embodiment also includes an end-to-end joint optimization module that spans both the client and server sides. This module embeds the aforementioned processes into the end-to-end training framework of vertical federated learning. This module integrates multiple sub-modules such as embedding learning, feature perturbation, gradient noise addition, and main task loss optimization, ensuring consistency between the privacy control mechanism and the model training process, forming a closed-loop optimization process.

[0111] Figure 2 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. The electronic device may include:

[0112] The memory 201, the processor 202, and the computer program stored on the memory 201 and capable of running on the processor 202.

[0113] When the processor 202 executes the program, it implements the longitudinal federated learning method based on feature importance provided in the above embodiments.

[0114] Furthermore, electronic devices also include:

[0115] Communication interface 203 is used for communication between memory 201 and processor 202.

[0116] The memory 201 is used to store computer programs that can run on the processor 202.

[0117] The memory 201 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0118] If the memory 201, processor 202, and communication interface 203 are implemented independently, then the communication interface 203, memory 201, and processor 202 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0119] Optionally, in a specific implementation, if the memory 201, processor 202, and communication interface 203 are integrated on a single chip, then the memory 201, processor 202, and communication interface 203 can communicate with each other through an internal interface.

[0120] The processor 202 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention.

[0121] In this embodiment, the modules work collaboratively according to a vertical federated learning framework. First, during system initialization, the client analyzes the contribution of local original features through the feature importance evaluation module, generating a feature importance vector as input, which is then fed into the privacy budget generator module. This generator outputs a feature-level privacy budget vector, which is then processed by the embedding perturbation and proposal budget generation module to generate an embedding-level privacy budget vector. This vector guides the embedding vector to perform a Laplacian noise addition operation, thereby generating a privacy-preserving noisy embedding vector. Simultaneously, the embedding-level privacy budget vector is mapped to discrete levels, further forming a proposal privacy budget vector, which is then sent to the server. Upon receiving the proposal privacy budget vector, the gradient perturbation module on the server side allocates gradient perturbation intensity to each embedding dimension based on its level information, achieving refined noise addition to the gradient and updating the server-side model parameters. The client update module receives the noisy gradient from the server and performs local embedding model updates, privacy budget generator updates, and feature importance updates. During training, the end-to-end joint optimization module runs through both the client and server sides, and performs unified modeling and solving of the task loss function, privacy budget control target and perturbation strategy to achieve a dynamic balance between accuracy and privacy. It also continuously optimizes generator parameters and embedding representation quality, ultimately achieving the goal of synergistic improvement in performance and privacy.

[0122] In summary, this embodiment innovatively implements a client-server collaborative adaptive privacy control strategy by constructing a closed-loop mechanism of "feature importance assessment—differentiated privacy budget generation—embedding and uploading with noise—budget suggestion feedback—server-side gradient noise addition—local update." Compared with traditional unified privacy budget mechanisms, this embodiment has three significant advantages: First, through feature-level budget modeling and learning, the privacy protection strategy becomes more granular and adapted to model importance, improving the coordination between privacy protection and model performance; second, through feature selection and embedding with noise uploading, budget discretization feedback, etc., the communication and noise addition costs in federated learning are effectively reduced; finally, the constructed feedback closed loop supports online optimization and adaptive adjustment of the privacy strategy, breaking through the static limitations of privacy-accuracy trade-offs, and has stronger practicality and promotional value.

[0123] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0124] Those skilled in the art will understand that all or part of the steps of the methods implementing the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0125] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A longitudinal federated learning method based on feature importance, characterized in that, Includes the following steps: The client evaluates the original local features and generates a feature importance vector; Using the feature importance vector as input, the client constructs a lightweight neural network as a privacy budget generator, and outputs a feature-level privacy budget vector corresponding to the feature importance vector; The client generates an embedding-level privacy budget vector using a pseudo-model copy that has the same structure and shares parameters as the local embedded model. Based on the embedding-level privacy budget vector, the local embedded vector is noise-added to generate a privacy-preserving noisy embedding vector. The client generates a suggested privacy budget vector by discretizing the embedded-level privacy budget vector; The server receives a noisy embedding vector and a proposed privacy budget vector, and calculates the loss function through forward propagation. The server performs gradient calculations and updates the parameters; The server adds noise to the gradient based on the proposed privacy budget vector, generating a noisy gradient. The client receives the noisy gradient and performs local embedding model updates, privacy budget generator updates, and feature importance updates.

2. The longitudinal federated learning method based on feature importance according to claim 1, characterized in that, The local original characteristics were evaluated using Gini impurity.

3. The longitudinal federated learning method based on feature importance according to claim 1, characterized in that, Lightweight neural networks are specifically trained on a local loss function that includes a gradient guidance term, a total budget constraint term, and an allocation extremum penalty term.

4. The longitudinal federated learning method based on feature importance according to claim 1, characterized in that, The original local features are filtered bitwise and then fed into the local embedding model to generate embedding vectors. The local embedding vectors are then noise-added by applying Laplacian noise perturbation.

5. The longitudinal federated learning method based on feature importance according to claim 1, characterized in that, The specific method of discretization is as follows: Where, γ m To suggest privacy budget vectors, T1-T4 represent four levels, and τ1-τ3 represent the embedding-level privacy budget vectors corresponding to the quantiles of 25%, 50%, and 75% for the distribution of all embedding-level privacy budget vectors. For the embedded-level privacy budget vector, d ' m For the embedded dimension.

6. The longitudinal federated learning method based on feature importance according to claim 1, characterized in that, The process of generating noisy gradients is as follows: The server divides the total privacy budget that can be allocated to each client equally according to the number of clients. Combining the four levels of the proposed privacy budget vector, the server calculates the final budget for the gradient corresponding to each client. Noise is then added to the gradient of each dimension to generate noisy gradients.

7. A longitudinal federated learning device based on feature importance, characterized in that, include: The feature importance evaluation module is used by the client to evaluate the original local features and generate feature importance vectors; The privacy budget generation module is used to take the feature importance vector as input, and the client builds a lightweight neural network as a privacy budget generator to output the feature-level privacy budget vector corresponding to the feature importance vector. The embedding perturbation and suggestion budget generation module is used by the client to generate an embedding-level privacy budget vector using a pseudo-model copy with the same structure and shared parameters as the local embedding model. Based on the embedding-level privacy budget vector, the local embedding vector is noise-added to generate a privacy-preserving noisy embedding vector. The client generates a suggested privacy budget vector by discretizing the embedded-level privacy budget vector. The gradient perturbation module is used by the server to receive the noisy embedding vector and the proposed privacy budget vector, and to calculate the loss function during forward propagation. The server performs gradient calculations and updates parameters. The server adds noise to the gradient based on the proposed privacy budget vector to generate a noisy gradient; The client update module is used by the client to receive the noisy gradient and perform local embedding model updates, privacy budget generator updates, and feature importance updates. The end-to-end joint optimization module runs through both the client and server sides and is used to embed the above processes into the end-to-end training framework of vertical federated learning.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the feature importance-based longitudinal federated learning method according to any one of claims 1-6.

Images