Vertical federal learning feature selection method based on context awareness and application

Abstract

The invention discloses a vertical federal learning feature selection method based on context awareness, and belongs to the technical field of artificial intelligence and data privacy protection. According to the method, firstly, an unsupervised sparse network is utilized at a client to initialize the importance of local features so as to accelerate convergence and reduce calculation complexity; and then, obtaining the embedded representation of each client in a pre-training stage, and screening the embedded representation by combining context features through a server side, thereby indirectly identifying key features. In the feature selection stage, the client side performs local feature screening according to the significant embedded index issued by the server, and the influence of irrelevant features on calculation and communication is further reduced. According to the invention, an attention mechanism is introduced to dynamically evaluate contributions of different participants, so that fair weight distribution is realized. According to the method, through staged joint optimization, the communication and calculation cost in the federation training process is effectively reduced, and meanwhile, the prediction precision and interpretability of the model are improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and data privacy protection technology, specifically relating to feature selection methods in federated learning, and more particularly to a context-aware vertical federated learning feature selection method. Background Technology

[0002] Federated Learning (FL), as an emerging distributed machine learning framework, shows great promise in protecting data privacy, breaking down data silos, and promoting collaborative modeling among multiple parties. Unlike traditional centralized machine learning, which requires data to be aggregated on a single server, federated learning allows participants to retain data locally and train models only by exchanging intermediate results. Therefore, it has gained widespread attention in data-sensitive scenarios such as finance, healthcare, and smart cities.

[0003] Among the different partitioning models of federated learning, Vertical Federated Learning (VFL) has unique advantages. VFL is suitable for scenarios where the sample space overlaps but the feature space is complementary, that is, different participants have the same user group, but each possesses features in different dimensions. For example, in financial risk control, banks, insurance companies, and e-commerce platforms may all have partial data about the same user. If VFL can be used for collaborative modeling, the accuracy of predictions will be significantly improved.

[0004] While Virtual Functions (VFLs) hold promise for improving prediction performance and protecting privacy, they also have inherent limitations, particularly in the integration of feature selection and model training. Existing research remains insufficient in addressing communication and computational challenges, leading to the following challenges for VFLs in real-world applications: 1) Feature redundancy and unbalanced computational load. Since each client holds a large number of feature dimensions, these may include redundant or task-irrelevant features. These features consume local computing power during training and increase the communication burden between the server and client, thus reducing overall efficiency. This redundancy overhead is particularly severe for resource-constrained clients.

[0005] 2) Ignoring contextual information leads to insufficient model generalization. Existing VFL feature selection methods typically assume that feature importance remains consistent across all samples. However, in real-world scenarios, feature importance often depends on contextual information. For example, in medical prediction, the feature contributions of different age groups and genders may differ significantly. Ignoring contextual factors can easily lead to a decline in the model's predictive performance in subgroups, while also weakening the model's interpretability.

[0006] 3) Asymmetric feature distribution between server and client. In the VFL architecture, contextual features are typically stored centrally on the server side, while interpretive features are distributed across multiple clients. Since clients cannot directly access contextual features, this complicates context-guided feature selection during training and may introduce additional communication overhead.

[0007] 4) Difficulty in measuring and allocating contributions from participants. The quality of feature dimensions provided by different clients varies, with some clients contributing significantly to model training while others contribute less. The lack of a reasonable contribution evaluation mechanism not only easily leads to a waste of resources during training but also affects the enthusiasm and fairness of multi-party participation.

[0008] It is worth noting that VFL can only truly realize the advantages of collaborative modeling when communication is efficient, feature selection is reasonable, and the contributions of participants can be fairly evaluated.

[0009] If all the above factors are taken into account, the feature selection problem in VFL will encounter the following difficulties: a. Due to the widespread existence of redundant and irrelevant features, it remains difficult to solve the problem of reducing communication and computational overhead while ensuring model accuracy through a unified optimization framework.

[0010] b. The dynamic impact of contextual information on feature importance is difficult to characterize, especially when contextual features and explanatory features belong to different stakeholders. How to introduce contextual guidance in a distributed environment remains a challenge.

[0011] c. In multi-party collaboration scenarios, contribution evaluation involves fairness and incentive mechanisms. How to accurately quantify the contributions of each party without increasing additional communication overhead is a challenging problem with multi-dimensional complexity. Summary of the Invention

[0012] Purpose of the invention: To address the problems in existing technologies of vertical federated learning, such as excessively high communication and computation costs due to redundant features, insufficient model generalization and interpretability due to neglect of context information, and difficulty in reasonably evaluating the contributions of participants, this invention provides a context-aware feature selection method and application for vertical federated learning.

[0013] Technical Solution: A context-aware vertical federated learning feature selection method. This method uses context-guided feature selection, is geared towards multi-participant collaboration, and aims to minimize communication and computation costs while improving model prediction accuracy and interpretability. Based on a context-aware mechanism, features from each client are embedded on the server side for context-conditional selection and redistribution, and local feature selection is performed on the client side, realizing a context-driven joint optimization process. The method includes the following steps: (1) Without sharing sample alignment information and labels with the server, the client constructs a sparse autoencoder to perform unsupervised analysis on local features and establishes a feature importance measurement mechanism to provide prior reference for the subsequent feature selection stage. (2) The pre-training stage includes generating the global model and embeddings, specifically including: Each client inputs its own interpretive features, aligned with the server, into its local model to generate an initial local embedding vector, and then sends the embedding vector to the server. The server aggregates the embedding vectors sent by all clients, makes predictions using its top-level network model, and calculates the loss function and the gradient of the loss with respect to the embedding vectors of each client by combining the real labels. The server returns the calculated gradient vectors to the corresponding clients. After receiving the gradients, each client updates the parameters of its local model through the backpropagation algorithm. Repeat the above process until the global model converges to obtain the pre-trained embedding vectors; (3) The server performs embedding selection and contribution evaluation based on context features, selects the most important embedding components for the current task from the pre-trained embeddings of the client, and calculates the marginal contribution of each client through the attention mechanism; Given a fixed pre-trained embedding, the server generates a set of continuously differentiable gated variables for each sample, conditioned on contextual features, and enforces a cardinality constraint that at most only a few gated variables are preserved. One embedded dimension; (4) The client prunes irrelevant features based on the set of important embedding indices sent by the server, completes the feature-level redistribution, selects the optimal local feature subset, and performs local feature simplification operations, including constructing a new local optimization objective. The local optimization objective is to minimize the squared difference between the new embedding vector generated by the selected local features and the pre-trained embedding vector on the component corresponding to the index, and introduces a constraint function to ensure that the number of features finally selected is close to the preset upper limit.

[0014] Further, step (1) includes constructing an unsupervised local sparse network for training on local data, wherein the local data refers to local data that does not overlap with the server-side sample IDs, in order to obtain an initial importance score for all its explanatory features; the local sparse network includes a sparse autoencoder with a dynamic topology, which performs the following training process: (11) Weight optimization is performed with the goal of minimizing the mean square error between the input samples and the network reconstruction output, including updating the weights using stochastic gradient descent; (12) While optimizing the weights, in order to ensure the contrast between structural sparsity and importance, the client maintains a fixed sparsity and periodically performs drop-growth adjustment of the connection topology. Specifically, based on the connection weight and the importance of the connected neurons, some unimportant connections are dropped, and an equal number of new connections are regenerated around the most important neurons, so as to dynamically allocate sparse connection resources to the features with the richest information. (13) After training, the initial importance score of each explanatory feature is calculated based on the cumulative gradient and connection weight of each neuron in the network input layer.

[0015] To avoid the randomness of instantaneous gradients, the importance measurement mechanism adopts an exponentially sliding cumulative measurement. No. In the next iteration, the input neuron The importance update is represented as: , in This indicates the balance between instantaneous sensitivity and structural coupling. Indicates the client m The number of hidden layer neurons in the constructed sparse autoencoder, denoted by weight. , Indicates the reconstruction loss. Indicates input, Indicates the connection to the input layer. u The first neuron and the hidden layer v The weights of each neuron, and the importance of the output neuron, are determined by... and Combined to obtain; Connection weights Importance is defined as: , in For connecting associated input or output neurons, the importance of a fine connection is measured by the importance of the neuron it is attached to and its own amplitude.

[0016] Furthermore, the embedded selection process in step (3) includes: (31) The server receives and fixes all the pre-trained embedding vectors sent by the clients as the input for this stage; (32) Introduce a random gate mechanism to continuously relax the selection vector to achieve binary selection of embedded components and make the process differentiable; (33) The mean parameter of the random gate is dynamically generated by an embedding selector network whose input is the context features of the server; in this way, the selection probability of the embedding component depends on the specific context information of each sample. (34) Introduce a constraint function in the optimization objective on the server side. This function applies both sparsity constraints and binary constraints to the selection vector to ensure that the number of the finally selected embedded components is close to the preset upper limit and that their probability values ​​are close to 0 or 1. (35) The server trains the embedding selector network by optimizing a joint objective function that includes the task prediction loss and the above constraint function; (36) After training is completed, the server determines the most important embedded components for each client and sends its index set to the corresponding client.

[0017] Furthermore, the embedded component in step (3) includes a scalar attention module configured to evaluate the contribution of each client in the joint modeling.

[0018] Furthermore, the specific modeling and calculation process for step (3) is as follows: Server-side embedded network selection Given context input, output the gate mean for each embedded component. To maintain differentiability during training, a Gaussian perturbation is introduced and clipped to an interval, represented as: , The Gaussian perturbation of the gate mean for each embedded component, the perturbation factor. The value decreases with each iteration to gradually approach the hard gate; For the sample Client The selected embedded network is represented as ,in For element-wise multiplication, it indicates component retention or attenuation, and Z represents the context feature; Then Context features corresponding to sample i The signal is fed into the prediction network to obtain the output. To limit the number of selections per sample, a comprehensive constraint is used, with the following conditions: , The first term is used to encourage fewer non-zero values, the second term... The third term is used to encourage components to move away from the middle value and tend towards 0 or 1. Pull towards the target base In practice The cardinality constraint can be increased by setting it to 1 or by increasing it as training progresses. The objective function for the embedding selection phase is: , in It is for the samplei and client m , by context features Z Guided embedding selection vector. Represents all learnable parameters in the server's VFL network. The parameters representing the embedded selector network, It is a loss function. This represents a dynamic adjustment factor, which is small in the early stages and gradually increases thereafter, approaching the sparsity and cardinality targets, using a linear or piecewise weighted schedule. , T It is a hyperparameter used to control the constraint function. The rate at which losses are incorporated; To measure the marginal contribution of multiple parties to the current sample, scalar attention is introduced:

[0019] in It is a client m Processing samples i Attention weights at that time are used to quantify the client's embedding. The significance or contribution of this sample to the prediction. It is a client m For the sample i The original attention score, For trainable parameters, This is a nonlinear perturbation factor; if in the context... Below, the retained sub-embedded Able to explain ,but Larger The corresponding increase, based on all samples Calculate the average to obtain the client's... global contribution .

[0020] Furthermore, step (4) specifically outlines the local feature selection process as follows: (41) Each client receives the index of important embedded components from the server; (42) The client constructs a new local optimization objective that minimizes the squared difference between the new embedding vector generated from the filtered local features and the pre-trained embedding vector on the component corresponding to the index; (43) The feature selection process is implemented through a random gate mechanism controlled by a local feature selector network to make it differentiable; (44) Introduce a constraint function into the local optimization objective to ensure that the number of features selected in the end is close to the preset upper limit; (45) The client trains its feature selector network by optimizing the local objective function, and finally determines whether each input feature is retained or discarded, forming the final feature subset.

[0021] Furthermore, in step (4), the specific calculation process for the client to prune irrelevant features based on the important embedding index set sent by the server includes: Client selects network based on features For the sample Output gate mean Using Gaussian perturbation and clipping, the calculation expression is as follows: To obtain the selection variable The selected feature is represented as ; To anchor feature selection to the important embeddings determined by the server, the alignment error is defined as: , The above formula indicates that only in the key embedding dimension The above measures the deviation between the new embedding and the pre-trained embedding; The optimization goals for the client are: , in The basic objective in the comprehensive constraints is... Replace with feature upper limit To control the number of features that can be retained per sample, By employing piecewise or linear weighting, the optimization first satisfies alignment, and then emphasizes sparsity and binary values.

[0022] On the other hand, the present invention also provides a context-aware vertical federated learning collaboration system. The system realizes sample feature selection among clients based on the above method. Based on the unified awareness and management of context on the server side, the system aims to achieve a balance between minimizing the expected communication and computing costs of each client and maintaining prediction accuracy. First, the multi-source embedding representations of the samples are dynamically centralized to the associated server for context-guided embedding selection. Then, each client decides whether to participate in and perform local feature reallocation and pruning based on the embedding index set issued by the server, thereby realizing the reconfiguration and optimization of load and information channels under cross-party collaboration.

[0023] The implementation of the above method enables the construction of a context-aware vertical federated learning collaboration system. This system achieves sample feature selection among clients based on the method described above. Based on the unified awareness and management of context on the server side, the system aims to achieve a balance between minimizing the expected communication and computation costs of each client and maintaining prediction accuracy. First, the multi-source embedding representations of the samples are dynamically centralized to the associated server for context-guided embedding selection. Then, each client decides whether to participate in and execute local feature reallocation and pruning based on the embedding index set issued by the server, thereby realizing the reconfiguration and optimization of load and information channels under cross-party collaboration.

[0024] On the other hand, applications of the present invention include constructing a context-aware vertical federated learning recommendation system, and using a context-aware vertical federated learning feature selection method to achieve federated learning and training of medical image data among clients.

[0025] Beneficial effects: Compared with the prior art, the substantial progress and significant effects of the present invention are as follows: 1) Significantly improves model performance and interpretability. Existing VFL feature selection methods typically assume that feature importance is fixed across all samples, which leads to decreased model performance and reduced interpretability in real-world scenarios. This invention innovatively introduces the concept of contextual features, using contextual information held by the server to dynamically guide the feature selection process on the client side, enabling the model to adapt to changes in feature importance across different subgroups.

[0026] 2) Effectively reduces communication and computational overhead. This invention designs a communication-efficient multi-stage framework. Through a "pre-training-selection" separation mode, only a one-time transmission of important embedding indices is needed after pre-training to guide the client to complete local feature selection, avoiding the huge communication cost of repeatedly exchanging embeddings and gradients during the selection process in traditional methods. Furthermore, the method explicitly introduces constraints on the number of selected embeddings and features, thereby achieving quantitative control over communication and computational loads, reducing the burden on resource-constrained clients, and improving the training efficiency of the entire federated learning model.

[0027] 3) Accelerate model convergence and optimize final accuracy. This invention designs a novel feature importance initialization phase, allowing each client to train an unsupervised local sparse network using its locally held, non-overlapping sample data with the server. This phase can efficiently identify the preliminary importance of key features within several training rounds before the formal federated learning task begins. This pre-exploration based on local data provides a superior starting point for subsequent feature selection and model training, avoiding a completely random search, thus significantly accelerating the convergence speed of the feature selection process and ultimately contributing to improved global model accuracy. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the overall framework of the context-aware vertical federated feature selection method described in this invention; Figure 2 The figure shown is an accuracy result of the method described in the embodiment on a classification task dataset; Figure 3 The figure shown is an R² score result of the method described in the embodiment on the regression task dataset; Figure 4 The figure shown is a graph illustrating the communication overhead of the method described in the embodiment on different datasets. Detailed Implementation

[0029] To illustrate the technical solutions disclosed in this invention in detail, further explanation will be provided below with reference to the accompanying drawings and specific embodiments.

[0030] To address the high communication and computation costs in vertical federated learning scenarios, and the failure of existing participants (servers and clients) to effectively leverage the synergistic advantages of "server-side contextual information + client-side interpretive features," which leads to difficulties in eliminating feature redundancy, the inability to reflect contextual differences in model selection, and the inability to optimize the overall system utility, this invention provides a context-aware feature selection method for multi-participant collaboration in vertical federated learning. This method establishes a vertical federated learning collaboration platform based on the server-side unified awareness and management of context. The goal is to achieve a balance between minimizing the expected communication and computational costs of each participant and maintaining prediction accuracy. First, the multi-source embedding representations of samples are dynamically centralized to the associated server for context-guided embedding selection. Then, each client, based on the significant embedding index issued by the server, decides whether to participate and performs local feature reallocation and pruning, thereby reconfiguring and optimizing the load and information channels in cross-party collaboration.

[0031] The method described in this invention is based on a context-driven "embedding-feature" joint selection mechanism to achieve vertical federated feature management for multi-participant collaboration. Unlike existing technologies and traditional client-side independent computation, this invention also needs to consider the following issues: (1) The dynamic modulation relationship of context on feature importance; (2) Complementarity and redundancy among multiple embeddings; (3) Dual cost constraints of communication and local computing and fair contribution.

[0032] To achieve collaborative equilibrium among all participants at the lowest expected cost, this invention constructs a joint optimization scheme, comprehensively determining: the upper limit of cardinality for server-side embedding selection, the upper limit of cardinality for client-side feature selection, cross-party contribution allocation weights (for fair aggregation and incentives), and the rhythm and frequency of encoding and aggregation during training (for controlling effective computation speed and communication rounds). To this end, this invention establishes a novel context-guided constrained selection framework that organically combines continuously differentiable sparse / cardinality constraint optimization, a core-based contribution sharing approach (implemented through attention), and mapping rules from embedding indices to local features. This ensures interpretable, controllable, and efficient collaborative selection and redistribution without leaking the original data.

[0033] Combination Figure 1 The implementation process of the method of the present invention will be described in detail below.

[0034] Step 1: Build a context-aware vertical federated learning collaborative system and define a global optimization objective function. This step mathematically processes the feature collaborative modeling problem involving multiple participants, clarifying the joint optimization objective between the server and client sides regarding communication overhead, computational cost, and prediction accuracy.

[0035] Without sharing sample alignment information and labels with the server, each client first obtains an initial estimate of feature importance based solely on local data statistical patterns, serving as a good starting point for subsequent limited selection. This approach has three advantages: 1) complete localization and privacy protection; 2) reducing the search space and accelerating subsequent convergence through early sparsity reduction; and 3) prioritizing the filtering of obviously redundant dimensions for resource-constrained clients, reducing local computation time.

[0036] Client Construct a sparse autoencoder with the input dimension being the number of local features. Number of hidden layer neurons Record the weights. Forward mapping is defined as ,in It uses non-linear activations (such as ReLU / GELU). This structure reconstructs the input with a low-dimensional bottleneck, which is beneficial for characterizing the importance of features with reconstructibility.

[0037] Using mean squared error as the reconstruction loss: , Update weights using stochastic gradient descent: , in This is the learning rate. In practice, it can be combined with momentum or Adam optimization to achieve stable convergence.

[0038] To ensure a contrast between structural sparsity and importance, the client maintains a fixed sparsity. , And execute a two-step process of "discarding-growing" at a fixed period: first delete the proportion that is... The least important connections are then replaced with new connections of the same weight between the most important neuron pairs, with the weights of these new connections initialized to 0. This process moves connection resources to information-intensive parts without changing the overall sparsity, improving reconfigurability and gradient utilization efficiency.

[0039] To avoid the randomness of instantaneous gradients, importance is measured using an exponentially moving cumulative measure. In the next iteration, the input neuron The importance has been updated to , in Balancing instantaneous sensitivity (gradient) with structural coupling (weights). The importance of output neurons can be similarly determined by… and Combined to obtain. Connect. Importance is defined as: , in For connecting related input (or output) neurons, the importance of fine connections is emphasized because of the importance of the neurons they are attached to and their own amplitude.

[0040] After convergence, the importance of the input layer neurons is taken as the importance score of each local feature and ranked. This ranking does not depend on labels and alignment, thus ensuring privacy; at the same time, as a prior within the client for subsequent stages, it can significantly reduce the scope of limited choices and the number of training epochs.

[0041] It is important to note the initial sparsity. ; percentage of deleted lines Early stopping is used to verify that the reconstruction error or importance ranking is stable; to prevent jitter, [the following can be done]: Perform a mild L2 normalization. This stage is completely localized, with no cross-party communication, zero additional privacy risks, and no communication costs.

[0042] Step 2: The client performs initial training and establishes a feature importance measurement mechanism.

[0043] This step involves constructing a sparse autoencoder to perform unsupervised analysis on local features, obtaining a preliminary importance ranking, and providing prior reference for the subsequent feature selection stage.

[0044] In this stage, the server and client collaborate to build a vertical federated learning structure that is completely consistent with the subsequent training process, but no selection constraints are imposed on the embeddings or features at this stage. The main goal is to enable the client encoder to learn stable and representative embedding representations through sufficient model training, while the server's prediction network develops preliminary discriminative capabilities, thus laying a solid foundation for subsequent context-aware embedding selection. Unlike traditional centralized models, the pre-training stage needs to fully consider the distributed nature of VFL, meaning that the client and server need to frequently exchange intermediate information. The specific process can be divided into the following steps: (1) Client-side encoding and embedding generation: After receiving the index of the training sample, each client extracts the corresponding explanatory features from the local dataset. These features are achieved through the local encoder function. Convert to embedded representation ,here This represents the parameter set of the client encoder. The significance of this process is to map the high-dimensional and potentially redundant raw features to a low-dimensional, structured representation space, so that it can be fused with embeddings from other clients on the server side.

[0045] (2) Server-side fusion and prediction. After the client calculates the embedding locally, it will... Uploaded to the server. After receiving all the embeddings from the clients, the server concatenates or weights them and combines them with contextual features. Feed into the prediction network : ,in This represents the parameter set of the server's prediction network. The network can be a fully connected neural network or a more complex deep model. Its function is to synthesize complementary feature information from different clients and output the final prediction result. .

[0046] (3) Loss function calculation. The server calculates the loss function based on the predicted output and the true label. The difference between them is used to calculate the task loss function. ,in . These could be cross-entropy loss (for classification tasks), mean squared error loss (for regression tasks), etc. The definition of this loss function ensures that the server can minimize prediction error on a global scale, thereby driving the learning process of the entire system.

[0047] (4) Gradient Calculation and Parameter Update. To complete end-to-end training, the server needs to calculate the gradient of the loss function with respect to the embeddings of each client and return it to the corresponding client. Specifically, the server first calculates... And send the gradient information to the client. The client then performs backpropagation to update its local encoder parameters: ,in This is the learning rate. Similarly, the server uses the gradient of the global loss function to update the prediction network parameters: .

[0048] (5) Convergence and Consolidation. After several rounds of iterative training, the parameters of the client and server gradually converge. At this point, the system obtains a stable set of parameters. And save fixed pre-trained embeddings These embeddings are used as baseline representations in subsequent stages for context-aware embedding selection. It is important to emphasize that the embeddings generated in the pre-training stage are based on all features, thus possessing strong expressive power and providing a reliable reference for the subsequent sparsification process.

[0049] It's important to note that no selection constraints are imposed during the pre-training phase; therefore, its primary function is to ensure the model has good representational capabilities in the embedding space. In engineering implementation, different iteration counts and learning rates can be set according to task requirements to balance convergence speed and model performance. If there are a large number of clients in the system, parallel communication mechanisms can be used to improve training efficiency.

[0050] Step 3: The server performs embedding selection and contribution evaluation based on contextual features.

[0051] This step utilizes a context-guided embedding selection network to filter key information globally and calculates the marginal contribution of each client through an attention mechanism.

[0052] In fixed pre-trained embeddings Under the premise that the server uses context features As a condition, generate a set of continuously differentiable gated variables for each sample. And by using cardinality constraints, it is enforced that at most only one number can be preserved. This involves embedding multiple dimensions to achieve maximum predictive utility with minimal information retention; simultaneously, it calculates sample-level attention and aggregates it into the client's contribution.

[0053] Server-side embedded selection network Given context input, output the gate mean for each embedded component. To maintain differentiability for training, a Gaussian perturbation is introduced and clipped to an interval: Continuous relaxation can approximate a binary representation while also allowing end-to-end backpropagation. In implementation... It can be iteratively reduced (simulated annealing) to gradually approach the hard gate.

[0054] For the sample Client Selected to be embedded as ,in" "" indicates element-wise multiplication, representing component retention or attenuation. and Feeding into the prediction network yields .

[0055] To limit the number of selections per sample, a comprehensive constraint is used. , The first term encourages fewer non-zero values ​​(L1 sparsity), the second term... The third term encourages components to move away from the midpoint towards 0 / 1 (close to binary), and... Pull towards the target base In practice The value can be set to 1 or increased as training progresses, thereby increasing the rigidity of the cardinality constraint.

[0056] The objective function for the embedding selection phase is: , in In the early stages, the number of tasks is relatively small (prioritizing task convergence), and then the number is gradually increased (approaching the sparsity and cardinality targets). A linear or piecewise weighted schedule can be used. .

[0057] To measure the marginal contribution of multiple parties to the current sample, scalar attention is introduced:

[0058] in For trainable parameters, It is non-linear. Intuitively: if in context Below, the retained sub-embedded Able to explain ,but Larger The corresponding increase. (Based on all samples) Calculate the average to obtain the client's data. global contribution .

[0059] After training converges, the most frequently used option for each side or the option with the threshold value is retained. The set of embedded dimension indexes close to 1 This data is sent to the client. This collection is the most effective information channel under the given context, and it will be used to guide local feature trimming later.

[0060] Typical settings The value gradually decreased from 0.5 to 0.1. Grid search. If the context dimension is high, it can be used... A small projection / self-attention loop is used to reduce dimensionality and improve selection stability. To avoid underfitting due to excessive sparseness, this can be done in the first 30% of rounds. Set an upper limit (e.g., 0.3), and then linearly increase it to 1 for the next 70%.

[0061] Step 4: The client performs feature selection and local model pruning based on the significant embedding index.

[0062] This step prunes irrelevant features based on the embedded index set issued by the server, completing the reallocation at the feature level, thereby achieving optimal utilization of communication and computing resources.

[0063] The server has provided a set of embedded components that are important in the context. Based on this, the client learns a differentiable feature gate vector in the original feature space. and through with The selection is driven by embedding alignment: the retained feature subset, after being mapped by the local encoder, should reproduce the pre-training representation on these key embedding dimensions as much as possible.

[0064] Client selects network based on features For the sample Output gate mean Similarly, Gaussian perturbation and clipping are used. , to obtain the selection variable The selected feature is .

[0065] To anchor feature selection to the important embeddings determined by the server, an alignment error is defined: , That is, only in the key embedded dimension The deviation between the new embedding and the pre-trained embedding is measured. This avoids overfitting to irrelevant dimensions and maximizes the local reproduction of important channels in the context.

[0066] The optimization goals for the client are: , in Similar to Phase Three, but with the base target changed from Replace with feature upper limit This controls the number of features that can be retained per sample. Segmented or linear weighting can also be used to optimize the alignment first, and then emphasize sparsity and binary values.

[0067] After training stabilizes, components close to 0 are thresholded to 0, and components close to 1 are set to 1, resulting in a binary feature subset. Subsequently, the input channels with the values ​​set to 0 are physically removed, and the local encoder is applied to this pruned input dimension. Fine-tuning with a few epochs is performed to restore accuracy. Thereafter, both forward and backward passes on the client side are performed on a simplified feature set, significantly reducing computation time and memory usage.

[0068] It is important to note the upper limit of features. Can be set according to device / latency budget (e.g.) ); and The scheduling can be symmetrical with stage three. If local features are highly correlated, it is recommended to... A light correlation penalty or Group-Lasso is added beforehand to encourage group-based pruning, facilitating engineering implementation. Alignment errors only occur when... The computation is small, the gradient direction is clear, and it can accelerate local convergence.

[0069] We consider evaluating the feature selection method of our invention on seven datasets: classification tasks including syn_cls (26 explanatory features, 10,000 samples, 4 clients; context: “generation condition”), heart (11 features, 297 samples, 2 clients; context: “gender”, “age”), and MNIST (1,600 features, 480,000 samples, 4 clients; context: “rotation angle”); and regression tasks including syn_reg (26 features, 10,000 samples, 4 clients; context: “generation condition”), news (53 features, 39,644 samples, 5 clients; context: “category of the news”), apparatus (27 features, 19,762 samples, 4 clients; context: “timestamp”), and house (21 features, 15,940 samples, 3 clients; context: “longitude”, “latitude”). The experimental setup was as follows: explanatory features were distributed across all clients, while contextual features were held by the server; 75% of the total samples were shared between the client and server, and the remaining 25% were only available on the client; the training / test ratio was 8:2 in the shared samples. Each image in MNIST was rotated 45°–315°, and the rotation angle served as context; two artificial datasets each contained 10,000 samples and 30-dimensional features.

[0070] Figure 2The classification accuracy of the described feature selection method on three classification datasets is shown. Figure 3 The R² scores on four regression datasets are presented. It can be seen that it exhibits stable and excellent performance across all tasks and datasets. In classification tasks, it achieves high accuracy on the heart disease prediction and rotating MNIST datasets; in regression tasks, it maintains high R² values ​​on both real and synthetic data. The overall trend indicates that this method can fully utilize contextual information to guide the feature selection process, thereby achieving reliable predictive performance in different scenarios. This demonstrates that the method has good generalization ability and adaptability in multi-task and multi-data distribution settings.

[0071] like Figure 4 As shown, the communication overhead results of the method of this invention on seven different datasets are presented in logarithmic form. It can be seen that the communication overhead increases with the increase of dataset size and feature dimension. The communication overhead is significantly higher on large-scale datasets (such as MNIST) than on other datasets, while it remains at a low level on small-to-medium-sized datasets (such as heart, syn_cls, etc.). This result demonstrates that the proposed method effectively reduces unnecessary data transmission while maintaining model performance, thereby improving the overall communication efficiency and scalability of the system.

Claims

1. A context-aware vertical federated learning feature selection method, characterized in that, The method is a context-guided feature selection approach, oriented towards multi-participant collaboration, with the goal of minimizing communication and computation costs, improving model prediction accuracy and interpretability. Based on a context-aware mechanism, the features of each client are embedded in the server for context-conditional selection and redistribution, and local feature selection is performed on the client to achieve a context-driven joint optimization process. The method implementation includes the following steps: (1) Without sharing sample alignment information and labels with the server, the client constructs a sparse autoencoder to perform unsupervised analysis on local features and establishes a feature importance measurement mechanism to provide prior reference for the subsequent feature selection stage. (2) The pre-training stage includes generating the global model and embeddings, specifically including: Each client inputs its own interpretive features, aligned with the server, into its local model to generate an initial local embedding vector, and then sends the embedding vector to the server. The server aggregates the embedding vectors sent by all clients, performs predictions using the top-level network model, and calculates the loss function and its gradient with respect to each client's embedding vector by combining the real labels. The server returns the calculated gradient vectors to the corresponding clients. After receiving the gradients, each client updates the parameters of its local model through the backpropagation algorithm. Repeat the above process until the global model converges to obtain the pre-trained embedding vectors; (3) The server performs embedding selection and contribution evaluation based on context features, selects the most important embedding components for the current task from the pre-trained embeddings of the client, and calculates the marginal contribution of each client through the attention mechanism; Given a fixed pre-trained embedding, the server generates a set of continuously differentiable gated variables for each sample, conditioned on contextual features, and enforces a cardinality constraint that at most only a few gated variables are preserved. One embedded dimension; (4) The client prunes irrelevant features based on the set of important embedding indices sent by the server, completes the feature-level redistribution, selects the optimal local feature subset, and performs local feature simplification operations, including constructing a new local optimization objective. The local optimization objective is to minimize the squared difference between the new embedding vector generated by the selected local features and the pre-trained embedding vector on the component corresponding to the index, and introduces a constraint function to ensure that the number of features finally selected is close to the preset upper limit.

2. The vertical federated learning feature selection method according to claim 1, characterized in that, Step (1) includes constructing an unsupervised local sparse network for training on local data, where local data refers to local data that does not overlap with the server-side sample IDs, to obtain an initial importance score for all its explanatory features; the local sparse network includes a sparse autoencoder with a dynamic topology, which performs the following training process: (11) Weight optimization is performed with the goal of minimizing the mean square error between the input samples and the network reconstruction output, including updating the weights using stochastic gradient descent; (12) While optimizing the weights, in order to ensure the contrast between structural sparsity and importance, the client maintains a fixed sparsity and periodically performs drop-growth adjustment of the connection topology. Specifically, based on the connection weight and the importance of the connected neurons, some unimportant connections are dropped, and an equal number of new connections are regenerated around the most important neurons, so as to dynamically allocate sparse connection resources to the features with the richest information. (13) After training, the initial importance score of each explanatory feature is calculated based on the cumulative gradient and connection weight of each neuron in the network input layer.

3. The vertical federated learning feature selection method according to claim 1 or 2, characterized in that, To avoid the randomness of instantaneous gradients, the importance measurement mechanism adopts an exponentially sliding cumulative measurement. No. In the next iteration, the input neuron The importance update is represented as: ， in This indicates the balance between instantaneous sensitivity and structural coupling. Indicates the client m The number of hidden layer neurons in the constructed sparse autoencoder, denoted by weight. , Indicates the reconstruction loss. Indicates input, Indicates the connection to the input layer. u The first neuron and the hidden layer v The weights of each neuron, and the importance of the output neuron, are determined by... and Combined to obtain; Connection weights Importance is defined as: ， in For connecting associated input or output neurons, the importance of a fine connection is measured by the importance of the neuron it is attached to and its own amplitude.

4. The vertical federated learning feature selection method according to claim 1, characterized in that, The embedding selection process in step (3) includes: (31) The server receives and fixes all the pre-trained embedding vectors sent by the clients as the input for this stage; (32) Introduce a random gate mechanism to continuously relax the selection vector to achieve binary selection of embedded components and make the process differentiable; (33) The mean parameter of the random gate is dynamically generated by an embedding selector network whose input is the context features of the server; in this way, the selection probability of the embedding component depends on the specific context information of each sample. (34) Introduce a constraint function in the optimization objective on the server side. This function applies both sparsity constraints and binary constraints to the selection vector to ensure that the number of the finally selected embedded components is close to the preset upper limit and that their probability values ​​are close to 0 or 1. (35) The server trains the embedding selector network by optimizing a joint objective function that includes the task prediction loss and the above constraint function; (36) After training is completed, the server determines the most important embedded components for each client and sends its index set to the corresponding client.

5. The vertical federated learning feature selection method according to claim 4, characterized in that, The embedded component in step (3) includes a scalar attention module for evaluating the contribution of each client in the joint modeling.

6. The vertical federated learning feature selection method according to claim 1 or 4, characterized in that, The specific modeling and calculation process for step (3) is as follows: Server-side embedded network selection Given context input, output the gate mean for each embedded component. To maintain differentiability for training, a Gaussian perturbation is introduced and clipped to an interval, and the computation is expressed as: ， in The Gaussian perturbation of the gate mean of each embedded component follows a Gaussian distribution, and the perturbation factor is... The value decreases with each iteration to gradually approach the hard gate; For the sample Client The selected embedded network is represented as ,in Element-wise multiplication indicates that components are retained or attenuated. Z Indicates the features of the preceding and following text; Then With sample i Corresponding context features The signal is fed into the prediction network to obtain the output. To limit the number of selections per sample, a comprehensive constraint is used, the mathematical expression of which is: ， The first term is used to encourage fewer non-zero values, the second term... The third term is used to encourage components to move away from the middle value and tend towards 0 or 1. Pull towards the target base In practice The cardinality constraint can be increased by setting it to 1 or by increasing it as training progresses. The objective function for the embedding selection phase is: ， in It is for the sample i and client m , by context features Z Guided embedding selection vector. Represents all learnable parameters in the server's VFL network. The parameters representing the embedded selector network, It is a loss function. This represents a dynamic adjustment factor, which is small in the early stages and gradually increases thereafter, approaching the sparsity and cardinality targets, using a linear or piecewise weighted schedule. , T These are hyperparameters used to control the constraint functions. The rate at which losses are incorporated; To measure the marginal contribution of multiple parties to the current sample, scalar attention is introduced: in It is a client m Processing samples i Attention weights at that time are used to quantify the client's embedding. The significance or contribution of this sample to the prediction. It is a client m For the sample i The original attention score, For trainable parameters, It is a nonlinear disturbance factor; In the context Below, the retained sub-embedded Able to explain ,but Larger The corresponding increase, based on all samples Calculate the average to obtain the client's... global contribution .

7. The vertical federated learning feature selection method according to claim 1, characterized in that, Step (4) for the local feature selection process is as follows: (41) Each client receives the index of important embedded components from the server; (42) The client constructs a new local optimization objective that minimizes the squared difference between the new embedding vector generated from the filtered local features and the pre-trained embedding vector on the component corresponding to the index; (43) The feature selection process is implemented through a random gate mechanism controlled by a local feature selector network to make it differentiable; (44) Introduce a constraint function into the local optimization objective to ensure that the number of features selected in the end is close to the preset upper limit; (45) The client trains its feature selector network by optimizing the local objective function, and finally determines whether each input feature is retained or discarded, forming the final feature subset.

8. The vertical federated learning feature selection method according to claim 1 or 7, characterized in that, In step (4), the specific calculation process of the client pruning irrelevant features based on the important embedding index set sent by the server includes: Client selects network based on features For the sample Output gate mean Using Gaussian perturbation and clipping, the calculation expression is as follows: To obtain the selection variable The selected feature is represented as ; To anchor feature selection to the important embeddings determined by the server, the alignment error is defined as: , The above formula indicates that only in the key embedding dimension The above measures the deviation between the new embedding and the pre-trained embedding; The optimization goals for the client are: , in The basic objective in the comprehensive constraints is... Replace with feature upper limit To control the number of features that can be retained per sample, By employing piecewise or linear weighting, the optimization first satisfies alignment, and then emphasizes sparsity and binary values.

9. A context-aware vertical federated learning collaboration system, characterized in that, The system achieves sample feature selection among clients based on the method described in any one of claims 1-8. Based on the unified perception and management of the context on the server side, and with the goal of achieving a balance between minimizing the expected communication and computing costs of each client and maintaining prediction accuracy, the multi-source embedding representation of the sample is first dynamically centralized to the associated server for context-guided embedding selection. Then, each client decides whether to participate in and execute local feature reallocation and pruning based on the embedding index set issued by the server, thereby realizing the reconfiguration and optimization of load and information channels under cross-party collaboration.

10. A context-aware vertical federated learning recommendation system, characterized in that, The system implements federated learning and training of medical image data among clients based on the context-aware vertical federated learning feature selection method described in any one of claims 1-8.

Images