Annoy tree fast weighting K neighbor feature completion method, device and equipment based on longitudinal federated learning
By introducing Annoy trees and weighted K-nearest neighbors algorithm into vertical federated learning, the problem of missing data in vertical federated learning is solved, achieving efficient and safe feature completion, improving completion accuracy and reducing computational complexity.
Patent Information
- Application Number
- CN202511170336.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-20
- Publication Date
- 2025-11-11
AI Technical Summary
The problem of missing data in existing vertical federated learning scenarios has not been effectively solved. Traditional methods have limited accuracy and high computational complexity, making it difficult to achieve efficient and large-scale feature completion while protecting data privacy.
We employ a fast weighted K-nearest neighbor feature completion method based on vertical federated learning using Annoy trees. We align sample IDs through a privacy-preserving intersection protocol, construct representative centroid vectors and Annoy trees, and use Euclidean distance exponential decay weights for weighted mean completion, thereby reducing computational complexity and improving completion accuracy.
Under the vertical federated architecture, the computational complexity is optimized from O(n2) to O(nlog n), ensuring data security and significantly improving the completion accuracy in heterogeneous data distribution scenarios.
Smart Images
Figure CN120934737A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence technology and relates to a missing data completion method under a vertical federated learning framework. In particular, it relates to a technical solution that achieves efficient large-scale feature completion by improving the K-nearest neighbor algorithm with Gaussian weighted average while protecting data privacy. Background Technology
[0002] In the process of digital transformation, high-quality data is a core resource driving intelligent development across various fields. However, data gaps are prevalent in real-world scenarios. Studies show that over 60% of datasets contain missing values, severely limiting data analysis efficiency: not only reducing model generalization ability but also potentially leading to systemic decision-making biases. Simultaneously, the strengthening of global privacy regulations and cross-institutional trust barriers have resulted in data being distributed in "islands," making traditional centralized data completion solutions difficult to implement.
[0003] Federated learning provides a technical path for distributed data utilization under privacy protection through a collaborative paradigm of "data remains stationary while the model moves." Based on data distribution characteristics, federated learning can be divided into three categories: horizontal, vertical, and transfer-based federated learning. Vertical federated learning (VFL) supports multiple participants in collaborative modeling based on different feature dimensions of the same sample ID, and has spawned various efficient algorithms (such as Federated Forest and SecureBoost).
[0004] However, existing technologies have certain research gaps regarding the data missingness problem in vertical federated learning scenarios: there is relatively more research on data completion in horizontal federated learning (such as completion schemes based on generative adversarial networks), but these cannot be directly transferred to vertical federated architectures; mainstream vertical federated platforms (such as FATE) still rely on simple imputation methods such as maximum / minimum values (MAX / MIN), resulting in limited completion accuracy; and while a few vertical completion schemes (such as methods based on encrypted KNN) achieve privacy protection, their computational complexity is as high as O(n^2). 2 However, centralized completion methods are severely limited in efficiency on large-scale, high-dimensional datasets. In contrast, centralized completion methods are mature (statistical methods, machine learning, deep learning, and others), but their federation faces a triple challenge of algorithm adaptability, efficiency, and privacy protection. Summary of the Invention
[0005] To address the aforementioned problems, the technical solution adopted in this invention is: a fast weighted K-nearest neighbor feature completion method based on vertical federated learning using Annoy trees, comprising the following steps:
[0006] S1: Obtain the dataset to be completed;
[0007] S2: Use the privacy intersection protocol to align the sample IDs of the dataset to be completed. After obtaining the intersection dataset of Guest and host, the coordinator generates and distributes the homomorphic encryption public key.
[0008] S3: Guest and host construct a representative centroid vector based on local data, calculate the local normal plane vector, intercept, and encrypted intermediate distance matrix, and send them to the coordinator.
[0009] S4: After decryption, the coordinator summarizes the data, divides the sample nodes, and returns the index to Guest and host. It then recursively constructs the Annoy tree and constructs T Annoy trees based on steps S1-S4.
[0010] S5: Traverse T Annoy trees to obtain the nearest neighbors of the leaf nodes corresponding to the missing samples, and then use the weighted mean of Euclidean distance with exponential decay weights to complete the feature completion of the dataset to be completed.
[0011] Furthermore, the process by which the Guest and host construct representative centroid vectors based on local data is as follows:
[0012] Two initial sample points p and q are randomly selected as seed vectors;
[0013] The process of optimizing the sample point locations involves dynamically updating the centroid position using a distance-weighted average. The specific process is as follows:
[0014] Randomly select sample x c ;
[0015] Calculate the distance from sample points p and q to sample x. c Distance:
[0016] Update sample point p based on distance. new and q new Output the representative centroid vector p new ,q new .
[0017] Furthermore, the local plane vector ω, intercept z, and encrypted intermediate distance matrix D are calculated using the following formula:
[0018]
[0019] [D] is obtained by encrypting the distance matrix using the public key, where X is the feature matrix of the current node.
[0020] Furthermore: the process by which the coordinator decrypts and aggregates the data, divides it into sample nodes, returns the indexes to Guest and host, and recursively constructs the Annoy tree is as follows:
[0021] Coordinator C aggregates the intercepts to obtain the global intercept, aggregates the encrypted intermediate distance matrix from all parties, decrypts it using the private key, partitions the current sample set based on the calculation results of each sample, and obtains the left and right leaf node index sets ID. left ID right :
[0022] ID right =ID-ID left (5)
[0023] Where 's' represents the participating party;
[0024] Coordinator C will set the left and right leaf node indexes ID. left ID right Send to each participant. Each participant uses their ID. left ID right The number of sample indices is used to determine the splitting process. If it exceeds the threshold, return to step 2 and continue splitting; otherwise, stop splitting and save the Annoy tree.
[0025] Furthermore: the process of traversing T Annoy trees to obtain the nearest neighbors of the leaf nodes corresponding to the missing samples, and then using weighted mean imputation based on Euclidean distance with exponential decay weights, to finally achieve feature imputation of the dataset to be imputed is as follows:
[0026] When handling missing values, the entire tree structure is recursively traversed to locate the leaf node containing the ID. The ID pools of each node are merged to form a candidate set. Two filtering operations are also required: removing IDs with the same missing feature from the candidate set and excluding the query ID itself.
[0027] Iterate through the mask matrix to find the positions M(i,j) where the value is 1, and obtain the sample index r corresponding to the missing position. For each r, traverse each tree and obtain the index set ID of the leaf node containing index r. t The set of indexes IDs for all leaf nodes of the tree. t Obtain by taking the union
[0028] Based on index set ID r Remove the index of the sample with missing j-th feature value and the current index r, and take the sample corresponding to the remaining index as the nearest neighbor sample. For the candidate nearest neighbor sample x m Its weight w(d) i,m ) depends on its Euclidean distance (d) from the target sample im It exhibits exponential decay, and the final weighted mean is:
[0029]
[0030] Where σ is the standard deviation of the Gaussian function, used to control the decay rate of the weights. A fast weighted K-nearest neighbor feature completion device based on vertical federated learning of Annoy trees, comprising:
[0031] The acquisition module is used to acquire the dataset to be completed.
[0032] The public key generation module is used to align the sample IDs of the dataset to be completed using a privacy intersection protocol. After obtaining the intersection dataset of the Guest and host, the coordinator generates and distributes the homomorphic public key.
[0033] Calculation module: Used by the Guest and host to construct representative centroid vectors based on local data, calculate local normal plane vectors, intercepts, and encrypted intermediate distance matrices, and send them to the coordinator;
[0034] The Annoy tree construction module is used to decrypt and summarize the data, divide the sample nodes, return the index to Guest and host, and recursively construct the Annoy tree. Then, based on steps S1-S4, the construction of T Annoy trees is realized.
[0035] The completion module is used to traverse T Annoy trees to obtain the nearest neighbors of the leaf nodes corresponding to the missing samples, and then use Euclidean distance with exponential decay weights to perform weighted mean completion, ultimately realizing feature completion of the dataset to be completed.
[0036] A computer device includes: a processor and a memory, the memory storing a program module, characterized in that the program module runs on the processor to implement any of the methods described above.
[0037] This invention provides a method, apparatus, and device for fast weighted K-nearest neighbor feature completion based on Annoy trees using longitudinal federated learning. Through collaboration between the Guest, Host, and Coordinator parties, it first introduces an approximate nearest neighbor search technique and utilizes the high-dimensional space segmentation indexing mechanism of Annoy trees to reduce the computational complexity of the traditional KNN algorithm from O(n...) 2 First, the computational complexity of KNN is optimized to O(nlogn). Second, under the vertical federated architecture, each participant only needs to exchange gradient information and local statistics through an encrypted channel, strictly adhering to the principle that data does not leave the domain, thus ensuring data security. Finally, an innovative feature importance-aware dynamic weight optimization mechanism is constructed, which significantly improves the completion accuracy in heterogeneous data distribution scenarios by adaptively adjusting the contribution of nearest neighbor samples. This invention utilizes Annoy trees to reduce the computational complexity of KNN from O(nlogn) to O(nlogn). 2 The computation time is reduced to O(nlog n); under the vertical federated architecture, only encrypted gradients and statistics are interacted to ensure that data security does not leave the domain; the contribution of nearest neighbors is adjusted through a dynamic weight optimization mechanism to improve the accuracy of heterogeneous data completion. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 The architecture diagram of the fast Gaussian weighted K-nearest neighbor feature completion algorithm based on vertical federated learning Annoy tree provided in the embodiments of the present invention;
[0040] Figure 2 Here is a flowchart of an embodiment of the present invention:
[0041] Figure 3 This is a comparison chart of the time consumption of the embodiments of the present invention and the vertical federated KNN completion algorithm;
[0042] Figure 4 The root mean square error of the imputation values in the embodiments of the present invention and the four methods of maximum value imputation (MAX), minimum value imputation (MIN), and average value imputation (MEAN) built into the FATE platform are compared under a sample missing rate of 10%-30%.
[0043] Figure 5 The root mean square error of the completion values is calculated for five methods in this invention, namely, centralized mean completion (MEAN), linear regression completion (LR), random forest completion (RF), and standard ANNOY completion algorithm, at a sample missing rate of 10%-30%. Detailed Implementation
[0044] It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.
[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0046] A fast weighted K-nearest neighbor feature completion method based on vertical federated learning using Annoy trees includes the following steps:
[0047] S1: Obtain the dataset to be completed;
[0048] S2: Use the privacy intersection protocol to align the sample IDs of the dataset to be completed. After obtaining the intersection dataset of Guest and host, the coordinator generates and distributes the homomorphic encryption public key.
[0049] S3: Guest and host construct a representative centroid vector based on local data, calculate the local normal plane vector, intercept, and encrypted intermediate distance matrix, and send them to the coordinator.
[0050] S4: After decryption, the coordinator summarizes the data, divides the sample nodes, and returns the index to Guest and host. It then recursively constructs the Annoy tree and constructs T Annoy trees based on steps S1-S4.
[0051] S5: Traverse T Annoy trees to obtain the nearest neighbors of the leaf nodes corresponding to the missing samples, and then use the weighted mean of Euclidean distance with exponential decay weights to complete the feature completion of the dataset to be completed.
[0052] This method is suitable for datasets with high feature correlation.
[0053] The steps S1 / S2 / S3 / S4 / S5 are executed sequentially;
[0054] Furthermore, the process by which the Guest and host construct representative centroid vectors based on local data is as follows:
[0055] Two initial sample points p and q are randomly selected as seed vectors;
[0056] The process of optimizing the sample point locations involves dynamically updating the centroid position using a distance-weighted average. The specific process is as follows:
[0057] Randomly select sample x c ;
[0058] Calculate the distance from sample points p and q to sample x. c Distance:
[0059] Update sample point p based on distance. new and q new Output the representative centroid vector p new ,q new .
[0060] Furthermore, the local plane vector ω, intercept z, and encrypted intermediate distance matrix D are calculated using the following formula:
[0061]
[0062] [D] is obtained by encrypting the distance matrix using the public key, where X is the feature matrix of the current node.
[0063] Furthermore, the process by which the coordinator decrypts and aggregates the data, divides it into sample nodes, returns the indexes to Guest and host, and recursively constructs the Annoy tree is as follows:
[0064] Coordinator C aggregates the intercepts to obtain the global intercept, aggregates the encrypted intermediate distance matrix from all parties, decrypts it using the private key, partitions the current sample set based on the calculation results of each sample, and obtains the left and right leaf node index sets ID. left ID right :
[0065] ID right =ID-ID left (5)
[0066] Where 's' represents the participating party;
[0067] Coordinator C will set the left and right leaf node indexes ID. left ID right Send to each participant. Each participant uses their ID. left ID right The number of sample indices is used to determine the splitting process. If it exceeds the threshold, return to step 2 and continue splitting; otherwise, stop splitting and save the Annoy tree.
[0068] Furthermore, the process of traversing T Annoy trees to obtain the nearest neighbors of the leaf nodes corresponding to the missing samples, and then using weighted mean imputation based on Euclidean distance with exponential decay weights, ultimately achieving feature imputation of the dataset to be imputed, is as follows:
[0069] When dealing with missing values, the process involves recursively traversing all tree structures to locate the leaf node containing the ID, merging the ID pools of each node to form a candidate set, and then performing two filtering operations: removing IDs with the same missing features from the candidate set and excluding the query ID itself.
[0070] Iterate through the mask matrix to find the positions M(i,j) where the value is 1, and obtain the sample index r corresponding to the missing position. For each r, traverse each tree and obtain the index set ID of the leaf node containing index r. t The set of indexes IDs for all leaf nodes of the tree. t Obtain by taking the union
[0071] Based on index set ID r Remove the index of the sample with missing j-th feature value and the current index r, and take the sample corresponding to the remaining index as the nearest neighbor sample. For the candidate nearest neighbor sample x m Its weight w(d)i,m ) depends on its Euclidean distance (d) from the target sample im It exhibits exponential decay, and the final weighted mean is:
[0072]
[0073] Where σ is the standard deviation of the Gaussian function, used to control the decay rate of the weights. A fast weighted K-nearest neighbor feature completion device based on vertical federated learning of Annoy trees, comprising:
[0074] The acquisition module is used to acquire the dataset to be completed.
[0075] The public key generation module is used to align the sample IDs of the dataset to be completed using a privacy intersection protocol. After obtaining the intersection dataset of the Guest and host, the coordinator generates and distributes the homomorphic public key.
[0076] Calculation module: Used by the Guest and host to construct representative centroid vectors based on local data, calculate local normal plane vectors, intercepts, and encrypted intermediate distance matrices, and send them to the coordinator;
[0077] The Annoy tree construction module is used to decrypt and summarize the data, divide the sample nodes, return the index to Guest and host, and recursively construct the Annoy tree. Then, based on steps S1-S4, the construction of T Annoy trees is realized.
[0078] The completion module is used to traverse T Annoy trees to obtain the nearest neighbors of the leaf nodes corresponding to the missing samples, and then use Euclidean distance with exponential decay weights to perform weighted mean completion, ultimately realizing feature completion of the dataset to be completed.
[0079] A computer device includes: a processor and a memory, the memory storing a program module, characterized in that the program module runs on the processor to implement the method as described in any one of the claims.
[0080] Example 1: This embodiment of the invention introduces a fast weighted K-nearest neighbor feature completion method based on vertical federated learning using Annoy trees. Based on a typical vertical federated learning architecture, a vertical federated completion architecture is designed, such as... Figure 1 As shown, the vertical federation model of both parties is taken as an example.
[0081] The general process includes: S1: Public key issuance process; S2: Select sample points and record IDs, calculate normal vector and intercept; S3(1): A encrypts the intermediate distance and sends it to C; S3(2): B encrypts the intermediate distance and sends it to C; S4: Decrypt with private key and obtain left and right node IDs; S5: Return IDs to all parties; S6: If the number of IDs is less than the threshold, end; otherwise, return to S2.
[0082] The coordinator C generates a homomorphic encrypted public key and distributes it through a secure channel, while the private key is stored locally.
[0083] In the initial stage of building the tree data structure, each tree node needs to maintain a set of identifiers of the samples it governs, i.e., the ID set. All participating IDs need to be kept synchronized, which serves as the basis for branch decisions.
[0084] Each participant will send the intermediate results calculated based on the ID to the coordinator C. For low-sensitivity parameters (such as the intercept of the segmentation hyperplane), plaintext aggregation will be used.
[0085] Homomorphic encryption is implemented for normal vectors and distance parameters that involve the privacy of the original data. After being decrypted by the coordinating node's private key, secure aggregation is performed to ensure that the coordinating party cannot reverse-engineer the original feature data of the participants.
[0086] The embodiments of the present invention employ an ID positioning mechanism to achieve efficient nearest neighbor search: when processing missing values, all tree structures are recursively traversed to locate the leaf node where the ID is located, and the ID pools of each node are merged to form a candidate set.
[0087] Two more filtering steps are required:
[0088] 1. Remove IDs from the candidate set that have the same missing feature.
[0089] 2. Exclude the query ID itself
[0090] The feature values of the remaining valid IDs are weighted by a Gaussian kernel function, and the weighted mean is calculated to complete the missing value filling.
[0091] This invention comprises two parties: a Guest and a Host, with a semi-honest coordinator to assist both in completing the federated learning process. Since the Guest and Host data are not visible to each other, and their respective samples have overlapping and non-overlapping portions, ID alignment and intersection extraction of the samples must be performed before missing value completion. This operation can be implemented using the existing Private Set Intersection (PSI) protocol.
[0092] Therefore, this embodiment of the invention assumes that the dataset used for training is the intersection samples of both sides that have been aligned, denoted as X. g ∈R N×p and X h ∈R N×q ,in:
[0093] N represents the number of intersection samples; p and q represent the feature dimensions of Guest and Host, respectively.
[0094] The dataset may contain missing values, which are represented by NaN.
[0095] The basic process of the completion algorithm according to an embodiment of the present invention is described below. Assume that the feature matrix X of the current node contains N samples X = {x1; x2; ... x...} i ;...x n Given a sample index set of ID = {1, 2, ..., n}, construct T trees.
[0096] Step 1:
[0097] The coordinator C generates a homomorphic encrypted public key public_key and a private key private_key. The public key public_key is distributed to all parties through a secure channel, while the private key private_key is stored locally by the coordinator.
[0098] Step Two:
[0099] The initial sample point indices are selected heuristically and iteratively optimized to obtain a representative "centroid vector" p. new ,q new This makes them represent the two main clusters in the dataset.
[0100] S21. Select two initial sample indices, denoted as a and b, satisfying a ~ U(1,n) and b ~ U(1,n), where a ≠ b. Define a sample point p = x. a and q = x b Initialize the counters ic = 1 and jc = 1.
[0101] S22. Perform the sample point location optimization process, repeating the following steps L times, where L is the number of iterations:
[0102] S221: Randomly select sample x c The condition c ~ U(1,n) is satisfied, and c ≠ a ≠ b.
[0103] S222: Calculate the distance from sample point p and q to sample x c Distance:
[0104]
[0105] S223: Update sample point p based on distance. new and q new :
[0106]
[0107] S23. Finally, output the updated sample point p. new and q new As the two selected samples.
[0108] Step 3:
[0109] Based on sample point p new and q new Calculate the local normal plane vector ω, intercept z, and intermediate distance matrix D:
[0110]
[0111] The distance matrix is then encrypted using the public key public_key to obtain [D].
[0112] Step 4: Each participant sends the intercept z and the encrypted intermediate distance matrix [D] to the coordinator C.
[0113] Step 5: Coordinator C aggregates the intercepts to obtain the global intercept, aggregates the encrypted intermediate distance matrix of all parties, and decrypts it using the private key. Based on the calculation results of each sample, the current sample set is divided, and the left and right leaf node index sets (IDs) are obtained. left ID right :
[0114]
[0115] ID right =ID-ID left (5)
[0116] Where 's' represents the participating party.
[0117] Step Six: Coordinator C sets the left and right leaf node indexes to ID. left ID right Send to each participant. Each participant uses their ID. left ID right The number of sample indices is used for judgment. If it is greater than the threshold, return to step two and continue splitting; otherwise, stop splitting.
[0118] Step 7: Save the current tree. If the number of trees is less than T, reset the sample index set and return to Step 2.
[0119] Step 8: Traverse the mask matrix to find the positions M(i,j) where the value is 1, and obtain the sample index r corresponding to the missing position; for each r, traverse each tree and obtain the index set ID of the leaf node containing index r. t The set of indexes IDs for all leaf nodes of the tree. t Obtain by taking the union
[0120] Step Nine: Based on ID rRemove the index of the sample with missing j-th feature value and the current index r, and take the sample corresponding to the remaining index as the nearest neighbor sample. For candidate nearest neighbor sample x m Its weight w(d) i,m ) depends on its Euclidean distance (d) from the target sample i,m The weighted average decays exponentially. The final weighted average is:
[0121]
[0122] Here, σ is the standard deviation of the Gaussian function, used to control the decay rate of the weights. A larger σ value will result in a wider weight distribution, while a smaller σ value will cause the weights to concentrate near the center point.
[0123] Figure 2 A flowchart of the above algorithm is given, and a joint modeling approach is used as an example.
[0124] This invention addresses the missing value completion problem by using the publicly available real estate dataset "housing" (https: / / www.kaggle.com / datasets / rizwanrizwannazir / housing) from the Kaggle platform. This dataset contains 10 features and 20,640 samples, which are preprocessed and divided into Guest and Host data.
[0125] In the data generation phase, the sample size was set with an increment step of n=1000, gradually expanding from a baseline size of N=10,000 to N=17,000, with three missing rate levels of 10%, 20%, and 30% set for each size level. This study generates missing data based on the completely randomized missing data (MCAR) mechanism, removing different parts of the original values in the dataset by changing the random seed value. To ensure the stability of the experimental results, each experimental group was repeated 5 times to reduce the impact of randomness, and the final results are presented as average values. To systematically evaluate the comprehensive performance of this algorithm, quantitative verification was performed from two dimensions: computational efficiency and completion accuracy. All experiments were performed under the same hardware environment and data configuration, and the single completion time and root mean square error (RMSE) were used as the core evaluation indicators.
[0126] In the time complexity comparison experiment, the proposed algorithm is compared with the existing vertical federated KNN completion algorithm (KNN-VFL). To eliminate the random interference of missing patterns on computational resources, the experiment integrates the cumulative time consumption of three missing rates under the same sample size. Figure 3As shown, when the sample size increases from 16,000 to 17,000, the cumulative time consumption of this embodiment is reduced by at least 50% compared to KNN-VFL, significantly verifying the acceleration effect of the approximate indexing mechanism. The short time consumption of this embodiment stems from the approximate nearest neighbor search mechanism based on a multi-branch tree structure. This method constructs multiple binary splitting trees to hierarchically partition the high-dimensional space, and uses a random hyperplane to recursively split the data to form an index structure, reducing the time complexity from O(n^2) of brute-force search. 2 The computational complexity is reduced to O(nlog n), and a forest voting mechanism is used to balance accuracy and efficiency. It is noteworthy that, despite employing an approximate retrieval strategy, the RMSE of this embodiment only increases by 1.46% compared to a precise search. This indicates that the Annoy index maintains an order-of-magnitude advantage in computational efficiency without significantly impacting completion accuracy.
[0127] Given that fast approximate retrieval has a slight impact on accuracy, to further evaluate the algorithm's accuracy performance, the maximum value completion (MAX), minimum value completion (MIN), and average value completion (MEAN) built into the FATE platform were selected as benchmark comparison methods. It should be noted that this selection is based on the current limited availability of data completion methods in the field of longitudinal federated learning. Experimental results are as follows... Figure 4 As shown, under the three missing rate scenarios of 10%, 20%, and 30%, the RMSE values of the embodiments of the present invention are significantly lower than those of the three benchmark methods, demonstrating superior completion accuracy. For example, under the benchmark condition of a missing rate of 20%, the RMSE values of the embodiments of the present invention are reduced by an average of 63.70% (vs MIN), 27.66% (vs MEAN), and 94.14% (vs MAX).
[0128] To systematically verify the effectiveness of the proposed method, several classic centralized methods were selected as comparative experiments. A progressive centralized benchmark comparison framework was constructed, including Mean Imputation (MEAN), Linear Regression Imputation (LR), Random Forest Imputation (RF), and the standard ANNOY Imputation algorithm. In this framework, MEAN represents basic statistical imputation capability, LR reflects linear modeling performance, RF characterizes the ability to capture complex nonlinear relationships, and the standard ANNOY algorithm serves as the basic comparison for the embodiments of this invention. In particular, the experiments strictly distinguish between two data application paradigms: in the centralized imputation scenario, missing value imputation is performed only based on the Guest's independent dataset; while in the proposed federated imputation scheme, collaborative computation between the Guest and Host in a vertical federated learning framework is achieved through the embodiments of this invention. Figure 5Experimental results show that cross-institutional data collaboration significantly improves feature completion performance. After introducing host data into federated computation, the completion accuracy, while strictly protecting data privacy, not only did not decrease due to the distributed architecture, but actually surpassed traditional centralized methods, with RMSE values lower than all benchmark methods. Under the benchmark condition of a 20% missing rate, the RMSE values of the embodiments of this invention decreased by an average of 27.51% (vs MEAN), 25.28% (vs LR), 17.71% (vs RF), and 21.90% (vs ANNOY_centralized).
[0129] Finally, to further explore the impact of the weighting mechanism on algorithm performance, this embodiment of the invention verifies the necessity of the weighting strategy compared to uniform weighting through controlled variable experiments. Four control groups were designed, with Gaussian kernel weighting coefficients set to 0.3, 0.5, 0.7, and uniform weighting, respectively. The results show that although the improvement is small, the RMSE of all weighted groups is lower than that of the uniform weighting group, indicating that moderate distance decay weighting can effectively suppress outlier interference.
[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A fast weighted K-nearest neighbor feature completion method based on vertical federated learning using Annoy trees, characterized in that: Includes the following steps: S1: Obtain the dataset to be completed; S2: Use the privacy intersection protocol to align the sample IDs of the dataset to be completed. After obtaining the intersection dataset of Guest and host, the coordinator generates and distributes the homomorphic encryption public key. S3: Guest and host construct a representative centroid vector based on local data, calculate the local normal plane vector, intercept, and encrypted intermediate distance matrix, and send them to the coordinator. S4: After decryption, the coordinator summarizes the data, divides the sample nodes, and returns the index to Guest and host. It then recursively constructs the Annoy tree and constructs T Annoy trees based on steps S1-S4. S5: Traverse T Annoy trees to obtain the nearest neighbors of the leaf nodes corresponding to the missing samples, and then use the weighted mean of Euclidean distance with exponential decay weights to complete the feature completion of the dataset to be completed.
2. The fast weighted K-nearest neighbor feature completion method based on vertical federated learning of Annoy trees according to claim 1, characterized in that: The process by which the Guest and host construct representative centroid vectors based on local data is as follows: Two initial sample points p and q are randomly selected as seed vectors; The process of optimizing the sample point locations involves dynamically updating the centroid position using a distance-weighted average. The specific process is as follows: Randomly select sample x c ; Calculate the distance from sample points p and q to sample x. c Distance: Update sample point p based on distance. new and q new Output the representative centroid vector p new ,q new .
3. The method for fast weighted K-nearest neighbor feature completion based on longitudinal federated learning of Annoy trees according to claim 1, characterized in that: The local normal plane vector ω, intercept z, and encrypted intermediate distance matrix D are calculated using the following formula: [D] is obtained by encrypting the distance matrix using the public key, where X is the feature matrix of the current node.
4. The method for fast weighted K-nearest neighbor feature completion based on vertical federated learning of Annoy trees according to claim 1, characterized in that: The process by which the coordinator decrypts and aggregates the data, divides it into sample nodes, returns the index to Guest and host, and recursively constructs the Annoy tree is as follows: Coordinator C aggregates the intercepts to obtain the global intercept, aggregates the encrypted intermediate distance matrix from all parties, decrypts it using the private key, partitions the current sample set based on the calculation results of each sample, and obtains the left and right leaf node index sets ID. left ID right : Where 's' represents the participating party; Coordinator C will set the left and right leaf node indexes ID. left ID right Send to each participant. Each participant uses their ID. left ID right The number of sample indices is used to determine the splitting process. If it exceeds the threshold, return to step 2 and continue splitting; otherwise, stop splitting and save the Annoy tree.
5. The fast weighted K-nearest neighbor feature completion method based on longitudinal federated learning of Annoy trees according to claim 1, characterized in that: The process of traversing T Annoy trees to obtain the nearest neighbors of the leaf nodes corresponding to the missing samples, and then using weighted mean imputation based on Euclidean distance with exponential decay weights, finally achieves feature imputation of the dataset to be imputed as follows: When handling missing values, the entire tree structure is recursively traversed to locate the leaf node containing the ID. The ID pools of each node are merged to form a candidate set. Two filtering operations are also required: removing IDs with the same missing feature from the candidate set and excluding the query ID itself. Iterate through the mask matrix to find the positions M(i,j) where the value is 1, and obtain the sample index r corresponding to the missing position. For each r, traverse each tree and obtain the index set ID of the leaf node containing index r. t The set of indexes IDs for all leaf nodes of the tree. t Obtain by taking the union Based on index set ID r Remove the index of the sample with missing j-th feature value and the current index r, and take the sample corresponding to the remaining index as the nearest neighbor sample. For the candidate nearest neighbor sample x m Its weight w(d) i,m ) depends on its Euclidean distance (d) from the target sample i,m It exhibits exponential decay, and the final weighted mean is: Where σ is the standard deviation of the Gaussian function, used to control the decay rate of the weights.
6. A fast weighted K-nearest neighbor feature completion device based on vertical federated learning of Annoy trees, characterized in that: include: The acquisition module is used to acquire the dataset to be completed. The public key generation module is used to align the sample IDs of the dataset to be completed using a privacy intersection protocol. After obtaining the intersection dataset of the Guest and host, the coordinator generates and distributes the homomorphic public key. Calculation module: Used by the Guest and host to construct representative centroid vectors based on local data, calculate local normal plane vectors, intercepts, and encrypted intermediate distance matrices, and send them to the coordinator; The Annoy tree construction module is used to decrypt and summarize the data, divide the sample nodes, return the index to Guest and host, and recursively construct the Annoy tree. Then, based on steps S1-S4, the construction of T Annoy trees is realized. The completion module is used to traverse T Annoy trees to obtain the nearest neighbors of the leaf nodes corresponding to the missing samples, and then use Euclidean distance with exponential decay weights to perform weighted mean completion, ultimately realizing feature completion of the dataset to be completed.
7. A computer device, comprising: A processor and a memory, wherein the memory stores a program module, characterized in that the program module runs on the processor to implement the method as described in any one of claims 1-5.