Privacy protection-based longitudinal federated federated inference method and system and storage medium

Abstract

The application provides a privacy protection-based longitudinal federated joint inference method and system and a storage medium. The method comprises the following steps: a terminal initiates local inference based on a local target feature dataset and inference model parameters to generate intermediate results; the closest classification cluster is determined as a competitive cluster, similarity metrics are calculated and cluster indexes are disclosed; the similarity metrics and the intermediate results are processed through secret sharing to generate share information and exchange with other terminals; the share information is aggregated according to the disclosed cluster indexes to reconstruct the complete similarity metrics and inference output of each group of models on the corresponding cluster; the inference ability score is weighted and fused by using the similarity metrics to obtain an evaluation value, and the evaluation value is mapped to a model output weight through a nonlinear amplification function; finally, the outputs of each group of models are weighted and summed according to the weight to obtain the overall inference result. The application can improve the inference efficiency of the longitudinal federated model under the secret sharing mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federated learning, and in particular to a privacy protection-based longitudinal federated joint inference method and system and a storage medium. BACKGROUND

[0002] Machine learning has achieved remarkable success in many application fields such as medical diagnosis, stock prediction and fraud detection. The key to realizing a high-performance machine learning model is to train on large-scale high-quality data. However, in practice, high-quality data is generally distributed in different institutions in the form of longitudinal division, that is, these institutions hold a set of non-overlapping feature sets of public samples. If these data are collected at a central node for centralized training, it may cause serious privacy leakage problems. Therefore, privacy protection longitudinal machine learning is proposed, which can enable multiple parties to collaboratively build a machine learning model without exposing their own data.

[0003] In the prior art, secret sharing technology is usually used to realize privacy protection longitudinal machine learning. In these studies, each participant distributes the local input data in the form of secret sharing, and collaboratively builds and infers a unified machine learning model based on various computing primitives (such as multiplication and comparison operations) under secret sharing, thereby completing the joint prediction task without leaking their own private data.

[0004] However, since the entire inference process needs to be carried out in an encrypted state, it involves a large number of high-overhead secret sharing operation operations such as secure multiplication, secure comparison, etc., resulting in an inference efficiency much lower than plaintext calculation. More seriously, in a multi-party participation setting, the communication and computation complexity of the secret sharing mechanism is usually in square relationship with the number of participants, so that the inference delay increases sharply with the increase of the number of participants, further exacerbating the problem of low inference efficiency, limiting the feasibility and practicality of this type of method in actual large-scale application scenarios. SUMMARY

[0005] In view of this, the embodiments of the present application provide a privacy protection-based longitudinal federated joint inference method, system and storage medium to eliminate or improve one or more defects in the prior art. It can effectively solve the problem of sharp increase of inference delay and significant reduction of inference efficiency due to the square relationship between the communication and computation complexity of the secret sharing mechanism and the number of participants in a multi-party participation setting.

[0006] One aspect of the present application provides a privacy protection-based longitudinal federated joint inference method, which comprises the following steps:

[0007] The initiating terminal performs local inference calculation based on the locally held target feature data set and the locally held inference model parameter, and generates an intermediate inference result; the inference model parameter is a part of the parameter of the group model; each group model is held in a secret sharing form by each terminal in the terminal group to which the group model belongs; the number of terminals in each terminal group is the same, including the initiating terminal and the participating terminal, and the initiating terminal in each group is the same, and the participating terminal is different;

[0008] In the locally divided classification cluster, the classification cluster closest to the target feature data set is determined as a competitive classification cluster, the similarity measure between the target feature data set and other data sets under the competitive classification cluster is calculated, and the competitive classification cluster index corresponding to the competitive classification cluster is disclosed;

[0009] The similarity measure and the intermediate inference result are processed through the secret sharing mechanism to obtain corresponding similarity measure share information and intermediate inference result share information, which are distributed to each terminal in the terminal group to which the terminal belongs, and the share information sent by other terminals in the terminal group and the disclosed competitive classification cluster index are received;

[0010] According to the disclosed all competitive classification cluster indexes, the similarity measure share information and the intermediate inference result share information held by each terminal are aggregated to reconstruct the complete similarity measure of each group model on the corresponding competitive classification cluster and the complete inference output corresponding to each group model;

[0011] The inference ability score of each group model on the corresponding competitive classification cluster is weighted and fused through the complete similarity measure to obtain the inference ability evaluation value corresponding to each group model, and the model output weight is obtained through the mapping processing of the preset nonlinear amplification function;

[0012] The complete inference output of each group model is weighted and summed using the model output weight to obtain the final output result.

[0013] In some embodiments of the application, before determining the classification cluster closest to the target feature data set as the competitive classification cluster, calculating the similarity measure between the target feature data set and other data sets under the competitive classification cluster, and disclosing the competitive classification cluster index corresponding to the competitive classification cluster, the method further comprises:

[0014] Based on the locally held verification feature data set, the feature is classified, and the feature space is divided into at least one classification cluster; the verification feature data set and the target feature data set have the same sample space and the same feature dimension;

[0015] According to the classification cluster to which each feature in the verification feature data set belongs, an indication vector is generated and vector normalization processing is performed to obtain a normalized vector;

[0016] Performing local inference calculation on the verification feature dataset and the inference model parameters to generate a verification intermediate inference result;

[0017] Generating a corresponding random prediction value for the verification feature dataset;

[0018] Processing the verification intermediate inference result, the random prediction value, the normalized vector, and the corresponding true inference result of the verification feature dataset through a secret sharing mechanism to obtain verification share information corresponding to the verification intermediate inference result, random prediction share information corresponding to the random prediction value, vector share information corresponding to the normalized vector, and result share information corresponding to the true inference result;

[0019] Distributing the verification share information, the random prediction share information, the vector share information, and the result share information to each terminal in the terminal group to which the terminal belongs, and receiving the verification share information, the vector share information, and the result share information sent by other terminals in the terminal group to which the terminal belongs;

[0020] Based on the locally held and received verification share information, the random prediction share information, the vector share information, and the result share information, the inference ability score intermediate variable of each group model on the corresponding classification cluster is calculated; after aggregation, the inference ability score intermediate variable held by each terminal is reconstructed to obtain the inference ability score of each group model on the corresponding classification cluster.

[0021] In some embodiments of the present application, the terminal group is divided in advance, including:

[0022] According to the preset number of terminals corresponding to each group, the candidate terminal group is constructed in an enumerated form;

[0023] Obtaining feature distribution weights, and calculating a feature quality score based on the feature distribution weights and the feature distribution of the locally held verification feature dataset; wherein each terminal holds feature distribution weight share information, and the feature distribution weights are reconstructed by aggregating the feature distribution weight share information held by each terminal;

[0024] The feature quality score share information is obtained by processing the feature quality score through a secret sharing mechanism, distributed to each terminal in the corresponding candidate terminal group, and the feature quality score share information sent by each terminal is received;

[0025] Based on the locally held feature quality score share information and the received feature quality score share information, the share information corresponding to the total score of the feature quality corresponding to each candidate terminal group is calculated;

[0026] Receiving the feature quality total score share information sent by each terminal, reconstructing the feature quality total score corresponding to each candidate terminal group, selecting the preset number of groups with the highest feature quality total score from the candidate terminal groups through a secure sorting mechanism, as the final terminal group and notifying each terminal group of the result.

[0027] In some embodiments of the application, the feature distribution weight is obtained, including:

[0028] The standard deviation of each feature in the verification feature data set is calculated and mapped to a preset interval, and a local quantity vector is constructed according to the number of features in each interval;

[0029] The local quantity vector is processed through a secret sharing mechanism to obtain the corresponding local quantity vector share information, which is distributed to each terminal in the corresponding candidate terminal group, and the local quantity vector share information sent by each terminal is received;

[0030] Based on the local holding of the local quantity vector share information and the received local quantity vector share information, the global quantity vector share information is calculated;

[0031] After receiving the global quantity vector share information sent by each terminal, the global quantity vector is reconstructed, the low, medium and high level division positions are determined through dynamic search and are disclosed;

[0032] The feature distribution weight corresponding to each level is determined, the feature distribution weight share information is obtained through the secret sharing mechanism, and is distributed to each terminal.

[0033] In some embodiments of the application, the low, medium and high level division positions are determined through dynamic search, including:

[0034] Initializing the search range, including searching for the first search range of the low level and the intermediate level division from the first interval, and searching for the second search range of the intermediate level and the high level division from the last interval;

[0035] The total number of features in the preset interval in the current search range is calculated, and the proportion of the total number of features in the global feature total number is determined;

[0036] If the proportion is greater than or equal to the preset threshold, the interval in the current search range is divided into the corresponding level; otherwise, the search range is adjusted, including extending the first search range by one interval or extending the second search range by one interval, executing the steps of calculating the total number of features in the preset interval in the current search range and determining the proportion of the total number of features in the global feature total number, until the level division of all intervals is completed.

[0037] In some embodiments of the present application, the feature quality score is calculated based on the feature distribution weight and the feature distribution of the locally held verification feature dataset, and the calculation includes:

[0038] The feature quantity in each preset interval is multiplied by the corresponding feature distribution weight to obtain an interval feature score corresponding to each interval.

[0039] The interval feature scores are added to obtain the feature quality score.

[0040] In some embodiments of the present application, the preset nonlinear amplification function is a cubic function constructed in advance by the Lagrange interpolation method.

[0041] The model output weight is obtained by mapping processing through the preset nonlinear amplification function, and the mapping processing includes:

[0042] The mean of the inference ability evaluation values of all group models is calculated.

[0043] The ratio of the inference ability evaluation value of each group model to the mean is calculated.

[0044] The ratio and the corresponding inference ability evaluation value are taken as the input of the preset nonlinear amplification function to obtain the model output weight corresponding to each group model.

[0045] In some embodiments of the present application, each group model is obtained by pre-cooperative training of each terminal group in a self-supervised form and a preset inference task; the preset inference task includes a feature classification task of generating a classification label or a regression task of generating an output value.

[0046] Another aspect of the present application provides a privacy protection based longitudinal federated joint inference system, including a processor, a memory and a computer program / instruction stored on the memory, the processor being configured to execute the computer program / instruction, and when the computer program / instruction is executed, the system implements the steps of the privacy protection based longitudinal federated joint inference method as described above.

[0047] Another aspect of the present application provides a computer readable storage medium having a computer program / instruction stored thereon, and the computer program / instruction is executed by a processor to implement the steps of the privacy protection based longitudinal federated joint inference method as described above.

[0048] The privacy protection based longitudinal federated joint inference method and system can effectively solve the problem of sharp rise of inference delay and significant reduction of inference efficiency caused by the communication and calculation complexity of the secret sharing mechanism in the square relationship with the number of participants in the multi-party participation setting.

[0049] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0050] It will be understood by those skilled in the art that the objects and advantages of the present application can not be limited to those specifically described and that the present application can realize the above and other objects and advantages in light of the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application.

[0052] Figure 1 The flowchart of the privacy protection based longitudinal federated joint inference method provided by an embodiment of the present application.

[0053] Figure 2 The terminal grouping division schematic diagram provided by an embodiment of the present application.

[0054] Figure 3 The schematic diagram of preset feature distribution weight division provided by an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application with reference to the embodiments and drawings. Herein, the illustrative embodiments of the present application and the description thereof are used to explain the present application but not to limit the present application.

[0056] It should be further noted that, in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0057] It should be emphasized that the term "comprising / comprising" as used herein refers to the presence of a feature, element, step or component, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0058] It should be further noted that, unless otherwise specified, the term "connection" as used herein can not only refer to direct connection, but also represent indirect connection with the presence of intermediate objects.

[0059] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts or the same or similar steps.

[0060] The following explains some of the terms involved in this application:

[0061] Secret sharing is a cryptographic technique that allows multiple participants to jointly hold a part of a secret information, without any single party having enough information to reconstruct the secret. Only when a sufficient number of participants cooperate can the original secret information be recovered. This technology is very useful in protecting data privacy and security, especially in multi-party computation, distributed systems and federated learning scenarios.

[0062] The following describes in detail the method for longitudinal federated joint inference based on privacy protection provided by the present application.

[0063] As shown in Figure 1 The embodiments of the present application provide a method for longitudinal federated learning privacy protection joint output. The present embodiment takes the method used in the initiating terminal as an example for illustration. The initiating terminal can be a mobile phone, a computer, a server or a server cluster, and the present embodiment does not limit the implementation of the initiating terminal. The method comprises at least steps S101-S106:

[0064] Step S101, the initiating terminal performs local inference calculation based on the target feature data set held locally and the inference model parameters held locally, and generates an intermediate inference result.

[0065] The reasoning model parameters are part of the group model parameters. Each group model is held by each terminal in the terminal group in a secret sharing form. The number of terminals in each terminal group is the same, including the initiating terminal and the participating terminals, and the initiating terminal in each group is the same, and the participating terminals are different.

[0066] For example, refer to Figure 2 For example, one initiating terminal and four participating terminals are used for illustration, including the initiating terminal p1, the participating terminal p2, the participating terminal p3, the participating terminal p4, and the participating terminal p5. The five terminals are divided into two terminal groups, including the terminal group k1 and the terminal group k2. In the terminal group k1, the initiating terminal p1, the participating terminal p2, and the participating terminal p3 are included. In the terminal group k2, the initiating terminal p1, the participating terminal p4, and the participating terminal p5 are included.

[0067] In some embodiments of the present application, the initiating terminal refers to a terminal used for terminal group division, vertical federated joint reasoning process initiation, and global coordination. The participating terminal refers to a terminal that establishes an encrypted communication connection with the initiating terminal and cooperatively performs vertical federated joint reasoning. The participating terminal can be a mobile phone, a computer, a server, or a server cluster, and the implementation of the participating terminal is not limited in the present embodiment.

[0068] In some embodiments of the present application, the group model is a local joint model pre-trained by each terminal group based on supervised learning and a preset reasoning task.

[0069] The initiating terminal first initiates a construction request for a vertical machine learning model to each participating terminal, and agrees on a training and reasoning protocol of the machine learning model and a preset reasoning task. In the cooperative training process, each terminal group exchanges intermediate parameters of the model through a secret sharing framework.

[0070] In some embodiments of the present application, the preset reasoning task includes a feature classification task of generating a classification label. For example, determining whether a user is a high-risk customer, whether a patient has a certain disease, etc.

[0071] In the model training process of each terminal group, the initiating terminal and each participating terminal cooperatively construct a machine learning model based on a vertical federated learning framework, such as a logistic regression model, a decision tree, or a neural network model. The specific training process is as follows:

[0072] In the preparation stage, the initiating terminal sends information containing the training protocol to the participating terminals, which is used to determine the model type, the parameter initialization method, and the training round, etc. Each terminal prepares the locally held sample target feature data set and its corresponding sample label information according to the task requirements. The data sample spaces held by all terminals are consistent (i.e., the sample IDs are aligned), but the feature dimensions held by each terminal are different.

[0073] For example, referring to Figure 2 For example, the terminal group k1 and the sample data feature subset include the features of user 001 and user 002; the initiating terminal p1 holds the age features and gender features of user 001 and user 002; the participating terminal p2 holds the income features and debt ratio features of user 001 and user 002; the participating terminal p3 holds the loan times features and repayment status features of user 001 and user 002, and the initiating terminal p1 holds the sample classification labels of user 001 and user 002, which are used to indicate the risk states of user 001 and user 002, including high-risk labels, medium-risk labels and low-risk labels.

[0074] In the training phase, each terminal performs forward propagation calculation based on the locally held sample target feature data set to generate a local output result. Subsequently, according to different cases whether the sample label is held locally, a corresponding loss function calculation method is adopted. In the case where each terminal holds the sample label, the loss value is directly calculated based on the local output and the real label, and the local gradient is updated accordingly; in the case where only the initiating terminal holds the sample label, the initiating terminal is responsible for calculating the loss and generating global gradient information, and then the encrypted gradient information is distributed to other participating terminals through a secure mechanism. On this basis, using the secret sharing framework, each terminal encrypts the calculated local gradient or received encrypted gradient information and distributes it to other terminals in the same group; at the same time, it receives encrypted gradient information from other terminals in the group. The terminal group aggregates all received encrypted gradient information, and after decryption, the global gradient is formed, which is used for updating the model parameters. The above process is repeated until the predetermined training round is reached or the convergence condition is met. Finally, each terminal obtains a local joint model, which can be used for subsequent collaborative inference tasks in the inference phase.

[0075] In some other examples of the present application, the preset inference task includes a regression task of generating an output value. For example, predicting the credit score of a user, the sales of a product, the house price, etc. continuous numerical output.

[0076] In the model training process of each terminal group, the initiating terminal and each participating terminal cooperatively construct a regression model based on the vertical federated learning framework, such as a linear regression model, a random forest regression model or a deep neural network regression model, etc. The specific training process is as follows:

[0077] In the preparation phase, the initiating terminal sends information containing the training protocol to the participating terminals, which is used to determine the model type, parameter initialization method, and training round, etc. Each terminal prepares the locally held sample target feature dataset and its corresponding sample label information according to the task requirements. Among them, the data sample spaces held by all terminals remain consistent (i.e. sample ID alignment), but the feature dimensions held by each terminal are mutually exclusive.

[0078] For example, reference Figure 2 For example, terminal grouping k1 and sample data feature subset including features of user 001 and user 002; initiating terminal p1 holds the occupation features and marital status features of user 001 and user 002; participating terminal p2 holds the income features and total asset features of user 001 and user 002; participating terminal p3 holds the credit card usage rate features and repayment day features of user 001 and user 002, and initiating terminal p1 holds the sample regression labels of user 001 and user 002, which are used to indicate the credit score values of user 001 and user 002, such as 720 points, 680 points, etc. continuous prediction target.

[0079] In the training phase, each terminal performs forward propagation calculation based on the locally held sample target feature dataset to generate a local output result. Subsequently, according to different situations whether the sample label is held locally, the corresponding loss function calculation method is adopted. In the case where each terminal holds the sample label, the loss value (such as the difference term MSE) is directly calculated based on the local output and the real label, and the local gradient is updated accordingly; in the case where only the initiating terminal holds the sample label, the initiating terminal is responsible for calculating the loss and generating global gradient information, and then the encrypted gradient information is distributed to other participating terminals through a secure mechanism. On this basis, using the secret sharing framework, each terminal encrypts the calculated local gradient or received encrypted gradient information and distributes it to other terminals in the same group; at the same time, it receives encrypted gradient information from other terminals in the group. The terminal grouping aggregates all received encrypted gradient information, and after decryption, the global gradient is formed and used for model parameter update. The above process is repeated until the predetermined training round is reached or the convergence condition is met. Finally, each terminal obtains a local joint model, which can be used for subsequent collaborative inference tasks in the inference phase.

[0080] In some embodiments of the present application, the target feature data refers to object data required for various machine learning tasks, such as classification tasks or prediction tasks, etc. In the vertical federated learning framework, these tasks are collectively referred to as inference tasks, i.e. the process of generating output results based on input features by the model.

[0081] In actual business scenarios, the complete target feature data total set is usually composed of multiple feature dimensions, which are often scattered in different institutions or terminals. Due to different business divisions, each terminal holds a vertically divided target feature data set corresponding to the complete target feature data total set and related to its own function (for example, hospital A may have patient medical history information, while hospital B records corresponding test report data). The feature dimensions in each target feature data set are different, but the sample space is consistent (such as the same patient ID).

[0082] For example, taking the complete target feature data total set containing data of all patients in a certain region as an example, the initiating terminal is the terminal of hospital A, which holds patient medical history (feature dimensions include: diagnosis record, past medical history, medication record, etc.); participating terminals include terminals corresponding to hospitals B and C, which hold test data (feature dimensions include: blood glucose value, blood lipid level, imaging report, etc.) and genomic data (feature dimensions include: gene mutation type, single nucleotide polymorphism, etc.) of the same batch of patients, respectively. Since hospitals A, B, and C hold different feature dimensions due to diagnosis, testing, and genetic testing businesses, respectively, but all terminals hold the same patient ID.

[0083] Or, taking the complete target feature data total set containing user data of a certain bank and e-commerce platform as an example, the initiating terminal is the terminal corresponding to bank D, which holds user credit records (feature dimensions include: credit card limit, payment history, number of overdue payments, etc.); participating terminals include terminals corresponding to e-commerce platform E, which hold user consumption behavior (feature dimensions include: shopping frequency, product category preference, average single price, etc.), and terminals corresponding to social platform C, which hold user social relationships (feature dimensions include: number of friends, interaction frequency, social activity level, etc.). Since bank D, e-commerce platform E, and social platform C hold different feature dimensions due to credit, transaction, and social businesses, respectively, but all terminals hold the same user ID.

[0084] In addition, in order to effectively improve the model training efficiency and inference performance under the premise of protecting data privacy, before collaborative training of the group model, the feature quality total score of each potential terminal grouping is calculated based on the verification feature data set held by each terminal using the secret sharing mechanism, and the top several combinations with tighter feature clustering and stronger complementarity are selected as the final terminal grouping division through a secure sorting protocol, so that the terminals in each group have higher internal consistency in feature distribution, which not only helps the group model to learn key feature relationships more efficiently, but also improves the accuracy and stability of the output results in the subsequent inference stage. At the same time, the process runs under the privacy protection mechanism all the time, avoiding the leakage of original feature information, enhancing the security and adaptability of the system.

[0085] In some embodiments of the present application, before initiating the terminal to initiate the construction request or joint inference of the longitudinal machine learning model, the terminals are divided into multiple candidate terminal groups in a preset group size by enumeration.

[0086] For example, with a preset group size of 3 and a preset number of terminal groups of 2, the terminals include the initiating terminal p1, the participating terminal p2, the participating terminal p3, the participating terminal p4, and the participating terminal p5. The initiating terminal p1 divides the candidate terminal groups by enumeration, including: (p1, p2, p3), (p1, p2, p4), (p1, p2, p5), (p1, p3, p4), (p1, p3, p5), and (p1, p4, p5).

[0087] After dividing all the candidate terminal groups by enumeration, the initiating terminal determines the feature quality total score corresponding to each candidate terminal group under the secret sharing framework, in cooperation with the terminals in each candidate terminal group, to select the preset number of groups with the highest feature quality score from the candidate terminal groups as the final terminal group division result.

[0088] Specifically, the initiating terminal also performs the operation of pre-dividing the terminal groups, including: constructing candidate terminal groups in an enumeration form according to the preset number of terminals corresponding to each group; obtaining feature distribution weights and calculating the feature quality score based on the feature distribution weights and the feature distribution of the local held verification feature data set; wherein each terminal holds feature distribution weight share information, and the feature distribution weights are reconstructed by aggregating the feature distribution weight share information held by each terminal; the feature quality score share information is obtained by processing the feature quality score through the secret sharing mechanism and distributed to each terminal in the corresponding candidate terminal group, and the feature quality score share information sent by each terminal is received; the feature quality total score share information corresponding to each candidate terminal group is calculated based on the local held feature quality score share information and the received feature quality score share information; the feature quality total score share information sent by each terminal is received, and the feature quality total score corresponding to each candidate terminal group is reconstructed; the preset number of groups with the highest feature quality total score are selected from the candidate terminal groups through the secure sorting mechanism, as the final terminal groups and the terminal group results are notified to each terminal group.

[0089] In some embodiments of the present application, based on the distribution of the features held by each terminal, the feature distribution weight is obtained to evaluate the total score of the feature quality of each candidate terminal group, which is used as an important basis for subsequent terminal grouping. By calculating the distribution of the features in different numerical intervals in each candidate terminal group, and combining the global statistical information to determine the level of each interval and the feature distribution weight, the total score of the feature quality of each group is comprehensively evaluated, so as to realize the quantitative sorting and optimal screening of the candidate terminal groups.

[0090] The feature distribution weight is obtained, including steps S1011-S1015:

[0091] In step S1011, the standard deviation of each feature in the verification feature data set is calculated and mapped to a preset interval, and a local quantity vector is constructed according to the number of features in each interval.

[0092] The preset interval refers to the distribution interval of the standard deviation. The initiating terminal calculates the number of features in the target feature data set held locally and mapped to the preset interval (for example, the preset interval includes [0, 0.1), [0.1, 0.2), [0.2, 0.3), [0.3, 0.4), [0.4, 0.5)), and a local quantity vector is constructed to indicate the distribution of the features held by the initiating terminal.

[0093] Meanwhile, the same mapping operation is also performed on the multiple participating terminals to obtain their respective local quantity vectors to indicate the distribution of the features held by each terminal.

[0094] For example, taking the preset interval as [0, 0.1), [0.1, 0.2), [0.2, 0.3), [0.3, 0.4), [0.4, 0.5), the local quantity vector generated by the initiating terminal is [10, 4, 6, 5, 6], that is, in the initiating terminal, the number of features mapped to [0, 0.1) is 10, the number of features mapped to [0.1, 0.2) is 4, the number of features mapped to [0.2, 0.3) is 6, the number of features mapped to [0.3, 0.4) is 5, and the number of features mapped to [0.4, 0.5) is 6.

[0095] In step S1012, the local quantity vector is processed by a secret sharing mechanism to obtain corresponding local quantity vector share information, which is distributed to each terminal in the candidate terminal group, and the local quantity vector share information sent by each terminal is received.

[0096] In step S1013, the global quantity vector share information is calculated based on the locally held local quantity vector share information and the received local quantity vector share information.

[0097] Step S1014, after receiving the global quantity vector share information sent by each terminal, the global quantity vector is reconstructed, and the low, medium and high level division positions are determined through dynamic search and disclosed.

[0098] The global quantity vector is used to indicate the total number of global features and the total number of features in each preset interval.

[0099] For example, refer to Figure 2 For example, the terminal group k1 includes the initiator terminal p1, the participant terminal p2 and the participant terminal p3, and the preset interval includes [0, 0.1), [0.1, 0.2), [0.2, 0.3), [0.3, 0.4), [0.4, 0.5). The local quantity vector of the initiator terminal p1 is [3, 1, 2, 0, 1], the local quantity vector of the participant terminal p2 is [2, 4, 1, 2, 0], and the local quantity vector of the initiator terminal p1 is [1, 2, 3, 1, 0]. The global quantity vector is [6, 7, 6, 3, 1].

[0100] Specifically, the low, medium and high level division positions are determined through dynamic search, including: initializing the search range, including the first search range for searching the low level and the medium level division from the first interval, and the second search range for searching the medium level and the high level division from the last interval; calculating the total number of features in the preset interval in the current search range, and determining the proportion of the total number of features in the global feature total number; in the case that the proportion is greater than or equal to the preset threshold, the interval in the current search range is divided into the corresponding level; otherwise, adjust the search range, including extending the first search range by one interval, or extending the second search range by one interval, execute the steps of calculating the total number of features in the preset interval in the current search range, and determining the proportion of the total number of features in the global feature total number, until the level division of all intervals is completed.

[0101] In actual implementation, the initiator terminal and each participant terminal can also cooperatively search and determine the low, medium and high level division positions under the secret sharing framework, and the embodiment does not limit the implementation manner of searching and determining the low, medium and high level division positions. The level division process is as follows: first, initialize the first search range S1 and the second search range S2, let S1=1, S2=1; then, all terminals cooperatively calculate the total number of features in the first S1 preset intervals in the global quantity vector under the framework of secret sharing, and compare the global feature total number; in the case that the total number of features in the first S1 preset intervals exceeds 20% of the global feature total number, S1 is taken as the division position of the low level and the medium level, otherwise S1=S1+1, continue the above process. Until the low level and the medium level division positions are determined.

[0102] In the case of determining the low-level, medium-level partition position, all terminals cooperatively calculate the feature total number in the last S2 preset intervals in the global quantity vector under the framework of secret sharing, and compare the global feature total number; in the case that the feature total number in the first S2 preset intervals exceeds 20% of the global feature total number, the position of the last S2 in the global quantity vector is taken as the medium-level, high-level partition position, otherwise S2=S2+1, and the above process is continued. Until the medium-level, high-level partition position is determined.

[0103] For example, refer to Figure 3 Taking the global quantity vector [23, 16, 11, 2, 2] as an example, the global feature total number is 54; the first search range S1 is initialized to 1, and the second search range S2 is initialized to 1; first, the feature total number 23 in the first S1 (1) preset intervals exceeds 20% of the global feature total number 52, so S1=1 is taken as the low-level and medium-level partition position, that is, the first preset interval is determined as the low-level interval; then, the feature total number 2 of the last S2 (1) preset intervals does not exceed 20% of the global feature total number 52, S2=2 is obtained, the feature total number 4 of the last S2 (2) preset intervals does not exceed 20% of the global feature total number 52, S2=3 is updated, and the feature total number 15 of the last S2 (3) preset intervals exceeds 20% of the global feature total number 52, so the third, fourth and fifth preset intervals are determined as the high-level interval, and the second preset interval is determined as the medium-level interval.

[0104] In step S1015, the feature distribution weight corresponding to each level is determined, the feature distribution weight share information is obtained by processing through the secret sharing mechanism, and is distributed to each terminal.

[0105] For example, the feature distribution weight corresponding to the low-level can be 1, the weight fraction corresponding to the medium-level can be 2, and the weight fraction corresponding to the high-level can be 3. In actual implementation, the feature distribution weight can also adopt other division forms, for example, the first interval, the second interval and the third interval are divided, and the embodiment does not limit the division form of the feature distribution weight and the value of the feature distribution weight corresponding to each feature distribution weight.

[0106] In some embodiments of the application, each terminal cooperatively calculates and determines the feature quality score corresponding to each candidate terminal group according to the local quantity vector held by itself and the public feature distribution weight, and the feature distribution weight distributed in the form of secret sharing.

[0107] For example, a candidate terminal group includes an initiating terminal P1, a participating terminal P2, and a participating terminal P3, where the local quantity vector of P1 is [3, 1, 2, 0, 1], the local quantity vector of P2 is [2, 4, 1, 2, 0], and the local quantity vector of P3 is [1, 2, 3, 1, 0]. The global quantity vector obtained by aggregation is [6, 7, 6, 3, 1]. According to the preset classification rule, the first interval is low level (weight 1), the second and third intervals are medium level (weight 2), and the fourth and fifth intervals are high level (weight 3). By summing the product of the feature quantity in each interval and the corresponding feature distribution weight, the feature quality score corresponding to the candidate terminal group is 44.

[0108] Specifically, the feature quality score is calculated based on the feature distribution weight and the feature distribution of the locally held verification feature dataset, including: multiplying the feature quantity in each preset interval by the corresponding feature distribution weight to obtain an interval feature score corresponding to each interval; and adding the interval feature scores to obtain the feature quality score.

[0109] In actual implementation, the information gain (Information Gain, IG) or entropy (Entropy) of each feature to the target variable can also be calculated as the basis of the feature quality score. For example, for a classification task, the information gain or Gini index is used, and for a regression task, the variance reduction amount is considered. This embodiment does not limit the calculation method of the feature quality score.

[0110] In step S102, in the locally divided classification cluster, the classification cluster closest to the target feature dataset is determined as the competitive classification cluster, the similarity measure between the target feature dataset and other datasets under the competitive classification cluster is calculated, and the competitive classification cluster index corresponding to the competitive classification cluster is disclosed.

[0111] In each terminal, the feature dimension of the locally held verification feature dataset is the same as the feature dimension of the locally held target feature dataset. The verification feature dataset is a vertically divided data subset corresponding to the global verification feature dataset. The feature dimensions of the verification feature datasets held by different terminals are different, but the sample spaces are consistent.

[0112] The global verification feature dataset is a complete dataset formed by vertically splicing the verification feature datasets held by each terminal, and its feature dimension covers all the features held by the terminals. The target feature dataset is a feature set selected according to the current task requirement, and its feature dimension is consistent with the global verification feature dataset, or is a subset thereof.

[0113] Since different terminal groups hold different feature dimensions, each group model has different inference capabilities on different classification clusters. For example, some terminal groups may hold features (such as income level, debt ratio, etc.) that have strong discriminative ability for a certain type of problem (such as high-risk user identification), so the inference performance on this classification cluster is better; while other terminal groups may lack these key features, resulting in weaker inference ability on this classification cluster.

[0114] Based on this, in order to improve the accuracy and efficiency of vertical federated joint inference, it is also necessary to determine the weight of the output result of each group model according to the pre-evaluated inference performance of each group model on different classification clusters, and the classification cluster corresponding to the target feature data set.

[0115] Specifically, before determining the classification cluster closest to the target feature data set as the competitive classification cluster, calculating the similarity measure between the target feature data set and other data sets under the competitive classification cluster, and disclosing the competitive classification cluster index corresponding to the competitive classification cluster, the following steps S1021 to S1027 are included:

[0116] Step S1021, based on the locally held verification feature data set, performing feature classification to divide the feature space into at least one classification cluster.

[0117] Specifically, the initiating terminal divides the feature space based on the locally held verification feature data set using a clustering algorithm (such as KMeans), obtaining a plurality of classification clusters; at the same time, the participating terminals in each terminal group also independently perform clustering operations based on their local verification feature data sets to generate classification clusters matching the local feature dimensions, thereby achieving multi-granularity division of the sample space from different feature perspectives.

[0118] For any terminal , the plurality of classification clusters obtained by division can be represented as:

[0119]

[0120] wherein, represents the th classification cluster in the terminal ; , represents the total number of classification clusters divided by the terminal .

[0121] Step S1022, generating an indicator vector according to the classification cluster to which each feature in the verification feature data set belongs, and performing vector normalization processing to obtain a normalized vector.

[0122] In some embodiments of the present invention, the classification clusters of each feature are converted into binary vector forms that are easy to calculate and compare.

[0123] Specifically, for each terminal Its validation feature dataset is divided into multiple classification clusters. For each cluster Generate an indicator vector The vector belongs to the set ,in, This represents the total number of features in a validation feature dataset.

[0124] Indicator Vector The first in Each element is represented as In the validation feature dataset, the first Features belong to classification clusters In this case, ,otherwise .

[0125] Next, in order to eliminate the impact of differences between different taxa and ensure the fairness and accuracy of subsequent calculations, it is also necessary to process each indicator vector. Normalization is performed.

[0126] Specifically, calculate each class cluster Number of samples For each indicator vector Divide it by the size of the corresponding class cluster. The normalized vector is obtained and represented as .

[0127] Step S1023: Perform local inference calculations on the verification feature dataset and inference model parameters to generate intermediate verification inference results.

[0128] Step S1024: Generate the verification feature dataset and generate the corresponding random prediction values.

[0129] In some embodiments of this invention, the initiating terminal generates random predicted values ​​corresponding to the locally held verification feature dataset, and distributes them to other terminals within the corresponding group in a secret sharing manner. These random predicted values ​​are numerical values ​​generated through a random mechanism, used to compare the effectiveness of the random guesses with the actual values ​​corresponding to the verification feature dataset, thereby more accurately measuring the effectiveness of the model.

[0130] In the vertical federal scenario, the initiating terminal first needs to determine a reasonable value range of the random prediction value. For a classification task, if the target is binary classification, a random number between 0 and 1 can be generated to represent the probability of each sample belonging to the positive class; if it is multi-classification, a random probability vector conforming to the class distribution can be generated. For a regression task, the range of the random number can be set according to the actual value range of the target variable, for example, if the range of the target variable is from 0 to 100, a random number in the same range is generated as the random prediction value.

[0131] Specifically, the random prediction value under the secret sharing can be represented as:

[0132]

[0133] In the formula, represents a prediction value vector randomly generated on the verification feature dataset held by the terminal group, in the form of secret sharing; represents the i-th value in the prediction value vector, also in the form of secret sharing.

[0134] Step S1025, the verification intermediate reasoning result, the random prediction value, the normalization vector and the real reasoning result corresponding to the verification feature dataset are processed through the secret sharing mechanism to obtain the verification share information corresponding to the verification intermediate reasoning result, the random prediction share information corresponding to the random prediction value, the vector share information corresponding to the normalization vector and the result share information corresponding to the real reasoning result.

[0135] Step S1026, the verification share information, the random prediction share information, the vector share information and the result share information are distributed to each terminal in the terminal group to which the terminal belongs, and the verification share information, the vector share information and the result share information sent by other terminals in the terminal group to which the terminal belongs are received.

[0136] Step S1027, based on the local holding and received verification share information, random prediction share information, vector share information and result share information, the reasoning ability score intermediate variable of each group model on the corresponding classification cluster is calculated; after the aggregation of the reasoning ability score intermediate variable held by each terminal, the reasoning ability score of each group model on the corresponding classification cluster is reconstructed.

[0137] In some embodiments of the application, for any terminal group, the normalization vector under the intra-group secret sharing framework can be represented as:

[0138]

[0139] The verification intermediate reasoning result under the intra-group secret sharing framework can be represented as:

[0140] ​

[0141] wherein, denotes the group model The verification intermediate inference result vector obtained by each terminal in the group model under the secret sharing framework through inference on the verification feature data set in the group can be expressed as: , denotes the number of group models or the number of terminal groups; denotes the th value.

[0142] The real inference result under the secret sharing in the group can be expressed as:

[0143]

[0144] wherein, denotes the real inference result vector corresponding to the terminal group verification feature data set, which exists in the form of secret sharing; denotes the th value in the real inference result, which also exists in the form of secret sharing.

[0145] The difference term between the random prediction value under the secret sharing, the verification intermediate inference result and the real inference result can be expressed by the following formula:

[0146]

[0147]

[0148] wherein, denotes the th value in the real inference result vector under the secret sharing; denotes the th value in the verification intermediate inference result vector under the secret sharing; denotes the th value in the random prediction value vector under the secret sharing.

[0149] The product of and the corresponding elements of , combined with the shared normalization vector corresponding to each group, cooperatively calculates the inference ability score of each group model on the classification cluster, which can be expressed by the following formula:

[0150]

[0151]

[0152] wherein, denotes the th group model in the classification cluster a reasoning capability score of the group model on the corresponding competing classification cluster; a shared normalized vector; a terminal a first classification cluster in the terminal a terminal a total number of classification clusters partitioned by the terminal a classification cluster a corresponding indication vector.

[0153] In step S103, the similarity measure and the intermediate reasoning result are processed by a secret sharing mechanism to obtain corresponding similarity measure share information and intermediate reasoning result share information, which are distributed to each terminal in the terminal group to which the terminal belongs, and the terminal receives share information sent by other terminals in the terminal group and public competing classification cluster indexes.

[0154] After the initiating terminal and each participating terminal locally determine the classification cluster to which the target feature data set belongs, the cluster index corresponding to the classification cluster is disclosed, and each determined classification cluster is regarded as a competing classification cluster.

[0155] Under the secret sharing framework, the similarity measure of the target feature data set and other data sets under the competing classification cluster can be represented by the following formula:

[0156]

[0157] In the formula, a terminal a similarity measure calculated locally by the terminal for the competing classification cluster to which the target feature data set belongs; a public competing classification cluster index; a candidate competing classification cluster, i.e., all non-own classification clusters of the target feature data set; a terminal a total number of candidate competing classification clusters considered by the terminal; a distance of the target feature data set to each candidate competing classification cluster.

[0158] In step S104, according to the public competing classification cluster indexes, the similarity measure share information and the intermediate reasoning result share information held by each terminal are aggregated to reconstruct the complete similarity measure of each group model on the corresponding competing classification cluster and the complete reasoning output corresponding to each group model.

[0159] In step S105, the reasoning capability scores of each group model on the corresponding competing classification cluster are weighted and fused by the complete similarity measure to obtain reasoning capability evaluation values corresponding to each group model, which are mapped and processed by a preset nonlinear amplification function to obtain model output weights.​​

[0160] wherein the complete similarity measure is multiplied by the inference ability score of each group model on the corresponding competitive classification cluster respectively, to obtain the inference ability evaluation value of each group model corresponding to each group model, which can be expressed by the following formula:

[0161]

[0162] In the formula, terminal grouping or group model corresponding inference ability evaluation value, in the form of secret sharing; denotes the number of terminals in the terminal grouping; denotes the terminal in the terminal grouping target feature data set in the formula between the target feature data set and the classification cluster to which it belongs; denotes the inference ability evaluation value of the group model on the classification cluster ; denotes the classification cluster index.

[0163] In some embodiments of the present application, the preset nonlinear amplification function is a cubic function constructed in advance by Lagrange interpolation method.

[0164] Specifically, the model output weight is obtained by mapping processing through the preset nonlinear amplification function, including: calculating the mean of the inference ability evaluation values of all group models; calculating the ratio of the inference ability evaluation value of each group model to the mean;

[0165] The ratio and the corresponding inference ability evaluation value are taken as the input of the preset nonlinear amplification function, to obtain the model output weight corresponding to each group model.

[0166] Specifically, the model output weight can be expressed by the following formula:

[0167]

[0168] In the formula, is a cubic function constructed by Lagrange interpolation method, expressed as ; is the mean of all inference ability evaluation values, expressed as , wherein denotes the number of group models, denotes the inference ability evaluation value of the th group model.

[0169] Step S106, the complete inference output of each group model is weighted and summed using the model output weight, to obtain the final output result.

[0170] Specifically, the final output result can be represented by the following formula:

[0171]

[0172] In the formula, represents the final output result, in the form of secret sharing; represents the inference ability evaluation value corresponding to the th group model, in the form of secret sharing; represents the complete inference output output by the th group model, in the form of secret sharing.

[0173] In summary, the privacy protection-based longitudinal federated joint inference method provided in the embodiment can effectively solve the problem of sharp rise in inference delay and significant reduction in inference efficiency caused by the square relationship between the communication and calculation complexity of the secret sharing mechanism and the number of participants in a multi-party participation setting. The parallel inference of multiple group models avoids the large amount of encryption calculation and communication overhead caused by the dependence on a single global model in the traditional method. By distributing the inference task to each terminal group for execution and only aggregating the weighted output result in the last stage, the high-cost operation frequency under the secret sharing framework is effectively reduced, thereby significantly improving the overall inference efficiency, especially for large-scale application scenarios with multiple participants. At the same time, the classification cluster division mechanism is introduced, and the similarity between the features and the local classification clusters is combined to dynamically evaluate the matching degree of the features and the data distribution of each terminal group. On this basis, the inference ability evaluation values of each group model on the corresponding classification cluster are further fused to generate personalized model output weights, so that the final output result is more consistent with the actual data distribution, and the generalization ability and output accuracy of the model are improved.

[0174] In one example, the privacy protection-based longitudinal federated joint inference method provided in the present application is simulated and experimented in terms of efficiency and accuracy. The data set information used is shown in Table 1.

[0175]

[0176] Table 1

[0177] Uniform feature engineering is performed on each real data set, including one-hot encoding for discrete features and maximum-minimum normalization for continuous features, and the data set is divided into 60% training set, 20% validation set and 20% test set.

[0178] In a multi-party collaboration scenario, the feature set of each data set is evenly divided into subsets before feature engineering, wherein each subset is held by a participant. At the same time, one party holds the real label set.

[0179] The list of parameters involved is shown in Table 2:

[0180]

[0181] Table 2

[0182] For the comparative method, the number of iterations and the tree depth are set to 200 and 10, respectively. In contrast, the number of iterations and the tree depth under the framework of the present application are set to be less than those in the comparative method. This is because the group model is trained using only a part of the dataset, so the model complexity of the group model can be lower than that of the model trained using the complete dataset.

[0183] In addition, all input data are converted to 128-bit fixed-point numbers, of which 10 bits are the decimal part. The comparative method involved is a single-model federated learning method, i.e., all participants collaboratively train one machine learning model using secret sharing. Both the framework of the present application and the comparative method are implemented using SPD semi-honest version of the framework.

[0184] Specifically, four classical machine learning models are selected for performance testing, namely linear regression (LR1), logistic regression (LR2), CART regression tree (RT), and CART classification tree (CT).

[0185] Among them, LR1 and LR2 are both linear models (Linear Model, LM), which find a set of parameters to approximate the mapping function between input features and real labels through optimization algorithms; while RT and CT are both decision trees (Decision Tree, DT), which construct binary trees to realize the mapping of input features to real labels by recursively partitioning the dataset using Gini impurity. In particular, LR1 and RT are used for regression tasks, while LR2 and CT are used for classification tasks. The training and inference protocols of these models are derived from existing works.

[0186] The evaluation metrics involved are as follows. For model accuracy, the Area Under Curve (AUC) and F1 score are used in classification tasks, while the Mean Absolute Error (MAE) is used in regression tasks. In particular, AUC and F1 score provide different perspectives on evaluating classification model accuracy. AUC is a widely used metric for assessing overall discriminative ability across various probability thresholds. In contrast, F1 score takes into account the performance of the model in terms of precision and recall, making it suitable for class- imbalanced scenarios. For training and inference efficiency, this paper not only counts the online running time (in minutes), but also counts the number of OT primitives introduced by the offline generation of Beaver triples (in billions). In particular, when evaluating the training efficiency of our framework, the competitive pre-evaluation is taken into account.

[0187] Table 3 shows the comparison results of the efficiency and accuracy of the framework of the present application and the comparative method. As shown in Table 3:

[0188]

[0189] Table 3

[0190] In particular, considering that the training efficiency and model accuracy of the proposed framework are a trade-off relationship, a set of parameter settings are selected on each dataset to make the model accuracy of the proposed framework closest to the comparative method, so as to intuitively reflect the advantage of the proposed framework in efficiency under the condition of obtaining the same model accuracy.

[0191] Table 4 gives the parameter settings:

[0192]

[0193] Table 4

[0194] It can be observed that the framework proposed in this application can significantly improve the training efficiency of LM and DT. This is not only due to the reduction of the computation and communication overhead of each secret sharing of the computing primitive by the task decomposition idea, but also due to the smaller training data size and model complexity of each base model relative to the complete model based on all training data. At the same time, it can be observed that in terms of inference efficiency, the framework proposed in this application performs significantly better on DT, but weaker on LM. The reason for this result can be explained as follows. The inference cost of the base model on one sample is smaller than that of the complete model, but the dynamic integration of the base model will produce some additional computation and communication overhead. Since the inference time of DT is longer, these additional overheads will not have a significant impact on the inference efficiency of DT. The case of LM is the opposite of DT. In terms of model accuracy, it can be observed that compared with the comparative method, the framework proposed in this application can have both efficiency advantage and close to or even higher prediction accuracy in most cases. This is due to the efforts in group generation and base model aggregation. On the one hand, a group of base models with low generalization error can be trained; on the other hand, when aggregating the outputs of the base models, the weights of the base models that can make correct predictions can be dynamically improved, thereby improving the prediction performance after aggregation.

[0195] Further, we respectively count the running time of the three key stages of group generation, base model training and competitiveness pre-evaluation.

[0196] Table 5 lists the online running time of these stages under different settings. It can be observed that the online running time required by group generation and competitiveness pre-evaluation is significantly lower than that of base model training, indicating that these two stages will not have a significant impact on the training efficiency of the proposed framework.

[0197] As shown in Table 5:

[0198]

[0199] Table 5

[0200] Corresponding to the above method, the application also provides a privacy protection based longitudinal federal joint inference system, which comprises a computer device, the computer device comprises a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory, when the computer instructions are executed by the processor, the system realizes the steps of the privacy protection based longitudinal federal joint inference method in the above method embodiment.

[0201] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the privacy-preserving vertical federated joint inference method described in the above-described method embodiments. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0202] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0203] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0204] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0205] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A privacy protection based longitudinal federated federated inference method, characterized in that, The method comprises the following steps: The method comprises the following steps: In the locally divided classification cluster, the classification cluster closest to the target feature data set is determined as the competition classification cluster, the similarity measure between the target feature data set and other data sets under the competition classification cluster is calculated, and the competition classification cluster index corresponding to the competition classification cluster is disclosed; The similarity measure, the intermediate reasoning result are processed through the secret sharing mechanism to obtain corresponding similarity measure share information and intermediate reasoning result share information, which are distributed to each terminal in the terminal group, and the share information sent by other terminals in the terminal group and the disclosed competition classification cluster index are received; According to the disclosed competition classification cluster index, the similarity measure share information and the intermediate reasoning result share information held by each terminal are aggregated to reconstruct the complete similarity measure of each group model on the corresponding competition classification cluster and the complete reasoning output corresponding to each group model; The reasoning ability score of each group model on the corresponding competition classification cluster is weighted and fused through the complete similarity measure to obtain the reasoning ability evaluation value corresponding to each group model, and the model output weight is obtained through the mapping processing of the preset nonlinear amplification function; The complete reasoning output of each group model is weighted and summed using the model output weight to obtain the final output result.

2. The method of claim 1, wherein, Before determining the classification cluster closest to the target feature data set as the competition classification cluster, calculating the similarity measure between the target feature data set and other data sets under the competition classification cluster, and disclosing the competition classification cluster index corresponding to the competition classification cluster, the method further comprises the following steps: Based on the locally held verification feature data set, the feature classification is performed to divide the feature space into at least one classification cluster; the verification feature data set and the target feature data set have the same sample space and the same feature dimension; According to the classification cluster to which each feature in the verification feature data set belongs, an indication vector is generated and vector normalization processing is performed to obtain a normalized vector; Local reasoning calculation is performed on the verification feature data set and the reasoning model parameter to generate a verification intermediate reasoning result; A corresponding random prediction value is generated for the verification feature data set; The verification intermediate reasoning result, the random prediction value, the normalized vector, and the true reasoning result corresponding to the verification feature data set are processed through the secret sharing mechanism to obtain verification share information corresponding to the verification intermediate reasoning result, random prediction share information corresponding to the random prediction value, vector share information corresponding to the normalized vector, and result share information corresponding to the true reasoning result; distributing the verification share information, the random prediction share information, the vector share information and the result share information to each terminal in the terminal group to which the terminal belongs, and receiving the verification share information, the vector share information and the result share information sent by other terminals in the terminal group; based on the locally held and received verification share information, the random prediction share information, the vector share information and the result share information, calculating an inference ability score intermediate variable of each group model on the corresponding classification cluster; after the inference ability score intermediate variables held by each terminal are aggregated, the inference ability score of each group model on the corresponding classification cluster is reconstructed.

3. The method of claim 1, wherein, pre-dividing the terminal group, comprising: constructing candidate terminal groups in an enumerated form according to the preset terminal quantity corresponding to each group; obtaining feature distribution weights, and calculating a feature quality score based on the feature distribution weights and the feature distribution of the locally held verification feature dataset; wherein each terminal holds feature distribution weight share information, and the feature distribution weights are reconstructed after aggregating the feature distribution weight share information held by each terminal; processing the feature quality score through a secret sharing mechanism to obtain feature quality score share information, distributing the feature quality score share information to each terminal in the candidate terminal group to which the terminal belongs, and receiving the feature quality score share information sent by each terminal; based on the locally held and received feature quality score share information, calculating share information corresponding to a feature quality total score corresponding to each candidate terminal group; receiving the feature quality total score share information sent by each terminal, reconstructing the feature quality total score corresponding to each candidate terminal group, selecting a preset number of groups with the highest feature quality total score from the candidate terminal groups through a secure sorting mechanism, and taking the groups as the final terminal groups and notifying each terminal group of the result.

4. The method of claim 3, wherein, the obtaining of the feature distribution weights comprises: calculating the standard deviation of each feature in the verification feature dataset and mapping to a preset interval, and constructing a local quantity vector according to the number of features in each interval; processing the local quantity vector through a secret sharing mechanism to obtain corresponding local quantity vector share information, distributing the local quantity vector share information to each terminal in the candidate terminal group to which the terminal belongs, and receiving the local quantity vector share information sent by each terminal; based on the locally held and received local quantity vector share information, calculating global quantity vector share information; after receiving the global quantity vector share information sent by each terminal, reconstructing the global quantity vector, determining the division positions of the low, medium and high levels through dynamic search and publicizing; determining the feature distribution weights corresponding to each level, processing the feature distribution weights through a secret sharing mechanism to obtain feature distribution weight share information, and distributing the feature distribution weight share information to each terminal.

5. The method of claim 4, wherein, the determination of the division positions of the low, medium and high levels through dynamic search comprises: initializing a search range, including a first search range for finding the division of the low level and the medium level from the first interval, and a second search range for finding the division of the medium level and the high level from the last interval; Calculate the total number of features in a preset interval in the current search range, and determine the proportion of the total number of features in the global feature distribution; If the proportion is greater than or equal to a preset threshold, then the interval in the current search range is divided into corresponding levels; otherwise, adjust the search range, including extending the first search range by one interval or extending the second search range by one interval, execute the steps of calculating the total number of features in a preset interval in the current search range, and determining the proportion of the total number of features in the global feature distribution, until all interval level divisions are completed.

6. The method of claim 4, wherein, The feature quality score is calculated based on the feature distribution weight and the feature distribution of the locally held verification feature dataset, including: Multiply the number of features in each preset interval by the corresponding feature distribution weight to obtain the interval feature score corresponding to each interval; Add the interval feature scores to obtain the feature quality score.

7. The method of claim 1, wherein, The preset nonlinear amplification function is a cubic function constructed in advance by Lagrange interpolation method; The mapping processing by the preset nonlinear amplification function obtains the model output weight, including: Calculate the mean of the inference ability evaluation values of all group models; Calculate the ratio of the inference ability evaluation value of each group model to the mean; The ratio and the corresponding inference ability evaluation value are used as the input of the preset nonlinear amplification function to obtain the model output weight corresponding to each group model.

8. The method of claim 1, wherein, Each group model is obtained by pre-training of each terminal group in a self-supervised form and a preset inference task; the preset inference task includes a feature classification task of generating a classification label or a regression task of generating an output value. 9.A privacy-preserving longitudinal federated federated inference system comprising a processor, a memory, and a computer program / instructions stored on the memory, wherein, The processor is used to execute the computer program / instruction, and when the computer program / instruction is executed, the system implements the steps of the method in any one of claims 1 to 8.

10. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the steps of the method in any one of claims 1 to 8.

Images