Information processing method, apparatus, device, storage medium, and program product

Abstract

The application discloses an information processing method, device and equipment, a storage medium and a program product. The method is applied to a first participant in a plurality of participants participating in federated learning, and the first participant holds a label. The method comprises the following steps: performing feature extraction on each data provided by the first participant locally to obtain a first feature vector of each data provided by the first participant; receiving a second feature vector of each data sent by a second participant in the plurality of participants, wherein the second feature vector is obtained by the second participant through local self-supervised training; wherein the first feature vector and the second feature vector are feature data of aligned samples, and one participant corresponds to one type of feature vector; and determining parameters for updating a federated learning model and a local model of each participant in the plurality of participants according to each first feature vector and each second feature vector. The application can effectively realize model updating and improve communication efficiency.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to an information processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] Vertical federated learning involves taking the users and data that have the same participants but different user data features and using them for joint machine learning training when there is little overlap in the data features of the participants but a lot of overlap in the users.

[0003] However, in vertical federated learning applications, there are scenarios where only one participant has a label, while the others do not. In practical applications, traditional deep learning methods cannot directly update model parameters, and traditional vertical federated learning architectures typically require a large amount of information synchronization and interaction during training.

[0004] Therefore, existing technologies cannot effectively update models and have low communication efficiency. Summary of the Invention

[0005] The main objective of this invention is to provide an information processing method, apparatus, device, storage medium, and program product, which aims to effectively achieve model updates and improve communication efficiency.

[0006] To achieve the above objectives, the present invention provides an information processing method, which is applied to a first participant among multiple participants in federated learning, the first participant holding a tag; the method includes:

[0007] After performing self-supervised training on each piece of data provided by the first participant locally, feature extraction is performed to obtain the first feature vector of each piece of data provided by the first participant.

[0008] The system receives the second feature vector of each data sent by the second participant among the plurality of participants. The second feature vector is obtained by the second participant through self-supervised training on its local machine. The first feature vector and the second feature vector are feature data of aligned samples, and each participant among the plurality of participants corresponds to a class of feature vectors.

[0009] Based on each of the first feature vectors and each of the second feature vectors, parameters are determined for updating the federated learning model and the local models of each of the plurality of participants, wherein the federated learning model is used to support information interaction among the plurality of participants.

[0010] Optionally, based on each of the first feature vectors and each of the second feature vectors, parameters for updating the federated learning model and the local models of each of the plurality of participants are determined, including:

[0011] Each of the first feature vectors is used as a sample of the first distribution, and each of the second feature vectors is used as a sample of the second distribution. A sample of the first distribution and a corresponding sample of the second distribution are used as a set of samples.

[0012] For each distribution of samples, the maximum mean difference distance of the samples in each distribution is determined by using the mapping function set on the sample space.

[0013] If the maximum mean difference distance is less than or equal to a preset distance threshold, then the parameters for updating the federated learning model and the local models of each participant are determined based on the samples of each distribution.

[0014] If the maximum mean difference distance is greater than a preset distance threshold, then based on the samples of each distribution and the corresponding maximum mean difference distance, the parameters for updating the federated learning model and the local models of each participant are determined, and a first target parameter and a second target parameter are determined. The first target parameter is used to update the local model of the first participant, and the second target parameter is used to update the local model of the second participant.

[0015] Optionally, by using a set of mapping functions on the sample space, the maximum mean difference distance of samples from each distribution on the mapping function set is determined, including:

[0016] For each mapping function in the set of mapping functions, based on the samples of the first distribution and the samples of the second distribution in each set of distributions, calculate the first mean of the function values ​​of the samples of the first distribution on the mapping function and the second mean of the function values ​​of the samples of the second distribution on the mapping function, respectively.

[0017] The mean difference distance on the mapping function is calculated based on the first mean and the second mean.

[0018] Obtain the maximum mean difference distance from the mean difference distances corresponding to each of the mapping functions.

[0019] Optionally, based on the samples from each distribution, parameters are determined for updating the federated learning model and the local models of each participant, including:

[0020] For each distribution of samples, the first distribution sample in the group is merged with the second distribution sample to obtain a new sample;

[0021] Based on each new sample and its corresponding label, parameters for updating the federated learning model and the local models of each participant are determined.

[0022] Optionally, determining the parameters for updating the federated learning model and the local models of each participant based on the samples of each distribution and the corresponding maximum mean difference distance, and determining the first target parameter and the second target parameter, includes:

[0023] For each group of samples, the parameters of the local model of the first participant are adjusted according to the maximum mean difference distance to obtain the first parameter;

[0024] The maximum mean difference distance is sent to the second participant, so that the second participant can adjust the parameters of the second participant's local model according to the maximum mean difference distance to obtain the second parameter;

[0025] For each distribution of samples, the first distribution sample in the group is merged with the second distribution sample to obtain a new sample;

[0026] Based on each new sample and the label corresponding to the new sample, parameters for updating the federated learning model and the local model of each participant are determined and sent to the second participant, so that the second participant determines the second target parameter based on the parameters for updating the federated learning model and the local model of each participant and the second parameter.

[0027] The first target parameter is determined based on the parameters used to update the federated learning model and the local models of each participant, as well as the first parameter.

[0028] Optionally, determining the parameters for updating the federated learning model and the local models of each participant based on the samples of each distribution and the corresponding maximum mean difference distance, and determining the first target parameter and the second target parameter, includes:

[0029] For each distribution of samples, the parameters of the local model of the first participant are adjusted according to the maximum mean difference distance to obtain the first target parameters and the corresponding new local model of the first participant. The first new distribution sample corresponding to the first distribution sample in the group is obtained through the new local model of the first participant. The new local model of the first participant is used to make the maximum mean difference distance less than the preset distance threshold.

[0030] The maximum mean difference distance is sent to the second participant, so that the second participant adjusts the parameters of the second participant's local model according to the maximum mean difference distance to obtain the second target parameters and the corresponding new local model of the second participant. The second new distribution sample corresponding to the second distribution sample in the group is obtained through the new local model of the second participant. The new local model of the second participant is used to make the maximum mean difference distance less than the preset distance threshold.

[0031] Receive the second new distribution sample sent by the second participant, and merge the first new distribution sample corresponding to the group with the second new distribution sample to obtain a new sample;

[0032] Based on each new sample and its corresponding label, parameters for updating the federated learning model and the local models of each participant are determined.

[0033] Optionally, based on each of the new samples and the labels corresponding to the new samples, parameters for updating the federated learning model and the local models of each of the multiple participants are determined, including:

[0034] Based on each of the new samples, a prediction result is obtained through a federated learning model;

[0035] Based on the prediction results, the label corresponding to the new sample, and the maximum mean difference distance corresponding to the new sample, the gradient of the loss function with respect to the intermediate layer output is determined;

[0036] The gradient of the loss function with respect to the intermediate layer output is used as a parameter for updating the federated learning model and the local models of each of the multiple participants. Optionally, the method further includes:

[0037] Based on the parameters used to update the federated learning model and the local model of each of the multiple participants, the federated learning model and the local model of the first participant are updated using the gradient descent algorithm.

[0038] The parameters used to update the federated learning model and the local models of each of the multiple participants are sent to the second participant so that the second participant can update its own local model.

[0039] Optionally, the method further includes:

[0040] Obtain multiple positive samples and multiple sub-samples corresponding to each positive sample;

[0041] Based on multiple positive samples and multiple sub-samples corresponding to each positive sample, a feature extraction model is obtained through self-supervised training.

[0042] The feature extraction model is used to extract the first feature vector of each data provided by the first participant.

[0043] The present invention also provides an information processing apparatus, which is applied to a first participant among multiple participants in federated learning, the first participant holding a tag; the apparatus includes:

[0044] The feature extraction module is used to extract features from each piece of data provided by the first participant after performing self-supervised training locally, so as to obtain the first feature vector of each piece of data provided by the first participant.

[0045] The receiving module is used to receive the second feature vector of each data sent by the second participant among the plurality of participants. The second feature vector is obtained by the second participant through self-supervised training on its local machine. The first feature vector and the second feature vector are feature data of aligned samples, and each participant among the plurality of participants corresponds to a class of feature vectors.

[0046] The processing module is configured to determine, based on each of the first feature vectors and each of the second feature vectors, the parameters for updating the federated learning model and the local models of each of the plurality of participants, wherein the federated learning model is used to support information interaction among the plurality of participants.

[0047] The present invention also provides an information processing device, the information processing device comprising: a memory, a processor, and an information processing program stored in the memory and executable on the processor, wherein the information processing program, when executed by the processor, implements the steps of the information processing method as described in any of the preceding claims.

[0048] The present invention also provides a computer-readable storage medium storing an information processing program, which, when executed by a processor, implements the steps of the information processing method as described in any of the preceding claims.

[0049] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0050] In this invention, the first labeled participant among the multiple participants in federated learning can process information. By performing self-supervised training on the data provided by the first participant locally and then extracting features, a first feature vector is obtained for each data provided by the first participant, and a second feature vector is obtained for each data sent by the second participant among the multiple participants. The second feature vector is obtained by the second participant through local self-supervised training. Based on the first and second feature vectors, parameters for updating the federated learning model and the local models of each participant among the multiple participants are determined. Federated learning is then achieved through information interaction among the multiple participants. Therefore, by employing self-supervised training to extract data features from unlabeled or labeled participants locally, feature vectors for each participant are obtained. Since each participant trains locally and, based on the feature vectors, fine-tuning through vertical federated learning yields the model update parameters, resulting in their respective final prediction models. This effectively achieves model updates and requires only a small number of communication interactions, reducing communication costs and improving communication efficiency in vertical federated learning scenarios. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present invention;

[0052] Figure 2 A flowchart illustrating an information processing method provided in an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of self-supervised training provided in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the structure of an information processing device provided in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the structure of an information processing device provided in an embodiment of the present invention.

[0056] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the invention to those skilled in the art.

[0058] In vertical federated learning applications, there exist scenarios where only one participant has a label, while the others do not. In practical applications, traditional deep learning methods cannot directly obtain model parameters (such as gradient information) for updates, and traditional vertical federated learning architectures typically require extensive information synchronization and interaction during training. Therefore, existing technologies cannot effectively achieve model updates and suffer from low communication efficiency.

[0059] To address this issue, the technical concept of this invention is to employ self-supervised learning, enabling each participant to autonomously learn a better feature extraction model without requiring labels. This fully utilizes the data from each party, especially in scenarios where both parties have limited labeled data or different target labels. Based on the self-supervised learned model, when each participant performs fine-tuning through vertical federation, only a small number of communication interactions are needed to obtain a better model, reducing communication costs and improving communication efficiency in vertical federation scenarios.

[0060] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features thereof can be combined with each other.

[0061] It should be noted that the term "multiple" in this application is used to mean two or more, and the specific number is not specifically limited here. For example, multiple participants can be two or more participants. Since multiple participants include a first participant and a second participant, the second participant among the multiple participants can be one or more.

[0062] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of the present invention. For example... Figure 1 As shown, two entities act as participants in the information processing, denoted as the first participant and the second participant (i.e., the second participant among multiple participants). For example, the first participant could be a bank, and the second participant could be an e-commerce platform. Both the first and second participants hold information on multiple users, and they are located in the same region. A natural person can be considered a user, or a bank account can be considered a user, and an e-commerce account can be considered a user.

[0063] Specifically, in combination Figure 1As shown in steps ①, ②, and ③, the users held by the first participant and the users held by the second participant are aligned. For example, if the first participant has information on 10 users and the second participant has information on 8 users, the user information of the two participants is aligned by selecting the same users: the first participant has information on users 1 to 5 and holds labels, and the second participant also has information on users 1 to 5. Each participant extracts the feature vectors corresponding to the user information through local self-supervised training. That is, the first participant extracts the feature vectors U corresponding to the information of users 1 to 5 through its feature extractor A. A (Using the first feature vector, where the user information of the first participant is represented by X) A This means that each user corresponds to a first feature vector U. A The second participant extracts the feature vectors U corresponding to the information of users 1 to 5 through its feature extractor B. B (Using the second feature vector, where the user information of the second participant is represented by X) B This means that each user corresponds to a second feature vector U. B ).

[0064] The second participant will assign each user's corresponding U B Transmitted to the first participant, the first participant will use U A and U B Data fusion is performed, such as combining or concatenating User-Agent (UA) and User-Binder (UB), and then the predicted result Y is obtained through a federated learning model (such as a classifier). pred And calculate the loss based on the prediction results and the label Y held by the first participant. The gradient of the loss function at the intermediate layer output is transmitted to the second participant. The first participant updates the classifier and Extractor A using gradient descent, while the second participant updates Extractor B using the chain rule and gradient descent. Through fine-tuning iterations, the final federated learning model is determined.

[0065] The information processing method provided in this invention allows for the extraction of data features from unlabeled or labeled participants through self-supervised training, yielding feature vectors for each participant. Since each participant trains locally, the self-supervised training process does not involve any communication. Furthermore, based on the feature vectors, the model's update parameters can be obtained through fine-tuning via vertical federated learning, leading to the final prediction model for each participant. This effectively achieves model updates and requires only a small number of communication interactions, reducing communication costs and improving communication efficiency in vertical federated learning scenarios.

[0066] Figure 2 This is a flowchart illustrating an information processing method provided in an embodiment of the present invention. The method is applied to the first participant among multiple participants in federated learning, such as... Figure 2 As shown, the method in this embodiment may include:

[0067] S201. After performing self-supervised training on each data provided by the first participant locally, feature extraction is performed to obtain the first feature vector of each data provided by the first participant.

[0068] S202, Receive the second feature vector of each data sent by the second participant among the plurality of participants, wherein the second feature vector is obtained by the second participant through local self-supervised training; wherein the first feature vector and the second feature vector are feature data of aligned samples, and each participant among the plurality of participants corresponds to a class of feature vectors.

[0069] The second participant may hold tags (e.g., a small number of tags) or no tags.

[0070] Each participant provides a type of feature vector. If we consider k participants, there are k types of feature vectors. For example, the participants include a first participant and m second participants. The first participant holds a label, and the m second participants may hold labels (e.g., a few labels) or have no labels. The first participant corresponds to the first feature vector, and each of the m second participants corresponds to a second feature vector, such as second feature vector 1, second feature vector 2, ..., second feature vector m.

[0071] The parameters for updating the federated learning model and the local models of each participant among the multiple participants are determined based on the first feature vector and the second feature vector. This can be achieved by analyzing the first feature vector and each of the first feature vectors (e.g., second feature vector 1, second feature vector 2, ..., second feature vector m) to determine the update parameters. Specifically, for each data point, the first feature vector and its matching second feature vectors are the aligned sample data.

[0072] The second participant is exemplified below. The information processing method is described in detail using the example of the second participant among multiple participants being a single participant without a label.

[0073] In a vertical federation scenario, each party possesses its own data features. Except for the labeled party, all other participants are unlabeled. Unlabeled participants cannot undergo supervised training. To fully utilize the data from unlabeled participants, each participant can employ self-supervised learning methods to learn an effective feature extraction model locally.

[0074] Specifically, both the first participant (labeled Client A) and the second participant (labeled Client B) can obtain feature extraction models through self-supervised local model training. For example, Client A's feature extraction model (or feature extractor) is Extractor A, and Client B's feature extraction model (or feature extractor) is Extractor B. Client A has both data and labels, while Client B only has data. Client A and Client B each have their own feature extractors, Extractor A and Extractor B, which output feature U respectively. A and U B Client B will output U B The data is transmitted to Client A in an encrypted manner. Furthermore, Client A has a classifier that operates on U. A and U B This generates prediction results.

[0075] In this process, each participant initializes its own model, Extractor A and Extractor B. Before self-supervised training, Client A and Client B align their data using encryption.

[0076] Alternatively, the training process of the feature extraction model can be achieved through the following steps:

[0077] Step a1: Obtain multiple positive samples and multiple sub-samples corresponding to each positive sample.

[0078] Step a2: Based on multiple positive samples and multiple sub-samples corresponding to each positive sample, a feature extraction model is obtained through self-supervised training.

[0079] The feature extraction model is used to extract the first feature vector of each data provided by the first participant.

[0080] Accordingly, Client B can train Extractor B through the above steps.

[0081] Specifically, the process of each participant training the feature extractor locally under their own supervision can be as follows:

[0082] The structure of a self-supervised training method is as follows: Figure 3As shown, a Query[q] (representing the image features, i.e., feature vectors, of an image) and a queue[k] (representing the image features, i.e., feature vectors, of multiple augmented images) are defined. This queue typically contains a single positive sample (e.g., an original image) and multiple negative samples (e.g., augmented images). Among the multiple negative samples, one matches the positive sample, while the others can be different augmented images. Feature representations are learned using contrastive loss. A queue is used to store and sample negative samples, containing multiple recently used batches of feature vectors for training. The queue is continuously updated; as a new batch of training feature vectors is added, the oldest batch is dequeued. Here, what is added to the queue is not the image itself, but rather the image features.

[0083] During training, the target sample (i.e., the Anchor sample) is denoted as [x]. q ](query), after passing through the encoder network (i.e., the encoder network)[f q Encoding yields q = f q (x q Subsequently, K+1 samples were sampled from the queue {k 0 ,k 1 ,k 2 ...} are used as keys. These keys are used with different queue encoder networks f k It is obtained through encoding. Because f k The change is very slow, therefore although {k 0 ,k 1 ,k 2 , ...} are encoded by different encoders, and the differences caused by the encoders are very small. For example, this embodiment uses a momentum update method to update f. k Its parameter is a smooth copy of the Query Encoder:

[0084] θ k =mθ k +(1-m)θ q

[0085] Assuming there is one positive sample k+ and the rest are negative samples, then according to the lnfoNCE (loss function), the loss is expressed as:

[0086]

[0087] Where q and k +There are multiple ways to construct it. For example, this embodiment uses a relatively simple method: (1) randomly resize an image; (2) perform two random cropping operations of 224*224 to obtain two images as q and k respectively. + (3) Perform enhancement operations, including random colorjittering, random horizontal flip, random grayscale conversion, etc.

[0088] The encoder network can select any deep learning network as the encoder network based on the characteristics of the participants and the characteristics of the task, and generate the final feature extractor, such as ResNet.

[0089] S203. Based on each of the first feature vectors and each of the second feature vectors, determine the parameters for updating the federated learning model and the local models of each of the multiple participants, wherein the federated learning model is used to support information interaction among the multiple participants.

[0090] As demonstrated by self-supervised training, each party initializes its own model extractor, which is a feature extractor that has been fully learned from local data and possesses excellent feature extraction capabilities. Therefore, each participant only needs to communicate minimally with the party holding the label to fine-tune the model, resulting in a better final model. By employing self-supervised training to learn and extract data features locally for both unlabeled and labeled participants, feature vectors for each participant are obtained. Since each participant learns and trains locally, the self-supervised training process does not generate any communication. Based on the feature vectors, the updated parameters of the model can be obtained through fine-tuning via vertical federated learning, leading to their respective final prediction models. This effectively achieves model updates with only a small number of communication interactions, reducing communication costs and improving communication efficiency in vertical federated learning scenarios.

[0091] Optionally, determining the parameters for updating the federated learning model and the local models of each of the plurality of participants, based on each of the first feature vectors and each of the second feature vectors, can be achieved through the following steps:

[0092] Step b1: Take each of the first feature vectors as samples of the first distribution, and take each of the second feature vectors as samples of the second distribution, and take a sample of the first distribution and a corresponding sample of the second distribution as a set of samples.

[0093] Step b2: For each distribution of samples, determine the maximum mean difference distance of the samples in the mapping function set on the sample space.

[0094] Step b3: If the maximum mean difference distance is less than or equal to a preset distance threshold, then determine the parameters used to update the federated learning model and the local models of each participant based on the samples of each distribution.

[0095] Step b4: If the maximum mean difference distance is greater than a preset distance threshold, then based on the samples of each distribution and the corresponding maximum mean difference distance, determine the parameters for updating the federated learning model and the local models of each participant, and determine the first target parameter and the second target parameter. The first target parameter is used to update the local model (i.e., the feature extraction model) of the first participant, and the second target parameter is used to update the local model of the second participant.

[0096] In this approach, each feature vector is treated as a sample of a distribution, and the first and second feature vectors can be viewed as samples from two different distributions. Since the samples corresponding to the first feature vectors of the first participant and the second feature vectors of the second participant are aligned, the first feature vector and its corresponding (e.g., the same user) second feature vector are considered as a set of samples from two distributions.

[0097] Then, for each set of samples, the maximum mean difference distance on the mapping function set is calculated, and the federated learning model and the local models of each participant are fine-tuned based on the maximum mean difference distance and the samples of each set of samples.

[0098] Specifically, taking a set of samples from different distributions as an example, if the maximum mean difference distance is less than or equal to a preset distance threshold, it means that the samples from the two distributions in the set are the same. Based directly on the samples from this set of distributions, the parameters used to update the federated learning model and the local models of each participant are determined.

[0099] If the maximum mean difference distance is greater than a preset distance threshold, it indicates that the sample distributions of the two distributions in this group are not the same, and their feature spaces differ significantly. Directly concatenating or adding the features of both distributions will reduce the representational power of the features, and using these features to train a classifier will also reduce the model's discriminative ability. Therefore, to accelerate convergence and improve model performance, the samples of this group of distributions can be combined with the maximum mean difference distance to update their respective local models and federated learning models. Optionally, the maximum mean difference distance of samples from each group of distributions on the mapping function set in the sample space can be determined through the following steps:

[0100] Step c1: For each mapping function in the set of mapping functions, based on the samples of the first distribution and the samples of the second distribution in each set of distributions, calculate the first mean of the function values ​​of the samples of the first distribution on the mapping function and the second mean of the function values ​​of the samples of the second distribution on the mapping function, respectively.

[0101] Step c2: Calculate the mean difference distance on the mapping function based on the first mean and the second mean.

[0102] Step c3: Obtain the maximum mean difference distance from the mean difference distances corresponding to each of the mapping functions.

[0103] Optionally, for each set of distributions, if the maximum mean difference distance is less than a preset distance threshold, then the sample distributions of the two distributions in that set are determined to be the same; otherwise, the sample distributions of the two distributions in that set are determined to be different. If the distributions are different, to accelerate model convergence or reduce the number of interactions between devices, this can be achieved through at least the following two methods:

[0104] Method 1: Based on the samples of each distribution and the corresponding maximum mean difference distance, determine the parameters used to update the federated learning model and the local models of each participant, as well as determine the first target parameter and the second target parameter. This can be achieved through the following steps:

[0105] Step d1: For each group of samples, adjust the parameters of the local model of the first participant according to the maximum mean difference distance to obtain the first parameter;

[0106] Step d2: Send the maximum mean difference distance to the second participant, so that the second participant can adjust the parameters of the second participant's local model according to the maximum mean difference distance to obtain the second parameter.

[0107] Step d3: For each distribution of samples, merge the first distribution sample and the second distribution sample in the group to obtain a new sample.

[0108] Step d4: Based on each new sample and the label corresponding to the new sample, determine the parameters for updating the federated learning model and the local model of each participant, and send them to the second participant, so that the second participant determines the second target parameter based on the parameters for updating the federated learning model and the local model of each participant and the second parameter.

[0109] Step d5: Determine the first target parameter based on the parameters used to update the federated learning model and the local models of each participant, as well as the first parameter.

[0110] Therefore, the local models are adjusted based on the maximum mean difference distance to narrow the gap. Simultaneously, the parameters of both the federated learning model and the local models are adjusted based on samples from the two distributions. The adjusted parameters of the local models are then summed or weighted to determine the target parameters of the local models. This simultaneous adjustment of the local and federated learning models reduces training time and accelerates the convergence process.

[0111] Method 2: Based on the samples of each distribution and the corresponding maximum mean difference distance, determine the parameters used to update the federated learning model and the local models of each participant, and determine the first target parameter and the second target parameter. This can be achieved through the following steps:

[0112] Step e1: For each group of distributed samples, adjust the parameters of the local model of the first participant according to the maximum mean difference distance to obtain the first target parameters and the corresponding new local model of the first participant. Then, obtain the first new distribution sample corresponding to the first distribution sample in the group through the new local model of the first participant. The new local model of the first participant is used to make the maximum mean difference distance less than the preset distance threshold.

[0113] Step e2: Send the maximum mean difference distance to the second participant, so that the second participant adjusts the parameters of the second participant's local model according to the maximum mean difference distance to obtain the second target parameters and the corresponding new local model of the second participant, and obtains the second new distribution sample corresponding to the second distribution sample in the group through the new local model of the second participant. The new local model of the second participant is used to make the maximum mean difference distance less than the preset distance threshold.

[0114] Step e3: Receive the second new distribution sample sent by the second participant, and merge the first new distribution sample corresponding to the group with the second new distribution sample to obtain a new sample.

[0115] Step e4: Determine the parameters for updating the federated learning model and the local models of each participant based on each new sample and the label corresponding to the new sample.

[0116] Therefore, based on the maximum mean difference distance, the samples of the two distributions in this group are narrowed down. That is, the local models of each distribution are adjusted based on the maximum mean difference distance until they converge to a maximum mean difference distance less than or equal to a preset distance threshold. Then, based on the two distribution samples that have been narrowed down, the parameters of the federated learning model are adjusted and then distributed to their respective local models. This reduces the training time of the model and accelerates the convergence process.

[0117] In the pre-training phase, each participant performs unsupervised training only on its own data and features. Therefore, for an aligned sample (samples from the first distribution and samples from the second distribution), the sample's features are distributed across different devices, and training is conducted independently on each device. Consequently, the feature vectors obtained from different devices are not in the same feature space. In this case, due to the significant difference in feature spaces, directly concatenating or adding the features from both sides will reduce the feature's representational power. Using this feature to train a classifier will also reduce the model's discriminative ability. Therefore, to avoid excessively long convergence times and low model performance, it is necessary to narrow down the feature space of the features from the first and second participants.

[0118] Specifically, if the maximum mean difference distance is greater than or equal to a preset distance threshold, the maximum mean difference distance corresponding to the samples of this distribution is optimized to narrow the two distribution spaces corresponding to the samples of the first distribution and the second distribution. The optimization process can involve using the maximum mean difference distance as a loss to adjust the local model parameters of the first participant and the second participant. The first participant can optimize U based on the maximum mean difference distance. A and U B The MMD distance is reduced, bringing U closer. A and U B The corresponding two distribution spaces. The Maximum Mean Discrepancy (MMD) measures the distance between two distributions in the reproducing Hilbert space and is a kernel learning method. The distance between two random variables is:

[0119]

[0120] Where k is a mapping used to map the original variable to a higher-dimensional space. X and Y represent samples from two distributions (X represents the sample from the first distribution, and Y represents the sample from the second distribution), F represents the set of mapping functions; m and n represent the number of elements in the sample from each distribution, x represents the element in the sample from the first distribution, and y represents the element in the sample from the second distribution.

[0121] Based on samples from two distributions, we find a mapping function k in the sample space, calculate the mean of the function values ​​of the samples from different distributions on k, and obtain the mean discrepancy (mean difference distance) between the two distributions corresponding to k by subtracting the two means. We find a k that maximizes this mean discrepancy, which is the MMD. Finally, we use the MMD as a test statistic to determine whether the two distributions are the same. If this value is sufficiently small (e.g., MMD is less than a preset distance threshold), the two distributions are considered the same; otherwise, they are considered different.

[0122] Optionally, the parameters for updating the federated learning model and the local models of each of the multiple participants can be determined based on each new sample and the label corresponding to the new sample. This can be achieved through the following steps:

[0123] Step e1: Based on each of the new samples, obtain the prediction results through the federated learning model.

[0124] Step e2: Based on the prediction results and the labels corresponding to the new samples, determine the gradient of the loss function with respect to the intermediate layer output.

[0125] Step e3: Use the gradient of the loss function with respect to the intermediate layer output as the parameter for updating the federated learning model and the local model of each participant among the multiple participants.

[0126] Among them, the first participant will bring the U closer A and U B The results are added together or concatenated, and then analyzed using a classifier to obtain the final prediction. Furthermore, various encryption methods can be used to process the data for U. B , To further enhance data security, differential privacy methods such as Dynamic Privacy (DP) or homomorphic encryption methods such as Heteromorphic Encryption (HE) are employed. Then, based on the prediction results and labels, the gradient of the loss function with respect to the intermediate layer output is determined to adjust or train the parameters of the federated learning model and the parameters of the local models of each participant, thereby obtaining the final model. Optionally, after determining the parameters used to update the federated learning model and the local models of each of the multiple participants, the method can also be implemented through the following steps:

[0127] Step f1: Based on the parameters used to update the federated learning model and the local model of each of the multiple participants, update the federated learning model and the local model of the first participant using the gradient descent algorithm.

[0128] Step f2: Send the parameters for updating the federated learning model and the local models of each of the multiple participants to the second participant, so that the second participant updates its respective local model.

[0129] In this process, the first participant obtains the prediction result and calculates the loss, then transmits the gradient of the loss function with respect to the intermediate layer output to the second participant. The first participant updates the classifier and Extractor A according to the gradient descent algorithm, while the second participant updates Extractor B according to the chain rule and the gradient descent algorithm.

[0130] Specifically, in combination Figure 1 As shown, the fine-tuning process of vertical federated learning can be as follows: each participant initializes its own model Extractor A and Extractor B; the first participant and the second participant obtain the features U of the data through their respective models Extractor A and Extractor B. A and U B The second participant will U B It is transmitted to the first participant, who then optimizes U. A and U B The MMD distance is shortened, bringing the two distribution spaces closer; the first participant will shorten the U... A and U B The results are summed or concatenated and passed through the classifier to obtain the final prediction. The first participant obtains the prediction result and calculates the loss, then transmits the gradient of the loss function with respect to the intermediate layer's output to the second participant. The first participant updates the classifier and Extractor A according to the gradient descent algorithm, while the second participant updates Extractor B according to the chain rule and gradient descent algorithm.

[0131] This invention employs self-supervised learning, enabling each participant to autonomously learn a superior feature extraction model without requiring labels. This fully utilizes the data from each party, especially in scenarios where both parties have limited labeled data or different target labels. Based on the self-supervised learned model, when each participant performs fine-tuning through vertical federation, only a few communication interactions are needed to obtain a better model, significantly reducing communication costs in vertical federation scenarios. During fine-tuning, the features of each participant are brought closer together in the feature space dimension, ensuring that samples from different participants are on the same distribution. This accelerates the convergence of the federated learning model, improves model performance, and simultaneously reduces the number of communication interactions between devices.

[0132] Therefore, this invention can be applied to vertical federated learning. The training process can be as follows: First, each participant performs self-supervised training without any communication. Then, each participant fine-tunes its final prediction model based on the self-supervised training model using a vertical federated method. By using self-supervised learning pre-training, a large amount of unlabeled data can be fully utilized while significantly reducing data communication. It also solves the problem of inconsistent feature dimensions caused by self-training by each party and ensures data and model security using differential privacy method DP or homomorphic encryption method HE.

[0133] Figure 4 This is a schematic diagram of an information processing device provided in an embodiment of the present invention. The device is applied to a first participant among multiple participants in federated learning, and the first participant holds a tag.

[0134] like Figure 4 As shown, the information processing device may include:

[0135] The feature extraction module 401 is used to perform feature extraction on each data provided by the first participant after performing self-supervised training locally, so as to obtain the first feature vector of each data provided by the first participant.

[0136] The receiving module 402 is used to receive the second feature vector of each data sent by the second participant among the plurality of participants. The second feature vector is obtained by the second participant through self-supervised training on its local machine. The first feature vector and the second feature vector are aligned data, and each participant among the plurality of participants corresponds to a type of feature vector.

[0137] The processing module 403 is used to determine, based on each of the first feature vectors and each of the second feature vectors, the parameters for updating the federated learning model and the local models of each of the multiple participants, wherein the federated learning model is used to support information interaction among the multiple participants.

[0138] Optionally, the processing module 403 includes a first processing unit, a second processing unit, and a third processing unit; the first processing unit is used to treat each of the first feature vectors as samples of a first distribution, and each of the second feature vectors as samples of a second distribution, and to treat a sample of the first distribution and a corresponding sample of the second distribution as samples of a set of distributions; the second processing unit is used to determine, for each set of distributions, the maximum mean difference distance of the samples of each set of distributions on the mapping function set on the sample space; the third processing unit is used to determine the parameters for updating the federated learning model and the local model of each participant among the multiple participants based on the samples of each set of distributions and the corresponding maximum mean difference distance.

[0139] Optionally, the second processing unit includes a first processing subunit, a second processing subunit, a third processing subunit, and a fourth processing subunit; the first processing subunit is configured to, for each mapping function in the mapping function set, calculate, based on samples from the first distribution and samples from the second distribution in each set of samples, a first mean of the function values ​​of the samples from the first distribution and a second mean of the function values ​​of the samples from the second distribution on the mapping function, respectively; the second processing subunit is configured to calculate the mean difference distance on the mapping function based on the first mean and the second mean; the third processing subunit is configured to, when the maximum mean difference distance is less than or equal to a preset distance threshold, determine the parameters for updating the federated learning model and the local models of each participant based on samples from each set of distributions; the fourth processing subunit is configured to, when the maximum mean difference distance is greater than the preset distance threshold, determine the parameters for updating the federated learning model and the local models of each participant based on samples from each set of distributions and the corresponding maximum mean difference distance, and determine a first target parameter and a second target parameter, wherein the first target parameter is used to update the local model of the first participant, and the second target parameter is used to update the local model of the second participant. Optionally, the determining module is further configured to, when the maximum mean difference distance is greater than or equal to a preset distance threshold, optimize the maximum mean difference distance corresponding to the samples of the group distribution to bring the two distribution spaces corresponding to the samples of the first distribution and the samples of the second distribution closer together.

[0140] Optionally, the third processing subunit is specifically used for:

[0141] For each distribution of samples, the first distribution sample in the group is merged with the second distribution sample to obtain a new sample;

[0142] Based on each new sample and its corresponding label, parameters for updating the federated learning model and the local models of each participant are determined.

[0143] Optionally, the fourth processing subunit is specifically used for:

[0144] For each group of samples, the parameters of the local model of the first participant are adjusted according to the maximum mean difference distance to obtain the first parameter;

[0145] The maximum mean difference distance is sent to the second participant, so that the second participant can adjust the parameters of the second participant's local model according to the maximum mean difference distance to obtain the second parameter;

[0146] For each distribution of samples, the first distribution sample in the group is merged with the second distribution sample to obtain a new sample;

[0147] Based on each new sample and the label corresponding to the new sample, parameters for updating the federated learning model and the local model of each participant are determined and sent to the second participant, so that the second participant determines the second target parameter based on the parameters for updating the federated learning model and the local model of each participant and the second parameter.

[0148] The first target parameter is determined based on the parameters used to update the federated learning model and the local models of each participant, as well as the first parameter.

[0149] Optionally, the fourth processing subunit is also specifically used for:

[0150] For each distribution of samples, the parameters of the local model of the first participant are adjusted according to the maximum mean difference distance to obtain the first target parameters and the corresponding new local model of the first participant. The first new distribution sample corresponding to the first distribution sample in the group is obtained through the new local model of the first participant. The new local model of the first participant is used to make the maximum mean difference distance less than the preset distance threshold.

[0151] The maximum mean difference distance is sent to the second participant, so that the second participant adjusts the parameters of the second participant's local model according to the maximum mean difference distance to obtain the second target parameters and the corresponding new local model of the second participant. The second new distribution sample corresponding to the second distribution sample in the group is obtained through the new local model of the second participant. The new local model of the second participant is used to make the maximum mean difference distance less than the preset distance threshold.

[0152] Receive the second new distribution sample sent by the second participant, and merge the first new distribution sample corresponding to the group with the second new distribution sample to obtain a new sample;

[0153] Based on each new sample and its corresponding label, parameters for updating the federated learning model and the local models of each participant are determined.

[0154] Optionally, the fourth processing unit is specifically used for:

[0155] Based on each of the new samples, a prediction result is obtained through a federated learning model;

[0156] Based on the prediction results and the labels corresponding to the new samples, the gradient of the loss function with respect to the intermediate layer output is determined;

[0157] The gradient of the loss function with respect to the intermediate layer output is used as a parameter for updating the federated learning model and the local model of each participant among the multiple participants.

[0158] Optionally, the device further includes: an update module; the update module is configured to:

[0159] Based on the parameters used to update the federated learning model and the local model of each of the multiple participants, the federated learning model and the local model of the first participant are updated using the gradient descent algorithm.

[0160] The parameters used to update the federated learning model and the local models of each of the multiple participants are sent to the second participant, so that the second participant updates its respective local model.

[0161] Optionally, the device further includes: a self-supervised training module; the self-supervised training module is used for:

[0162] Obtain multiple positive samples and multiple sub-samples corresponding to each positive sample;

[0163] Based on multiple positive samples and multiple sub-samples corresponding to each positive sample, a feature extraction model is obtained through self-supervised training.

[0164] The feature extraction model is used to extract the first feature vector of each data provided by the first participant.

[0165] The information processing apparatus provided in any of the foregoing embodiments is used to execute the technical solutions of any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0166] Figure 5 This is a schematic diagram of the structure of an information processing device provided in an embodiment of the present invention. Figure 5 As shown, the device may include: a memory 501, a processor 502, and an information processing program stored in the memory 501 and executable on the processor 502. When the information processing program is executed by the processor 502, it implements the steps of the information processing method as described in any of the foregoing embodiments.

[0167] Alternatively, the memory 501 can be either standalone or integrated with the processor 502.

[0168] The implementation principle and technical effects of the device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0169] This invention also provides a computer-readable storage medium storing an information processing program, which, when executed by a processor, implements the steps of the information processing method as described in any of the foregoing embodiments.

[0170] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0171] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0172] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention.

[0173] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0174] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0175] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0176] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0177] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0178] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0179] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0180] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An information processing method, characterized in that, The method is applied to a first participant among multiple participants in federated learning, the first participant holding a label; the method includes: After performing self-supervised training on each piece of data provided by the first participant locally, feature extraction is performed to obtain the first feature vector of each piece of data provided by the first participant. The system receives the second feature vector of each data sent by the second participant among the plurality of participants. The second feature vector is obtained by the second participant through self-supervised training on its local machine. The first feature vector and the second feature vector are feature data of aligned samples, and each participant among the plurality of participants corresponds to a class of feature vectors. Based on each of the first feature vectors and each of the second feature vectors, parameters are determined for updating the federated learning model and the local models of each of the multiple participants, wherein the federated learning model is used to support information interaction among the multiple participants. Based on each of the first feature vectors and each of the second feature vectors, parameters for updating the federated learning model and the local models of each of the plurality of participants are determined, including: Each of the first feature vectors is used as a sample of the first distribution, and each of the second feature vectors is used as a sample of the second distribution. A sample of the first distribution and a corresponding sample of the second distribution are used as a set of samples. For each distribution of samples, the maximum mean difference distance of the samples in each distribution is determined by using the mapping function set on the sample space. If the maximum mean difference distance is less than or equal to a preset distance threshold, then the parameters for updating the federated learning model and the local models of each participant are determined based on the samples of each distribution. If the maximum mean difference distance is greater than a preset distance threshold, then based on the samples of each distribution and the corresponding maximum mean difference distance, the parameters for updating the federated learning model and the local models of each participant are determined, and a first target parameter and a second target parameter are determined. The first target parameter is used to update the local model of the first participant, and the second target parameter is used to update the local model of the second participant. By using a set of mapping functions on the sample space, the maximum mean difference distance of samples from each distribution on the mapping function set is determined, including: For each mapping function in the set of mapping functions, based on the samples of the first distribution and the samples of the second distribution in each set of distributions, calculate the first mean of the function values ​​of the samples of the first distribution on the mapping function and the second mean of the function values ​​of the samples of the second distribution on the mapping function, respectively. The mean difference distance on the mapping function is calculated based on the first mean and the second mean. Obtain the maximum mean difference distance from the mean difference distances corresponding to each of the mapping functions.

2. The method according to claim 1, characterized in that, Based on the samples from each distribution, determine the parameters used to update the federated learning model and the local models of each participant, including: For each distribution of samples, the first distribution sample in the group is merged with the second distribution sample to obtain a new sample; Based on each new sample and its corresponding label, parameters for updating the federated learning model and the local models of each participant are determined.

3. The method according to claim 1, characterized in that, The step of determining the parameters for updating the federated learning model and the local models of each participant based on the samples of each distribution and the corresponding maximum mean difference distance, and determining the first target parameter and the second target parameter, includes: For each group of samples, the parameters of the local model of the first participant are adjusted according to the maximum mean difference distance to obtain the first parameter; The maximum mean difference distance is sent to the second participant, so that the second participant can adjust the parameters of the second participant's local model according to the maximum mean difference distance to obtain the second parameter; For each distribution of samples, the first distribution sample in the group is merged with the second distribution sample to obtain a new sample; Based on each new sample and the label corresponding to the new sample, parameters for updating the federated learning model and the local model of each participant are determined and sent to the second participant, so that the second participant determines the second target parameter based on the parameters for updating the federated learning model and the local model of each participant and the second parameter. The first target parameter is determined based on the parameters used to update the federated learning model and the local models of each participant, as well as the first parameter.

4. The method according to claim 1, characterized in that, The step of determining the parameters for updating the federated learning model and the local models of each participant based on the samples of each distribution and the corresponding maximum mean difference distance, and determining the first target parameter and the second target parameter, includes: For each distribution of samples, the parameters of the local model of the first participant are adjusted according to the maximum mean difference distance to obtain the first target parameters and the corresponding new local model of the first participant. The first new distribution sample corresponding to the first distribution sample in the group is obtained through the new local model of the first participant. The new local model of the first participant is used to make the maximum mean difference distance less than the preset distance threshold. The maximum mean difference distance is sent to the second participant, so that the second participant adjusts the parameters of the second participant's local model according to the maximum mean difference distance to obtain the second target parameters and the corresponding new local model of the second participant. The second new distribution sample corresponding to the second distribution sample in the group is obtained through the new local model of the second participant. The new local model of the second participant is used to make the maximum mean difference distance less than the preset distance threshold. Receive the second new distribution sample sent by the second participant, and merge the first new distribution sample corresponding to the group with the second new distribution sample to obtain a new sample; Based on each new sample and its corresponding label, parameters for updating the federated learning model and the local models of each participant are determined.

5. The method according to any one of claims 2-4, characterized in that, Based on each new sample and its corresponding label, determine the parameters for updating the federated learning model and the local models of each of the multiple participants, including: Based on each of the new samples, a prediction result is obtained through a federated learning model; Based on the prediction results and the labels corresponding to the new samples, the gradient of the loss function with respect to the intermediate layer output is determined; The gradient of the loss function with respect to the intermediate layer output is used as a parameter for updating the federated learning model and the local model of each participant among the multiple participants.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: Based on the parameters used to update the federated learning model and the local model of each of the multiple participants, the federated learning model and the local model of the first participant are updated using the gradient descent algorithm. The parameters used to update the federated learning model and the local models of each of the multiple participants are sent to the second participant so that the second participant can update its own local model.

7. The method according to any one of claims 1-4, characterized in that, The method further includes: Obtain multiple positive samples and multiple sub-samples corresponding to each positive sample; Based on multiple positive samples and multiple sub-samples corresponding to each positive sample, a feature extraction model is obtained through self-supervised training. The feature extraction model is used to extract the first feature vector of each data provided by the first participant.

8. An information processing apparatus for implementing the method according to any one of claims 1-7, characterized in that, The device is applied to a first participant among multiple participants in federated learning, the first participant holding a tag; the device includes: The feature extraction module is used to extract features from each piece of data provided by the first participant after performing self-supervised training locally, so as to obtain the first feature vector of each piece of data provided by the first participant. The receiving module is used to receive the second feature vector of each data sent by the second participant among the plurality of participants. The second feature vector is obtained by the second participant through self-supervised training on its local machine. The first feature vector and the second feature vector are feature data of aligned samples, and each participant among the plurality of participants corresponds to a class of feature vectors. The processing module is configured to determine, based on each of the first feature vectors and each of the second feature vectors, the parameters for updating the federated learning model and the local models of each of the plurality of participants, wherein the federated learning model is used to support information interaction among the plurality of participants.

9. An information processing device, characterized in that, The information processing device includes: a memory, a processor, and an information processing program stored in the memory and executable on the processor, wherein when the information processing program is executed by the processor, it implements the steps of the information processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an information processing program, which, when executed by a processor, implements the steps of the information processing method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-7.

Images