Model training method, device and equipment and readable storage medium

Abstract

The invention discloses a model training method, device and equipment and a readable storage medium, a first node obtains the characteristics of each first alignment sample and the characteristics of each first non-alignment sample through a deployed first encoder, and each marking category corresponding to each first alignment sample is marked according to each marking category corresponding to each first non-alignment sample; determining a first prototype vector of the annotation category according to the features of each first aligned sample contained in the annotation category, determining a first loss for each first non-aligned sample according to the features of the first non-aligned sample and the first prototype vector of the annotation category to which the first non-aligned sample belongs, and determining a second loss according to the features of the first non-aligned sample and the first prototype vector of the annotation category to which the first non-aligned sample belongs; and training the first encoder according to the first loss corresponding to each first non-aligned sample. Therefore, by introducing each first non-aligned sample into the sample for training the first encoder, the scale of the training sample is expanded, the accuracy of the trained first encoder is improved, and the privacy security of the sample is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a model training method, device, equipment and readable storage medium. Background Art

[0002] As people pay more attention to privacy data and multi-party secure computing, deep learning has been widely used in speech recognition, image recognition, target detection, risk identification and other fields to automatically learn effective feature representations from data and improve the accuracy of prediction models.

[0003] At present, the model can be distributedly trained using Vertical Federated Learning (VFL). In vertical federated learning, the training samples held by each node participating in model training can have the same sample space and different feature spaces. For example, a bank holds the credit data of a certain user group, and an e-commerce platform holds the shopping data of the same user group. The credit data and shopping data of the user group are training samples with the same sample space but different feature spaces. The bank and the e-commerce platform can jointly train a credit assessment model based on the credit data and shopping data of the user group.

[0004] Generally, in order to improve the privacy security of training samples, the encrypted alignment method can be used in advance to align the sample space of the training samples held by each node, so as to determine the aligned samples with the same sample space from the training samples held by each node, and then use the aligned samples of each node to jointly train the model. Still taking the above as an example, the shopping platform can send the encrypted user identifier corresponding to the shopping data it holds to the bank, and the bank can determine the credit data with the same user identifier from the credit data held by the bank itself based on the received encrypted user identifier as the aligned sample held by the bank. Afterwards, the bank can return the encrypted user identifier corresponding to the aligned sample it holds to the shopping platform, and the shopping platform can determine the aligned sample held by the shopping platform from the shopping data it holds based on the received encrypted user identifier. Thus, the bank and the e-commerce platform can jointly train the model based on the aligned samples with the same user identifier.

[0005] However, usually, the sample size of the aligned samples held by each node is much smaller than the sample size of the training samples held by each node. The model can only be trained based on the aligned samples with a smaller sample size, resulting in a lower accuracy of the trained model.

[0006] Based on this, this specification provides a model training method. Summary of the invention

[0007] This specification provides a model training method, device, equipment and readable storage medium to partially solve the above-mentioned problems existing in the prior art.

[0008] This manual adopts the following technical solutions:

[0009] This specification provides a model training method, which is applied to a first node, where at least a first encoder is deployed, and the method includes:

[0010] Determine each first aligned sample, each labeled category of each first aligned sample, and each first non-aligned sample;

[0011] Inputting the first aligned samples to the first encoder respectively to obtain features of the first aligned samples, and inputting the first non-aligned samples to the first encoder respectively to obtain features of the first non-aligned samples;

[0012] For each annotation category corresponding to each of the first aligned samples, determining a first prototype vector corresponding to the annotation category for representing a feature center of the annotation category according to features of each of the first aligned samples included in the annotation category;

[0013] For each first non-aligned sample, determining a first loss corresponding to the first non-aligned sample according to a difference between a first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and a feature of the first non-aligned sample;

[0014] The first encoder is trained at least according to a first loss corresponding to each of the first non-aligned samples.

[0015] This specification provides a model training method, the method is applied to a second node, the second node deploys a second encoder, and the method includes:

[0016] Determining each second aligned sample and each second non-aligned sample;

[0017] Inputting the second aligned samples to the second encoder respectively to obtain features of the second aligned samples, and inputting the second non-aligned samples to the second encoder respectively to obtain features of the second non-aligned samples;

[0018] Sending the features of each second aligned sample to the first node, and receiving the second prototype vector corresponding to each labeled category returned by the first node, wherein the second prototype vector is determined by the first node according to the features of each second aligned sample and the labels of each first aligned sample, and is used to characterize the feature center of each labeled category corresponding to the second aligned sample;

[0019] For each second non-aligned sample, determining a third loss corresponding to the second non-aligned sample according to a difference between a second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and a feature of the second non-aligned sample;

[0020] The second encoder is trained at least according to a third loss corresponding to each of the second non-aligned samples.

[0021] This specification provides a model training device, which is applied to a first node, where at least a first encoder is deployed, and the device includes:

[0022] A first sample determination module, used to determine each first aligned sample, each labeled category of each first aligned sample, and each first non-aligned sample;

[0023] A first feature determination module, configured to input the first aligned samples to the first encoder to obtain features of the first aligned samples, and input the first non-aligned samples to the first encoder to obtain features of the first non-aligned samples;

[0024] A first prototype vector determination module is used to determine, for each annotation category corresponding to each of the first aligned samples, a first prototype vector corresponding to the annotation category and used to characterize a feature center of the annotation category according to features of each of the first aligned samples included in the annotation category;

[0025] A first loss determination module is used to determine, for each first non-aligned sample, a first loss corresponding to the first non-aligned sample according to a difference between a first prototype vector corresponding to a labeled category to which the first non-aligned sample belongs and a feature of the first non-aligned sample;

[0026] The first training module is used to train the first encoder at least according to the first loss corresponding to each of the first non-aligned samples.

[0027] This specification provides a model training device, which is applied to a second node, where a second encoder is deployed, and the device includes:

[0028] A second sample alignment module, used for determining each second aligned sample and each second non-aligned sample;

[0029] A second feature determination module, configured to input the second aligned samples to the second encoder to obtain features of the second aligned samples, and input the second non-aligned samples to the second encoder to obtain features of the second non-aligned samples;

[0030] A receiving module, configured to send the features of each second aligned sample to the first node, and receive a second prototype vector corresponding to each labeled category returned by the first node, wherein the second prototype vector is determined by the first node according to the features of each second aligned sample and the labels of each first aligned sample, and is used to characterize the feature center of each labeled category corresponding to the second aligned sample;

[0031] A third loss determination module is used to determine, for each second non-aligned sample, a third loss corresponding to the second non-aligned sample according to a difference between a second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and a feature of the second non-aligned sample;

[0032] The second training module is used to train the second encoder at least according to the third loss corresponding to each second non-aligned sample.

[0033] This specification provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned model training method is implemented.

[0034] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned model training method when executing the program.

[0035] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0036] In the model training method provided in this specification, the first node obtains the features of each first aligned sample and each first non-aligned sample through the deployed first encoder, and for each labeling category corresponding to each first aligned sample, the first prototype vector of the labeling category is determined according to the features of each first aligned sample contained in the labeling category, and for each first non-aligned sample, the first loss is determined according to the features of the first non-aligned sample and the first prototype vector of the labeling category to which it belongs, and the first encoder is trained with the first loss corresponding to each first non-aligned sample. It can be seen that introducing each first non-aligned sample into the sample used to train the first encoder expands the scale of the training samples and improves the accuracy of the first encoder after training. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification.

[0038] In the figure:

[0039] Figure 1A flowchart of a model training method in this specification;

[0040] Figure 2 is a schematic diagram of a model training system in this specification;

[0041] Figure 3 A flowchart of a model training method in this specification;

[0042] Figure 4 A flowchart of a model training method in this specification;

[0043] Figure 5 is a schematic diagram of a model training system in this specification;

[0044] Figure 6 A schematic diagram of a model training device provided in this specification;

[0045] Figure 7 A schematic diagram of a model training device provided in this specification;

[0046] Figure 8 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0048] In addition, it should be noted that all actions of acquiring signals, information or data in this manual are performed in compliance with the relevant local data protection laws and policies and with the authorization given by the owner of the corresponding device.

[0049] It should be noted that, in the absence of conflict, the features in the following embodiments and implementations may be combined with each other.

[0050] Deep learning is an important branch of computer science and artificial intelligence, and is a further extension of neural networks. It has been widely used in speech recognition, image recognition, transaction risk identification, and target detection by automatically learning effective feature representations from data and improving the accuracy of prediction models.

[0051] With the rapid expansion of deep learning data and the increasing attention to data security, how different institutions can share data and achieve win-win cooperation at the model level while ensuring security is an important issue in digital transformation. Federated learning has been the most important paradigm in joint modeling in recent years. Especially in the scenario of transaction risk identification in the financial industry, the method of federated learning that uses multiple nodes to execute model training jobs in parallel has been widely used. While ensuring the privacy and security of data, it uses data from multiple nodes to improve the training accuracy of the model, thereby accelerating the accuracy of the model in practical application.

[0052] Among them, vertical federated learning can meet the premise that the local data stored by each party does not leave the domain, and conduct multi-party joint modeling under this premise, and realize joint training of the model by exchanging intermediate model results (such as the representation of the original data) between different parties.

[0053] In traditional vertical federated learning, the training samples held by each node participating in model training may contain aligned samples with the same sample space and different feature spaces. By aggregating the features of these aligned samples, the feature diversity of the samples can be enhanced, thereby training a model with better performance. As mentioned above, aligned samples with the same sample space can be determined from the training samples held by each node according to the encrypted alignment method. However, in practice, for each node participating in the joint model training, the sample size of the aligned samples determined by the encrypted alignment that can be used for the joint model training is much smaller than the sample size of the complete training samples held by each node. Even in vertical federated learning, the node with sample annotations (active party) will aggregate the features of the aligned samples from other nodes (passive party) for model training, and the size of the aligned samples participating in the model training is still small. In general, the performance of a machine learning model depends on the size of the samples used for model training, which limits the sufficiency and generalization ability of model learning, and thus affects the performance of the trained model.

[0054] Based on this, this specification provides a model training method, which uses non-aligned samples to expand the sample size of training samples used in model training, thereby improving the performance of the model.

[0055] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0056] In practice, there are demands for privacy computing and data security in various scenarios. Generally speaking, the characteristics of data stored by a single data holder are subject to the type of business performed between the data holder and the user. Therefore, even if the data holder has a relatively broad user group, the data stored by the data holder will have the problem of single characteristics. Therefore, the method of vertical federated learning can be used to integrate the data stored by multiple data holders and expand the breadth and dimension of data characteristics, so as to jointly train and obtain a model with better performance while ensuring data security.

[0057] In the embodiment of this specification, since non-aligned samples can be used to expand the scale of samples used for model training, under the framework of vertical federated learning, the first node with annotations can independently complete the model training process based on aligned samples and non-aligned samples, such as Figure 1 As shown, the model training process can also be completed by the first node with annotations and the second node without annotations, such as Figure 3 shown.

[0058] Figure 1 A flowchart of a model training method provided in this specification, which is executed solely by the first node.

[0059] S100: Determine each first aligned sample, each labeled category of each first aligned sample, and each first non-aligned sample.

[0060] A model training method is provided in an embodiment of the present specification, and the execution process of the model training method can be executed by a first node for model training, wherein the first node can be an electronic device such as a server, or a server cluster. In the process of executing the model training method, the first encoder to be trained can be deployed on the first node, and after the training of the first encoder is completed, the trained first encoder can still be deployed on the first node, or it can be deployed on other nodes trusted by the first node.

[0061] In this specification, the first node may be a data holder that stores data and data annotations. Depending on the application scenario, the models that need to be jointly trained may have different functions. For example, in a speech recognition scenario, the target model may be a speech recognition model, and the data stored in the first node may be the voice data of the voice chat between the chat robot and the user; or in an image classification scenario, the target model may be an image classification model, and the data stored in the first node may be images of pedestrians captured by urban road surveillance cameras; or in a transaction risk control scenario, the target model may be a transaction risk identification model, and the data stored in the first node may be the user's fund flow data in the bank.

[0062] In the framework of vertical federated learning, in the traditional model training method, the first node will determine the first aligned sample saved by the first node and the second aligned sample saved by the second node based on encrypted alignment with other data holders (second nodes), and then perform model training based on the first aligned sample and the second aligned sample, wherein the first node only obtains the labeled category of the first aligned sample.

[0063] However, in the embodiments of the present specification, the first node can still obtain the labels of each first aligned sample after determining each first aligned sample saved by itself based on the encrypted alignment method with the second node under the framework of vertical federated learning, and use all the other samples except the first aligned samples among the training samples saved by itself as first non-aligned samples. Without obtaining the labeled category of each first non-aligned sample, each first non-aligned sample is introduced into the model training process under the framework of vertical federated learning, so as to expand the sample scale that can be used for model training and improve the performance of the trained model.

[0064] Therefore, in this specification, the first aligned samples are samples selected by the first node and the second node from the training samples saved by the first node based on the encrypted alignment method, and have the same sample space as the training samples saved by the second node. The labeling category of the first aligned samples can be obtained by manual labeling, or can be automatically labeled based on other machine learning models, which is not limited in this specification. The first non-aligned samples are the other samples in the training samples saved by the first node, except for the first aligned samples.

[0065] For example, if the purpose of model training is to train a credit assessment model, the training sample saved by the first node can be the credit data of user [X1, X2, X3], and the training sample saved by the second node can be the shopping transaction data of user [X1, X4, X5]. Then, through encryption alignment, it can be determined that the first aligned sample is the credit data of user X1. The loan data may include loan amount, loan type, repayment record (monthly repayment amount, actual repayment date, whether repayment is on time, overdue record and overdue days). The label of the first aligned sample can be the credit score of user X1, and the first non-aligned sample is the credit data of user X2 and the credit data of user X3.

[0066] It should be noted that, in this specification, the first node masters the labeling category of the first aligned sample, and the labeling category of the first non-aligned sample can be obtained by the first node or not. Considering the difficulty of obtaining labeled data, the following takes the example of the first node only obtaining the labeling of the first aligned sample to explain the specific technical solution.

[0067] In the embodiments of this specification, the encryption alignment method for determining the first alignment sample may be any existing type of encryption alignment method, such as an encryption entity alignment method based on an RSA algorithm and a hash mechanism, and this specification does not limit this.

[0068] S102: Input the first aligned samples to the first encoder respectively to obtain features of the first aligned samples, and input the first non-aligned samples to the first encoder respectively to obtain features of the first non-aligned samples.

[0069] In this specification, the first node is deployed with at least a first encoder, and the first encoder is used to extract features from the samples respectively, so that the first aligned sample is input into the first encoder to obtain the features of the first aligned sample output by the first encoder, and the first non-aligned sample is input into the first encoder to obtain the features of the first non-aligned sample output by the first encoder.

[0070] The first encoder may be an encoder that has never been trained, that is, the parameters of the first encoder are only initialized and not adjusted. Of course, the first encoder may also be an encoder pre-trained based on other samples, that is, the parameters of the first encoder may be optimized and adjusted in advance, and this specification does not limit this.

[0071] S104: For each annotation category corresponding to each of the first aligned samples, determine a first prototype vector corresponding to the annotation category for representing a feature center of the annotation category according to features of each of the first aligned samples included in the annotation category.

[0072] For the first node, in the process of expanding the scale of samples used for training the model based on each first non-aligned sample, each first non-aligned sample is actually an unlabeled sample and cannot be directly applied in the supervised learning model training process.

[0073] Based on this, in this specification, based on the clustering hypothesis, it is assumed that the data distribution of the training samples (including each first aligned sample and each first non-aligned sample) mastered by the first node contains multiple separate clusters, and the samples in the same cluster are more likely to have the same annotation category, and the feature vector of the cluster center of each sample contained in each annotation category is used as the prototype vector of each annotation category. Since the annotation category of each first aligned sample is obtained, the first aligned samples contained in each annotation category can be determined according to the annotation category of each first aligned sample, and then for each annotation category, the feature center of the annotation category is determined according to the features of each first aligned sample contained in the annotation category, and the feature vector of the feature center of the annotation category is used as the first prototype vector of the annotation category. It can be seen that the prototype vector of each annotation category is actually obtained based on the feature aggregation of each first aligned sample belonging to the same annotation category.

[0074] The specific implementation method of determining the prototype vector of each labeled category can be as follows:

[0075] The first scheme for determining the first prototype vector of each annotation category is as follows: initializing the initial prototype vector of each annotation category, calculating the distance between each first alignment sample and the initial prototype vector of each annotation category respectively, and for each first alignment sample, determining the prototype vector closest to the first alignment sample based on the calculated distances between the first alignment sample and the initial prototype vector of each annotation category, and judging whether the annotation category corresponding to the prototype vector closest to the first alignment sample is the same as the annotation category of the first alignment sample.

[0076] If yes, then the prototype vector closest to the first alignment sample and the first update parameter is updated. The following formula is an optional implementation scheme:

[0077] μ′ z =μ z* +η(f a,i -μ z* )

[0078] Among them, μ z* is the initial prototype vector closest to the i-th first aligned sample, whose label category is z, η is the first update parameter, μ′ z is the updated prototype vector.

[0079] If not, the nearest prototype vector is updated according to the first alignment sample and the second update parameter. The following formula is an optional implementation scheme:

[0080] μ′ z =μ z* -η(f a,i -μ z* )

[0081] Wherein, -η is the second update parameter.

[0082] The prototype vector μ z* Update to μ′ z , the above process is iterated until each first aligned sample is traversed, thereby obtaining the first prototype vector of each labeled category.

[0083] The second method for determining the first prototype vector of each annotation category is to initialize the initial prototype vector of each annotation category, and for each annotation category, update the initial prototype vector of the annotation category according to the statistical value of the features of each first aligned sample contained in the annotation category to obtain the first prototype vector of the annotation category. The following is an optional method for determining the first prototype vector:

[0084]

[0085] in, is the initial prototype vector, ρ is the update parameter, Na z is the number of first aligned samples, f a,z,i is the feature of the first aligned sample, μ′ z is the first prototype vector.

[0086] S106: For each first non-aligned sample, determine a first loss corresponding to the first non-aligned sample according to a difference between a first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and a feature of the first non-aligned sample.

[0087] Furthermore, since each first non-aligned sample is actually an unlabeled sample, in order to introduce the first non-aligned samples into the first aligned samples that can be used for model training and participate in the model training process of supervised learning, it is possible to first determine to which annotation category each first non-aligned sample belongs among the annotation categories corresponding to the first aligned samples.

[0088] Afterwards, since the features of the first non-aligned sample are obtained through the first encoder, the prototype vectors corresponding to each labeled category can represent the feature center of each labeled category. Therefore, the parameters of the first encoder can be optimized by minimizing the difference between the features of the first non-aligned sample and the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs. Thus, in this step, the first loss corresponding to the first non-aligned sample is determined based on the difference between the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and the features of the first non-aligned sample.

[0089] An optional scheme for determining the labeling category to which each first non-aligned sample belongs is: for each first non-aligned sample, based on the features of the first non-aligned sample and the similarities between the first prototype vectors of the various labeling categories, the labeling category corresponding to the first prototype vector with the greatest similarity is taken as the labeling category to which the first non-aligned sample belongs.

[0090] However, considering that the first prototype vector of each annotation category is determined based on the features of each first aligned sample, and the features of each first aligned sample and the features of each first non-aligned sample are determined based on the first encoder to be trained, only taking the annotation category corresponding to the first prototype vector with the greatest similarity as the annotation category to which the first non-aligned sample belongs may have the problem of low accuracy.

[0091] Therefore, another optional solution for determining the labeling category of each first non-aligned sample is: for each labeling category, according to the first prototype vector of the labeling category and the characteristics of each first non-aligned sample, determine the probability distribution of each first non-aligned sample belonging to the labeling category as the first local prior probability of the labeling category. The probability of each first non-aligned sample belonging to the labeling category is represented by the first local prior probability of each labeling category, so that based on the first local prior probability of each labeling category, the probability distribution of each first non-aligned sample belonging to each labeling category can be deduced, so that the labeling category to which each first non-aligned sample belongs can be determined based on the probability distribution.

[0092] Specifically, the first local prior probability of each labeled category is determined as follows:

[0093] Initialize the initial prior probability of each labeled category, determine the similarities between the features of each first non-aligned sample and the first prototype vector of the labeled category for each labeled category, and determine the similarity distribution between the features of each first non-aligned sample and the first prototype vector of the labeled category based on the similarities, update the initial prior probability of the labeled category based on the similarity distribution, and obtain the first local prior probability of the labeled category.

[0094] The first local prior probability of each labeled category is introduced here to determine the distribution of each first non-aligned sample among different labeled categories in the current round of model iteration, that is, the approximate proportion of each labeled category containing the first non-aligned sample. Since the first prototype vector is the feature center of each category, it can be regarded as a typical feature vector of the labeled category. The similarity distribution between the features of each first non-aligned sample and the first prototype vector of the labeled category is determined by any existing type of distance measurement or similarity score. In the similarity distribution, the similarity between the features of each first non-aligned sample and the first prototype vector of the labeled category is recorded. The greater the similarity between the features of the first non-aligned sample and the first prototype vector of the labeled category, the higher the possibility that the first non-aligned sample belongs to the labeled category. Conversely, if the similarity between the features of the first non-aligned sample and the first prototype vector of the labeled category is smaller, the possibility that the first non-aligned sample belongs to the labeled category is low. Compared with the aforementioned method of taking the label category corresponding to the first prototype vector with the largest similarity as the label category to which the first non-aligned sample belongs, determining the label category to which the first non-aligned sample belongs based on the similarity distribution can more objectively reflect the possibility that the first non-aligned sample belongs to each label category, thereby improving the accuracy and robustness when processing the attribution of the unlabeled first non-aligned sample.

[0095] S108: Train the first encoder at least according to the first losses corresponding to the first non-aligned samples.

[0096] After determining the first loss corresponding to each first non-aligned sample, the total loss for training the first encoder can be determined based on the sum of the first losses of the first non-aligned samples, and the first encoder is trained with minimization of the total loss as the training goal, thereby obtaining a trained first encoder.

[0097] Since both the first aligned samples and the first non-aligned samples are used in the training process of the first encoder, the scale of the samples used in the training process of the first encoder is expanded, thereby improving the performance of the first encoder, so that the first encoder has a more accurate feature extraction capability, thereby improving the generalization and accuracy of the target model constructed based on the trained first encoder.

[0098] In an optional embodiment of the present specification, the first node may also be deployed with a classifier to be trained, which can be used to classify samples according to input sample features. Thus, during the model training process, the first node may input the features of each first aligned sample into the classifier to be trained, and obtain the predicted category of each first aligned sample output by the classifier, and then determine the prediction loss based on the predicted category of each first aligned sample and the labeled category of each first aligned sample, thereby training the classifier based on the prediction loss. In addition, since the features of each first aligned sample are obtained based on the first encoder to be trained, after determining the prediction loss, the first encoder may also be trained based on the prediction loss and the first loss of each first non-aligned sample. Thus, after obtaining the trained first encoder and the trained classifier, the two may be combined to form a trained target model, thereby realizing the inference prediction process based on the target model.

[0099] In the model training method provided in this specification, the first node obtains the features of each first aligned sample and the features of each first non-aligned sample through the deployed first encoder, and for each labeled category corresponding to each first aligned sample, determines the first prototype vector of the labeled category according to the features of each first aligned sample contained in the labeled category, and for each first non-aligned sample, determines the first loss according to the features of the first non-aligned sample and the first prototype vector of the labeled category to which it belongs, and trains the first encoder with the first loss corresponding to each first non-aligned sample.

[0100] It can be seen that introducing each first non-aligned sample into the sample used to train the first encoder expands the scale of the training samples and improves the accuracy of the trained first encoder.

[0101] In an optional embodiment of this specification, Figure 1 In step S104, the first prototype vector corresponding to each annotation category can be obtained based on the following implementation:

[0102] Step 1: Obtain the first prototype vector corresponding to each labeled category obtained in the last iterative training as the first historical prototype vector corresponding to each labeled category.

[0103] Specifically, similar to the second scheme for determining the first prototype vector of each labeled category in the aforementioned step S104, this scheme is to superimpose the features of each first aligned sample obtained in the current iterative training on the basis of the first prototype vector obtained in the previous iterative training to determine the first prototype vector obtained in the current iterative training.

[0104] Therefore, firstly, the first prototype vector corresponding to each annotation category obtained in the previous iterative training of the current iterative training is obtained, and it is used as the first historical prototype vector corresponding to each annotation category, so that in subsequent steps, the first prototype vector of each annotation category can be updated separately based on the first historical prototype vector of each annotation category.

[0105] Step 2: for each annotation category corresponding to each of the first aligned samples, according to the features of each of the first aligned samples included in the annotation category, determine the statistical value of the features of each of the first aligned samples included in the annotation category.

[0106] In this specification, the method of performing statistics on the features of each first aligned sample included in each labeled category may be any existing statistical method such as average, weighted average, variance, etc., and this specification does not limit this.

[0107] Step 3: Determine the first prototype vector corresponding to the annotation category obtained by the current iterative training according to the statistical values ​​of the features of each first aligned sample contained in the annotation category and the first historical prototype vector corresponding to the annotation category.

[0108] An optional implementation is as follows:

[0109]

[0110] Among them, is the first historical prototype vector, is the first prototype vector obtained in the current iterative training after update, Na z is the number of first aligned samples contained in the labeled category, f a,z,i is the feature of the first aligned sample contained in the labeled category, and ρ is the update parameter.

[0111] Among them, in the above optional implementation, the statistical method for the features of each first aligned sample contained in the labeled category is averaging, that is, based on the average value of the features of each first aligned sample contained in the labeled category and the first historical prototype vector corresponding to the labeled category, the first prototype vector corresponding to the labeled category obtained by the current iterative training is determined.

[0112] In an optional embodiment of this specification, in Figure 1When the first loss corresponding to each first non-aligned sample is determined in step S106, the first loss includes the difference between the feature of the first non-aligned sample and the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs. In this specification, the expected probabilistic dual transport cost (PDTC) consumed by moving from the point where the feature of the first non-aligned sample is located to the feature center of the labeled category to which the first non-aligned sample belongs (i.e., the point where the corresponding first prototype vector of the labeled category is located) can be introduced in the feature space to characterize the difference between the feature of the first non-aligned sample and the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs, thereby effectively utilizing the first non-aligned sample to enhance the learning ability of the first encoder. Therefore, the scheme for determining the first loss corresponding to the first non-aligned sample based on PDTC can be as follows:

[0113] Step 1: For each first non-aligned sample, according to the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and the features of the first non-aligned sample, determine the first conditional probability of aligning the first non-aligned sample to the feature center of the labeled category to which the first non-aligned sample belongs.

[0114] Specifically, taking the aforementioned determination of the labeled categories to which each first non-aligned sample belongs based on the first local prior probability of each labeled category as an example, for each first non-aligned sample, the expected migration cost of migrating from the feature of the first non-aligned sample to the first prototype vector of the labeled category to which the first non-aligned sample belongs can be decomposed based on the chain rule to obtain the first prototype vector of the labeled category to which the first non-aligned sample belongs. and the features of the first non-aligned sample The joint distribution of

[0115] The first conditional probability of aligning (or moving) the first non-aligned sample to the feature center of the labeled category to which the first non-aligned sample belongs is actually the probability of migrating from the feature of the first non-aligned sample to the first prototype vector of the labeled category to which the first non-aligned sample belongs. A specific implementation method can be as follows:

[0116]

[0117] in, is the first local prior probability of the labeled category to which the first non-aligned sample belongs. The similarity between the feature of the first non-aligned sample and a first prototype vector of the labeled category to which the first non-aligned sample belongs is estimated.

[0118] Step 2: Determine a first sub-loss according to the first conditional probability and the difference between a first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and a feature of the first non-aligned sample.

[0119] Furthermore, the first conditional probability is multiplied by the difference between the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and the feature of the first non-aligned sample to obtain the first sub-loss.

[0120] A method to determine the first sub-loss The implementation method is as follows:

[0121]

[0122] in, is the difference between the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and the feature of the first non-aligned sample, specifically as follows:

[0123]

[0124] Step 3: Determine a second conditional probability of aligning the feature center of the labeled category to which the first non-aligned sample belongs to the first non-aligned sample according to the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and the feature of the first non-aligned sample.

[0125] In the aforementioned steps, what is actually happening is the process of migrating the first non-aligned sample to the feature center of the labeled category to which it belongs. However, if only this migration process is performed, there may be a risk of ignoring the feature centers of certain labeled categories. That is, when migrating the first non-aligned sample to the feature center of the labeled category to which it belongs, there may be some labeled categories where the first non-aligned sample does not exist near the feature center. Therefore, in order to determine that there are at least some first aligned samples near the feature center of each labeled category and avoid missing the feature center of any labeled category, this specification also introduces the first loss into the process of reverse migration from the feature center of the labeled category to the first non-aligned sample. Specifically, assuming that the number of first non-aligned samples in each round of iterative training is B, that is, the first non-aligned sample is Based on the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and the features of the first non-aligned sample, calculate the probability of migrating from the feature center of the labeled category to the first non-aligned sample for:

[0126]

[0127] Step 4: Determine a second sub-loss according to the second conditional probability and the difference between the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and the feature of the first non-aligned sample.

[0128] Furthermore, the second conditional probability is multiplied by the difference between the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and the feature of the first non-aligned sample to obtain the second sub-loss.

[0129] A method to determine the second sub-loss The implementation method is as follows:

[0130]

[0131]

[0132] Step 5: Determine a first loss corresponding to the first non-aligned sample according to the first sub-loss and the second sub-loss.

[0133] Thus, the first loss corresponding to the first non-aligned sample can be determined by fusing the first sub-loss and the second sub-loss.

[0134]

[0135] In one or more embodiments of the present specification, in order to further expand the scale of samples used for training the model, the first node may also rely on the framework of vertical federated learning and perform the model training process in combination with the training samples held by the second node. Figure 2 A schematic diagram of a model training system applicable to a model training method provided in this specification, in which there are two parties executing the model training method, namely a first node and a plurality of second nodes, and the number of second nodes can be one or more. Among them, the first node holds training samples that can participate in model training and annotations corresponding to the training samples, so the first node is regarded as the active party. The second node only holds training samples that can participate in model training, but does not hold corresponding annotations, so each second node is regarded as a passive party. Since the model training method provided in this specification is based on vertical federated learning, therefore, during the training process of the model, the first node and the second node will not transmit the aligned samples or the non-aligned samples themselves, but the data related to the model training, such as the features (representations) of the samples, the vectors related to the samples, the gradients of the model, the losses, etc., so as to protect the privacy security of the training samples held by each node.

[0136] exist Figure 2 In the model training system shown, the first node is deployed with at least a first encoder, which is used to determine the characteristics of a first aligned sample and a first non-aligned sample held by the first node, and the second node is deployed with a second encoder, which is used to determine the characteristics of a second aligned sample and a second non-aligned sample held by the second node.

[0137] Figure 3 This is a schematic diagram of the interactive process of a model training method provided in this manual. The two parties of the interaction are Figure 2 The first node and the second node in the model training system shown are used as an example to illustrate the specific technical solution by taking the number of the second node as one.

[0138] S200: The first node determines a first aligned sample, a labeled category of the first aligned sample, and a first non-aligned sample.

[0139] In the model training method provided in this specification, both the first node and the second node can be data holders that store data. Depending on the application scenario, the models that need to be jointly trained also have different functions. For example, in a speech recognition scenario, the goal of model training can be to train a speech recognition model. The data stored in the first node can be voice data of voice chat between a chat robot and a user, and the data stored in the second node can be voice data of voice conversation between a vehicle-mounted voice system and a user; or in an image classification scenario, the goal of model training can be to train an image classification model. The data stored in the first node can be images of pedestrians captured by urban road surveillance cameras, and the data stored in the second node can be images of consumers captured by indoor surveillance cameras in shopping malls; for example, in a transaction risk control scenario, the goal of model training can be to train a transaction risk identification model. The data stored in the first node can be the user's bank fund flow data, and the data stored in the second node can be transaction data of users purchasing goods from merchants.

[0140] It can be seen that the data stored by different nodes (data holders) may overlap in the user dimension, but there are differences in the data feature dimension. The model trained based on the data held by only one node may have low data accuracy and cannot be generalized and applied in actual scenarios.

[0141] To this end, in this specification, a first encoder is deployed on the first node and a second encoder is deployed on the second node. The second encoder obtains the characteristics of the data held by the second node, and the second node sends the characteristics of the data to the first node, that is, the data held by the first node and the data held by the second node are combined, so that the original data stored in the second node will not be leaked due to the need to be transmitted to the first node, thereby improving the privacy and security of the data.

[0142] Generally, the training samples held by the first node are determined based on the data stored by the first node and the scenario to which the target model is applicable. For example, in the field of financial risk control, when the first node is a bank, the bank may store the transaction record data and loan data of the user. If the target model is a credit assessment model, the training samples held by the first node need to be determined based on the loan data of the user stored by the bank.

[0143] Similarly, the second node will also determine the training samples held by the second node based on its own stored data and the scenarios to which the target model is applicable. For example, in the field of financial risk control, the second node is a shopping platform, and the user's shopping records stored on the shopping platform may include the user's installment payment transaction records and full payment transaction records. When the target model is still the aforementioned credit assessment model, it is necessary to determine the training samples held by the second node based on the user's installment payment transaction records stored on the shopping platform.

[0144] In this specification, the first node grasps the labeling category of the first aligned sample, and the second node does not grasp the labeling data.

[0145] S202: The second node determines each second aligned sample and each second non-aligned sample.

[0146] Steps S200 to S202 are actually the first node and the second node determining samples with the same sample space as aligned samples from each other's training samples through encryption alignment, i.e., the first aligned samples and the second aligned samples. Therefore, this specification does not limit the order of executing steps S200 and S202.

[0147] At the same time, since the first node is the party that has the labeled data, the first node also needs to obtain the labeled category of the first aligned sample.

[0148] In addition, among the training samples mastered by the first node, other samples except the first aligned samples can be used as first non-aligned samples, and among the training samples mastered by the second node, other samples except the second aligned samples can be used as second non-aligned samples. Specifically, the first aligned sample and the second aligned sample have the same sample space, but have different feature spaces. The sample spaces where the first non-aligned sample and the second non-aligned sample are located are different from the sample spaces where the first aligned sample and the second aligned sample are located.

[0149] For example, the first node is a bank, which has the credit data of users [X1, X2, X3], and the second node is a shopping platform, which has the installment payment transaction records of users [X1, X4, X5]. Then, through encryption alignment, it can be determined that the credit data of user X1 held by the bank overlaps with the installment payment transaction records of user X1 held by the shopping platform in sample space. Therefore, the credit data of user X1 held by the bank can be used as the first aligned sample of the first node, and the installment payment transaction records of user X1 held by the shopping platform can be used as the second aligned sample of the second node. The remaining credit data of users X2 and X3 are the first non-aligned samples, and the installment payment transaction records of users X4 and X5 are the second non-aligned samples.

[0150] In this specification, the encryption alignment method for determining the alignment samples of the first node and the second node may be any existing type of encryption alignment method, such as an encryption entity alignment method based on the RSA algorithm and the hash mechanism, etc., and this specification does not limit this.

[0151] S204: The first node inputs the first aligned samples to the first encoder respectively to obtain features of the first aligned samples, and inputs the first non-aligned samples to the first encoder respectively to obtain features of the first non-aligned samples.

[0152] The first encoder deployed by the first node may be an encoder that has never been trained, that is, the parameters of the first encoder are only initialized and not adjusted. Of course, the first encoder may also be an encoder pre-trained based on other samples, that is, the parameters of the first encoder may be optimized and adjusted in advance, and this specification does not limit this.

[0153] S206: The second node inputs the second aligned samples to the second encoder respectively to obtain features of the second aligned samples, and inputs the second non-aligned samples to the second encoder respectively to obtain features of the second non-aligned samples.

[0154] At the same time, since a second encoder is deployed on the second node, after encryption alignment, the second node can input the second aligned sample into the second encoder to obtain the characteristics of the second aligned sample output by the second encoder, and input the second unaligned sample into the second encoder to obtain the characteristics of the second unaligned sample output by the second encoder.

[0155] Among them, the second encoder deployed at the second node is similar to the first encoder deployed at the first node, and will not be described in detail here.

[0156] Likewise, this specification does not limit the order of executing steps S204 and S206, and the two can be performed simultaneously.

[0157] S208: The second node sends the features of each second aligned sample to the first node.

[0158] S210: The first node receives features of each second aligned sample, and for each annotation category corresponding to each first aligned sample, determines a first prototype vector corresponding to the annotation category for characterizing a feature center of the annotation category based on the features of each first aligned sample included in the annotation category, and determines a second prototype vector corresponding to the annotation category for characterizing a feature center of the annotation category based on the features of each second aligned sample included in the annotation category.

[0159] The first node determines the first prototype vector corresponding to each annotation category. Figure 1 Step S104 is shown and will not be described again here.

[0160] In addition, the first node determines the second prototype vector corresponding to each labeled category in a manner similar to the first prototype vector. Similarly, there are the following solutions:

[0161] First, initialize the initial prototype vector of each annotation category, calculate the distance between each second alignment sample and the initial prototype vector of each annotation category, and for each second alignment sample, determine the prototype vector closest to the second alignment sample based on the calculated distances between the second alignment sample and the initial prototype vector of each annotation category, and determine whether the annotation category corresponding to the prototype vector closest to the second alignment sample is the same as the annotation category of the second alignment sample. If so, update the prototype vector closest to the first alignment sample and the first update parameter to obtain an updated prototype vector (for a specific implementation, refer to S104); if not, update the prototype vector closest to the first alignment sample and the second update parameter to obtain an updated prototype vector (for a specific implementation, refer to S104); iterate the above process until each first alignment sample is traversed, thereby obtaining the first prototype vector of each annotation category.

[0162] Second, initialize the initial prototype vector of each annotation category. For each annotation category, update the initial prototype vector of the annotation category according to the statistical values ​​of the features of each first aligned sample contained in the annotation category to obtain the first prototype vector of the annotation category. The specific implementation method can refer to the above step S104.

[0163] Thirdly: obtaining the second prototype vector corresponding to each annotation category obtained in the last iterative training as the second historical prototype vector corresponding to each annotation category; determining the annotation category of each second alignment sample according to the annotation category of each first alignment sample and the correspondence between each first alignment sample and each second alignment sample; for each annotation category, determining the statistical value of the features of each second alignment sample contained in the annotation category according to the features of each second alignment sample contained in the annotation category; determining the second prototype vector corresponding to the annotation category obtained in the current iterative training according to the statistical value of the features of each second alignment sample contained in the annotation category and the second historical prototype vector corresponding to the annotation category; the specific implementation method may refer to the above-mentioned implementation method of determining the first prototype vector in the current iterative training based on the first historical prototype vector, which will not be elaborated here.

[0164] S212: Return the second prototype vector of each labeled category to the second node.

[0165] S214: The first node determines, for each first non-aligned sample, a first loss corresponding to the first non-aligned sample according to a difference between a first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and a feature of the first non-aligned sample.

[0166] Here and above Figure 1 The step S106 shown is similar and will not be described in detail here.

[0167] S216: Train the first encoder at least according to the first losses corresponding to the first non-aligned samples.

[0168] Here and above Figure 1 The step S108 shown is similar and will not be described in detail here.

[0169] S218: The second node determines, for each second non-aligned sample, a third loss corresponding to the second non-aligned sample according to a difference between a second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and a feature of the second non-aligned sample.

[0170] Specifically, similar to the aforementioned step S106, each second non-aligned sample is also an unlabeled sample. In order to introduce each second non-aligned sample into the model training process of supervised learning, it is necessary to first determine that each second non-aligned sample belongs to each labeled category in each labeled category corresponding to the first aligned sample, and then determine the third loss based on the feature of each second non-aligned sample and the difference between the second prototype vector of the labeled category to which it belongs. The third loss can be used to optimize the parameters of the second encoder.

[0171] Similar to the aforementioned S106, the scheme for determining the labeling category of the second non-aligned sample can be implemented in the following two optional ways:

[0172] First, for each second non-aligned sample, according to the similarities between the features of the second non-aligned sample and the second prototype vectors of each labeling category, the labeling category corresponding to the second prototype vector with the greatest similarity is taken as the labeling category to which the second non-aligned sample belongs.

[0173] Secondly, for each labeled category, according to the second prototype vector of the labeled category and the characteristics of each second non-aligned sample, a probability distribution for characterizing that each second non-aligned sample belongs to the labeled category is determined as the second local prior probability corresponding to the labeled category. The probability that each second non-aligned sample belongs to the labeled category is characterized by the second local prior probability of each labeled category, so that based on the second local prior probability of each labeled category, the probability distribution that each second non-aligned sample belongs to each labeled category can be deduced, so that the labeled category to which each second non-aligned sample belongs can be determined based on the probability distribution.

[0174] The second local prior probability of each labeled category is determined as follows:

[0175] Initialize the initial prior probability of each labeled category, determine the similarities between the features of each second non-aligned sample and the second prototype vector of the labeled category for each labeled category, and determine the similarity distribution between the features of each second non-aligned sample and the second prototype vector of the labeled category based on the similarities, update the initial prior probability of the labeled category based on the similarity distribution, and obtain the second local prior probability of the labeled category. Similarly, similar to the determination of the first local prior probability in S106, the second local prior probability of each labeled category is determined based on the similarity between the features of each second non-aligned sample and the second prototype vector of each labeled category, which can more objectively reflect the possibility of each second non-aligned sample belonging to each labeled category, thereby improving accuracy and robustness when processing the attribution of unlabeled second non-aligned samples.

[0176] Optionally, similar to the first node determining the first loss of each first non-aligned sample, the third loss of each second non-aligned sample can be determined according to the PDTC solution. The specific implementation method can refer to the above, and the solution is as follows:

[0177] Step 1: for each second non-aligned sample, according to the second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and the feature of the second non-aligned sample, determine the third conditional probability of aligning the second non-aligned sample to the feature center of the labeled category to which the second non-aligned sample belongs;

[0178] Step 2: Determine a third sub-loss according to the third conditional probability and the difference between the second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and the feature of the second non-aligned sample;

[0179] Step 3: determining a fourth conditional probability of aligning the feature center of the labeled category to which the second non-aligned sample belongs to the second non-aligned sample according to the second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and the feature of the second non-aligned sample;

[0180] Step 4: determining a fourth sub-loss according to the fourth conditional probability and a difference between the second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and the feature of the second non-aligned sample;

[0181] Step 5: Determine a third loss corresponding to the second non-aligned sample according to the third sub-loss and the fourth sub-loss.

[0182] The specific implementation of the above steps can all refer to the above implementation of determining the first loss of each first non-aligned sample, which will not be described in detail here.

[0183] S220: Train the second encoder at least according to the third loss corresponding to each of the second non-aligned samples.

[0184] Thus, at least the trained first encoder deployed on the first node and the trained second encoder deployed on the second node can constitute a target model, thereby achieving a target model with better performance through training under the framework of vertical federated learning.

[0185] In addition, it can be understood that the above steps S214 to S216 are the process of the first node determining the first loss and training the first encoder, and steps S218 to S220 are the process of the second node determining the second loss and training the second encoder. This specification does not limit the execution order of these two processes, which can be performed one after the other or at the same time. This specification also does not limit whether step S212 is performed before steps S214 to S216. Step S212 only needs to be performed before steps S218 to S220.

[0186] Optionally, in an optional embodiment of the present specification, the first node may also be deployed with a classifier to be trained, which can be used to classify samples according to input sample features. Thus, the target model obtained by training the first node and the second node includes at least a first encoder, a second encoder and a classifier. Based on this, in the model training process, especially after the first node receives the features of each second aligned sample sent by the second node in the above step S210, before the second node trains the second encoder in S218, the first node can input the features of each first aligned sample and the features of each second aligned sample into the classifier to be trained, and the prediction result output by the classification includes the predicted category of each first aligned sample and the predicted category of each second aligned sample. Since each first aligned sample and each second aligned sample have the same sample space, the labeled category of each first aligned sample can be directly used as the labeled category of each second aligned sample, thereby, based on the difference between the predicted category of each first aligned sample and the labeled category of each first aligned sample, and the difference between the predicted category of each second aligned sample and the labeled category of each second aligned sample, the function value is determined by a preset loss function as the second loss. Among them, the preset loss function can be any type of existing loss function, such as L1 norm loss, mean square error loss, cross entropy loss, etc., and this specification does not limit this.

[0187] Afterwards, the first encoder can be trained based on the second loss and the first loss corresponding to each first non-aligned sample, and the classifier can be trained based on the second loss. At the same time, before step S218, the second loss can be sent to the second node, and after the second node determines the third loss corresponding to each second non-aligned sample, the second encoder can be trained based on the second loss and the third loss corresponding to each second non-aligned sample. A trained target model can be constructed based on the trained first encoder, the trained second encoder, and the trained classifier, so as to achieve a target model with better performance under the framework of vertical federated learning.

[0188] It can be seen that based on Figure 3 The scheme shown uses the aligned samples and non-aligned samples held by the first node and the second node to jointly train the model. Even if the sample size of the first aligned sample and the second aligned sample is small, the non-aligned sample can expand the sample size, so that the model training based on vertical federated learning has a larger scale of training samples, thereby improving the accuracy and performance of the trained target model. In addition, during the model training process, there is no need to transmit aligned samples or non-aligned samples between the first node and the second node, which improves the privacy security of the training samples.

[0189] In an optional embodiment of this specification, in Figure 3Before determining the first loss corresponding to each first non-aligned sample in step S212, the annotation type to which each first non-aligned sample belongs is first determined. If, when determining the annotation type to which each first non-aligned sample belongs, the first local prior probability corresponding to each annotation category for characterizing the probability distribution of each first non-aligned sample belonging to each annotation category is used, at this time, the number of first non-aligned samples respectively included in each annotation category indicated by the first local prior probability may be significantly uneven, resulting in the trained model (which may be the first encoder here) tending to predict the majority category and ignoring the minority category and the unseen category, thereby affecting the generalization ability of the model and the recognition accuracy of the minority category.

[0190] In practice, the situation where the number of first non-aligned samples contained in each labeled category is significantly unequal may be caused by data collection bias, such as the easy acquisition of samples of certain categories and the difficulty in collecting data of other categories during data collection, resulting in the number of samples of easily acquired categories being significantly greater than the number of samples of difficult-to-acquire categories; or caused by the natural distribution of data, such as the sample data collected in a specific actual scenario itself has a natural imbalance, such as the number of users with normal credit (or high credit score) in the user credit assessment scenario will be much greater than the number of users with abnormal credit (or low credit score). At the same time, the second non-aligned samples saved by the second node also have the above problem. In addition, since there can be three or more nodes participating in the model training process at the same time in vertical federated learning, this class imbalance problem includes class imbalance within the node (referring to the imbalance of the number of samples of different labeled categories in the non-aligned samples owned by a single node) and class imbalance between nodes (referring to the imbalance of the number of samples of different labeled categories in the non-aligned samples owned by each node).

[0191] Based on the above problems, in this specification, a hybrid prior guidance method is adopted. After the second node determines the second local prior probability of each labeled category based on each second non-aligned sample held by itself and the second prototype vector corresponding to each labeled category sent by the first node, the second node sends the second local prior probability of each labeled category to the first node. The first node summarizes the first local prior probability of each labeled category determined by the first node itself and the second prior probability of each labeled category sent by the second node, and calculates the global prior probability by statistics. After that, the first node will send the calculated global prior probability corresponding to each labeled category to the second node. At this time, the first node can update the first local prior probability of each labeled category based on the global prior probability of each labeled category. At the same time, the second node can update the second local prior probability of each labeled category based on the global prior probability of each labeled category. In this way, by introducing the global prior probability of each labeled category, the possible category distribution differences between different nodes can be balanced. Especially when the scale of the second non-aligned sample of the second node is small or the category representation is insufficient, by incorporating global information, the model can make more generalized predictions and reduce the overfitting problem caused by local bias, thereby improving the accuracy and robustness of the trained model.

[0192] The above process can be specifically implemented through the following steps: Figure 4 As shown:

[0193] S300: The second node sends the second local prior probability of each labeled category to the first node.

[0194] S302: The first node receives the second local priori probability of each annotation category. For each annotation category, according to the first local priori probability of the annotation category and the second local priori probability of the annotation category, a global priori probability corresponding to the annotation category is determined.

[0195] S304: The first node updates, for each labeled category, the first local prior probability of the labeled category according to the global prior probability corresponding to the labeled category, to obtain the updated first local prior probability of the labeled category, and for each first non-aligned sample, determines the probability distribution that the first non-aligned sample belongs to each labeled category according to the updated first local prior probabilities corresponding to each labeled category, and determines the labeled category to which the first non-aligned sample belongs according to the probability distribution that the first non-aligned sample belongs to each labeled category.

[0196] S306: The first node sends the global prior probability corresponding to each labeled category to the second node.

[0197] S308: The second node updates, for each labeled category, the second local prior probability of the labeled category according to the global prior probability corresponding to the labeled category, to obtain the updated second local prior probability of the labeled category, and for each second non-aligned sample, determines the probability distribution that the second non-aligned sample belongs to each labeled category according to the updated second local prior probabilities corresponding to each labeled category, and determines the labeled category to which the second non-aligned sample belongs according to the probability distribution that the second non-aligned sample belongs to each labeled category.

[0198] The execution order of steps S306 to S308 and step S304 is not limited in this specification.

[0199] In an optional embodiment of the present specification, the first node is also deployed with a gating network and multiple adaptation networks, wherein the adaptation network is used to map the features of the first aligned sample and the features of the second aligned sample to the same feature space, so as to facilitate the fusion of the features of the first aligned sample and the features of the second aligned sample. The gating network is used to measure the feature contributions of the first aligned sample and the second aligned sample respectively during the model training process to determine the weights corresponding to the first aligned sample and the second aligned sample respectively, thereby alleviating the imbalance in the feature distribution of aligned samples contributed by different nodes. Thus, the target model obtained based on the training of the first node and the second node may include a first encoder, a second encoder, each adaptation network, a gating network and a classifier, such as Figure 5 shown.

[0200] Based on the above model structure, during the model training process, the first node can actually train the adaptation network of the first node, the adaptation network of the second node, and the gating network, so that the first node and the second node jointly execute Figure 3 In the process of the scheme shown, when determining the predicted category of each first aligned sample and the predicted category of each second aligned sample based on the classifier, the following scheme may be specifically performed:

[0201] First, the features of each first aligned sample are input into the adaptation network corresponding to the first node to obtain the target features of each first aligned sample.

[0202] Secondly, the features of each second aligned sample are input into the adaptation network corresponding to the second node to obtain the target features of each second aligned sample.

[0203] In practice, the first encoder deployed on the first node and the second encoder deployed on the second node may have different model structures, and the feature distributions of the first aligned sample and the second aligned sample are different, resulting in that the features of the first aligned sample and the features of the second aligned sample may not be in the same feature space. To this end, an adaptation network corresponding to the first node may be deployed on the first node, and an adaptation network corresponding to the second node may be deployed, and the above two steps are performed, so that the target features of each first aligned sample and the target features of each second aligned sample are in the same feature space, which is convenient for subsequent weight measurement or feature fusion.

[0204] An optional method for determining the target feature of each first aligned sample or the target feature of each second aligned sample is as follows:

[0205]

[0206] in, is the target feature, R m is the adaptation network corresponding to the first node or the adaptation network corresponding to the second node, is the feature of the first aligned sample or the feature of the second aligned sample, It is a parameter of the adaptation network corresponding to the first node or a parameter of the adaptation network corresponding to the second node.

[0207] Afterwards, the target features of the first aligned samples and the target features of the second aligned samples are taken as inputs and input into the gating network to obtain the weights of the first aligned samples and the weights of the second aligned samples output by the gating network.

[0208] Considering that in each round of iterative training, using different non-aligned samples will cause dynamic changes in data distribution, it is not ideal to directly splice the features of each first aligned sample contributed by the first node and the features of the second aligned sample contributed by the second node with fixed weights. Therefore, in this specification, a parameterized gating network is introduced to determine the contribution of data from different data sources (i.e., each first aligned sample and each second aligned sample) to the final prediction. The contribution degree is the weight of each first aligned sample output by the gating network, and the weight of each second aligned sample.

[0209] The use of a gating network to output the weights of each first aligned sample and the weights of each second aligned sample can also be used to alleviate the problem of inconsistent feature contributions caused by the imbalanced distribution of each first aligned sample and each second aligned sample in the category. This can adapt to the changes in the importance of different features in different rounds and improve the fusion efficiency.

[0210] Then, according to the weights of the first aligned samples and the weights of the second aligned samples, weighted splicing is performed on the target features of the first aligned samples and the target features of the second aligned samples, and the splicing result is input into the classifier to obtain the prediction result output by the classifier.

[0211] Thus, after obtaining the prediction result output by the classifier, the second loss can be determined based on the prediction category of each first aligned sample and the prediction category of each second aligned sample contained in the prediction result, combined with the labeled category of each first aligned sample. When the first encoder is trained with the second loss and the first loss of each first non-aligned sample, and when the second encoder is trained with the second loss and the third loss of each second non-aligned sample, the adaptation network of the first node, the adaptation network of the second node, and the gating network can also be trained respectively based on the second loss. Then, according to the trained first encoder, the trained second encoder, the adaptation network corresponding to the trained first node, the adaptation network corresponding to the trained second node, the trained gating network, and the trained classifier, the trained target model is determined.

[0212] In one or more embodiments of this specification, the following two solutions can be used to predict the data to be predicted based on the trained target model:

[0213] First: Based on Figure 2 In the model structure shown, the first node, in response to a prediction request, inputs the acquired data to be predicted into the first encoder of the trained target model to obtain the first feature of the data to be predicted, and then sends the data to be predicted to the second node, so that the second node inputs the data to be predicted into the second encoder to obtain the second feature of the data to be predicted, and returns the second feature of the data to be predicted, and then the first node receives the second feature of the data to be predicted. Figure 2 A trained classifier is optionally deployed in the first node shown, and then the first node takes the first feature of the data to be predicted and the second feature of the data to be predicted as input, inputs the trained classifier, and obtains the prediction result of the data to be predicted output by the trained classifier.

[0214] Second, based on Figure 5In the model structure shown, the first node responds to the prediction request, inputs the acquired data to be predicted into the first encoder of the trained target model, obtains the first feature of the data to be predicted, and then sends the data to be predicted to the second node, so that the second node inputs the data to be predicted into the trained second encoder, obtains the second feature of the data to be predicted, and returns the second feature of the data to be predicted to the first node. Then, the first node receives the second feature of the data to be predicted, and inputs the first feature of the data to be predicted into the adaptation network corresponding to the trained first node, obtains the first target feature of the data to be predicted, and inputs the second feature of the data to be predicted into the adaptation network corresponding to the trained second node, obtains the second target feature of the data to be predicted, and then uses the first target feature of the data to be predicted and the second target feature of the data to be predicted as input, inputs them into the trained gating network, obtains the weight of the first target feature, and the weight of the second target feature, and further, according to the weight of the first target feature and the weight of the second target feature, weighted concatenation is performed on the first target feature and the second target feature, and the concatenation result is input into the trained classifier to obtain the prediction result of the data to be predicted output by the classifier.

[0215] The above is a model training method provided in one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding model training device, such as Figure 6 and Figure 7 shown.

[0216] Figure 6 A schematic diagram of a model training device provided in this specification specifically includes:

[0217] A first sample determination module 400, configured to determine each first aligned sample, each labeled category of each first aligned sample, and each first non-aligned sample;

[0218] A first feature determination module 402 is used to input the first aligned samples to the first encoder to obtain the features of the first aligned samples, and input the first non-aligned samples to the first encoder to obtain the features of the first non-aligned samples;

[0219] A first prototype vector determination module 404 is used to determine, for each annotation category corresponding to each of the first aligned samples, a first prototype vector corresponding to the annotation category and used to characterize a feature center of the annotation category according to features of each of the first aligned samples included in the annotation category;

[0220] A first loss determination module 406 is configured to determine, for each first non-aligned sample, a first loss corresponding to the first non-aligned sample according to a difference between a first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and a feature of the first non-aligned sample;

[0221] The first training module 408 is configured to train the first encoder at least according to the first loss corresponding to each of the first non-aligned samples.

[0222] Optionally, the first prototype vector determination module 404 is specifically used to obtain the first prototype vector corresponding to each annotation category obtained in the previous iterative training as the first historical prototype vector corresponding to each annotation category; for each annotation category corresponding to the first alignment samples, determine the statistical value of the features of each first alignment sample contained in the annotation category according to the features of each first alignment sample contained in the annotation category; determine the first prototype vector corresponding to the annotation category obtained in the current iterative training according to the statistical value of the features of each first alignment sample contained in the annotation category and the first historical prototype vector corresponding to the annotation category.

[0223] Optionally, the device further comprises:

[0224] The second prototype vector determination module 410 is specifically used to receive the features of each second aligned sample sent by the second node; for each labeled category corresponding to the first aligned samples, determine the second prototype vector corresponding to the labeled category for characterizing the feature center of the labeled category according to the features of each second aligned sample contained in the labeled category; send the second prototype vector corresponding to each labeled category to the second node, so that the second node trains the second encoder according to the second prototype vector corresponding to each labeled category.

[0225] Optionally, the second prototype vector determination module 410 is specifically used to obtain the second prototype vector corresponding to each annotation category obtained in the previous iterative training as the second historical prototype vector corresponding to each annotation category; determine the annotation category of each second alignment sample according to the annotation category of each first alignment sample and the correspondence between each first alignment sample and each second alignment sample; for each annotation category, determine the statistical value of the feature of each second alignment sample included in the annotation category according to the feature of each second alignment sample included in the annotation category; determine the second prototype vector corresponding to the annotation category obtained in the current iterative training according to the statistical value of the feature of each second alignment sample included in the annotation category and the second historical prototype vector corresponding to the annotation category.

[0226] Optionally, the device further comprises:

[0227] The first category determination module 412 is specifically used to determine, for each labeled category, a first local priori probability corresponding to the labeled category for characterizing the probability distribution of each first non-aligned sample belonging to the labeled category according to the first prototype vector of the labeled category and the characteristics of each first non-aligned sample; for each first non-aligned sample, determine the probability distribution of each labeled category respectively belonging to the first local priori probability corresponding to each labeled category, and determine the labeled category to which the first non-aligned sample belongs according to the probability distribution of each labeled category respectively belonging to the first non-aligned sample.

[0228] Optionally, the first category determination module 412 is specifically used to receive each second local prior probabilities sent by the second node, the second local prior probabilities are used to characterize the probability distribution of each second non-aligned sample belonging to each labeled category; for each labeled category, according to the first local prior probability of the labeled category and the second local prior probability of the labeled category, determine the global prior probability corresponding to the labeled category; according to the global prior probability corresponding to the labeled category, update the first local prior probability of the labeled category to obtain the updated first local prior probability of the labeled category; for each first non-aligned sample, according to the updated first local prior probabilities corresponding to the each labeled category, determine the probability distribution of the first non-aligned sample belonging to each labeled category.

[0229] Optionally, the first node further deploys a classifier;

[0230] Optionally, the first training module 408 is specifically used to receive features of each second aligned sample sent by the second node; obtain a prediction result through the classifier according to the features of each first aligned sample and the features of each second aligned sample, and determine the second loss according to the prediction result and the labeled category of each first aligned sample; train the first encoder according to the first loss corresponding to each first non-aligned sample and the second loss.

[0231] Optionally, the first node further deploys an adaptation network corresponding to the first node, an adaptation network corresponding to the second node, and a gating network;

[0232] Optionally, the first training module 408 is specifically used to input the features of the first aligned samples into the adaptation network corresponding to the first node to obtain the target features of the first aligned samples; input the features of the second aligned samples into the adaptation network corresponding to the second node to obtain the target features of the second aligned samples; input the target features of the first aligned samples and the target features of the second aligned samples as inputs to the gating network to obtain the weights of the first aligned samples and the weights of the second aligned samples output by the gating network; perform weighted splicing on the target features of the first aligned samples and the target features of the second aligned samples according to the weights of the first aligned samples and the weights of the second aligned samples, and input the splicing results into the classifier to obtain the prediction results output by the classifier.

[0233] Figure 7 A schematic diagram of a model training device provided in this specification specifically includes:

[0234] A second sample alignment module 500, configured to determine each second aligned sample and each second non-aligned sample;

[0235] A second feature determination module 502 is used to input the second aligned samples to the second encoder to obtain the features of the second aligned samples, and input the second non-aligned samples to the second encoder to obtain the features of the second non-aligned samples;

[0236] A receiving module 504 is used to send the features of each second aligned sample to the first node, and receive the second prototype vector corresponding to each labeled category returned by the first node, wherein the second prototype vector is determined by the first node according to the features of each second aligned sample and the labels of each first aligned sample, and is used to characterize the feature center of each labeled category corresponding to the second aligned sample;

[0237] A third loss determination module 506 is configured to determine, for each second non-aligned sample, a third loss corresponding to the second non-aligned sample according to a difference between a second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and a feature of the second non-aligned sample;

[0238] The second training module 508 is configured to train the second encoder at least according to the third loss corresponding to each of the second non-aligned samples.

[0239] Optionally, the device further comprises:

[0240] The second category determination module 510 is specifically used to determine, for each labeled category, according to the second prototype vector of the labeled category and the characteristics of each second non-aligned sample, a second local prior probability corresponding to the labeled category for characterizing the probability distribution of each second non-aligned sample belonging to the labeled category; for each second non-aligned sample, according to each second local prior probabilities corresponding to each labeled category, determine the probability distribution that the second non-aligned sample belongs to each labeled category respectively, and determine the labeled category to which the second non-aligned sample belongs according to the probability distribution that the second non-aligned sample belongs to each labeled category respectively.

[0241] Optionally, the second category determination module 510 is specifically used to send the second local prior probabilities corresponding to the respective labeled categories to the first node, and receive the global prior probabilities corresponding to the respective labeled categories returned by the first node, wherein the global prior probability is determined by the first node based on the first local prior probabilities corresponding to the respective labeled categories and the second local prior probabilities corresponding to the respective labeled categories; for each labeled category, according to the global prior probability corresponding to the labeled category, the second local prior probability of the labeled category is updated to obtain the updated second local prior probability of the labeled category; for each second non-aligned sample, according to the updated second local prior probabilities of the respective labeled categories, the probability distribution of the second non-aligned sample belonging to the respective labeled categories is determined.

[0242] Optionally, the second training module 508 is specifically used to receive a second loss sent by the first node, where the second loss is obtained by the first node through a classifier based on the characteristics of each first aligned sample and the characteristics of each second aligned sample, and is determined based on the difference between the predicted result and the labeled category of the first aligned sample; and the second encoder is trained based on the second loss and a third loss corresponding to each second non-aligned sample.

[0243] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 or Figure 3 The model training method shown.

[0244] This manual also provides Figure 8 The schematic structure diagram of the electronic device shown in FIG. Figure 8 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 or Figure 3Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0245] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0246] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0247] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0248] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0249] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0250] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0251] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0252] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0253] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0254] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0255] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0256] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0257] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0258] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0259] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0260] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A model training method, the method is applied to a first node, the first node at least deploys a first encoder, the method comprising: Determine each first aligned sample, each labeled category of each first aligned sample, and each first non-aligned sample; Inputting the first aligned samples to the first encoder respectively to obtain features of the first aligned samples, and inputting the first non-aligned samples to the first encoder respectively to obtain features of the first non-aligned samples; For each annotation category corresponding to each of the first aligned samples, determining a first prototype vector corresponding to the annotation category for representing a feature center of the annotation category according to features of each of the first aligned samples included in the annotation category; For each first non-aligned sample, determining a first loss corresponding to the first non-aligned sample according to a difference between a first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and a feature of the first non-aligned sample; The first encoder is trained at least according to a first loss corresponding to each of the first non-aligned samples.

2. The method according to claim 1, wherein for each annotation category corresponding to each of the first aligned samples, determining a first prototype vector corresponding to the annotation category for representing a feature center of the annotation category according to features of each of the first aligned samples contained in the annotation category, specifically comprises: Obtaining the first prototype vector corresponding to each of the labeled categories obtained in the last iterative training as the first historical prototype vector corresponding to each of the labeled categories; For each labeled category corresponding to each of the first aligned samples, according to the features of each of the first aligned samples included in the labeled category, determine a statistical value of the features of each of the first aligned samples included in the labeled category; According to the statistical values ​​of the features of each first aligned sample included in the labeled category and the first historical prototype vector corresponding to the labeled category, a first prototype vector corresponding to the labeled category obtained by the current iterative training is determined.

3. The method of claim 1, further comprising: receiving features of each second alignment sample sent by the second node; For each labeled category corresponding to each of the first aligned samples, determining a second prototype vector corresponding to the labeled category for representing a feature center of the labeled category according to features of each of the second aligned samples included in the labeled category; The second prototype vector corresponding to each labeled category is sent to the second node, so that the second node trains the second encoder according to the second prototype vector corresponding to each labeled category.

4. The method according to claim 3, wherein for each annotation category corresponding to each of the first aligned samples, determining a second prototype vector corresponding to the annotation category for representing the feature center of the annotation category according to the features of each of the second aligned samples contained in the annotation category, specifically comprises: Obtaining the second prototype vector corresponding to each labeled category obtained in the last iterative training as the second historical prototype vector corresponding to each labeled category; Determine the labeling category of each second aligned sample according to the labeling category of each first aligned sample and the corresponding relationship between each first aligned sample and each second aligned sample; For each annotation category, according to the features of each second aligned sample included in the annotation category, determine the statistical value of the features of each second aligned sample included in the annotation category; According to the statistical value of the features of each second aligned sample included in the labeled category and the second historical prototype vector corresponding to the labeled category, the second prototype vector corresponding to the labeled category obtained by the current iterative training is determined.

5. The method according to claim 1, before determining the first loss corresponding to each first non-aligned sample according to the difference between the first prototype vector corresponding to the labeled category to which the first non-aligned sample belongs and the feature of the first non-aligned sample, the method further comprises: For each labeled category, according to a first prototype vector of the labeled category and features of each of the first non-aligned samples, determine a first local prior probability corresponding to the labeled category for characterizing a probability distribution that each of the first non-aligned samples belongs to the labeled category; For each first non-aligned sample, the probability distribution of the first non-aligned sample belonging to each labeled category is determined according to the first local prior probabilities corresponding to the labeled categories, and the labeled category to which the first non-aligned sample belongs is determined according to the probability distribution of the first non-aligned sample belonging to each labeled category.

6. The method according to claim 5, wherein for each first non-aligned sample, determining the probability distribution of the first non-aligned sample belonging to each labeled category according to the first local prior probabilities corresponding to each labeled category, specifically comprises: receiving each second local priori probability sent by the second node, where the second local priori probability is used to characterize the probability distribution that each second non-aligned sample belongs to each labeled category; For each annotation category, determine the global prior probability corresponding to the annotation category according to the first local prior probability of the annotation category and the second local prior probability of the annotation category; According to the global prior probability corresponding to the annotation category, the first local prior probability of the annotation category is updated to obtain the updated first local prior probability of the annotation category; For each first non-aligned sample, the probability distribution that the first non-aligned sample belongs to each of the labeled categories is determined according to the updated first local prior probabilities corresponding to each of the labeled categories.

7. The method of claim 1, wherein the first node further deploys a classifier; The step of training the first encoder at least according to the first loss corresponding to each of the first non-aligned samples specifically includes: receiving features of each second alignment sample sent by the second node; According to the features of the first aligned samples and the features of the second aligned samples, a prediction result is obtained by the classifier, and a second loss is determined according to the prediction result and the labeled category of the first aligned samples; The first encoder is trained according to the first loss corresponding to each of the first non-aligned samples and the second loss.

8. The method of claim 7, wherein the first node further deploys an adaptation network corresponding to the first node, an adaptation network corresponding to the second node, and a gating network; The obtaining a prediction result by the classifier according to the features of each of the first aligned samples and the features of each of the second aligned samples specifically includes: Inputting the features of each of the first aligned samples into the adaptation network corresponding to the first node to obtain the target features of each of the first aligned samples; Inputting the features of each second aligned sample into the adaptation network corresponding to the second node to obtain the target features of each second aligned sample; Taking the target features of the first aligned samples and the target features of the second aligned samples as inputs, and inputting them into the gating network, obtaining the weights of the first aligned samples and the weights of the second aligned samples output by the gating network; According to the weights of the first aligned samples and the weights of the second aligned samples, weighted splicing is performed on the target features of the first aligned samples and the target features of the second aligned samples, and the splicing result is input into the classifier to obtain the prediction result output by the classifier.

9. A model training method, the method being applied to a second node, the second node deploying a second encoder, the method comprising: Determining each second aligned sample and each second non-aligned sample; Inputting the second aligned samples to the second encoder respectively to obtain features of the second aligned samples, and inputting the second non-aligned samples to the second encoder respectively to obtain features of the second non-aligned samples; Sending the features of each second aligned sample to the first node, and receiving the second prototype vector corresponding to each labeled category returned by the first node, wherein the second prototype vector is determined by the first node according to the features of each second aligned sample and the labels of each first aligned sample, and is used to characterize the feature center of each labeled category corresponding to the second aligned sample; For each second non-aligned sample, determining a third loss corresponding to the second non-aligned sample according to a difference between a second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and a feature of the second non-aligned sample; The second encoder is trained at least according to a third loss corresponding to each of the second non-aligned samples.

10. The method of claim 9, wherein before determining the third loss corresponding to each second non-aligned sample according to the difference between the second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and the feature of the second non-aligned sample, the method further comprises: For each labeled category, according to the second prototype vector of the labeled category and the characteristics of each second non-aligned sample, determine a second local prior probability corresponding to the labeled category for characterizing the probability distribution that each second non-aligned sample belongs to the labeled category; For each second non-aligned sample, the probability distribution of the second non-aligned sample belonging to each labeled category is determined according to the second local prior probabilities corresponding to the labeled categories, and the labeled category to which the second non-aligned sample belongs is determined according to the probability distribution of the second non-aligned sample belonging to each labeled category.

11. The method according to claim 10, wherein for each second non-aligned sample, determining the probability distribution of the second non-aligned sample belonging to each labeled category according to each second local prior probability corresponding to each labeled category, specifically comprises: Sending the second local prior probabilities corresponding to the respective annotation categories to the first node, and receiving the global prior probabilities corresponding to the respective annotation categories returned by the first node, wherein the global prior probabilities are determined by the first node according to the first local prior probabilities corresponding to the respective annotation categories and the second local prior probabilities corresponding to the respective annotation categories; For each annotation category, according to the global prior probability corresponding to the annotation category, the second local prior probability of the annotation category is updated to obtain the updated second local prior probability of the annotation category; For each second non-aligned sample, the probability distribution that the second non-aligned sample belongs to each of the labeled categories is determined according to the updated second local prior probabilities of each of the labeled categories.

12. The method according to claim 9, wherein the training of the second encoder at least according to the third loss corresponding to each of the second non-aligned samples comprises: receiving a second loss sent by the first node, where the second loss is determined by the first node obtaining a prediction result through a classifier according to the features of each first aligned sample and the features of each second aligned sample, and according to a difference between the prediction result and the labeled category of the first aligned sample; The second encoder is trained according to the second loss and a third loss corresponding to each of the second non-aligned samples.

13. A model training device, the device is applied to a first node, the first node at least deploys a first encoder, the device comprising: A first sample determination module, used to determine each first aligned sample, each labeled category of each first aligned sample, and each first non-aligned sample; A first feature determination module, configured to input the first aligned samples to the first encoder to obtain features of the first aligned samples, and input the first non-aligned samples to the first encoder to obtain features of the first non-aligned samples; A first prototype vector determination module is used to determine, for each annotation category corresponding to each of the first aligned samples, a first prototype vector corresponding to the annotation category and used to characterize a feature center of the annotation category according to features of each of the first aligned samples included in the annotation category; A first loss determination module is used to determine, for each first non-aligned sample, a first loss corresponding to the first non-aligned sample according to a difference between a first prototype vector corresponding to a labeled category to which the first non-aligned sample belongs and a feature of the first non-aligned sample; The first training module is configured to train the first encoder at least according to a first loss corresponding to each of the first non-aligned samples.

14. A model training device, the device is applied to a second node, the second node deploys a second encoder, the device comprises: A second sample alignment module, used for determining each second aligned sample and each second non-aligned sample; A second feature determination module, configured to input the second aligned samples to the second encoder to obtain features of the second aligned samples, and input the second non-aligned samples to the second encoder to obtain features of the second non-aligned samples; A receiving module, configured to send the features of each second aligned sample to the first node, and receive a second prototype vector corresponding to each labeled category returned by the first node, wherein the second prototype vector is determined by the first node according to the features of each second aligned sample and the labels of each first aligned sample, and is used to characterize the feature center of each labeled category corresponding to the second aligned sample; A third loss determination module is used to determine, for each second non-aligned sample, a third loss corresponding to the second non-aligned sample according to a difference between a second prototype vector corresponding to the labeled category to which the second non-aligned sample belongs and a feature of the second non-aligned sample; The second training module is used to train the second encoder at least according to the third loss corresponding to each second non-aligned sample.

15. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 8 or 9 to 12.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 8 or 9 to 12 when executing the program.

Images