Method and device for updating model based on secret sharing algorithm, equipment and medium

Abstract

The embodiment of the invention discloses a method and device for updating a target model based on a secret sharing algorithm, equipment and a medium, the target model comprises a sub-model based on a factorization machine, and the scheme can comprise the following steps: obtaining a first data fragment; the first data fragment comprises a first sample fragment of training sample data and a first parameter fragment of a model parameter of the sub-model; based on the first data fragment and the second data party holding the second data fragment, using a first secret sharing algorithm to calculate and obtain a first predicted value fragment; based on the first predicted value fragment and the second data party holding the second predicted value fragment, a second secret sharing algorithm is used to calculate a first residual fragment; and updating the first parameter fragment by using the first residual fragment to obtain an updated target model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and medium for updating a model based on a secret sharing algorithm. Background Art

[0002] With the popularization of computing devices and the implementation of applications related to daily life, the speed and volume of digital information generated have increased dramatically. For example, online video-on-demand services record users' playback behavior, social network services collect friend relationships, and travel service platforms obtain passengers' location information. Service providers use this data to improve application performance and service quality, thereby improving user experience through computing methods such as training or improving recommendation systems.

[0003] These informationized data bring many conveniences to people's lives, but some data or even most of the data may contain sensitive information, which is inconvenient to share with other service providers or cannot obtain data held by other service providers. This may cause the recommendation system to not meet the actual needs of users and the user experience to be poor. Therefore, how to update model training while ensuring data privacy and security is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of this specification provide a method, apparatus, device and medium for updating a target model based on a secret sharing algorithm to solve the data privacy and security issues existing in existing model updating methods.

[0005] To solve the above technical problems, the embodiments of this specification are implemented as follows: An embodiment of the present specification provides a method for updating a target model based on a secret sharing algorithm. The target model may include a sub-model based on a factorization machine. The method may involve a first data entity and a second data entity. The method may be applied to the first data entity and may include: Acquire a first data slice; the first data slice includes a first sample slice of training sample data and a first parameter slice of a model parameter of the sub-model; the training sample data includes a plurality of first feature data items for a target object held by the second data party; A first prediction value shard is calculated based on the first data shard and the second data party holding the second data shard using a first secret sharing algorithm; the first prediction value shard is used to represent a data shard of the prediction value of the target model; A first residual shard is calculated based on the first predicted value shard and the second data party holding the second predicted value shard using a second secret sharing algorithm; the first residual shard is a data shard used to represent the residual of the target model; the second predicted value shard is another data shard of the predicted value of the target model; The first parameter slice is updated using the first residual slice to obtain an updated target model.

[0006] An embodiment of the present specification provides a device for updating a target model based on a secret sharing algorithm, wherein the target model includes a sub-model based on a factorization machine, and the device may include: A data acquisition module, used to acquire a first data slice; the first data slice includes a first sample slice of training sample data and a first parameter slice of a model parameter of the sub-model; the training sample data includes a plurality of first feature data items for a target object held by the second data party; A prediction value calculation module, configured to calculate a first prediction value slice based on the first data slice and the second data party holding the second data slice using a first secret sharing algorithm; the first prediction value slice is used to represent a data slice of the prediction value of the target model; A residual calculation module, used to calculate a first residual shard based on the first prediction value shard and the second data party holding the second prediction value shard using a second secret sharing algorithm; the first residual shard is a data shard used to represent the residual of the target model; the second prediction value shard is another data shard of the prediction value of the target model; A parameter updating module is used to update the first parameter slice using the first residual slice to obtain an updated target model.

[0007] An apparatus for updating a target model based on a secret sharing algorithm provided in an embodiment of this specification may include: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Acquire a first data slice; the first data slice includes a first sample slice of training sample data and a first parameter slice of a model parameter of the sub-model; the training sample data includes a plurality of first feature data items for a target object held by the second data party; A first prediction value shard is calculated based on the first data shard and the second data party holding the second data shard using a first secret sharing algorithm; the first prediction value shard is used to represent a data shard of the prediction value of the target model; A first residual shard is calculated based on the first predicted value shard and the second data party holding the second predicted value shard using a second secret sharing algorithm; the first residual shard is a data shard used to represent the residual of the target model; the second predicted value shard is another data shard of the predicted value of the target model; The first parameter slice is updated using the first residual slice to obtain an updated target model.

[0008] An embodiment of the present specification provides a computer-readable medium on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement a method for updating a target model based on a secret sharing algorithm.

[0009] At least one embodiment of the present specification can achieve the following beneficial effects: the target model to be updated in the embodiments of the present specification can include a sub-model based on a factor decomposition machine, the first data party and the second data party can collaboratively calculate the model prediction value based on the training data using the first secret sharing algorithm, the first data party and the second data party can respectively obtain data slices of the prediction value, and both parties can also use the second secret sharing algorithm to calculate the residual of the model, the first data party can obtain the data slice of the residual, and then can use the data slice of the residual to update the slice of the model parameter obtained in the process of calculating the model prediction value, and then can update the target model. In this way, if the training data contains feature data held by the second data party, the feature data will not be directly exposed to the first data party, but the first data party can also update the model, and then can perform model training updates under the premise of ensuring data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 A schematic diagram of an application scenario of a method for updating a target model based on a secret sharing algorithm provided in an embodiment of this specification; Figure 2 A flowchart of a method for updating a target model based on a secret sharing algorithm provided in an embodiment of this specification; Figure 3 A schematic diagram of the structure of a device for updating a target model based on a secret sharing algorithm provided in an embodiment of this specification; Figure 4 A schematic diagram of the structure of a device for updating a target model based on a secret sharing algorithm provided in an embodiment of this specification. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solutions and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of one or more embodiments of this specification.

[0013] In order to more clearly understand the technical solutions provided in this specification, some technical terms are explained here: Factorization Machine (FM): is a common marketing recommendation algorithm. The factorization machine learns a hidden weight vector for each feature, and when features are crossed, the inner product of the two feature hidden vectors is used as the weight of the crossed features.

[0014] Deep Factorization Machine (DeepFM): is a common recommendation marketing algorithm. The Deep Factorization Machine is an improvement on the FM algorithm, with both memory and generalization capabilities, as well as automatic feature combination capabilities. It can be understood that the Deep Factorization Machine contains the basic structure of the Factorization Machine, which is based on the Factorization Machine combined with the Deep Neural Network DNN. The DeepFM model includes the FM model as a sub-model, and also includes the Deep Neural Network DNN as another sub-model.

[0015] Hidden variables / embedding: It is a basic structure of machine learning / deep learning. The embedding in factorization machine and deep factorization machine captures the dense representation of features.

[0016] Secret Sharing: is a privacy protection technology. Secret sharing hides secret values ​​by splitting them into multiple random parts and distributing them to different parties. It can include secret addition, secret multiplication, secret inner product, etc. Secret addition can also be called additive secret sharing algorithm, which can be used for secret sharing to calculate the sum of at least two data without leaking the calculated data; secret multiplication and secret inner product can also be called multiplicative secret sharing algorithm, which can be used for secret sharing to calculate the product of at least two data or vectors without leaking the calculated data. Such as triple secret sharing algorithm, etc.

[0017] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0018] In the prior art, in order to obtain a model or recommendation system that better meets the needs, it may be necessary to combine the data of other data parties for model training. A common method is a centralized factorization machine or deep factorization machine. The general idea is to first collect the user interaction data to a central server, and then centrally train the factorization machine or deep factorization machine. However, this method does not use privacy protection technology, and directly trains and recommends on the original data. As the protection of personal data has attracted people's attention, privacy protection laws and regulations have been promulgated one after another, giving rise to the problem of data islands, resulting in these privacy-containing data being isolated between different organizations and institutions, and cannot be sent directly to each other, and will not be used in model training.

[0019] In order to solve the defects in the prior art, this solution provides the following embodiments.

[0020] Figure 1 Schematic diagram of an application scenario of a method for updating a target model based on a secret sharing algorithm provided in an embodiment of this specification. Figure 1As shown, the scheme may include a first data party 1 and a second data party 2. Among them, the first data party 1 can use the feature data held by the second data party 2 to train the model. Specifically, the first data party 1 can obtain the data slice containing the feature data of the second data party and the data slice of the corresponding model parameters, and then cooperate with the second data party 2 to calculate the predicted value of the model using the secret sharing algorithm. The first data party 1 can obtain a data slice of the model predicted value, and the second data party 2 can obtain another data slice of the model predicted value. Then, based on the obtained data slice of the predicted value, the residual is calculated using the secret sharing algorithm. The first data party 1 can obtain a data slice of the residual, and the second data party can also obtain another data slice of the residual. Then, both parties can update the slices of the model parameters held by each party based on the obtained residual slice, and then the model update can be completed. In this way, the feature data of the second data party will not be directly exposed to the first data party, and data security can be guaranteed. If the feature data of the second data party contains privacy data, the model can also be updated on the basis of complying with relevant regulations, which is also conducive to training a model that better meets business needs and improves the experience of business users.

[0021] Next, a method for updating a target model based on a secret sharing algorithm provided in an embodiment of the specification will be specifically described with reference to the accompanying drawings.

[0022] Figure 2 A flowchart of a method for updating a target model based on a secret sharing algorithm provided in an embodiment of this specification. From a program perspective, the execution subject of the process can be a program or application client mounted on an application server. The target model in the embodiment of this specification can include a sub-model based on a factorization machine, the method can involve at least a first data party and a second data party, and the execution subject of this embodiment can be a server of the first data party or a server for processing data required by the first data party.

[0023] like Figure 2 As shown, the process may include the following steps.

[0024] Step 202: Obtain a first data slice.

[0025] The first data segment may include training sample data The first sample fragment and the model parameters of the sub-model The first parameter slice The training sample data may include several first feature data items for the target object held by the second data party. .

[0026] The model parameters may be the initial parameters of the model set by the second data party or the first data party, or the model parameters determined before the current model update training. In the embodiment of this specification, the second data party may shard the first feature data, and the data party holding the model parameters may shard the model parameters, wherein the data may be sharded according to a preset privacy computing protocol. For any feature data in each item of the first feature data, a data shard of the arbitrary feature data may be provided to the first data party, or a data shard of the model parameters may be provided to the first data party, so that the first data shard held by the first data party may include the first sample shard of the training sample data. and the first parameter slice of the model parameters . First sample fragment The first feature fragment that can contain the first feature data .

[0027] In practical applications, the training data can be in matrix form or in vectorized matrix form, and the first data shard can also be in matrix form. Data shards can be understood as a number of sub-data obtained based on the original data, the dimension or size of each sub-data is the same as the original data, and the sum of each sub-data is equal to the original data. The training data or model parameters can be sharded according to the number of participants in the secret sharing algorithm.

[0028] Step 204: Based on the first data shard, use a first secret sharing algorithm with the second data party holding the second data shard to calculate a first prediction value shard.

[0029] Among them, the first prediction value slice can be used to represent a data slice of the prediction value of the target model; the second data slice may include a second feature slice of the training sample data and a second parameter slice of the model parameter. The model prediction value may represent the output result obtained after the training data is input into the model. In the embodiment of this specification, the second data party may hold a second data slice corresponding to the first data slice held by the first data party, and the second data slice may include a second sample slice of the training data. And the model parameters The second parameter of the slice Then the first data party and the second data party can collaboratively use the secret sharing algorithm to calculate the model prediction value , so that the first data party can get the model prediction value A data shard, which can be called the first prediction value shard .

[0030] In practical applications, if there are multiple data parties participating in the secret sharing algorithm, each data party can obtain a data slice of the model prediction result. Here, only the first data party and the second data party are used as examples for illustration, and are not used to limit the number of data parties participating in the calculation. The above-mentioned sample slices can be data slices obtained by slicing the sample data, or they can be data slices obtained by adding some disturbances on the basis of the obtained data slices, such as adding or subtracting a preset disturbance amount to obtain feature slices. Similarly, parameter slices can be data slices obtained by slicing the model parameters, or they can be parameter slices obtained by adding some disturbances on the basis of the obtained data slices. The slicing processing and disturbance processing here can be performed according to the existing method or computing protocol, and are not specifically limited here.

[0031] Step 206: Based on the first prediction value slice, the second data party holding the second prediction value slice uses a second secret sharing algorithm to calculate a first residual slice.

[0032] Among them, the first residual slice can be used to characterize a data slice of the residual of the target model; the second predicted value slice can be another data slice of the model predicted value. In machine learning, the residual refers to the difference between the actual observation value and the model predicted value, which can be a positive number or a negative number. The actual observation value can be understood as the label value of the training sample, which is known. In the embodiment of this specification, the first data party and the second data party can use a secret sharing algorithm to calculate the residual of the model, so that the first data party can obtain a data slice of the residual, and the second data party can obtain another data slice of the residual. The sum of each residual data slice is equal to the residual.

[0033] Step 208: Use the first residual slice to update the first parameter slice to obtain an updated target model.

[0034] The first data party can use the obtained residual slice to update the held first parameter slice according to the preset parameter update rules to obtain an updated target model. The first parameter slice obtained by the first data party can be understood as the model parameter to be updated, and the first residual slice can be understood as the residual used to update the model parameter. The model parameter can be updated based on the existing parameter update method, such as calculating the gradient based on the residual. As an implementation method, the second data party in the embodiment of this specification can also obtain another residual slice of the residual, which can be called the second residual slice, and the second data party can also update the second parameter slice based on the second residual slice.

[0035] It should be understood that the order of some steps in the methods described in one or more embodiments of this specification can be interchanged according to actual needs, or some steps can be omitted or deleted.

[0036] Figure 2 In the method, the target model to be updated may include a sub-model based on a factorization machine. The first data party and the second data party may collaboratively calculate the model prediction value based on the training data using the first secret sharing algorithm. Both parties may obtain data slices of the prediction value. Both parties may also use the second secret sharing algorithm to calculate the residual of the model. The first data party may obtain data slices of the residual, and then use the data slices of the residual to update the slices of the model parameters obtained in the process of calculating the model prediction value, and then update the target model. In this way, if the training data contains feature data held by the second data party, the feature data will not be directly exposed to the first data party, but the first data party can also update the model, and then the model training and update can be performed under the premise of ensuring data security.

[0037] In actual applications, different institutions or servers may have different features and are unwilling to share them directly with each other, but these features can be mutually reinforcing, such as transaction features, credit features, and user portraits, which can collaboratively improve the quality of recommendations in order to provide users with better services. The recommendation algorithm can choose a common factor decomposition machine or a deep factor decomposition machine. The target model in the embodiments of this specification can be a model that includes a factor decomposition machine. The privacy protection method used is selected to have both formal privacy guarantees, model accuracy that is equal to non-privacy, and secret sharing with low computational cost. The method in the embodiments of this specification can effectively ensure the security of training data. The above-mentioned training sample data may include transaction data, credit data, user portrait data, etc., and these data are used as training sample data only after the user authorizes their use.

[0038] The model training method provided in the embodiments of this specification can collaboratively build a recommendation system under the premise of protecting data privacy across institutions. It can be applied to scenarios where one participant holds some features and labels, and other participants each hold some features to collaboratively build a recommendation system in a way that protects the data privacy of all parties. For example, one participant holds the transaction record and credit score of the user in the first application, and the other participant holds the bank's transaction record. They hope to collaboratively improve the recommendation system used to determine the user's credit score under the premise of protecting the data privacy of all parties, which can also be understood as an evaluation system.

[0039] It can also be applied to scenarios where one participant holds a label and the other participants each hold some features, and collaboratively build a recommendation system in a way that protects the data privacy of each party. For example, an advertising provider places an advertisement in an advertising space of a terminal application. The server of the terminal application holds the user's click record in the terminal application. The advertising provider has its own features for the user. They hope to collaboratively improve the efficiency of personalized advertising delivery under the premise of protecting the data privacy of all parties. One of the participants in this scenario may not have the user's feature data, but only hold the label value, and the other participants may hold the user's feature data. This scenario can be understood as a special case of the above scenario, in which the feature dimension held by one of the participants can be 0. In these application scenarios, the above-mentioned first data party can be one of the above-mentioned participants, and the second data party can be another participant. For example, the first data party can be a participant holding a label, and the second data party can be a participant holding the user's feature data; for another example, the second data party can be a participant holding a label, and the first data party can be a participant holding the user's feature data.

[0040] It can be understood that the steps executed by the data parties participating in the secret sharing algorithm are symmetrical or similar, and the execution subject in the above embodiment can represent any party participating in the secret sharing algorithm.

[0041] After the model training or parameter update meets the requirements, the target model can be used to process the corresponding business, such as recommendation, evaluation business, etc. When actually processing the recommendation or evaluation business, the data slices of the acquired feature data can be provided to the first data party and the second data party respectively, and then the first data party and the second data party can obtain the predicted value slices based on their respective updated parameters, and then the predicted value slices can be provided to the evaluation party or the prediction party, which can also be the first data party or the second data party, and the predicted value can be obtained by adding up the predicted value slices for subsequent business processing.

[0042] The factorization machine FM is a model based on second-order polynomial regression. It can be a conventional linear regression with a binary combination of features added. The equation of the factorization machine can be expressed as: (1) Among them, n represents the number of feature items of the training sample, which can also be understood as the feature dimension, and xi is the feature value of the i-th feature; , , are model parameters; where and It can represent the first-order feature weight; Can represent the second-order feature weight. can represent the linear model part, Can represent feature cross-model parts.

[0043] On this basis, in the embodiments of this specification, the above formula (1) can be transformed by vectorizing the feature relationship to obtain: (2) It can represent the hidden vector corresponding to the i-th feature or called parameter vector. The dimension of the hidden vector is , .

[0044] The second-order feature crossover can be further simplified to obtain: (3) in, Represents a k-dimensional vector The f-th dimension element in .

[0045] In practical applications, if one party holds the feature data, the parameter terms and feature data terms in the above factorization machine formula can be replaced with the data of the data party. Taking the above formula (3) as an example, assuming that the second data party holds the feature data matrix , the equation of the factorization machine can be expressed as: (4) in, The characteristic data matrix that can represent the second data The j-th dimension feature data in , where the value of j can be 1 to m; A linear shift parameter for characteristic data of a second data cube may be represented; It can represent the j-th dimension feature data in the feature data held by the second data party The corresponding linear parameters; It can be expressed for the j-th dimension feature data The k-dimensional vector The f-th dimension element in .

[0046] Based on the same idea, if the first data party and the second data party both hold some feature data, as an implementation method, assuming that the first data party holds the feature data matrix , the second data party holds the feature data matrix , the equation of the factorization machine can be expressed as: (5) in, The characteristic data matrix of the first data can be represented The i-th dimension feature data in , the value of i can be 1 to n; A linear shift parameter for characteristic data of the first data party may be represented; It can represent the i-th dimension feature data in the feature data held by the first data party The corresponding linear parameters; It can be expressed for the feature data of the i-th dimension The k-dimensional vector The f-th dimension element in . Can represent the i-th dimension feature data The corresponding hidden vector is Can represent the j-th dimension feature data The corresponding latent vector. For other parameters, please refer to the description of formula (4) and will not be repeated here.

[0047] For ease of understanding, the above formula (5) can be understood as three sub-parts, where the calculation result of the first row is only related to the data of the first data party, and can represent the sub-model part of the factor decomposition machine for the feature data of the first data party; the calculation result of the second row is only related to the data of the second data party, and can represent the sub-model part of the factor decomposition machine for the feature data of the second data; the calculation result of the third row is related to the data of both parties, and can represent the cross-part of the features of both the first data party and the second data party.

[0048] It is understandable that in actual applications, the data held by the data party and the corresponding parameters can also be applied to the above formulas (1) and (2), and the factor decomposition equation for the data party can also be obtained, which is similar to the above formula (3) as an example, and will not be described in detail here. The sub-model based on the factor grading machine included in the target model in the embodiment of this specification can be a representation of any one of the above formulas, which is not specifically limited here.

[0049] based on Figure 2 The present specification also provides some specific implementation schemes of the method, which are described below.

[0050] As an implementation method, if the training data includes the first feature data of the second data party, the model parameters may include the first linear parameters and the first latent vector parameters, and the target model to be updated may include the first sub-model of the factor decomposition machine for the first feature data, such as the above formula (4). Optionally, the above-mentioned first prediction value shard is calculated based on the first data shard and the second data party holding the second data shard using the first secret sharing algorithm, which may specifically include: based on the first feature shard, the first linear parameter shard and the first latent vector parameter shard, the first prediction value shard is calculated using the additive secret sharing algorithm and the multiplicative secret sharing algorithm with the second data party.

[0051] Among them, the first linear parameter slice The first linear parameter in the first submodel In practical applications, each training data may correspond to a linear parameter, and the first linear parameter slice may be a set or matrix of data slices containing linear parameters corresponding to each training data. can be the first latent vector in the first sub-model data sharding.

[0052] The second data party may hold another data shard of the first linear parameter, which may be referred to as a second linear parameter shard. , can also hold another data slice of the first latent vector, which can be called the second latent vector parameter slice The two parties can predict the value of the calculation model in collaboration based on data sharding and the preset secret sharing algorithm protocol.

[0053] In practical applications, if the factor decomposition machine model also includes a linear offset parameter, the first data cube can also obtain a first linear offset parameter for the first feature data. The first linear offset slice , the second data party can get the second linear offset fragment In this way, the first data party can calculate the predicted value of the target model by using the secret sharing algorithm in collaboration with the second data party based on the first feature slice, the first linear offset slice, the first linear parameter slice and the first latent vector parameter slice.

[0054] In practical applications, the factor decomposition machine can be a linear model with a feature cross model added to it. As an implementation method, the target model can include a first sub-model of the factor decomposition machine for the first feature data; the first sub-model includes a first linear sub-model and a first feature cross sub-model; the prediction value of the target model is calculated based on the first data shard and the second data party holding the second data shard using a first secret sharing algorithm to obtain a first prediction value shard, which can specifically include: Based on the first feature slice and the first linear parameter slice, a first linear sub-prediction value slice is calculated with the second data party using a multiplicative secret sharing algorithm; the first linear sub-prediction value slice is used to represent a data slice of the prediction value of the linear sub-model; Based on the first feature shard and the first latent vector parameter shard, the first feature cross-sub prediction value shard is calculated with the second data party using a multiplicative secret sharing algorithm and an additive secret sharing algorithm; the first feature cross-sub prediction value shard is used to represent a data shard of the prediction value of the first feature cross-sub model; the second data party holds the second latent vector parameter shard, and the first latent vector parameter shard and the second latent vector parameter shard are data shards of the first latent vector parameter in the model parameters; Based on the first linear sub-prediction value slice and the first feature cross sub-prediction value slice, the first prediction value slice is calculated with the second data party using an additive secret sharing algorithm.

[0055] Among them, the second data cube can hold the second linear parameter slice corresponding to the first linear parameter slice; the first linear parameter slice and the second linear parameter slice are data slices of the first linear parameter in the model parameters. The second data cube can also hold the second linear sub-prediction value slice and the second feature cross-prediction value slice; the first linear sub-prediction value slice and the second linear sub-prediction value slice are the calculation results of the linear feature sub-model, which can be called the data slices of the prediction value of the linear feature sub-model; the first feature cross-prediction value slice and the second unilateral feature cross-prediction value slice are the calculation results or prediction value data slices of the feature cross-model.

[0056] The linear sub-model of the factorization machine can be expressed as the product of the training sample data and the corresponding linear parameters. If there is a linear offset parameter, the linear sub-model can also be expressed as the sum of the product of the training sample data and the corresponding linear parameters and the linear offset parameter. The linear offset parameter can be zero or non-zero data, which can be set according to actual needs.

[0057] The feature cross sub-model can be the calculation part of the training data and the corresponding latent vector, as shown in the above formula (1): part, or the equivalent deformation part of this part. The above formula (2) In this part, formula (3) This part, etc., will not be listed here one by one.

[0058] If the training data includes the first feature data, the above feature cross sub-model can represent the calculation part of the first feature data and the corresponding latent vector, such as This part.

[0059] Assume that the training sample data includes several first feature data items for the target object held by the second data party The second data cube can also set the initial parameters of the model, assuming that the first sub-linear offset parameters are included , the first sublinear parameter And the first latent vector The second data party can provide the first feature data and data slices of each parameter to the first data party according to the protocol of the preset secret sharing algorithm, or provide the data slices to the first data party after further processing. If a triple secret sharing algorithm is used, the data slices obtained by the first data party can be data slices after disturbance processing.

[0060] Specifically, the first data slice obtained by the first data party may include the above-mentioned first feature slice, the first linear parameter slice and the first latent vector parameter slice, wherein the above-mentioned first feature slice may include the first sub-feature slice of the first feature data , the first linear parameter slice may include the first sub-linear parameter The first sublinear parameter slice of , the first latent vector parameter slice may include the first sub-latent vector parameter The first sub-latent vector element slice The first feature slice may also include a first sub-offset parameter slice of a first sub-offset parameter Correspondingly, the second data slice obtained by the second data party may include a second feature slice, a second linear parameter slice, and a second latent vector parameter slice, wherein the second feature slice may include a second sub-feature slice of the first feature data , the second linear parameter slice may include the first sub-linear parameter The second sublinear parameter shard , the first sub-latent vector parameter slice may include the second sub-latent vector parameter The first sub-latent vector element slice , and may also include a second sub-offset parameter fragment of the first sub-offset parameter .

[0061] Here, the above formula (4) is used as an example to illustrate that the first data party and the second data party can be based on data sharding and And parameter sharding and , use secret sharing matrix multiplication to calculate the product of each first feature data and the corresponding linear parameter, that is, , so that the first data party can obtain a multiplication slice according to the preset secret sharing calculation protocol , the second data party can obtain another multiplication slice according to the preset secret sharing calculation protocol Then, both parties can use the additive secret sharing algorithm to calculate the sum of the products of each feature data and the corresponding linear parameters based on the obtained product slices, that is, , so that the first data party can get a product and a slice , the second data party can get another product and fragment As another implementation method, the first data party may obtain the multiplication of each feature data into slices. By adding them locally, the product and the slice can also be obtained. Similarly, the second data cube can also add the product slices locally to obtain another product and the slice.

[0062] The first data party and the second data party can also be fragmented based on the offset parameter and , and the resulting product and slice and , the additive secret sharing algorithm can be used to calculate the predicted value of the linear sub-model in the factorization machine, that is, , so that the first data cube can obtain the first linear model data slice , the second data party can get another linear model data slice .

[0063] For the feature intersection part in formula (4), the first data cube and the second data cube can be segmented based on the latent vector elements and And feature data sharding and , using the multiplicative secret sharing algorithm to calculate the hidden vector elements With characteristic data product , the product can be regarded as an atomic term. In this way, the first data party can obtain a data slice of the atomic term, which can be called the first atomic term slice. , the second data party can get another data shard, which can be called the third atomic item shard Then, both parties can perform local addition based on the data fragments of the atomic items corresponding to each feature data, or both parties can use the additive secret sharing algorithm to calculate the sum of the atomic items corresponding to each feature data, so that the first data party can obtain the data fragments of the sum of the atomic items. , the second data party can get another data shard Then, based on these two data shards, the first data party and the second data party can use the multiplicative secret sharing algorithm to calculate the square of the sum of the atomic terms, and the first data party can obtain the data shard of the sum of the atomic terms. , the second data party can get another data shard .

[0064] Based on the data sharding of the above atomic terms, the first data party and the second data party can calculate the square of the atomic term using secret sharing matrix multiplication according to the preset secret algorithm protocol. , so that the first data square can get the data slice of the square of the atomic term , the second data party can get another data shard ; Then the first data party and the second data party can calculate the sum of the squares of the atomic terms corresponding to each data feature locally. Both parties can also use the additive secret sharing algorithm to calculate the sum of the squares of the atomic terms corresponding to each data feature. In this way, the first data party can obtain a data slice of the sum of the squares of the atomic terms. , the second data party can get another data shard .

[0065] The first data party obtains data fragments based on and , and the second data party based on the obtained data fragmentation and , both parties can calculate the difference between the sum of the squares of the atomic terms and the sum of the squares of the atomic terms based on the additive secret sharing algorithm, so that the first data party can obtain the data fragmentation of the intermediate data , the second data party can get another data shard ;in, Represents the feature data for the jth dimension The k-dimensional vector The f-th dimension element in can be iteratively calculated based on the above-mentioned similar algorithm to obtain the k-dimensional vector Then the first data party and the second data party can calculate the sum of the data slices of the intermediate data corresponding to the elements of each dimension in their respective local areas, or use the additive secret sharing algorithm to calculate the sum of the intermediate data corresponding to the elements of each dimension, so that the first data party can obtain the first feature cross sub-model data slice of the feature cross sub-model. , the second data party can also get another corresponding data shard .

[0066] The first data party is based on the data sharding of the obtained linear sub-model And data sharding for feature cross sub-models , the second data party is based on the other data shards of the obtained linear sub-model And additional data sharding for feature cross sub-models , both parties can use the additive secret sharing algorithm to calculate the sum of the linear sub-model and the feature cross sub-model, so that the first data party can obtain the first prediction value fragment of the model prediction value , the second data entity can obtain the second predicted value slice of the model predicted value .

[0067] In practical applications, if the feature data is in matrix form, It can be understood as , can represent the first product above, where represents the linear parameter matrix, for The j-th dimension element in ; Represents the first feature data matrix, which can contain several first feature data , for The j-th dimension element in . Feature intersection Can be equal to ,in, represents the latent vector matrix, for The @ represents matrix multiplication, and the * represents bitwise multiplication, that is, multiplying the elements at corresponding positions.

[0068] If there is a linear offset parameter, the first prediction value slice is calculated based on the first linear sub-prediction value slice and the first feature cross-prediction value slice with the second data party using an additive secret sharing algorithm. It can be: the first sub-offset parameter slice based on the linear offset parameter, the first linear sub-prediction value slice and the first feature cross-prediction value slice are calculated with the second data party using an additive secret sharing algorithm to obtain the first prediction value slice.

[0069] If the training data also includes the second feature data held by the first data party The model parameters may also include model parameters for the second feature data, which may specifically include the second linear parameter , the second latent vector parameter , and can also include a second linear offset parameter The first feature slice may also include a third sub-feature slice of the second feature data The first linear parameter slice may also include a third sub-linear parameter slice of the second sub-linear parameter , the first latent vector parameter slice may also include a third sub-latent vector element slice of the second sub-latent vector parameter The first linear parameter slice may also include a third sub-offset parameter slice of the second sub-offset parameter Correspondingly, the second data party obtains the corresponding data slice, so that the above second feature slice can also include the fourth sub-feature slice of the second feature data The second linear parameter slice may also include a fourth sub-linear parameter slice of the second sub-linear parameter The second latent vector parameter slice may also include a fourth sub-latent vector element slice of the second sub-latent vector parameter The second linear parameter slice may also include a fourth sub-offset parameter slice of the second sub-offset parameter .

[0070] Similar to the above method of calculating the model prediction value for the first feature data, the first data party and the second data party can also use the secret sharing algorithm to calculate the prediction value of the factor decomposition machine model for the second feature data, and the first party can also obtain the data slices of the prediction value of each part in the factor decomposition machine for the second feature data, and further obtain the data slices of the prediction value of the factor decomposition machine model for the second feature data. No further details are given here.

[0071] If the training data also includes the second feature data of the first data party, the prediction value of the above-mentioned linear sub-model may include the prediction value of the linear sub-model for the first feature data and the prediction value of the linear sub-model for the second feature data, which can also be understood as the sum of the two, which can be calculated by the first data party and the second data party based on the data slices of the above-mentioned two prediction values ​​using the additive secret sharing algorithm. The above-mentioned first linear sub-prediction value slice may include the first linear sub-slice of the prediction value of the linear sub-model for the first feature data and the second linear sub-slice of the prediction value of the linear sub-model for the second feature data, which can also be understood as the sum of the two, which can also be calculated by the first data party and the second data party based on the data slices of the above-mentioned two prediction values ​​using the additive secret sharing algorithm. If the first data party holds the first linear sub-slice and the second linear sub-shard , the second data party can hold the third linear sub-shard and the fourth linear sub-shard In this way, both parties can use the additive secret sharing algorithm to calculate the sum of the predicted values ​​of the two linear sub-models, and the first data party can obtain the data shards of the sum , the second data party can obtain another data shard .

[0072] Similarly, the prediction value of the above-mentioned first feature cross-submodel may include the prediction value of the feature cross-submodel for the first feature data and the prediction value of the feature cross-submodel for the second feature data, which can also be understood as the sum of the two, and the sum can also be calculated by the first data party and the second data party based on the data slices of the above-mentioned two prediction values ​​using the additive secret sharing algorithm; the above-mentioned first feature cross-sub prediction value slice may include the first cross-slice of the prediction value of the feature cross-submodel for the first feature data and the second cross-slice of the prediction value of the feature cross-submodel for the second feature data, which can also be understood as the data slice of the sum of the two prediction values. If the additive secret sharing algorithm is used to calculate the sum, the specific calculation process is similar to the above-mentioned calculation process, which will not be repeated here.

[0073] The calculation process of the predicted value of the feature cross sub-model for the first feature data can refer to the above content. The calculation of the predicted value of the feature cross sub-model for the second feature data is similar to the above process. For example, the first data party can obtain the third sub-feature fragment And the third latent vector parameter slice third sub-latent vector element slice , the second data party can obtain the fourth sub-feature fragment And the fourth sub-latent vector element slice , both parties can calculate the predicted value of the feature cross sub-model for the second feature data according to the secret sharing calculation protocol. Among them, the first data party and the second data party can calculate the product of the second feature data and the corresponding latent vector based on these data slices, which can be called the second atomicity , so that the first data entity can obtain the second atomic fragment of the atomic item , the second data party can obtain another fourth atomic item fragment of the atomic item The first data cube and the second data cube can calculate the predicted value of the feature cross sub-model for the second feature data based on the atomic item sharding. The specific calculation process can be similar to the above-mentioned process of the predicted value of the feature cross sub-model for the first feature data. It is only necessary to change the processing parameters to the parameters related to the second feature data, which will not be repeated here.

[0074] The second data party can obtain data slices corresponding to the data slices obtained by the first data party. For example, the second linear sub-prediction value slice obtained by the second data party can include the third linear sub-slice of the prediction value of the linear sub-model for the first feature data and the fourth linear sub-slice of the prediction value of the linear sub-model for the second feature data, which can also be understood as the sum of the two. The second feature cross-sub-prediction value slice includes the third cross-slice of the prediction value of the feature cross-sub-model for the first feature data and the fourth cross-slice of the prediction value of the feature cross-sub-model for the second feature data, which can also be understood as another data slice of the sum of the two prediction values.

[0075] As an implementation method, the training sample data in the embodiment of this specification may also include several items of second feature data of the target object held by the first data party; the sub-model may also include a two-way feature cross sub-model; the two-way feature cross sub-model can be used to represent the product of the product of the first feature data and the corresponding latent vector and the product of the second feature data and the corresponding latent vector, such as the model formula shown in the third line of the above formula (5).

[0076] Optionally, the method in the embodiment of the present specification may further include: based on the first sample slice and the first latent vector parameter slice, using the multiplicative secret sharing algorithm with the second data party to calculate a first bilateral feature cross sub-prediction value slice. The first bilateral feature cross sub-prediction value slice may represent a data slice of the calculation result or prediction value of the bilateral feature cross sub-model. The second data party may obtain another data slice, which may be referred to as a second bilateral feature cross sub-prediction value slice.

[0077] The first data party and the second data party can use the multiplicative secret sharing algorithm to calculate the feature cross-sub prediction value fragments. As an implementation method, the first data party can locally calculate the product of the second feature data matrix and the second latent vector matrix. , and then split a data of the product into pieces according to the preset multiplicative secret sharing algorithm protocol Provided to the second data party, the first data party can hold another data shard of the product Similarly, the second data cube can locally calculate the product of the first feature data matrix and the first latent vector matrix , and then split a data slice of the product Provided to the first data party according to the preset calculation protocol, the second data party can hold another data fragment of the product In this way, the first data party and the second data party can use the multiplicative secret sharing algorithm to calculate the prediction value of the feature cross sub-model of both parties based on the above four data fragments, so that the first data party can obtain the first feature cross sub-prediction value fragment of the prediction value. ; The second data party can obtain the second two-party feature cross-sub-prediction value fragment .

[0078] In practical applications, , where the first latent vector And the second latent vector It can contain k-dimensional elements, so the above-mentioned bilateral feature cross sub-model can be further expressed as Here you can It is called the second atomic term. The calculation of this atomic term can refer to the above-mentioned calculation of the predicted value of the feature cross sub-model for the second feature data. In this way, the first data party can obtain the data fragment of the second atomic term. , the second data party can obtain another data fragment of the second atomic item , the data shard can also be the data shard obtained by calculating the predicted value of the feature cross sub-model for the second feature data, without the need for repeated calculations by the first data party or the second data party. Further, the first data party can calculate the data shard locally or using the additive secret sharing algorithm based on the data shard of the atomic item And the second data party can get the data shard Similarly, the first data party can obtain data shards And the second data party can get the data shard The above four data slices may also be data generated by the first data party or the second data party in the process of calculating the predicted value of the feature cross sub-model. In this way, both parties can use the multiplicative secret sharing algorithm to calculate the feature cross sub-model of both parties, so that the first data party can obtain the first feature cross sub-prediction value slice of both parties, and the second data party can obtain the second feature cross sub-prediction value slice of both parties.

[0079] The first linear sub-prediction value slice and the first feature cross-prediction value slice are used to calculate the first prediction value slice with the second data party using the additive secret sharing algorithm, which may specifically include: Based on the first linear sub-prediction value slice, the first feature cross-prediction value slice and the first bilateral feature cross-prediction value slice, the first prediction value slice is calculated with the second data party using an additive secret sharing algorithm.

[0080] In the embodiments of the present specification, the sub-model included in the target model may include a linear part, a feature intersection part and a double feature intersection part. The first data party and the second data party may use an additive secret sharing algorithm to calculate the sum of each part, that is, the predicted value of the target model.

[0081] The first data party can hold the first linear sub-shard , the second linear sub-shard , First cross-slice , Second cross-slice And the first two-way feature cross-prediction value shard ; The second data party can hold the third linear sub-shard , the fourth linear sub-shard , the third cross-slice , the fourth cross-shard And the second two-way feature cross-prediction value shard In this way, both parties can calculate the predicted value of the target model based on these data slices using the additive secret sharing algorithm, and the first data party and the second data party can obtain the data slices of the predicted value.

[0082] In practical applications, if it is necessary to use a secret sharing algorithm to calculate the sum of multiple data, two data can be selected first, and after obtaining the data slice of the sum of the two data, further calculation can be performed in combination with slices of other data, so that the data slices of the sum of multiple data can be gradually obtained. For example, when using an additive secret sharing algorithm to calculate the predicted value of the target model, the linear sub-model for the first feature data and the linear sub-model for the second feature data can be calculated based on the first linear sub-slice and the second linear sub-slice, and the third linear sub-slice and the fourth linear sub-slice held by the second data party, and the additive secret sharing algorithm can be used to calculate the sum of the linear sub-model for the first feature data and the linear sub-model for the second feature data, so that the first data party and the second data party can respectively obtain a data slice of the sum, and then combine other data slices, such as the first cross slice held by the first data party and the third cross slice held by the second data party, and further perform sum calculation, and so on, the predicted value slice of the predicted value of the final target model can be obtained. There is no specific limitation on which two parts are calculated first and which part is summed, as long as the predicted value of the target model can be calculated in the end.

[0083] In the embodiments of this specification, the secret sharing algorithm can be used to calculate each part of the target model, and both parties only obtain the data fragments of each part of the model, which can avoid the first data party or the second data party from inferring the original data or labels based on the obtained data, and can further ensure the security of the data. In addition, the secret sharing algorithm is used in the embodiments of this specification to calculate the predicted value of the model, without the need to use technologies such as homomorphic encryption, which can effectively reduce the computing cost and avoid data security issues caused by attacks on public and private key pairs.

[0084] In practical applications, the calculation result of the factor decomposition machine model for the first feature data in the target model may only be related to the first feature data, and the calculation result of the factor decomposition machine model for the second feature data may only be related to the second feature data. The cross-sub-model part of the two-party feature is related to both the first data feature and the second data feature. In order to improve the calculation efficiency, the calculation of the part related only to the data party can be completed locally by the data party, and the secret sharing algorithm is used for the part involving the calculation of the two-party data. As an implementation method, the training sample data may also include several items of second feature data of the target object held by the first data party; the target model includes the sum of the first sub-model of the factor decomposition machine for the first feature data, the second sub-model of the factor decomposition machine for the second feature data, and the third sub-model representing the cross-feature of the two parties. As shown in the above formula (5), the first sub-model can be understood as the calculation formula on the first line on the right side of the equal sign of formula (5), the second sub-model can be understood as the calculation formula on the second line, and the third sub-model can be understood as the calculation formula on the third line.

[0085] The above-mentioned calculating the first prediction value slice based on the first data slice and the second data party holding the second data slice by using the first secret sharing algorithm may specifically include: Obtaining a first sub-prediction value slice; the first sub-prediction value slice is obtained by slicing the first sub-prediction value of the first sub-model locally calculated based on the first feature data and the corresponding model parameters by the second data party; Obtaining a second sub-prediction value slice; the second sub-prediction value slice is obtained by slicing the second sub-prediction value of the second sub-model locally calculated based on the second feature data and the corresponding model parameters by the first data party; Based on the first sample slice and the first latent vector parameter slice in the first parameter slice, a first bilateral feature cross sub-prediction value slice is calculated with the second data party using a multiplicative secret sharing algorithm; the first bilateral feature cross sub-prediction value slice is used to represent a data slice of a third sub-prediction value of a third sub-model; the first sample slice may include a first sub-feature slice of the first feature data and a third sub-feature slice of the second feature data; the first latent vector parameter slice may include a first sub-latent vector element slice of the latent vector corresponding to the first feature data and a third latent vector element slice of the latent vector corresponding to the second feature data; Based on the first sub-prediction value segment, the second sub-prediction value segment and the first two-party feature cross sub-prediction value segment, the first prediction value segment is calculated using an additive secret sharing algorithm with the second data party.

[0086] In the embodiments of this specification, the first data cube can calculate the predicted value of the second sub-model locally, and then provide the data slice of the predicted value to the second data cube. Similarly, the second data cube can also calculate the predicted value of the first sub-model locally, and then provide the data slice of the predicted value to the first data cube. For the convenience of explanation, the first sub-model is represented as , and its predicted value is expressed as , the second sub-model is the first sub-model expressed as , and its predicted value is expressed as , so that the first data party can obtain the first sub-prediction value slice And the second sub-prediction value shard , the second data party can obtain the third sub-prediction value fragment And the fourth sub-prediction value slice .

[0087] The specific process of calculating the prediction value of the third sub-model, as well as the process of calculating the prediction value of the target model based on the first sub-prediction value shard, the second sub-prediction value shard, the first two-party feature cross-sub-prediction value shard and the second data party using the additive secret sharing algorithm are similar to the previous introduction and can be used as a reference, so they will not be repeated here.

[0088] Compared with the secret sharing algorithm, local computing can effectively reduce the amount of computing and improve computing efficiency. In addition, in the embodiment of this specification, the first data party obtains the data slice of the predicted value of the first sub-model provided by the second data party, and the original feature data of the second data party will not be leaked. Similarly, the second data party can obtain the data slice of the predicted value of the second sub-model provided by the first data party, and the original feature data of the first data party will not be leaked, which can also ensure the security of the data.

[0089] In practical applications, at least one of the first sub-prediction value slice and the second sub-prediction value slice may also be calculated based on a secret sharing algorithm, and the details may refer to the previously introduced content. It can be understood that in the embodiments of this specification, if the first sub-prediction value slice is obtained based on local calculation, the second sub-prediction value slice may also be obtained based on a secret sharing algorithm; if the second sub-prediction value slice is obtained based on local calculation, the first sub-prediction value slice may also be obtained based on a secret sharing algorithm, which is not specifically limited here and can be set according to actual needs.

[0090] In practical applications, a deep factor decomposition machine model can be obtained by adding a deep calculation model to the factor decomposition machine. In the embodiments of this specification, the target model can include a deep factor decomposition machine model. Optionally, if the sub-model is a deep factor decomposition machine model, the first prediction value shard is obtained by calculating the first data shard with the second data party holding the second data shard using the first secret sharing algorithm, which may specifically include: Based on the first data shard, calculate with the second data party using a fourth secret sharing algorithm to obtain a fifth sub-prediction value shard; the fifth sub-prediction value shard is used to represent a data shard of a prediction value of a factor decomposition machine sub-model in the deep factor decomposition machine model; Based on the first data shard, calculate with the second data party using the fifth secret sharing algorithm to obtain a sixth sub-prediction value shard; the sixth sub-prediction value shard is used to represent a data shard of a prediction value of a deep sub-model in the deep factor decomposition machine model; Based on the fifth sub-prediction value slice and the sixth sub-prediction value slice, the first prediction value slice is calculated with the second data party using the sixth secret sharing algorithm.

[0091] The formula for calculating the prediction value of the deep factor decomposition machine model can be to add the deep sub-model prediction value hidden to any of the above formulas (1) to (5). The deep sub-model prediction value hidden can be the product of the training data and the corresponding latent vector and the product of the depth parameter.

[0092] If the training data includes the first feature data, the prediction value hidden of the deep sub-model can be expressed as: ; The corresponding prediction value of the deep factor classifier model can be expressed as: (6) If the training data also includes the second feature data, the prediction value hidden of the deep sub-model can be expressed as: ; The corresponding prediction value of the deep factor classifier model can be expressed as: (7) in, The depth parameter representing the nth layer can be set according to actual needs. In practical applications, the depth parameter can also be in matrix form. For the introduction of other parameters, please refer to the content of the embodiment described above.

[0093] The fifth sub-prediction value segment It can represent the predicted value of the factorization machine model The second data party can obtain the seventh prediction value slice of the prediction value. , where the factorization machine model can be understood as the above formula (6) or (7) except The other parts can also be expressed in any form in formulas (1) to (5). Here, in order to To distinguish, you can use It represents the predicted value of the factor decomposition machine sub-model in the deep factor decomposition machine model. The calculation process of this predicted value can be found in the above introduction to calculating the predicted value of the target model, which will not be repeated here.

[0094] The sixth sub-prediction value shard can represent a data shard of the prediction value of the deep sub-model in the deep factorization machine model , the second data party can obtain the eighth sub-prediction value fragment of the prediction value .

[0095] The calculation of the prediction value of the deep sub-model can also be calculated through the secret sharing algorithm. Specifically, if the training data includes the first feature data, the first data party can obtain the data shard , the second data party can get the corresponding data shard For the specific acquisition process, please refer to the above content. If the training data also includes the second feature data, the first data party can use the acquired data segmentation as well as , and holds the corresponding data shard as well as The second data party can use the additive secret sharing algorithm to calculate , so that the first data party can get the data shard , the second data party can get the data shard In the embodiment of the present specification, the first data of the depth parameter can also be obtained to obtain the first depth parameter slice matrix. , which can contain data slices of various depth parameters, such as Wait, the second data party can obtain the corresponding second depth parameter slice matrix , which may include additional data slices for each depth parameter, and then the first data party and the second data party may calculate the depth sub-model prediction value using a multiplicative secret sharing algorithm, and the product may be calculated step by step, and the first data party may obtain the sixth sub-prediction value slice , the second data party can get the eighth sub-prediction value fragment .

[0096] The above-mentioned sixth secret sharing algorithm may include an additive secret sharing algorithm. The first data party can calculate the prediction value of the target model using the additive secret sharing algorithm based on the acquired fifth sub-prediction value slice and the sixth sub-prediction value slice, and the second data party holding the seventh sub-prediction value slice and the eighth sub-prediction value slice. The first data party can obtain the first prediction value slice, and the second data party can obtain the second prediction value slice.

[0097] In the embodiments of this specification, calculating the model prediction value may include multiple steps. For the later processed steps, if the steps include the calculation results calculated in the earlier processed steps, the previous calculation results may be used. As an implementation mode, the above-mentioned calculation based on the first sample slice and the first latent vector parameter slice and the second data party using the multiplicative secret sharing algorithm to obtain the first two-party feature cross sub-prediction value slice may specifically include: Obtain a first atomic item slice and a second atomic item slice; the first atomic item slice and the second atomic item slice are data slices generated in the process of calculating the first feature cross-sub-prediction value slice; the first atomic item slice is a data slice of the product of the first feature data and the corresponding latent vector parameter; the second atomic item slice is a data slice of the product of the second feature data and the corresponding latent vector parameter; Based on the first atomic item fragment and the second atomic item fragment, the first two-party feature cross sub-prediction value fragment is calculated using a multiplicative secret sharing algorithm with the second data party.

[0098] As described above, in the process of calculating the predicted value of the feature cross sub-model for the first feature data or the second feature data, the first data party can obtain the first atomic term fragment , you can also get the second atomic term fragment , the second data party can obtain the corresponding third atomic item fragment , we can also get the fourth atomic term fragment In this way, the first data party can use the multiplicative secret sharing algorithm to calculate the prediction value of the bilateral feature cross sub-model, the first data party can obtain the first bilateral feature cross sub-prediction value slice of the prediction value, and the second data party can obtain the second bilateral feature cross sub-prediction value slice.

[0099] The residual is the difference between the predicted value and the label value. In the embodiment of this specification, the first data party and the second data party can calculate the residual based on the data slices of the obtained predicted value using a secret sharing algorithm. Optionally, the first residual slice is calculated based on the first predicted value slice and the second data party uses a second secret sharing algorithm to obtain the first residual slice, which may specifically include: Obtain a first label value slice; the first label value slice is an actual observation value corresponding to the training sample data; Based on the first predicted value shard and the first label value shard, the first residual shard is calculated with the second data party using an additive secret sharing algorithm; the first residual shard is a data shard of the residual between the model predicted value and the actual observed value.

[0100] In the embodiments of this specification, the second secret sharing algorithm may include an additive secret sharing algorithm. The additive secret sharing algorithm can be understood as calculating the sum of at least two target data according to the secret computing protocol, but the target data will not be exposed to the data party participating in the calculation, and the data party participating in the calculation can obtain the data slice of the sum of the at least two target data, but will not obtain the original target data, and can calculate the sum of several original data on the basis of ensuring the privacy of the original data.

[0101] In the embodiments of this specification, the first data party may hold the label value of the training data, and may provide a data slice of the label value to the second data party, and the first data party may hold another data slice. The label value may be understood as an actual observation value, which may be determined based on historical data or determined by labeling the training data based on expert experience. The specific form of the label value may be determined based on the application requirements of the target model, and is not specifically limited here.

[0102] Assume that the first data party holds the tag value A data shard , the second data party holds another data shard of the tag value , and in the process of calculating the predicted value according to the secret sharing algorithm, the first data party can obtain a data slice of the predicted value , the second data party can get another data slice of the predicted value Based on these four data, both parties can use the additive secret sharing algorithm to calculate the difference between the predicted value and the label value, that is, the residual , so that the first data party can get the first residual fragment , the second data party can get the second residual fragment .

[0103] In practical applications, the difference between the predicted value and the label value can also be normalized, such as using the Sigmoid function, and the residual can also be processed using the chain rule to facilitate subsequent model calculations.

[0104] In practical applications, it is necessary to combine the data of other data parties to update the model. It may be that the amount of training data held by the party itself is insufficient, but it holds actual observation values. Optionally, in the embodiment of this specification, the first data party can hold the actual observation values ​​corresponding to the training sample data.

[0105] During the residual calculation process, the first data party can shard the actual observation value according to the protocol of the preset secret sharing algorithm to obtain a first label value shard, and send another data shard of the actual observation value, namely the second label value shard, to the second data party, so that both parties can collaboratively calculate the residual based on the acquired data shards.

[0106] The parameters in the factorization machine model may include linear offset parameters, linear parameters, and latent vector scores. The first parameter slice obtained by the first data party may include at least one data slice of a first offset parameter slice of a linear offset parameter, a first linear parameter slice of a linear parameter, and a first latent vector parameter slice of a latent vector parameter.

[0107] If the first data party holds the first offset parameter slice, the above-mentioned use of the first residual slice to update the first parameter slice may specifically include: the first residual slice may be used as the first gradient slice to update the first offset parameter slice; the first gradient slice represents a data slice for updating the first gradient of the linear offset parameter in the factor decomposition machine.

[0108] The first offset parameter slice may be subtracted from the first gradient slice to obtain an updated first offset parameter slice. The first gradient slice may be equal to the first residual slice.

[0109] If there is a linear offset parameter, the first offset parameter fragment held by the first data party may include the first sub-offset parameter fragment The first data entity may update the parameter slice using the first residual slice based on the following calculation formula, where the calculation formula may be expressed as: ,in ; Here it is assumed that the training parameters include the first feature data, Indicates the updated first sub-offset parameter slice, Indicates the first sub-offset parameter slice, represents the first gradient slice, Represents the first residual slice.

[0110] Similarly, the second data party can also update the parameter slice it holds according to the obtained second residual slice. Specifically, the following formula can be used to update the parameters: ,in ; Indicates the updated second sub-offset parameter slice, Represents the second sub-offset parameter, represents the fourth gradient slice, Represents the second residual slice.

[0111] If the training data also includes second feature data, the target model may also include a second offset parameter , the first offset parameter fragment held by the first data party may include a third sub-offset parameter fragment , the second data party can hold the fourth sub-offset parameter fragment , these offset parameter slices can also be updated based on the residual slice. Optionally, the first data party can update the third sub-offset parameter slice. Specifically: ,in ; Indicates the updated third sub-offset parameter slice, Indicates the third sub-offset parameter fragment, represents the fifth gradient slice, Represents the first residual slice.

[0112] The second data party can also update the fourth sub-offset parameter slice, specifically: ,in ; Indicates the updated fourth sub-parameter slice, Indicates the fourth sub-offset parameter slice, represents the sixth gradient slice, Represents the first residual slice.

[0113] Regarding the updating of linear parameters, the above-mentioned use of the first residual slice to update the first parameter slice in the embodiment of this specification may specifically include: based on the first residual slice and the first sample slice, a second gradient slice can be calculated with the second data party using a multiplicative secret sharing algorithm; and the first linear parameter slice is updated using the second gradient slice.

[0114] The second gradient slice may represent a data slice of the second gradient for updating the linear parameters in the factor decomposition machine. In practical applications, the second gradient slice represents a data slice of the product sum of the residual and each item of sample data.

[0115] If the training data includes the first feature data, the first data party and the second data party can obtain the data fragmentation , , as well as , using the multiplicative secret sharing algorithm to calculate the second gradient , so that the first data party can be the second gradient shard of the gradient , the second data party can get the seventh gradient slice of the gradient .

[0116] The first data party may use the second gradient slice to update the first linear parameter slice, and the second data party may use the seventh gradient slice to update the second linear parameter slice. Specifically, the first data party may calculate the updated first sub-linear parameter slice ,in represents the updated first sub-linear parameter slice, Represents the first sub-linear parameter shard. The second data cube can also calculate the updated second sub-linear parameter shard ,in represents the updated second sub-linear parameter slice, Represents the second sublinear parameter slice.

[0117] If the training data also includes the second feature data, the linear parameter slice obtained by the first data party may also include the data slice of the linear parameter corresponding to the second feature data, such as the third sub-linear parameter slice , the second data party can obtain the corresponding fourth sub-linear parameter slice In the embodiment of the present specification, both parties can also use the multiplicative secret sharing algorithm to calculate the fifth gradient of the data slice for updating the linear parameter corresponding to the second feature data based on the feature slice and the residual slice of the second feature data. Specifically, the fifth gradient , the first data party can obtain data fragmentation as well as , and holding data shards as well as The second data party uses the multiplicative secret sharing algorithm to calculate the fifth gradient, so that the first data party can obtain the eighth gradient slice of the fifth gradient , the second data party can get the ninth gradient slice of the gradient .

[0118] In this way, the first data party can use the eighth gradient slice to update the third sub-linear parameter slice, and the second data party can use the ninth gradient slice to update the fourth sub-linear parameter slice. Specifically, the first data party can calculate the updated third sub-linear parameter slice ; The second data party can also calculate the updated fourth sub-linear parameter fragment .

[0119] Optionally, in the embodiments of this specification, the latent vector parameters in the model parameters can also be updated based on the residual slices. The above-mentioned updating of the parameter slice using the first residual slice specifically includes: based on the first sample slice and the first latent vector parameter slice in the first parameter slice, using the multiplicative secret sharing algorithm and the additive secret sharing algorithm with the second data party to calculate the third gradient slice; and using the third gradient slice to update the first latent vector parameter slice.

[0120] The third gradient slice may represent a data slice of the third gradient for updating the latent vector parameter in the factor decomposition machine. In practical applications, the process of determining the gradient for updating the latent vector based on the residual may be existing, and in the embodiments of this specification, the gradient may be calculated using a secret sharing algorithm according to a preset gradient calculation rule.

[0121] As an implementation method, the calculation formula of the third gradient can be , where the intermediate parameter , represents the third gradient, represents the training sample data matrix, Represents the original or updated latent vector matrix. If the training data contains the first feature data, the calculation formula of the third gradient can be , where the first intermediate parameter In practical applications, the third gradient may be data in matrix form.

[0122] In the embodiment of this specification, the first data party can slice the first latent vector matrix according to the parameter slices held. And the first characteristic matrix slice , with the matrix holding the second latent vector slice And the second characteristic matrix slice The second data party can use the multiplicative secret sharing algorithm and the additive secret sharing algorithm to calculate the third gradient, so that the first data party can obtain the third gradient slice of the third gradient , the second data party can also obtain the tenth gradient slice of the third gradient In practical applications, gradient slices can also be data slices in the form of matrices. Matrix slices can be understood as a matrix containing multiple element slices. For example, the first feature matrix slice can contain each second sub-feature slice. , the first latent vector matrix slice can contain the first sub-latent vector element slices .

[0123] In practical applications, the updated latent vector parameter can be obtained by subtracting the corresponding gradient from the latent vector parameter to be updated. Optionally, in the embodiment of this specification, the first data party can subtract the third gradient slice from the first latent vector parameter slice to obtain the updated first latent vector parameter slice.

[0124] For example, the first data party can calculate the updated first latent vector parameter slice ; The second data party can also calculate the updated second latent vector parameter shard .

[0125] If the training data includes several items of second feature data of the target object held by the first data party; the first sample slice may include a first sub-feature slice of the first feature data and a third sub-feature slice of the second feature data; based on the first sample slice and the first latent vector parameter slice in the first parameter slice, the third gradient slice is calculated by using the multiplicative secret sharing algorithm and the additive secret sharing algorithm with the second data party, specifically including: Determine a first intermediate shard; the first intermediate shard is a data shard of a derivative of a feature cross submodel in a factorization machine for the first feature data; Based on the third sub-feature slice and the third sub-latent vector element slice, a first factor slice of a first influencing factor is calculated with the second data party using a multiplicative secret sharing algorithm; the third sub-latent vector element slice is a data slice of the second latent vector corresponding to the second feature data; the first influencing factor is used to represent the influence of the second feature data and the second latent vector on the third gradient; Based on the first intermediate slice, the first factor slice and the first residual slice, the third gradient slice is calculated with the second data party using a multiplicative secret sharing algorithm and an additive secret sharing algorithm.

[0126] If the training data also includes the second feature data, the second feature data will also affect the update of the first latent vector parameters, where the third gradient can be expressed as , where the second intermediate parameter The first data party can be based on data sharding , , , , and holding data shards , , , The second data party can use the multiplicative secret sharing algorithm and the additive secret sharing algorithm to calculate the third gradient, so that the first data party can obtain the data slice of the third gradient, that is, the third gradient slice , the second data party can also obtain another data slice of the third gradient, that is, the tenth gradient slice In practical applications, gradient slicing can also be data slicing in the form of a matrix.

[0127] Based on the logic similar to the above, both parties can also calculate the fourth gradient for updating the latent vector parameter corresponding to the second feature data. The fourth gradient can be expressed as , where the third intermediate parameter , the fourth intermediate parameter Similarly, the first data party can be based on data sharding , , , , and holding data shards , , , The second data party can use the multiplicative secret sharing algorithm and the additive secret sharing algorithm to calculate the fourth gradient, so that the first data party can obtain the eleventh gradient slice of the fourth gradient , the second data party can also obtain the twelfth gradient slice of the third gradient . Further, the first data party may subtract the eleventh gradient slice from the third latent vector matrix slice to obtain an updated third latent vector parameter slice. Specifically, the first data party may calculate the updated third latent vector parameter slice Similarly, the second data party can also calculate the updated fourth latent vector parameter slice .

[0128] In the embodiment of this specification, in the process of calculating the gradient, the calculation can be based on the atomicity calculated in the process of calculating the predicted value before or other determined data, which can realize data reuse and reduce the amount of calculation.

[0129] If the target model includes a deep factor decomposition sub-model, the model parameters include a depth parameter; in the embodiment of this specification, the first data party and the second data party may also use a secret sharing algorithm to determine a gradient for updating the depth parameter and update the depth parameter. Optionally, the method in this specification may also include: Obtain a second factor shard; the second factor shard represents a data shard of the product of at least part of the depth parameters among the depth parameters and the sum of the products of the training data and the corresponding latent vector parameters; Based on the second factor slice and the first residual slice, a first depth gradient slice is calculated with the second data party using a multiplicative secret sharing algorithm; the first depth gradient slice represents a data slice for updating the gradient of the depth parameter; The updating of the parameter slice by using the first residual slice may include: A first depth parameter slice of the depth parameter is updated using the first depth gradient slice.

[0130] Among them, the first data party can obtain the first depth parameter slice of the depth parameter, the second data party can obtain the second depth parameter slice of the depth parameter, and the second factor slice can be calculated by the first data party and the second data party based on the first depth parameter slice using the multiplicative secret sharing algorithm. The deep sub-model included in the deep factor decomposition machine model can include an n-layer deep network, and the depth parameter can include the parameters of each layer of the deep network. The second data party can also obtain the corresponding second depth gradient slice, and can also use the second depth gradient slice to update the obtained second depth parameter slice of the depth parameter.

[0131] In the embodiments of this specification, the model parameters can be updated based on the gradient descent method, wherein the first data cube and the second data cube can be used to collaboratively perform the calculation of the gradient descent method. For the update of the depth parameter, the first data cube and the second data cube can collaboratively calculate the gradient for updating the depth parameter.

[0132] Depth gradient corresponding to the depth parameter ,in represents the residual, represents the first variable, , represents the training data, represents the latent vector corresponding to the training data, Represents the second variable.

[0133] Among them, for the depth gradient of the depth parameter of the nth layer, that is, the last layer, the second variable , that is, the first variable here is equal to the predicted value hidden of the above-mentioned deep sub-model, and the calculated data obtained in the process of calculating the predicted value can be reused when calculating the second variable.

[0134] The depth gradient of the depth parameter of the kth layer in the 2nd to n-1th layers, the second variable .

[0135] The depth gradient of the depth parameter of layer 1, the second variable .

[0136] If the training data includes the first feature data, the above ; If the training data also includes the second feature data, the above .

[0137] Similar to the calculation logic of the prediction value of the above-mentioned calculation model, in the embodiment of this specification, the first data party and the second data party can also calculate the depth gradient of the depth parameter based on the secret sharing algorithm. Specifically, the first data party can calculate the second variable based on the depth parameter slice with the second data party using the multiplicative secret sharing algorithm, so that the first data party can obtain the data slice of the second variable. , the second data party can get another data slice of the second variable ; The first data party obtains data fragments based on as well as Data Sharding , the first variable can be calculated with the second data party using a multiplicative secret sharing algorithm , so that the first data party can obtain the data slice of the first variable , the second data party can obtain another data slice of the first variable ,in The data slices can be calculated by the first data party based on the feature slices and the latent vector parameter slices with the second data party using a secret sharing algorithm, or can be data obtained from the process of calculating the predicted value; the first data party can be based on the data slices of the first variable And the first residual fragment Calculate the first depth gradient using the multiplicative secret sharing algorithm with the second data party , so that the first data square can obtain the first depth gradient The first depth gradient slice , the second data can get the first depth gradient The second depth gradient slice .

[0138] Then the first data party can subtract the corresponding first depth gradient slice from the parameter slice of the original depth parameter to obtain an updated depth parameter slice; similarly, the second data party can also subtract the corresponding second depth gradient slice from the parameter slice of the original depth parameter to obtain an updated depth parameter slice.

[0139] In the embodiment of this specification, if the target model includes a deep factorization machine model, the depth parameter will also affect the update of the latent vector parameter. Optionally, the method in the embodiment of this specification may also include: Obtain a third intermediate slice; the third intermediate slice represents a data slice of the product of each of the depth parameters; Based on the first sample slice and the third intermediate slice, calculate with the second data party using a multiplicative secret sharing algorithm to obtain a third factor slice of a second influencing factor; the second influencing factor is used to indicate the degree of influence of the depth parameter on the gradient of the latent vector parameter used to update the factor decomposition machine; The calculating, based on the first residual slice and the first sample slice, with the second data party using a multiplicative secret sharing algorithm to obtain a second gradient slice may specifically include: Based on the first residual slice and the first sample slice, calculate with the second data party using a multiplicative secret sharing algorithm to obtain an intermediate gradient slice; Based on the third factor shard and the first residual shard, obtain a fourth factor shard of a third influencing factor using a multiplicative secret sharing algorithm with the second data party; the third influencing factor represents the product of the second influencing factor and the residual; Based on the fourth factor shard and the intermediate gradient shard, the second gradient shard is calculated with the second data party using an additive secret sharing algorithm.

[0140] The first data party can be segmented based on the depth parameter , … Slice with depth parameter , … The second data party uses the multiplicative secret sharing algorithm to calculate the product of each depth parameter , so that the first data party can obtain a data slice of the product, namely the third intermediate slice , the second data party can get the fourth intermediate fragment Then the first data party can also calculate the second impact factor based on the first sample slice and the third intermediate slice with the second data party holding the second sample slice and the fourth intermediate slice using the multiplicative secret sharing algorithm , so that the first data party can get the third factor shard , the second data party can obtain another data shard of the second influencing factor, which can be called the fifth factor shard ; And the first data party can also calculate the intermediate gradient slice based on the first residual slice and the first sample slice with the second data party using the multiplicative secret sharing algorithm, and the intermediate gradient slice can represent the initial gradient calculated based on the latent vector and the sample data when there is no depth parameter For details, please refer to the calculation process of the gradient of the latent vector parameters mentioned above.

[0141] In the embodiment of the present specification, the influence factor of the depth parameter can be added to the initial gradient to obtain a composite gradient for updating the latent vector. Specifically, the composite gradient ,in, The influencing factor of the depth parameter, i.e., the third influencing factor, can be the product of the residual and the second influencing factor. The first data cube and the second data cube can be based on data sharding. , , as well as The third impact factor is calculated using the multiplicative secret sharing algorithm, so that the first data party can obtain the fourth factor shard , the second data party can get the sixth factor shard , and then based on these data shards and data shards , , both parties can use the additive secret sharing algorithm to calculate the composite gradient, so that the first data party can obtain the second gradient fragment , the second data party can also get the corresponding gradient fragment .

[0142] In practical applications, the product of each depth parameter has been calculated in the process of calculating the predicted value, and the data value of the third intermediate slice can also be a slice of the product of several depth parameters obtained in the process of calculating the reused predicted value. This can reduce repeated calculations and improve calculation efficiency.

[0143] It can be understood that the first data party and the second data party introduced in the embodiments of this specification use a secret sharing algorithm to calculate prediction values, residuals, gradients, products, data and other data to be calculated. In essence, the first data party and the second data party obtain data slices of these data to be calculated through the secret sharing algorithm without actually calculating the true values ​​of these data to be calculated.

[0144] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 3 The following is a schematic diagram of a device for updating a target model based on a secret sharing algorithm according to an embodiment of the present specification. The target model includes a sub-model based on a factorization machine, such as Figure 3 As shown, the device may include: The data acquisition module 302 is used to acquire a first data slice; the first data slice includes a first sample slice of the training sample data and a first parameter slice of the model parameter of the sub-model; the training sample data includes a plurality of first feature data items for the target object held by the second data party; A prediction value calculation module 304 is used to calculate a first prediction value slice based on the first data slice and the second data party holding the second data slice using a first secret sharing algorithm; the first prediction value slice is used to represent a data slice of the prediction value of the target model; The residual calculation module 306 is used to calculate the first residual shard based on the first prediction value shard and the second data party holding the second prediction value shard using the second secret sharing algorithm; the first residual shard is a data shard used to represent the residual of the target model; the second prediction value shard is another data shard of the prediction value of the target model; The parameter updating module 308 is used to update the first parameter slice using the first residual slice to obtain an updated target model.

[0145] The above-mentioned device may be a device of the first data party, and the second data party may also have a corresponding device.

[0146] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 4 A schematic diagram of a device for updating a target model based on a secret sharing algorithm provided in an embodiment of this specification. Figure 4 As shown, the device 400 may include: At least one processor 410; and a memory 430 communicatively connected to the at least one processor; wherein, The memory 430 stores instructions 420 executable by the at least one processor 410. The instructions are executed by the at least one processor 410 to enable the at least one processor 410 to: Acquire a first data slice; the first data slice includes a first sample slice of training sample data and a first parameter slice of a model parameter of the sub-model; the training sample data includes a plurality of first feature data items for a target object held by the second data party; A first prediction value shard is calculated based on the first data shard and the second data party holding the second data shard using a first secret sharing algorithm; the first prediction value shard is used to represent a data shard of the prediction value of the target model; A first residual shard is calculated based on the first predicted value shard and the second data party holding the second predicted value shard using a second secret sharing algorithm; the first residual shard is a data shard used to represent the residual of the target model; the second predicted value shard is another data shard of the predicted value of the target model; The first parameter slice is updated using the first residual slice to obtain an updated target model.

[0147] Based on the same idea, the embodiment of this specification also provides a computer-readable medium corresponding to the above method. The computer-readable medium stores computer-readable instructions, which can be executed by a processor to implement the above method for updating the target model based on the secret sharing algorithm.

[0148] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. Figure 4 As for the device shown, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0149] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0150] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0151] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0152] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices. For the convenience of description, the above devices are described separately by function into various units. Of course, when implementing this application, the functions of each unit may be implemented in the same or more software and / or hardware.

[0153] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0156] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0157] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0158] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0159] It should be understood by those skilled in the art that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0160] The above is only the embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for updating a target model based on a secret sharing algorithm, wherein the target model includes a sub-model based on a factorization machine, the method involving a first data cube and a second data cube, the method being applied to the first data cube, comprising: Obtain a first data slice; The first data slice includes a first sample slice of training sample data and a first parameter slice of a model parameter of the sub-model; The training sample data includes a plurality of first feature data items for the target object held by the second data party; Based on the first data shard and the second data party holding the second data shard, a first prediction value shard is calculated using a first secret sharing algorithm; The first predicted value slice is used to represent a data slice of the predicted value of the target model; Based on the first prediction value slice and the second data party holding the second prediction value slice, a first residual slice is calculated using a second secret sharing algorithm; The first residual slice is a data slice for representing the residual of the target model; the second predicted value slice is another data slice of the predicted value of the target model; The first parameter slice is updated using the first residual slice to obtain an updated target model.

2. According to the method of claim 1, the sub-model comprises a linear sub-model and a feature cross sub-model; the first parameter slice comprises a first linear parameter slice of a linear parameter and a first latent vector parameter slice of a latent vector parameter; The calculating based on the first data slice and the second data party holding the second data slice using a first secret sharing algorithm to obtain a first prediction value slice specifically includes: Based on the first sample slice and the first linear parameter slice, a first linear sub-prediction value slice is calculated with the second data party using a multiplicative secret sharing algorithm; the first linear sub-prediction value slice is used to represent a data slice of the prediction value of the linear sub-model; Based on the first sample slice and the first latent vector parameter slice, the first feature cross sub-prediction value slice is calculated with the second data party using a multiplicative secret sharing algorithm and an additive secret sharing algorithm; the first feature cross sub-prediction value slice is used to represent a data slice of the prediction value of the first feature cross sub-model; Based on the first linear sub-prediction value slice and the first feature cross sub-prediction value slice, the first prediction value slice is calculated with the second data party using an additive secret sharing algorithm.

3. According to the method of claim 2, the training sample data further includes a plurality of second feature data of the target object held by the first data party; the sub-model further includes a two-way feature cross sub-model; the two-way feature cross sub-model is used to represent the product of the product of the first feature data and the corresponding latent vector and the product of the second feature data and the corresponding latent vector; the method further includes: Based on the first sample slice and the first latent vector parameter slice, a first bilateral feature cross sub-prediction value slice is calculated with the second data party using a multiplicative secret sharing algorithm; the first bilateral feature cross sub-prediction value slice is used to represent a data slice of the prediction value of the bilateral feature cross sub-model; The step of calculating the first prediction value slice based on the first linear sub-prediction value slice and the first feature cross-prediction value slice with the second data party using an additive secret sharing algorithm specifically includes: Based on the first linear sub-prediction value slice, the first feature cross-prediction value slice and the first bilateral feature cross-prediction value slice, the first prediction value slice is calculated with the second data party using an additive secret sharing algorithm.

4. The method according to claim 1, wherein the training sample data further comprises a plurality of second feature data of the target object held by the first data party; the target model comprises a sum of a first sub-model of a factor decomposition machine for the first feature data, a second sub-model of a factor decomposition machine for the second feature data, and a third sub-model representing the intersection of the features of both parties; The above-mentioned calculating the first prediction value slice based on the first data slice and the second data party holding the second data slice by using the first secret sharing algorithm may specifically include: Get the first sub-prediction value slice; The first sub-prediction value slice is obtained by slicing the first sub-prediction value of the first sub-model locally calculated based on the first feature data and the corresponding model parameters by the second data entity; Get the second sub-prediction value slice; The second sub-prediction value shard is obtained by sharding the second sub-prediction value of the second sub-model locally calculated based on the second feature data and the corresponding model parameters by the first data entity; Based on the first sample slice and the first latent vector parameter slice in the first parameter slice, a first bilateral feature cross sub-prediction value slice is calculated with the second data party using a multiplicative secret sharing algorithm; the first bilateral feature cross sub-prediction value slice is used to represent the data slice of the third sub-prediction value of the third sub-model; the first sample slice includes the first sub-feature slice of the first feature data and the third sub-feature slice of the second feature data; the first latent vector parameter slice includes the first sub-latent vector element slice of the latent vector corresponding to the first feature data and the third latent vector element slice of the latent vector corresponding to the second feature data; Based on the first sub-prediction value slice, the second sub-prediction value slice and the first two-party feature cross sub-prediction value slice, the first prediction value slice is calculated with the second data party using an additive secret sharing algorithm.

5. The method according to claim 1, wherein if the sub-model is a deep factorization machine model, The calculating based on the first data slice and the second data party holding the second data slice using a first secret sharing algorithm to obtain a first prediction value slice specifically includes: Based on the first data shard, calculate with the second data party using a fourth secret sharing algorithm to obtain a fifth sub-prediction value shard; The fifth sub-prediction value slice is used to represent a data slice of the prediction value of the factor decomposition machine sub-model in the deep factor decomposition machine model; Based on the first data shard, calculate with the second data party using the fifth secret sharing algorithm to obtain a sixth sub-prediction value shard; the sixth sub-prediction value shard is used to represent a data shard of a prediction value of a deep sub-model in the deep factor decomposition machine model; Based on the fifth sub-prediction value slice and the sixth sub-prediction value slice, the first prediction value slice is calculated with the second data party using the sixth secret sharing algorithm.

6. The method according to claim 1, wherein the first residual shard is calculated based on the first prediction value shard and the second data party uses a second secret sharing algorithm to obtain the first residual shard, specifically comprising: Get the first label value fragment; The first label value slice is the actual observation value corresponding to the training sample data; Based on the first prediction value fragment and the first label value fragment, the first residual fragment is calculated by using an additive secret sharing algorithm with the second data party; The first residual slice is a data slice of the residual between the model prediction value and the actual observation value.

7. According to the method according to any one of claims 1 to 6, the first data entity holds actual observation values ​​corresponding to the training sample data.

8. According to the method of claim 1, the first parameter slice comprises a first offset parameter slice of a linear offset parameter, a first linear parameter slice of a linear parameter, and a first latent vector parameter slice of a latent vector parameter; The updating of the first parameter slice by using the first residual slice specifically includes: Using the first residual slice as a first gradient slice to update the first offset parameter slice; the first gradient slice represents a data slice for updating a first gradient of a linear offset parameter; and / or, based on the first residual slice and the first sample slice, a second gradient slice is calculated with the second data party using a multiplicative secret sharing algorithm; the second gradient slice represents a data slice for updating a second gradient of a linear parameter; Updating the first linear parameter slice using the second gradient slice; And / or, based on the first sample slice and the first latent vector parameter slice in the first parameter slice, the third gradient slice is calculated by the second data party using a multiplicative secret sharing algorithm and an additive secret sharing algorithm; the third gradient slice represents a data slice of a third gradient for updating the latent vector parameter in the factorization machine; The first latent vector parameter slice is updated using the third gradient slice.

9. The method according to claim 8, if the training data further includes a plurality of second feature data of the target object held by the first data party; the first sample slice includes a first sub-feature slice of the first feature data and a third sub-feature slice of the second feature data; The calculating, based on the first sample slice and the first latent vector parameter slice in the first parameter slice, and the second data party using a multiplicative secret sharing algorithm and an additive secret sharing algorithm to obtain a third gradient slice specifically includes: Determine a first intermediate shard; The first intermediate shard is a data shard of the derivative of the feature cross sub-model in the factorization machine for the first feature data; Based on the third sub-feature slice and the third sub-latent vector element slice, calculate with the second data party using a multiplicative secret sharing algorithm to obtain a first factor slice of a first influencing factor; The third latent vector element slice is a data slice of the second latent vector corresponding to the second feature data; the first influence factor is used to indicate the influence of the second feature data and the second latent vector on the third gradient; Based on the first intermediate slice, the first factor slice and the first residual slice, the third gradient slice is calculated with the second data party using a multiplicative secret sharing algorithm and an additive secret sharing algorithm.

10. The method according to claim 8, wherein if the sub-model is a deep factorization machine model, the model parameters include a depth parameter; and the method further comprises: Get the second factor shard; The second factor slice represents a data slice of the product of at least part of the depth parameters among the depth parameters and the sum of the products of the training data and the corresponding latent vector parameters; Based on the second factor slice and the first residual slice, calculate with the second data party using a multiplicative secret sharing algorithm to obtain a first depth gradient slice; The first depth gradient slice represents a data slice for updating the gradient of the depth parameter; The updating of the parameter slice by using the first residual slice comprises: A first depth parameter slice of the depth parameter is updated using the first depth gradient slice.

11. The method according to claim 9, further comprising: Get the third intermediate shard; The third intermediate slice represents a data slice of the product of each of the depth parameters; Based on the first sample slice and the third intermediate slice, calculate with the second data party using a multiplicative secret sharing algorithm to obtain a third factor slice of a second influencing factor; the second influencing factor is used to indicate the degree of influence of the depth parameter on the gradient of the latent vector parameter used to update the factor decomposition machine; The calculating, based on the first residual slice and the first sample slice, with the second data party using a multiplicative secret sharing algorithm to obtain a second gradient slice specifically includes: Based on the first residual slice and the first sample slice, calculate with the second data party using a multiplicative secret sharing algorithm to obtain an intermediate gradient slice; Based on the third factor shard and the first residual shard, obtain a fourth factor shard of a third influencing factor using a multiplicative secret sharing algorithm with the second data party; the third influencing factor represents the product of the second influencing factor and the residual; Based on the fourth factor shard and the intermediate gradient shard, the second gradient shard is calculated with the second data party using an additive secret sharing algorithm.

12. The method according to claim 3, wherein the step of calculating the bilateral feature cross sub-model based on the first sample slice and the first latent vector parameter slice using a multiplicative secret sharing algorithm with the second data party to obtain a first bilateral feature cross sub-prediction value slice may specifically include: Get the first atomic item fragment and the second atomic item fragment; The first atomic item slice and the second atomic item slice are data slices generated in the process of calculating the first feature cross-sub-prediction value slice; the first atomic item slice is a data slice of the product of the first feature data and the corresponding latent vector parameter; the second atomic item slice is a data slice of the product of the second feature data and the corresponding latent vector parameter; Based on the first atomic item fragment and the second atomic item fragment, the first two-party feature cross sub-prediction value fragment is calculated using a multiplicative secret sharing algorithm with the second data party.

13. A device for updating a target model based on a secret sharing algorithm, wherein the target model includes a sub-model based on a factorization machine, the device comprising: A data acquisition module, used to acquire a first data slice; The first data slice includes a first sample slice of training sample data and a first parameter slice of a model parameter of the sub-model; The training sample data includes a plurality of first feature data items for the target object held by the second data party; A prediction value calculation module, configured to calculate a first prediction value slice based on the first data slice and the second data party holding the second data slice by using a first secret sharing algorithm; The first predicted value slice is used to represent a data slice of the predicted value of the target model; A residual calculation module, configured to calculate a first residual slice based on the first prediction value slice and the second data party holding the second prediction value slice by using a second secret sharing algorithm; The first residual slice is a data slice for representing the residual of the target model; the second predicted value slice is another data slice of the predicted value of the target model; A parameter updating module is used to update the first parameter slice using the first residual slice to obtain an updated target model.

14. A device for updating a target model based on a secret sharing algorithm, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Acquire a first data slice; the first data slice includes a first sample slice of training sample data and a first parameter slice of a model parameter of the sub-model; the training sample data includes a plurality of first feature data items for a target object held by the second data party; A first prediction value shard is calculated based on the first data shard and the second data party holding the second data shard using a first secret sharing algorithm; the first prediction value shard is used to represent a data shard of the prediction value of the target model; A first residual shard is calculated based on the first predicted value shard and the second data party holding the second predicted value shard using a second secret sharing algorithm; the first residual shard is a data shard used to represent the residual of the target model; the second predicted value shard is another data shard of the predicted value of the target model; The first parameter slice is updated using the first residual slice to obtain an updated target model.

15. A computer-readable medium having computer-readable instructions stored thereon, wherein the computer-readable instructions can be executed by a processor to implement the method for updating a target model based on a secret sharing algorithm according to any one of claims 1 to 12.

Images