Distributed prediction method and system thereof

Abstract

Embodiments in the specification provide a distributed prediction method and system thereof. Two or more user nodes store respective user models, each of which corresponds to a part of a prediction model. One or more user nodes obtain local prediction results based on at least the user models and local feature data, and a prediction node obtains a prediction result for identification information of an object to be predicted based on the local prediction results of the one or more user nodes. In this way, the data privacy of each party can be effectively protected.

Description

[0001] Divisional Statement

[0002] This application is a divisional application of Chinese Invention Patent Application No. 201911272294.2, filed on December 11, 2019, entitled “Model Training Method, Distributed Prediction Method and System Thereof”. TECHNICAL FIELD

[0003] Embodiments of the present specification relate to the field of information technology, in particular to a distributed prediction method and system thereof. BACKGROUND

[0004] With the development of artificial intelligence technology, machine learning models have been gradually applied in the fields of risk assessment, speech recognition, natural language processing, etc. In the medical, financial and other fields, different enterprises or institutions have different sample data. If these sample data are jointly trained, the model accuracy can be effectively improved, bringing huge economic benefits to enterprises. However, the sample data owned by different enterprises or institutions contain a large amount of user privacy or even business secrets. Once the sample data is leaked, it will cause a series of negative effects.

[0005] Therefore, it is currently desirable to provide a joint training and prediction scheme that can effectively protect user data privacy. SUMMARY

[0006] One of the embodiments of the present specification provides a model training method, wherein the method is performed by a center node, and the method comprises: obtaining encrypted sample data from two or more user nodes; decrypting the encrypted sample data in a trusted execution environment; performing model training in the trusted execution environment using the decrypted sample data to obtain a trained model; splitting the trained model to obtain user models of the two or more user nodes; and sending the user models of the two or more user nodes to the respective user nodes.

[0007] One of the embodiments of the present specification provides a model training system, wherein corresponding to a center node, the system comprises: a data acquisition module configured to obtain encrypted sample data from two or more user nodes; a decryption module configured to decrypt the encrypted sample data in a trusted execution environment; a training module configured to perform model training in the trusted execution environment using the decrypted sample data to obtain a trained model; a model splitting module configured to split the trained model to obtain user models of the two or more user nodes; and a model distribution module configured to send the user models of the two or more user nodes to the respective user nodes.

[0008] One of the embodiments of the present specification provides a model training device, comprising at least one processor and at least one storage device, the storage device is used to store instructions, when the at least one processor executes the instructions, the model training method shown in any embodiment of the present specification is realized.

[0009] One of the embodiments of the present specification provides a distributed prediction method, wherein the prediction node is one of two or more user nodes, or a node other than the two or more user nodes and in communication connection with the two or more user nodes, the two or more user nodes respectively save their own user models, and each user model corresponds to a part of the prediction model; the method comprises: sending a prediction request for the identification information of the to-be-predicted object to one or more user nodes; receiving the local prediction result of the one or more user nodes, the local prediction result is obtained based on at least the user model of the corresponding user node and the feature data of the identification information of the to-be-predicted object; obtaining the prediction result for the identification information of the to-be-predicted object based on the local prediction result of the one or more user nodes.

[0010] One of the embodiments of the present specification provides a distributed prediction system, wherein the system corresponds to a prediction node, the prediction node is one of two or more user nodes, or a node other than the two or more user nodes and in communication connection with the two or more user nodes, the two or more user nodes respectively save their own user models, and each user model corresponds to a part of the prediction model; the system comprises: a prediction request sending module, configured to send a prediction request for the identification information of the to-be-predicted object to one or more user nodes; a local prediction result receiving module, configured to receive the local prediction result of the one or more user nodes, the local prediction result is obtained based on at least the user model of the corresponding user node and the feature data of the identification information of the to-be-predicted object; a prediction module, configured to obtain the prediction result for the identification information of the to-be-predicted object based on the local prediction result of the one or more user nodes.

[0011] One of the embodiments of the present specification provides a distributed prediction device, comprising at least one processor and at least one storage device, the storage device is used to store instructions, when the at least one processor executes the instructions, the distributed prediction method executed by the prediction node shown in any embodiment of the present specification is realized.

[0012] One of the embodiments of the present specification provides a distributed prediction method, wherein the method is performed by a certain user node of two or more user nodes, the two or more user nodes save respective user models, and each user model corresponds to a part of a prediction model; the method comprises: receiving a prediction request for identification information of a to-be-predicted object sent by a prediction node, the prediction node comprising one of the two or more user nodes or a node other than the two or more user nodes; obtaining feature data from the local based on the to-be-predicted object identification information; obtaining a local prediction result based on at least the user model and the feature data; returning the local prediction result to the prediction node.

[0013] One of the embodiments of the present specification provides a distributed prediction system, wherein the method corresponds to a certain user node of two or more user nodes, the two or more user nodes save respective user models, and each user model corresponds to a part of a prediction model; the system comprises: a prediction request receiving module, configured to receive a prediction request for identification information of a to-be-predicted object sent by a prediction node, the prediction node comprising one of the two or more user nodes or a node other than the two or more user nodes; a local prediction module, configured to obtain feature data from the local based on the to-be-predicted object identification information, and obtain a local prediction result based on at least the user model and the feature data; and a local prediction result returning module, configured to return the local prediction result to the prediction node.

[0014] One of the embodiments of the present specification provides a distributed prediction device, comprising at least one processor and at least one storage device, the storage device is used to store instructions, when the at least one processor executes the instructions, the distributed prediction method performed by the user node as shown in any embodiment of the present specification is realized. BRIEF DESCRIPTION OF DRAWINGS

[0015] The present specification will be further illustrated in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same numbers represent the same structures, wherein:

[0016] Figure 1 The application scenario diagram of the machine learning system according to some embodiments of the present specification;

[0017] Figure 2 The interaction diagram of the model training method according to some embodiments of the present specification;

[0018] Figure 3 The interaction diagram of the distributed prediction method according to some embodiments of the present specification;

[0019] Figure 4A user model diagram for a user of user node A, according to some embodiments of the present specification;

[0020] Figure 5 A user model diagram for a user of user node B, according to some embodiments of the present specification;

[0021] Figure 6 An exemplary block diagram of a model training system, according to some embodiments of the present specification;

[0022] Figure 7 An exemplary block diagram of a distributed prediction system, according to some embodiments of the present specification;

[0023] Figure 8 An exemplary block diagram of a distributed prediction system, according to some embodiments of the present specification. DETAILED DESCRIPTION

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some examples or embodiments of the present specification, and for those skilled in the art, the present specification can also be applied to other similar scenarios without creative labor. Unless it is clear from the language context or otherwise indicated, the same reference numbers in the drawings represent the same structure or operation.

[0025] It should be understood that the "system", "device", "unit" and / or "module" used herein is a method for distinguishing different components, elements, parts, sections or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0026] As shown in the specification and claims, unless the context clearly indicates otherwise, "a", "an", "one", and / or "this" do not necessarily mean singular, but can also include plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements explicitly identified, and these steps and elements do not constitute an exclusive list, and the method or device can also include other steps or elements.

[0027] Flowcharts are used in the present specification to illustrate the operations performed by the system according to the embodiments of the present specification. It should be understood that the preceding or subsequent operations are not necessarily performed in sequence. On the contrary, each step can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or more steps of the operation can be removed from these processes.

[0028] Figure 1FIG. 1 illustrates an application scenario of a machine learning system according to some embodiments of the present specification. As shown in FIG. 1, the machine learning system 100 can include two or more user nodes 110, a center node 120, and a network 140. Figure 1

[0029] The user nodes 110 can refer to devices of data providers, which are configured to provide data required in a training phase and a prediction phase. In some embodiments, the data providers can include one or more of enterprises, institutions, individuals, etc. In the training phase, the user nodes 110 can provide sample data to the center node 120. Different user nodes 110 can provide different feature items for the same sample identification information (e.g., sample ID). After the center node 120 trains a model, each user node 110 can obtain a user model from the center node 120, and the user model of each user node 110 corresponds to a part of the complete model. Similarly, in the prediction phase, different user nodes 110 can also provide different feature items for the same identification information of an object to be predicted (e.g., ID of the object to be predicted). For the same identification information of the object to be predicted, each user node 110 inputs the feature item held by itself into the user model and obtains a prediction result output by the user model, and finally the prediction result of the object to be predicted can be obtained based on the prediction results output by the user models of one or more user nodes 110.

[0030] The center node 120 can refer to a computing device with a trusted execution environment of a platform party, where the platform party can refer to a party that coordinates two or more data providers to participate in joint training. In some embodiments, the platform party can refer to a government, an enterprise, an institution, etc. The center node 120 can train a model based on the sample data provided by the user nodes 110 in the trusted execution environment, and split the model into multiple user models. The splitting can be performed on the structure and / or parameters of the model. Further, the center node 120 can distribute the user models to the user nodes 110. The trusted execution environment can effectively ensure that the data does not be leaked, so that the model training in the trusted execution environment can effectively protect the user privacy in the sample data of each party.

[0031] In some embodiments, the model mentioned in the present specification can include any one of a linear regression model, a decision tree, a neural network, etc.

[0032] It should be noted that before the user nodes 110 provide the sample data to the center node 120, the sample data needs to be encrypted to prevent data leakage in the transmission process. Further, the center node 120 can decrypt the sample data in the trusted execution environment and perform model training.

[0033] ​It is worth mentioning that the machine learning system 100 can include a prediction node 130. The prediction node 130 can be used to initiate a prediction request for the identification information of the to-be-predicted object, and obtain the prediction result of the identification information of the to-be-predicted object. In some embodiments, any user node 110 can serve as the prediction node 130. In some embodiments, the prediction node 130 can be a node other than the two or more user nodes 110, for example, the center node 120 or an external node of the machine learning system 100.

[0034] In some embodiments, one or more nodes mentioned in the present specification can be a single device of a party, or can belong to a device cluster of a party. In some embodiments, the device cluster can be centralized or distributed. In some embodiments, the device cluster can be regional or remote.

[0035] The network 140 can facilitate the exchange of data and / or information. In some embodiments, one or more components in the machine learning system 100 (e.g., the user nodes 110, the center node 120, the prediction node 130) can send data and / or information to other components in the machine learning system 100 through the network 140. In some embodiments, the network 140 can be any type of wired or wireless network. For example, the network 140 can include a cable network, a wired network, a fiber-optic network, a telecommunication network, an intranet, an internetwork, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near-field communication (NFC) network, or the like, or any combination thereof. In some embodiments, the network 140 can include one or more network access points. For example, the network 140 can include wired and / or wireless network access points, such as base stations and / or multiple internetwork exchange points 140-1, 140-2, through which one or more components of the machine learning system 100 can connect to the network 140 to exchange data and / or information.

[0036] Figure 2 An interaction diagram for a model training method according to some embodiments of the present specification is shown. It should be noted that although Figure 2 Only two user nodes are shown, but the training principles of the present specification are also applicable to three or more user nodes. The method is performed by the center node and can include a data acquisition step, a data decryption step, a model training step, a model splitting step, and a user model distribution step.

[0037] In the data acquisition step, encrypted sample data from two or more user nodes is acquired.

[0038] As Figure 2As shown, user node A and user node B send respective encrypted sample data to the center node. It should be understood that the encryption of the sample data can ensure the security of the sample data during transmission.

[0039] In the data decryption step, the encrypted sample data is decrypted in the trusted execution environment.

[0040] Since the decrypted sample data is in plaintext, the decryption can be performed in the trusted execution environment to prevent the decrypted sample data from being leaked.

[0041] It should be understood that, in order to prevent data related to user privacy from being leaked, one or more steps in the process shown in any embodiment of the present specification can be performed in the trusted execution environment. For example, as shown, the model training step can be performed in the trusted execution environment, and details can be referred to the description of the model training step in the present specification. For another example, in some embodiments, the model splitting step can be performed in the trusted execution environment, and details can be referred to the description of the model splitting step in the present specification. Figure 2

[0042] In the model training step, the decrypted sample data is used to train the model in the trusted execution environment, and a trained model is obtained.

[0043] Similarly, since the sample data is used in the model training process, the model training can be performed in the trusted execution environment to prevent the sample data from being leaked in the model training process.

[0044] In the model splitting step, the trained model is split to obtain user models of two or more user nodes.

[0045] In some embodiments, the center node can split the model parameters of the trained model by bits based on the feature items of the sample data of the two or more user nodes. Wherein, the splitting by bits refers to splitting out the model parameters (hereinafter referred to as user parameters) associated with the feature items of the sample data of a user node in position to obtain a user model of the user node, wherein the user model has the split-out user parameters.

[0046] ​For example, for any user node, the center node can split the parameter multiplied by the feature item of the sample data of the user node to obtain the user model of the user node. That is, the user model of the user node has the product item of the feature item of the sample data of the user node. In addition, in some embodiments, the constant term of the linear regression model can be split into two or more sub-constant terms as the constant terms of the user models of two or more user nodes. In yet some embodiments, the prediction node other than the two or more user nodes can also save the constant value needed for calculating the predicted value, and the constant term of the user model of the two or more user nodes and the sum of the constant value and the constant term can be equal to the constant term of the trained model. For example, assuming that the trained linear regression model is as follows where Y represents the predicted value, represents the feature item of the sample data of the user node A, represents the feature item of the sample data of the user node B, a1, a2, a3, b1, b2, and c are model parameters of the linear regression model, and c is the constant term, the user model of the user node A can be as follows the user model of the user node B can be as follows where Y a and Y b represent the predicted values of the user models of the user nodes A and B, respectively, and a0 and b0 represent the constant terms of the user models of the user nodes A and B, respectively. In some embodiments, c = a0 + b0 can be met. Accordingly, the prediction result Y = Y a + Y b for the identification information of the to-be-predicted object. In some embodiments, if the prediction node other than the two or more user nodes saves the constant value c0 needed for calculating the predicted value, c = a0 + b0 + c0 can be met. Accordingly, the prediction result Y = Y a + Y b + c0 for the identification information of the to-be-predicted object.

[0047] For example, a decision tree can include decision nodes that determine which branch (e.g., left child node / right child node) to enter and leaf nodes that determine the final decision result (i.e., prediction result). When making a prediction, starting from the root node, each time only a branch is selected based on the feature item and parameters (e.g., threshold value) corresponding to the current node, where the feature item and parameters (e.g., threshold value) can constitute the decision condition for determining which branch to enter. After selecting the branch, the next child node connected to the selected branch is entered. In this way, until the leaf node is reached, the prediction value is obtained. In some embodiments, the splitting of the decision tree can satisfy: for any user node, the user model of the user node has a complete structure, and if the feature item corresponding to the decision node does not belong to the user node, the decision condition (including the feature item and parameters) of the decision node is unknown to the user node. In this way, any user node can obtain the decision results of one or more decision tree nodes on the path from the root node to the leaf node, where the decision results of the local decision tree nodes on the path can be obtained based on the user model and locally obtained feature data, and the decision results of other decision tree nodes on the path can be obtained by initiating a prediction request for the same to-be-predicted object to other user nodes. In this way, the prediction result can be obtained based on the decision results of one or more decision tree nodes on the path. In some embodiments, the final decision result (i.e., prediction result) corresponding to any leaf node in the user model can be known to any user node. In yet some embodiments, the final decision result (i.e., prediction result) corresponding to any leaf node in the user model can be known only to one user node. For the splitting of the decision tree, reference can also be made to Figure 4 , Figure 5 and related descriptions.

[0048] In some embodiments, the center node can perform operation splitting on the model parameters of the trained model. Wherein, the operation splitting can mean that the model parameter slices corresponding to the split user nodes satisfy a certain operation relationship with the model parameters of the model before splitting, so that the model parameters can be split based on the operation relationship.

[0049] For example, the user models of the two or more split user nodes can be neural networks with the same structure, where the structure can include one or more of the number of layers, the number of nodes in each layer, the activation function of each node, the connection relationship between nodes, etc. However, the user models of the two or more user nodes can have different neural network parameters, i.e., the model parameter slices of the two or more split user nodes.

[0050] It should be noted that the present specification does not limit how to split the model, and any model splitting rule that satisfies the following condition can be considered as the disclosed range of the present specification: there is a joint prediction rule that matches the model splitting rule, so that based on the prediction results of the split user models and the joint prediction rule, the prediction result of the model before splitting can be obtained, which is the same or equivalent (e.g., approximately the same) as the prediction result of the model before splitting. Specifically, reference can be made to the content of the present specification related to the splitting and joint prediction of the same model.

[0051] It should be noted that in some embodiments, as shown in Figure 2 the model splitting step can be performed in a trusted execution environment. In this way, the trained model can be prevented from being leaked during the splitting process, and in addition, the feature items of the sample data of the user nodes can be prevented from being leaked when the model is split based on the feature items of the sample data of the user nodes.

[0052] In the model distribution step, the user models of the two or more user nodes are respectively sent to the user nodes.

[0053] In some embodiments, the center node can encrypt the user models, and then send the encrypted user models to the user nodes respectively. After receiving the encrypted user models, the user nodes can decrypt the user models. In this way, the security of the trained model during transmission can be ensured.

[0054] One or more steps disclosed in the present specification involve the encryption / decryption of interactive data, such as the encryption / decryption of sample data and the encryption / decryption of user models. To this end, the center node and the user nodes can agree on an encryption / decryption algorithm and a secret key before interacting with the data. In some embodiments, an asymmetric encryption / decryption algorithm can be used, and the center node and the two or more user nodes can generate their own public-private key pairs before interacting with the data, and the center node and the user nodes can send their own public keys to each other. Based on this, either the center node or the user nodes can encrypt the data using the public key of the other party before sending the data to the other party, and then the other party can decrypt the received encrypted data based on its own private key.

[0055] When the user models are distributed to the user nodes, the user nodes can use their own user models to participate in joint prediction. Figure 3 An interaction diagram of the distributed prediction method according to some embodiments of the present specification is shown. As described above, each user model corresponds to a part of the prediction model. The prediction model can refer to a model trained based on the sample data of the two or more user nodes. Regarding the method of obtaining the prediction model and the user model, reference can be made to the description of the model training method in the foregoing embodiments. It should be noted that although Figure 3Only two user nodes are shown, but the prediction principle of the present disclosure is also applicable to three or more user nodes.

[0056] In some embodiments, the node that initiates the prediction request and finally obtains the prediction result can be referred to as a prediction node. The prediction node can be one of the user nodes, or a node other than the user nodes. When the prediction node is a node other than the user nodes, the prediction node does not participate in the joint prediction, but only needs to send a prediction request containing the identification information of the to-be-predicted object to the relevant user nodes, receive the local prediction results returned by the user nodes, and obtain the prediction result for the identification information of the to-be-predicted object based on the local prediction results. When the prediction node is one of the user nodes, it means that the prediction node not only initiates the prediction request and aggregates the local prediction results, but also participates in the joint prediction. The difference will be introduced below.

[0057] When executed by the prediction node, the distributed prediction method can include a prediction request sending step, a local prediction result receiving step, and a prediction step.

[0058] In the prediction request sending step, a prediction request for the identification information of the to-be-predicted object is sent to one or more user nodes.

[0059] In some embodiments, the identification information of the to-be-predicted object can include the ID of the to-be-predicted object. In some embodiments, the ID can include a name, a mobile phone number, an ID number, a number, and the like. The prediction node can send a prediction request to each user node that has the feature data of the to-be-predicted object. When the prediction node does not participate in the joint prediction, the prediction node needs to send a prediction request to more than two user nodes. When the prediction node participates in the joint prediction, the prediction node can send a prediction request to only one other user node, or can send a prediction request to multiple other user nodes.

[0060] In the local prediction result receiving step, the local prediction result of one or more user nodes is received, which is obtained based on at least the user model of the corresponding user node and the feature data of the identification information of the to-be-predicted object.

[0061] For detailed steps of obtaining the local prediction result by the user node, reference can be made to the related description of the local prediction step executed by the user node hereinafter.

[0062] In the prediction step, the prediction result for the identification information of the to-be-predicted object is obtained based on the local prediction result of at least one user node.

[0063] It should be noted that one or more user nodes jointly predict based on the same identification information of the to-be-predicted object.

[0064] In some embodiments, the local prediction result of any user node corresponds to a part of the prediction result for the identification information of the to-be-predicted object. Accordingly, the prediction node can obtain the prediction result for the identification information of the to-be-predicted object based on the local prediction results of two or more user nodes.

[0065] Specifically, when the prediction node is one of the two or more user nodes, the prediction node can initiate a prediction request for the identification information of the to-be-predicted object to the other user nodes. After each user node receives the prediction request, the user node locally obtains the feature data corresponding to the identification information of the to-be-predicted object, obtains a local prediction result based on the user model and the locally obtained feature data, and returns the local prediction result to the prediction node except for the prediction node. Further, the prediction node can obtain the prediction result for the identification information of the to-be-predicted object based on the local prediction result of the prediction node and the local prediction results of the other user nodes (i.e., the local prediction results of the two or more user nodes).

[0066] When the prediction node is a node other than the two or more user nodes, the prediction node can initiate a prediction request for the identification information of the to-be-predicted object to all user nodes. After each user node receives the prediction request, the user node locally obtains the feature data corresponding to the identification information of the to-be-predicted object, obtains a local prediction result based on the user model and the locally obtained feature data (this process is hereinafter referred to as local prediction), and returns the local prediction result to the prediction node. Further, the prediction node can obtain the prediction result for the identification information of the to-be-predicted object based on the local prediction results of the two or more user nodes.

[0067] For example only, the process of the prediction node cooperating with multiple user nodes for joint prediction can be as shown in FIG. 1. Figure 3 As shown in FIG. 1, the prediction node sends a prediction request for X (X represents the identification information of the to-be-predicted object) to user node A and user node B respectively. After user node A and user node B receive the prediction request for X, the user nodes perform local prediction respectively. User node A obtains a local prediction result S A and sends the local prediction result S A to the prediction node, and user node B obtains a local prediction result S B and sends the local prediction result S B to the prediction node. Further, the prediction node obtains a prediction result for X based on the local prediction result S A and the local prediction result S BThe prediction result S for X is obtained. It can be understood that the above process is for the scenario that the prediction node is a node other than a user node, and when the prediction node is one of the user nodes, the prediction process is substantially similar, and the difference is only that the prediction node will perform local prediction as other user nodes, and the prediction node will determine the final prediction result based on the local prediction results of other user nodes as well as its own local prediction result.

[0068] In some embodiments, the prediction result for the identification information of the to-be-predicted object can be obtained based on the local prediction results of multiple user nodes. For example, the prediction node can add the local prediction results of two or more user nodes to obtain the prediction result for the identification information of the to-be-predicted object, and details can be referred to the related content of the linear regression model in this specification. For another example, for a neural network, each user node can split and distribute the operation of the locally owned feature data for the to-be-predicted object to other user nodes, each user node can calculate a shard of the final prediction result based on its own feature data shard and the obtained feature data shard, and user model parameters, and send the shard to the prediction node, and the prediction node can add the local prediction results (the shards) of each user node to obtain the prediction result for the identification information of the to-be-predicted object. For another example, the prediction node can find the intersection of the local prediction results of two or more user nodes to obtain the prediction result for the identification information of the to-be-predicted object.

[0069] Taking a decision tree as an example, the prediction result corresponding to a leaf node can be obtained based on the decision branches of all decision nodes on the path from the root node (one of the decision nodes) to the leaf node based on the feature data corresponding to the node. According to the related description of this specification Figure 2 , any user node has a complete model structure, but some decision node information (such as feature items and / or their segmentation thresholds) is unknown. A user node can determine all leaf nodes that can be reached based on the feature data for the identification information of the to-be-predicted object owned by the user node through the local user model (such as a decision tree), and take the leaf nodes as the local prediction result of the user node. In some embodiments, the prediction node can send a prediction request to all other user nodes and receive the local prediction results returned by all other user nodes, find the intersection of the local prediction results returned by each user node, and thus obtain the final prediction result.

[0070] The following describes Figure 4 and Figure 5 an example process of obtaining a prediction result for the identification information of a to-be-predicted object (such as a driver) by a user node A and a user node B. For example only, the user model owned by the user node A is as follows: Figure 4As shown, it has three decision nodes and four leaf nodes, wherein the information of two decision nodes, i.e. “driving age” and “traffic violation times”, is known, and the information of one decision node is unknown. The user model owned by user node B is as shown in Figure 5 As shown, it has three decision nodes and four leaf nodes, wherein the information of one decision node, i.e. “number of accidents last year”, is known, and the information of the other two decision nodes is unknown. It is not difficult to understand that the part of the same decision tree owned by user node A and user node B respectively. Both of them need to be jointly predicted to obtain the final prediction result.

[0071] After receiving the identification information of the object to be predicted, user node A obtains the driving age and traffic violation times of the driver from the local. If the driving age of the driver is 2 years and the traffic violation times is 0, the prediction result can be determined based on the model as shown in Figure 4 The threshold condition of the root node is not met, and the prediction result can go to the right. However, the information of the next decision node is unknown, and the prediction result can go to the left or the right. Therefore, the possible leaf nodes reached by user node A are leaf node 3 and leaf node 4.

[0072] After receiving the identification information of the object to be predicted, user node B obtains the number of accidents last year of the driver from the local. If the number of accidents last year of the driver is 1, the prediction result can be determined based on the model as shown in Figure 5 The information of the root node is unknown, and the prediction result can go to the left or the right. If it goes to the left, the possible leaf nodes reached are leaf node 1 and leaf node 2. If it goes to the right, the leaf node reached is leaf node 3.

[0073] After the local prediction results of user node A and user node B are intersected by the prediction node, it can be determined that the final prediction result for the driver is “dangerous” of leaf node 3.

[0074] Continue to combine Figure 3 For the user node (such as user node A or user node B) participating in the prediction in response to the prediction request of the prediction node, the execution can include a prediction request receiving step, a local prediction step and a local prediction result returning step.

[0075] In the prediction request receiving step, the prediction request of the prediction node for the identification information of the object to be predicted is received.

[0076] In the local prediction step, the feature data is obtained from the local based on the identification information of the object to be predicted, and the local prediction result is obtained based on at least the user model and the locally obtained feature data.

[0077] In some embodiments, the user node inputs the locally obtained feature data into the user model to obtain a local prediction result output by the user model, e.g., a user model obtained by splitting a linear regression model, a user model obtained by splitting a decision tree, etc. The process of making a local prediction based on a user model obtained by splitting a linear regression model or a user model obtained by splitting a decision tree can be found elsewhere in this specification.

[0078] In some embodiments, the user node can obtain a local prediction result based on the user model, the feature data, and a prediction intermediate result from one or more other user nodes. Taking a neural network as an example, for the same identification information of the to-be-predicted object, each user node can split its own feature data into N (N is an integer not less than 2, representing the number of more than two user nodes) feature data shards, send N-1 feature data shards to the other N-1 user nodes respectively, obtain an input shard of the prediction model based on the remaining feature data shard and the feature data shards from the other N-1 user nodes, and calculate an output shard (i.e., the local prediction result of the user node) of the prediction model by the secret sharing algorithm with the input shards and parameter shards of the other N-1 user nodes. Further, the output of the prediction model (i.e., the final prediction result) can be obtained based on the output shards of the N user nodes. Wherein, for any user node, one or more intermediate calculation results received from other user nodes in the collaborative calculation process can be regarded as prediction intermediate results.

[0079] In the local prediction result returning step, the local prediction result is returned to the prediction node.

[0080] For more details about the prediction request receiving step, the local prediction step, and the local prediction result returning step, please refer to the description of the local prediction result of any user node in the foregoing embodiments, which corresponds to a part of the prediction result for the identification information of the to-be-predicted object.

[0081] It should be noted that the above description of the process is only for example and illustration, and does not limit the scope of the present specification. Those skilled in the art can make various modifications and changes to the process under the guidance of the present specification. However, these modifications and changes are still within the scope of the present specification.

[0082] Figure 6 An exemplary block diagram of a model training system according to some embodiments of the present specification is shown. The system 600 corresponds to a center node. In some embodiments, the system 600 can include a data acquisition module 610, a decryption module 620, a training module 630, a model splitting module 640, and a model distribution module 650.

[0083] In some embodiments, the data obtaining module 610 can be configured to obtain encrypted sample data from two or more user nodes. For the specific implementation of the data obtaining module 610, reference can be made to the relevant description of the data obtaining step. That is, the data obtaining step can be performed by the data obtaining module 610.

[0084] In some embodiments, the decryption module 620 can be configured to decrypt the encrypted sample data in a trusted execution environment. For the specific implementation of the decryption module 620, reference can be made to the relevant description of the data decryption step. That is, the data decryption step can be performed by the decryption module 620.

[0085] In some embodiments, the training module 630 can be configured to train a model in the trusted execution environment using the decrypted sample data to obtain a trained model. In some embodiments, the model comprises at least one of a linear regression model, a decision tree, and a neural network. For the specific implementation of the training module 630, reference can be made to the relevant description of the model training step. That is, the model training step can be performed by the training module 630.

[0086] In some embodiments, the model splitting module 640 can be configured to split the trained model to obtain user models of the two or more user nodes. In some embodiments, the center node can split or operate the model parameters of the trained model bit by bit based on the feature items of the sample data of the two or more user nodes. For the specific implementation of the model splitting module 640, reference can be made to the relevant description of the model splitting step. That is, the model splitting step can be performed by the model splitting module 640.

[0087] In some embodiments, the model distribution module 650 can be configured to send the user models of the two or more user nodes to the respective user nodes.

[0088] For the specific implementation of the model distribution module 650, reference can be made to the relevant description of the model distribution step. That is, the model distribution step can be performed by the model distribution module 650.

[0089] In some embodiments, the model training system can further comprise an encryption module. In some embodiments, the encryption module can be configured to encrypt the user models of the two or more user nodes before the model distribution module sends the user models to the respective user nodes.

[0090] Figure 7FIG. 7 is an exemplary block diagram of a distributed prediction system according to some embodiments of the present disclosure. The system 700 corresponds to a prediction node. Two or more user nodes each maintain a respective user model, each of which corresponds to a portion of a prediction model. In some embodiments, the system 700 can include a prediction request sending module 710, a local prediction result receiving module 720, and a prediction module 730.

[0091] In some embodiments, the prediction request sending module 710 can be configured to send a prediction request for identification information of an object to be predicted to one or more user nodes. For implementation of the prediction request sending module 710, reference can be made to the description of the prediction request sending step. That is, the prediction request sending step can be performed by the prediction request sending module 710.

[0092] In some embodiments, the local prediction result receiving module 720 can be configured to receive a local prediction result of one or more user nodes, which is based at least on a user model of the corresponding user node and feature data of the identification information of the object to be predicted.

[0093] For implementation of the local prediction result receiving module 720, reference can be made to the description of the local prediction result receiving step. That is, the local prediction result receiving step can be performed by the local prediction result receiving module 720.

[0094] In some embodiments, the prediction module 730 can be configured to obtain a prediction result for the identification information of the object to be predicted based on the local prediction result of one or more user nodes. In some embodiments, the prediction module 730 can be further configured to obtain the prediction result based on the local prediction results of two or more user nodes.

[0095] For implementation of the prediction module 730, reference can be made to the description of the prediction step. That is, the prediction step can be performed by the prediction module 730.

[0096] In some embodiments, the model can include at least one of a linear regression model, a decision tree, and a neural network.

[0097] Figure 8 FIG. 8 is an exemplary block diagram of a distributed prediction system according to some embodiments of the present disclosure. The system 800 corresponds to a user node of two or more user nodes. Two or more user nodes each maintain a respective user model, each of which corresponds to a portion of a prediction model. In some embodiments, the system 800 can include a prediction request receiving module 810, a local prediction module 820, and a local prediction result returning module 830.

[0098] In some embodiments, the prediction request receiving module 810 can be configured to receive a prediction request for the to-be-predicted object identification information sent by a prediction node, which can include one of the two or more user nodes or a node other than the two or more user nodes.

[0099] For the specific implementation of the prediction request receiving module 810, reference can be made to the related description of the prediction request receiving step. That is, the prediction request receiving step can be performed by the prediction request receiving module 810.

[0100] In some embodiments, the local prediction module 820 can be configured to obtain feature data from the local based on the to-be-predicted object identification information, and obtain a local prediction result based on at least the user model and the feature data. In some embodiments, the local prediction module 820 can be further configured to obtain the local prediction result based on the user model, the feature data, and a prediction intermediate result from one or more other user nodes.

[0101] For the specific implementation of the local prediction module 820, reference can be made to the related description of the local prediction step. That is, the local prediction step can be performed by the local prediction module 820.

[0102] In some embodiments, the local prediction result returning module 830 can be configured to return the local prediction result to the prediction node.

[0103] For the specific implementation of the local prediction result returning module 830, reference can be made to the related description of the local prediction result returning step. That is, the local prediction result returning step can be performed by the local prediction result returning module 830.

[0104] In some embodiments, the model can include at least one of a linear regression model, a decision tree, and a neural network.

[0105] It should be understood that, Figures 6 to 8The system and its modules disclosed above can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of hardware and software. The hardware part can be implemented by using special logic; the software part can be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or a specially designed hardware. Those skilled in the art can understand that the above-mentioned method and system can be implemented by using computer executable instructions and / or contained in processor control code, such as carrier media, such as magnetic disk, CD or DVD-ROM, programmable memory, such as read-only memory (firmware), or data carrier, such as optical or electronic signal carrier. The system and its modules of the present specification can not only be implemented by hardware circuit, such as ultra large scale integrated circuit or gate array, semiconductor, such as logic chip, transistor, or programmable hardware device, such as field programmable gate array, programmable logic device, but also by software, such as executed by various types of processors, and also by a combination of the above-mentioned hardware circuit and software (for example, firmware).

[0106] It should be noted that the above description of the system and its modules is for the convenience of description, and cannot limit the present specification within the scope of the embodiments. It can be understood that, for those skilled in the art, after understanding the principle of the system, the modules can be combined or connected with other modules to form a subsystem without departing from the principle. For example, in some embodiments, Figure 6 The decryption module 620 and the training module 630 disclosed in the above embodiment can be different modules in a system, or can be a module to realize the functions of the two modules. For another example, in some embodiments, Figure 8 The local prediction module 820 disclosed in the above embodiment can be a module, or can be decomposed into a local feature data acquisition module and a local prediction result acquisition module, wherein the local feature data acquisition module is used to acquire feature data from the local based on the object identification information to be predicted, and the local prediction result acquisition module is used to obtain a local prediction result based on at least the user model and the feature data. Variations such as this are within the protection scope of the present specification.

[0107] The beneficial effects that the embodiments of the present specification can bring include but are not limited to: (1) centralized model training by the central node with a trusted execution environment, which not only protects the privacy of the sample data of each party, but also ensures a high training speed; (2) each user node locally performs prediction (i.e., distributed prediction), without transmitting the local feature data of the to-be-predicted object to the outside, thereby protecting the data privacy of each party. It should be noted that different embodiments can have different beneficial effects, and in different embodiments, the beneficial effects that can be produced can be any one or a combination of several of the above, or any other beneficial effects that can be obtained.

[0108] The above has described the basic concepts. Obviously, the above detailed disclosure is only taken as an example for the person skilled in the art, and does not constitute a limitation on the embodiments of the present specification. Although it is not explicitly stated here, the person skilled in the art can make various modifications, improvements and corrections to the embodiments of the present specification. Such modifications, improvements and corrections are suggested in the embodiments of the present specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of the present specification.

[0109] Meanwhile, specific words are used in the present specification to describe the embodiments of the present specification. As "one embodiment", "an embodiment", and / or "some embodiments" means a certain feature, structure or characteristic related to at least one embodiment of the present specification. Therefore, it should be emphasized and noted that the "an embodiment" or "one embodiment" or "one alternative embodiment" mentioned in different places in the present specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of the present specification can be properly combined.

[0110] In addition, the person skilled in the art can understand that aspects of the embodiments of the present specification can be described and illustrated by several patentable categories or cases, including any new and useful processes, machines, products or combinations of matter, or any new and useful improvements to them. Accordingly, various aspects of the embodiments of the present specification can be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software can be referred to as "data block", "module", "engine", "unit", "component" or "system". In addition, aspects of the embodiments of the present specification can be manifested as a computer product located in one or more computer readable media, including computer readable program code.

[0111] Computer storage media can include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program code, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, solid state drives (SSDs) that use flash memory, phase-change (PC) RAM, any other memory technology, or any other medium that can be used to store and / or transfer computer readable instructions and / or data. Computer storage media can also include, but is not limited to, any medium that facilitates computer readable instructions and / or data being communicated from one place to another, for example, by way of a communication link, a wireless channel, and / or any combination of the aforementioned. Computer storage media can be any available physical medium that can be accessed by a general purpose or special purpose computing device. Computer storage media can be implemented in a computer program product.

[0112] Computer program code for carrying out operations of embodiments of the present specification can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, or the like, conventional procedural programming languages, such as the "C" programming language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic program code can be downloaded from an on-demand computing platform, such as Amazon Web Services, Microsoft Azure, or Google Cloud Platform, or other on-demand computing platforms. In some embodiments, electronic program code can be downloaded from a cloud computing environment, or used as a service, such as Software as a Service (SaaS).

[0113] In addition, the order of execution or sequence of actions in which elements of embodiments of the present specification are described is not limited to the order given in the embodiments of the present specification unless otherwise specified by the claims. Further, use of ordinal terms such as "first" and "second" and "third" and "fourth" and "last" and the like does not limit the embodiments of the present specification to any particular sequence. Embodiments of the present specification can be implemented with other technologies as long as they do not depart from the spirit of the embodiments of the present specification. Although some embodiments of the present disclosure have been discussed through various examples in the above disclosure, it should be understood that such details are only for illustrative purposes, and additional claims are not limited to the disclosed embodiments, but rather, the claims are intended to cover all modifications and equivalent combinations that fall within the spirit and scope of the embodiments of the present specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented by software solutions, such as installing the described system on existing processing devices or mobile devices.

[0114] For simplicity of explanation, the illustrative embodiments herein are presented and described as pertaining specifically to an example embodiment. However, the example embodiments presented and described herein are not intended to be restrictive of the application as such. The scope of the application is defined by the appended claims and their equivalents. In addition, it should be noted that while the present application is concerned with methods, systems, and computer program products, the application is equally applicable and useful for other types of computing devices and environments.

[0115] Every patent, patent application, publication, document, article, book, specification, and drawing cited or referred to in this disclosure is hereby incorporated by reference in its entirety. To the extent the content of the incorporated references is inconsistent with the content of this disclosure, the content of this disclosure is controlling. Note that if a description, definition, and / or a term used in any incorporated reference is contradicted by one used in the present application, the description, definition, and / or term used in the present application prevails.

[0116] Finally, it should be noted that the embodiments described herein are merely examples of implementations and that alternative implementations are possible. Accordingly, the embodiments described herein are not limited to the examples described and illustrated herein.

Claims

1. A distributed prediction method, wherein, The method is performed by a prediction node, the prediction node being one of two or more user nodes or a node other than the two or more user nodes and being communicatively connected to the two or more user nodes, the two or more user nodes each storing a respective user model, each user model corresponding to a part of a prediction model, the prediction model being a machine learning model trained by sample data, the machine learning model including at least one of a linear regression model, a decision tree, and a neural network, the user model being obtained by splitting model parameters of the prediction model, and the method comprising: sending, to one or more user nodes, a prediction request for identification information of an object to be predicted; receiving local prediction results of the one or more user nodes, the local prediction results being based at least on user models of the corresponding user nodes and feature data for the identification information of the object to be predicted; obtaining a prediction result for the identification information of the object to be predicted based on the local prediction results of the one or more user nodes.

2. The method of claim 1, wherein, The obtaining of the prediction result for the identification information of the object to be predicted based on the local prediction results of the one or more user nodes includes: obtaining the prediction result based on the local prediction results of the two or more user nodes.

3. A distributed prediction system, wherein, The system corresponds to a prediction node, the prediction node being one of two or more user nodes or a node other than the two or more user nodes and being communicatively connected to the two or more user nodes, the two or more user nodes each storing a respective user model, each user model corresponding to a part of a prediction model, the prediction model being a machine learning model trained by sample data, the machine learning model including at least one of a linear regression model, a decision tree, and a neural network, the user model being obtained by splitting model parameters of the prediction model, and the system comprising: a prediction request sending module configured to send, to one or more user nodes, a prediction request for identification information of an object to be predicted; a local prediction result receiving module configured to receive local prediction results of the one or more user nodes, the local prediction results being based at least on user models of the corresponding user nodes and feature data for the identification information of the object to be predicted; a prediction module configured to obtain a prediction result for the identification information of the object to be predicted based on the local prediction results of the one or more user nodes.

4. The distributed prediction system of claim 3, wherein, The prediction module is further configured to: obtain the prediction result based on the local prediction results of the two or more user nodes.

5. A distributed prediction apparatus wherein, The system includes at least one processor and at least one storage device storing instructions that, when executed by the at least one processor, implement the method of claim 1 or 2.

6. A distributed prediction method, wherein, The method is performed by a user node of two or more user nodes, the two or more user nodes storing respective user models, each user model corresponding to a part of a prediction model, the prediction model being a machine learning model trained by sample data, the machine learning model comprising at least one of: a linear regression model, a decision tree, and a neural network, the user model being obtained by splitting model parameters of the prediction model; the method comprising: receiving a prediction request for identification information of a to-be-predicted object sent by a prediction node, the prediction node comprising one of the two or more user nodes or a node other than the two or more user nodes; obtaining feature data from the local based on the identification information of the to-be-predicted object; obtaining a local prediction result based on at least the user model and the feature data; returning the local prediction result to the prediction node.

7. The method of claim 6, wherein, The obtaining of the local prediction result based on at least the user model and the feature data comprises: obtaining the local prediction result based on the user model, the feature data, and a prediction intermediate result from another user node or other user nodes.

8. A distributed prediction system, wherein, The system corresponds to a user node of two or more user nodes, the two or more user nodes storing respective user models, each user model corresponding to a part of a prediction model, the prediction model being a machine learning model trained by sample data, the machine learning model comprising at least one of: a linear regression model, a decision tree, and a neural network, the user model being obtained by splitting model parameters of the prediction model; the system comprising: a prediction request receiving module configured to receive a prediction request for identification information of a to-be-predicted object sent by a prediction node, the prediction node comprising one of the two or more user nodes or a node other than the two or more user nodes; a local prediction module configured to obtain feature data from the local based on the identification information of the to-be-predicted object, and obtain a local prediction result based on at least the user model and the feature data; a local prediction result returning module configured to return the local prediction result to the prediction node.

9. The system of claim 8, wherein, The local prediction module is further configured to: obtain the local prediction result based on the user model, the feature data, and a prediction intermediate result from another user node or other user nodes.

10. A distributed prediction apparatus wherein, The system comprises at least one processor and at least one storage device storing instructions, when the at least one processor executes the instructions, the method of claim 6 or 7 is implemented.

Images