Model training method, data reconstruction method, and electronic device

Abstract

The present disclosure provides a model training method, a data reconstruction method, an apparatus, a device and a storage medium, which can be applied to the field of artificial intelligence technology, and specifically applied to the field of distributed machine learning technology. The method comprises: determining a first target training sample corresponding to first target label information from a training sample set, wherein the first target label information corresponds to second target label information, and the second target label information is label information corresponding to a second target training sample; inputting the first target training sample into a first pseudo-client model to obtain first pseudo-intermediate layer output information; inputting the first pseudo-intermediate layer output information into a first server model to obtain first output information; and adjusting model parameters of the first pseudo-client model according to the first output information and the first target label information.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the field of distributed machine learning technology, and particularly to a model training method, data reconstruction method, apparatus, device, medium, and program product. Background Technology

[0002] The protection of personal and institutional privacy has received increasing attention, leading to stricter regulations on data privacy and security. Data dispersed among numerous users cannot be directly uploaded to cloud computing centers for traditional centralized machine learning model training due to legal and regulatory restrictions. This dispersed data creates data silos, preventing the exchange of data and hindering the full potential and value of big data. Therefore, ensuring data privacy and security is a crucial technical challenge that needs to be addressed in related technologies. Summary of the Invention

[0003] In view of the above problems, this disclosure provides a model training method, data reconstruction method, apparatus, device, medium and program product.

[0004] According to a first aspect of this disclosure, a method for training a first pseudo-client model is provided. The first deep learning model includes a first pseudo-client model and a first server model, wherein the first deep learning model is deployed on a server, comprising:

[0005] A first target training sample corresponding to the first target label information is determined from the training sample set, wherein the first target label information matches the second target label information, the second target label information is the label information corresponding to the second target training sample, the second target training sample is used to train the first real client model in the second deep learning model, the second deep learning model also includes the first server model, and the first real client model is deployed on the client.

[0006] The first target training sample is input into the first pseudo-client model to obtain the output information of the first pseudo intermediate layer;

[0007] The first pseudo-intermediate layer output information is input into the first server model to obtain the first output information; and

[0008] Based on the first output information and the first target label information mentioned above, adjust the model parameters of the first pseudo-client model.

[0009] According to embodiments of this disclosure, determining the first target training sample corresponding to the first target label information from the training sample set includes:

[0010] In response to receiving second target label information from the aforementioned client, the first target training sample corresponding to the first target label information is determined from the aforementioned training sample set based on the first target label information that is consistent with the second target label information.

[0011] According to embodiments of this disclosure, adjusting the model parameters of the first pseudo-client model based on the first output information and the first target label information includes:

[0012] The first output information and the first target label information are input into the first loss function to obtain the first loss value;

[0013] Based on the first loss value mentioned above, the first gradient is obtained;

[0014] Input the first gradient into the first pseudo-client model to obtain the second gradient;

[0015] Based on the second gradient mentioned above, adjust the model parameters of the first pseudo-client model.

[0016] The second aspect of this disclosure provides a training method for a second pseudo-client model, wherein a third deep learning model includes a second pseudo-client model and a second server model, the second pseudo-client model includes a first pseudo-client module and a second pseudo-client module, and the third deep learning model is deployed on a server; the method includes:

[0017] A third target training sample corresponding to the third target label information is determined from the training sample set. The third target label information corresponds to the fourth target label information. The fourth target label information is the label information corresponding to the fourth target training sample. The fourth target training sample is used to train the second real client model in the fourth deep learning model. The second real client model includes a first real client module and a second real client module. The fourth deep learning model also includes the second server model. The second real client model is deployed on the client.

[0018] The training samples for the third target are input into the first pseudo-client module to obtain the output information of the second pseudo-intermediate layer.

[0019] The output information of the second pseudo-intermediate layer is input into the second server model to obtain the output information of the third pseudo-intermediate layer.

[0020] The output information of the third pseudo-intermediate layer is input into the second pseudo-client module to obtain the second output information; and

[0021] Based on the second output information and the third target label information mentioned above, adjust the model parameters of the second pseudo-client model.

[0022] According to embodiments of this disclosure, the above-described determination of the third target training sample corresponding to the third target label information from the training sample set includes:

[0023] In response to receiving the fourth target training sample from the aforementioned client, the third target label information is randomly determined; and

[0024] Based on the aforementioned third target label information, the aforementioned third target training samples corresponding to the aforementioned third target label information are determined from the aforementioned training sample set.

[0025] According to embodiments of this disclosure, adjusting the model parameters of the second pseudo-client model based on the second output information and the third target label information includes:

[0026] The second output information and the third target label information are input into the second loss function to obtain the second loss value;

[0027] Based on the second loss value mentioned above, the third and fourth gradients are obtained;

[0028] The third gradient is input into the first pseudo-client module through the second server model to obtain the fifth gradient;

[0029] Based on the fourth gradient described above, adjust the model parameters of the second pseudo-client module; and

[0030] Based on the fifth gradient mentioned above, adjust the model parameters of the first pseudo-client module.

[0031] A third aspect of this disclosure provides a method for training a reconstructed model, comprising:

[0032] The training samples of the fifth target are input into the pseudo-client model to obtain the output information of the fourth pseudo-intermediate layer;

[0033] The output information of the fourth pseudo-intermediate layer is input into the fifth deep learning model to obtain the reconstructed sample; and

[0034] Based on the training samples for the fifth objective and the reconstructed samples, the model parameters of the fifth deep learning model are adjusted to obtain the reconstructed model.

[0035] The aforementioned pseudo-client model includes one of the aforementioned first pseudo-client model and the aforementioned second pseudo-client model.

[0036] The fourth aspect of this disclosure provides a data reconstruction method, comprising:

[0037] Input the target data into the reconstruction model to obtain the original reconstruction data;

[0038] Input the above original reconstructed data into the pseudo-client model to obtain intermediate reconstructed data;

[0039] If the difference between the target data and the intermediate reconstructed data meets the preset conditions, the original reconstructed data is determined as the target reconstructed data.

[0040] The aforementioned pseudo-client model includes one of the aforementioned first pseudo-client model and the aforementioned second pseudo-client model.

[0041] According to embodiments of this disclosure, the above data reconstruction method further includes:

[0042] If the difference between the target data and the intermediate reconstructed data does not meet the preset conditions, the original reconstructed data is adjusted to obtain new intermediate reconstructed data until the difference between the target data and the new intermediate reconstructed data meets the preset conditions.

[0043] The fifth aspect of this disclosure provides a training apparatus for a first pseudo-client model, comprising: a determining module, a first obtaining module, a second obtaining module, and an adjusting module. The determining module is used to determine a first target training sample corresponding to first target label information from a training sample set, wherein the first target label information matches second target label information, the second target label information being the label information corresponding to the second target training sample, and the second target training sample being used to train a first real client model in a second deep learning model. The second deep learning model further includes a first server model, and the first real client model is deployed on a client. The first obtaining module is used to input the first target training sample into the first pseudo-client model to obtain first pseudo intermediate layer output information. The second obtaining module is used to input the first pseudo intermediate layer output information into the first server model to obtain first output information. The adjusting module is used to adjust the model parameters of the first pseudo-client model according to the first output information and the first target label information.

[0044] A sixth aspect of this disclosure provides a training apparatus for a second pseudo-client model, comprising: a determining module, a first obtaining module, a second obtaining module, a third obtaining module, and an adjusting module. The determining module is used to determine a third target training sample corresponding to third target label information from a training sample set. The third target label information corresponds to fourth target label information, which is the label information corresponding to the fourth target training sample. The fourth target training sample is used to train a second real client model in a fourth deep learning model. The second real client model includes a first real client module and a second real client module. The fourth deep learning model also includes a second server model, and the second real client model is deployed on a client. The first obtaining module is used to input the third target training sample into the first pseudo-client module to obtain second pseudo intermediate layer output information. The second obtaining module is used to input the second pseudo intermediate layer output information into the second server model to obtain third pseudo intermediate layer output information. The third obtaining module is used to input the third pseudo intermediate layer output information into the second pseudo client module to obtain second output information. The adjusting module is used to adjust the model parameters of the second pseudo client model according to the second output information and the third target label information.

[0045] A seventh aspect of this disclosure provides a training apparatus for a reconstructed model, comprising: a first obtaining module, a second obtaining module, and an adjustment module. The first obtaining module is used to input a fifth target training sample into a pseudo-client model to obtain the output information of a fourth pseudo-intermediate layer. The second obtaining module is used to input the output information of the fourth pseudo-intermediate layer into a fifth deep learning model to obtain reconstructed samples. The adjustment module is used to adjust the model parameters of the fifth deep learning model based on the fifth target training sample and the reconstructed samples to obtain the reconstructed model, wherein the pseudo-client model includes one of a first pseudo-client model and a second pseudo-client model.

[0046] An eighth aspect of this disclosure provides a data reconstruction apparatus, comprising: a first obtaining module, a second obtaining module, and a determining module. The first obtaining module is used to input target data into a reconstruction model to obtain original reconstructed data. The second obtaining module is used to input the original reconstructed data into a pseudo-client model to obtain intermediate reconstructed data. The determining module is used to determine the original reconstructed data as the target reconstructed data if the difference between the target data and the intermediate reconstructed data satisfies a preset condition. The pseudo-client model includes one of a first pseudo-client model and a second pseudo-client model.

[0047] A ninth aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the methods described above.

[0048] The tenth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described above.

[0049] The eleventh aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0050] According to the model training method, data reconstruction method, apparatus, device, medium, and program product provided in this disclosure, the pseudo-client model obtained through the model training method enables the server to acquire the functions of the real client model without disrupting the main learning thread, without being detected by the real client, and under the premise that the structure of the real client model is unknown and there are several orders of magnitude of training samples. The reconstructed model obtained through the model training method, and then according to the data reconstruction method, can realize the reconstruction of the privacy data of the real client model. At the same time, the real client sends the intermediate layer output obtained through the privacy data to the server, instead of sending the privacy data directly to the server, so that the data distributed by various parties no longer become data silos, and mutual flow is realized again. This realizes the full realization of the effectiveness and value of big data, and protects the technical effect of data privacy and security. Attached Figure Description

[0051] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0052] Figure 1 The illustrations depict application scenarios of model training methods, data reconstruction methods, apparatuses, devices, media, and program products according to embodiments of the present disclosure.

[0053] Figure 2 A flowchart illustrating a training method for a first pseudo-client model according to an embodiment of the present disclosure is shown schematically.

[0054] Figure 3 The illustration shows a schematic diagram of the first pseudo-client model function acquisition and inference execution according to an embodiment of the present disclosure;

[0055] Figure 4 A flowchart illustrating a training method for a second pseudo-client model according to an embodiment of the present disclosure is shown schematically.

[0056] Figure 5 The illustration shows a schematic diagram of the second pseudo-client model function acquisition and tag inference according to an embodiment of the present disclosure;

[0057] Figure 6A flowchart illustrating a training method for a reconstructed model according to an embodiment of the present disclosure is shown schematically;

[0058] Figure 7 A flowchart illustrating a data reconstruction method according to an embodiment of the present disclosure is shown schematically.

[0059] Figure 8 The illustration schematically depicts a privacy data reconstruction diagram according to an embodiment of the present disclosure;

[0060] Figure 9 The illustration shows a schematic diagram comparing the results of a label alignment experiment according to an embodiment of the present disclosure;

[0061] Figure 10 A schematic diagram illustrating a comparison of delayed start experiment results according to an embodiment of the present disclosure is provided.

[0062] Figure 11 A schematic diagram illustrating a model splitting strategy according to an embodiment of the present disclosure is shown.

[0063] Figure 12 The diagram illustrates the accuracy and loss convergence curves of a PCAT pseudo-client model according to an embodiment of the present disclosure.

[0064] Figure 13(a) schematically illustrates the reconstruction results of input data on the MNIST dataset according to an embodiment of the present disclosure;

[0065] Figure 13(b) schematically illustrates the reconstruction results of input data on the CIFAR-10 dataset according to an embodiment of the present disclosure;

[0066] Figure 13(c) schematically illustrates the reconstruction results of input data on the Tiny ImageNet dataset according to an embodiment of the present disclosure;

[0067] Figure 14 This schematically illustrates a structural block diagram of a training apparatus for a first pseudo-client model according to an embodiment of the present disclosure;

[0068] Figure 15 This schematic diagram illustrates a structural block diagram of a training apparatus for a second pseudo-client model according to an embodiment of the present disclosure;

[0069] Figure 16 A flowchart illustrating a training apparatus for a reconstructed model according to an embodiment of the present disclosure is shown schematically.

[0070] Figure 17 A flowchart illustrating a data reconstruction apparatus according to an embodiment of the present disclosure is shown schematically; and

[0071] Figure 18A block diagram schematically illustrates an electronic device suitable for implementing a model training method and a data reconstruction method according to embodiments of the present disclosure. Detailed Implementation

[0072] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0073] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0074] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0075] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0076] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0077] In the technical solution disclosed herein, the acquisition, collection, storage, use, processing, transmission, provision, disclosure, and application of data all comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0078] In implementing this disclosure, it was discovered that data scattered across numerous users cannot be directly uploaded to cloud computing centers for traditional centralized machine learning model training. Therefore, the data distributed across various parties becomes data silos, unable to flow between them, preventing big data from fully realizing its effectiveness and value. Against this backdrop, split learning technology emerged. Split learning is a new paradigm of distributed machine learning that utilizes splitting to divide deep neural networks into client-side and server-side models. The client only needs to perform lightweight computations and output intermediate layer activations, avoiding the output of raw data and thus protecting the original data. However, acquiring the client-side model functionality in split learning presents the problem of disrupting the split learning framework and being detected by the client.

[0079] Figure 1 The diagram illustrates an application scenario of the model training method and data reconstruction method according to embodiments of the present disclosure.

[0080] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0081] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0082] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0083] For example, the first real client model can be deployed on the first terminal device 101, the second terminal device 102, and the third terminal device 103. The second target label information and the second target training samples can be obtained from the original input data through the first terminal device 101, the second terminal device 102, and the third terminal device 103. At the same time, the second target label information and the second target training samples are sent to the server, without directly sending the original input data to the server, thus protecting the privacy and security of the data.

[0084] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0085] For example, the first pseudo-client model and the first server model can be deployed on server 105. Server 105 can determine the first target training sample corresponding to the first target label information from the training sample set, input the first target training sample into the first pseudo-client model to obtain the first pseudo intermediate layer output information, input the first pseudo intermediate layer output information into the first server model to obtain the first output information, and adjust the model parameters of the first pseudo-client model based on the first output information and the first target label information.

[0086] It should be noted that the model training method and data reconstruction method provided in this embodiment can generally be executed by server 105. Correspondingly, the model training device and data reconstruction device provided in this embodiment can generally be located in server 105. The model training method and data reconstruction method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the model training device and data reconstruction device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0087] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0088] The following will be based on Figure 1 The described scene, through Figures 2-13. The model training method and data reconstruction method of the disclosed embodiments are described in detail.

[0089] Figure 2 A flowchart illustrating a training method for a first pseudo-client model according to an embodiment of the present disclosure is shown schematically.

[0090] According to an embodiment of this disclosure, a first pseudo-client model is deployed on a server.

[0091] like Figure 2 As shown, the method 200 includes operations S210 to S240.

[0092] In operation S210, the first target training sample corresponding to the first target label information is determined from the training sample set.

[0093] According to embodiments of this disclosure, the first deep learning model may include a first pseudo-client model and a first server model, wherein the first deep learning model is deployed on a server.

[0094] According to embodiments of this disclosure, the first target label information is matched with the second target label information, the second target label information is the label information corresponding to the second target training sample, the second target training sample can be used to train the first real client model in the second deep learning model, the second deep learning model can also include a first server model, and the first real client model is deployed on the client.

[0095] According to embodiments of this disclosure, the training sample set may include a first image set. The training sample set may characterize the training sample set in the server.

[0096] According to an embodiment of this disclosure, before training the first pseudo-client model, the first pseudo-client model is randomly initialized, and a time period is set as the training time for the first pseudo-client model.

[0097] According to embodiments of this disclosure, the raw data of the first real client model may include a second image set.

[0098] According to embodiments of this disclosure, based on a second image set, raw data is input into a first real client model, forward propagation is performed to obtain intermediate layer outputs and second target label information of images in the second image set, and the intermediate layer outputs and second target label information are sent to a server. The second target training samples may include intermediate layer outputs, and the raw data may represent the privacy data of the first real client model. The real client sends the intermediate layer outputs obtained through the privacy data to the server, but does not send the privacy data directly to the server, thus protecting data privacy and security.

[0099] According to embodiments of this disclosure, when the label information of the first real client model is known, the server selects first target label information that matches the second target label information from the training sample set based on the received second target label information, and selects a first target training sample corresponding to the first target label information. Selecting first target label information that matches the second target label information can achieve label alignment.

[0100] In operation S220, the first target training sample is input into the first pseudo-client model to obtain the output information of the first pseudo intermediate layer.

[0101] According to an embodiment of this disclosure, a first target training sample determined from a first image set is input into a first pseudo-client model, and forward propagation is performed to obtain the output information of a first pseudo intermediate layer.

[0102] In operation S230, the output information of the first pseudo intermediate layer is input into the first server model to obtain the first output information.

[0103] According to embodiments of this disclosure, the first output information can characterize the output value obtained based on the first target training sample. The first pseudo-intermediate layer output information is input into the first server model to obtain the output value corresponding to the first pseudo-intermediate layer output information.

[0104] In operation S240, the model parameters of the first pseudo-client model are adjusted based on the first output information and the first target label information.

[0105] According to embodiments of this disclosure, under the constraints of the server, the gradient of the first pseudo-client model can be obtained based on the first output information and the first target label information. The model parameters of the first pseudo-client model are then adjusted based on the gradient of the first pseudo-client model.

[0106] According to embodiments of this disclosure, model training based on the first target label information can yield a first pseudo-client model. This allows the server to acquire the functionality of the first real client model without disrupting the main learning process, without being detected by the real client, and with the real client model structure unknown and having several orders of magnitude of training samples. Simultaneously, the real client sends the intermediate layer output obtained through privacy data to the server, without directly sending the privacy data to the server. This prevents data distributed across various parties from becoming isolated data silos, enabling mutual flow between them. This achieves the technical effect of fully leveraging the effectiveness and value of big data while protecting data privacy and security.

[0107] According to an embodiment of this disclosure, determining a first target training sample corresponding to the first target label information from a training sample set includes: in response to receiving second target label information from a client, determining a first target training sample corresponding to the first target label information from a training sample set based on first target label information that is consistent with the second target label information.

[0108] According to embodiments of this disclosure, a first target training sample corresponding to the first target label information is determined from the training sample set based on the first target label information that is consistent with the second target label information. This enables label alignment and improves the accuracy of the first pseudo-client model in obtaining the function of the first real client model.

[0109] According to embodiments of this disclosure, adjusting the model parameters of a first pseudo-client model based on first output information and first target label information includes: inputting the first output information and first target label information into a first loss function to obtain a first loss value; obtaining a first gradient based on the first loss value; inputting the first gradient into the first pseudo-client model to obtain a second gradient; and adjusting the model parameters of the first pseudo-client model based on the second gradient.

[0110] According to embodiments of this disclosure, first output information and first target label information are input into a first loss function, and forward propagation is performed to obtain a first loss value. Based on the first loss value, backpropagation is performed to obtain a first gradient, wherein the first gradient may include the gradient output by the first pseudo-intermediate layer. The first gradient is input into a first pseudo-client model, and backpropagation is performed to obtain a second gradient, wherein the second gradient may include the gradient of the first pseudo-client model.

[0111] According to embodiments of this disclosure, the gradient of the first pseudo-client model is obtained based on the first output information and the first target label information, and the model parameters of the first pseudo-client model are adjusted based on the gradient of the first pseudo-client model, thereby enabling the training of the first pseudo-client model.

[0112] Figure 3 The illustration shows a schematic diagram of the first pseudo-client model function acquisition and inference execution according to an embodiment of the present disclosure.

[0113] like Figure 3 As shown, with the label information of the first real client model known, we take the two-way split learning model as an example. This can characterize the first pseudo-client model. It can represent the first real client model. It can characterize the first server model. This can characterize the first pseudo-client model after training. It can characterize the first server model after training. X serverX can represent the first target training sample. priv The raw data can represent the first real client model, and the raw data can represent the privacy data of the first real client model. server It can represent the first target label information, y priv It can represent the second target label information, where y server =y priv This can characterize the achievement of label alignment. X text Pred can represent the original input of the reasoning process, while pred can represent the output information of the reasoning process.

[0114] According to embodiments of this disclosure, a first pseudo-client model and a first server model are deployed on a server, and a first real client model is deployed on a client.

[0115] According to an embodiment of this disclosure, before training the first pseudo-client model, the first pseudo-client model is randomly initialized, and a time period is set as the training time for the first pseudo-client model.

[0116] According to an embodiment of this disclosure, the first real client model performs forward propagation based on its own original data to obtain a second target training sample and a second target label information system, wherein the second target label information is the label information corresponding to the second target training sample, and the second target training sample may include the intermediate layer output information of the first real client model.

[0117] According to embodiments of this disclosure, when the label information of the first real client model is known, the intermediate layer output information and the second target label information are sent to the server. The real client sends the intermediate layer output obtained through privacy data to the server, but does not directly send the privacy data to the server, thus protecting data privacy and security. The server selects first target label information that matches the label information of the second target label information, and selects a first target training sample based on the first target label information. The first target label information and the second target label information achieve label alignment.

[0118] According to embodiments of this disclosure, a first target training sample is input into a first pseudo-client model, and forward propagation is performed to obtain the output information of a first pseudo-intermediate layer. The first pseudo-intermediate layer output information and the intermediate layer output information are then input into a first server model.

[0119] According to embodiments of this disclosure, a first server model obtains first output information based on the output information of a first pseudo-intermediate layer, and inputs the first output information and first target label information into a first loss function, performs forward propagation, and obtains a first loss value. Based on the first loss value, backpropagation is performed to obtain the gradient of the output of the first pseudo-intermediate layer.

[0120] According to an embodiment of this disclosure, the gradient output by the first pseudo-intermediate layer is input into the first pseudo-client model, backpropagation is performed to obtain the gradient of the first pseudo-client model, and the model parameters of the first pseudo-client model are adjusted according to the gradient of the first pseudo-client model to achieve training of the first pseudo-client model.

[0121] According to embodiments of this disclosure, the first server model performs forward propagation based on the intermediate layer output information to obtain the loss value of the first real client model, and performs backpropagation based on the loss value to obtain the gradient of the first server model and the gradient output by the intermediate layer. The model parameters of the first server model are then adjusted based on the gradient of the first server model.

[0122] According to an embodiment of this disclosure, the gradient output by the intermediate layer is sent to the first real client model. Backpropagation is performed based on the gradient output by the intermediate layer to obtain the gradient of the first real client model. The model parameters of the first real client model are adjusted based on the gradient of the first real client model.

[0123] According to embodiments of this disclosure, when the set time for training the first pseudo-client model is reached, model training ends, and the acquisition of the functionality of the first real client model is completed. Once the first pseudo-client model has completed acquiring the functionality of the first real client model, the server can independently perform inference tasks based on the first server model and the first pseudo-client model, without the participation of the first real client model.

[0124] Figure 4 A flowchart illustrating a training method for a second pseudo-client model according to an embodiment of the present disclosure is shown.

[0125] According to embodiments of this disclosure, the second pseudo-client model is deployed on a server.

[0126] like Figure 4 As shown, the method 400 includes operations S410 to S450.

[0127] In operation S410, the third target training sample corresponding to the third target label information is determined from the training sample set.

[0128] According to embodiments of this disclosure, the third deep learning model may include a second pseudo-client model and a second server model. The second pseudo-client model may include a first pseudo-client module and a second pseudo-client module. The third deep learning model is deployed on a server.

[0129] According to embodiments of this disclosure, the third target label information corresponds to the fourth target label information. The fourth target label information is the label information corresponding to the fourth target training sample. The fourth target training sample can be used to train the second real client model in the fourth deep learning model. The second real client model can include the first real client module and the second real client module. The fourth deep learning model can also include the second server model. The second real client model is deployed on the client.

[0130] According to embodiments of this disclosure, the training sample set may include a third image set. The training sample set may characterize the training sample set in the server.

[0131] According to an embodiment of this disclosure, before training the second pseudo-client model, the second pseudo-client model is randomly initialized, and a time period is set as the training time for the second pseudo-client model.

[0132] According to embodiments of this disclosure, a first pseudo-client module can represent a pseudo-bottom model, and a second pseudo-client module can represent a pseudo-top model. A first real client module can represent a bottom model, and a second real client module can represent a top model.

[0133] According to embodiments of this disclosure, the raw data of the first real client model may include a fourth image set.

[0134] According to embodiments of this disclosure, based on a fourth image set, the original data is input into the bottom model, forward propagation is performed to obtain the intermediate layer output of the images in the fourth image set and the fourth target label information, and the intermediate layer output is sent to the server. The fourth target training samples may include the intermediate layer output, and the original data may represent the privacy data of the second real client model. The real client sends the intermediate layer output obtained through the privacy data to the server, but does not send the privacy data directly to the server, thus protecting data privacy and security.

[0135] According to an embodiment of this disclosure, when the label information of the second real client model is unknown, the server randomly determines the label information of the third target based on the received fourth target training sample, and selects the third target training sample corresponding to the third target label information from the training sample set based on the third target label information, and inputs the third target label information into the pseudo top model.

[0136] When operating S420, the training sample of the third target is input into the first pseudo-client module to obtain the output information of the second pseudo-intermediate layer.

[0137] According to an embodiment of this disclosure, a third target training sample determined from a third image set is input into a pseudo-bottom model, and forward propagation is performed to obtain the output information of the second pseudo-intermediate layer.

[0138] When operating S430, the output information of the second pseudo intermediate layer is input into the second server model to obtain the output information of the third pseudo intermediate layer.

[0139] According to an embodiment of this disclosure, the second pseudo-intermediate layer output information is input into the second server model, and the second server model converts the received second pseudo-intermediate layer output information into the third pseudo-intermediate layer output information to be sent by the second server model.

[0140] When operating S440, the output information of the third pseudo intermediate layer is input into the second pseudo client module to obtain the second output information.

[0141] According to embodiments of this disclosure, the second output information can characterize the output value obtained from the training samples of the third target. Inputting the third pseudo-intermediate layer output information into the pseudo-top model yields the output value corresponding to the third pseudo-intermediate layer output information.

[0142] During operation of S450, the model parameters of the second pseudo-client model are adjusted based on the second output information and the third target label information.

[0143] According to embodiments of this disclosure, under server constraints, the gradients of the pseudo-top model and the pseudo-bottom model can be obtained based on the second output information and the third target label information. The model parameters of the pseudo-top model are adjusted based on the gradient of the pseudo-top model. The model parameters of the pseudo-bottom model are also adjusted based on the gradient of the pseudo-bottom model.

[0144] According to embodiments of this disclosure, model training based on third label information can yield a second pseudo-client model. This allows the server to acquire the functionality of the second real client model without disrupting the main learning process, without being detected by the real client, and with the real client model structure unknown and having several orders of magnitude of training samples. Simultaneously, the real client sends the intermediate layer output obtained through privacy data to the server, without directly sending the privacy data to the server. This prevents data distributed across various parties from becoming isolated data silos, enabling mutual flow between them. This achieves the technical effect of fully leveraging the effectiveness and value of big data while protecting data privacy and security.

[0145] According to embodiments of this disclosure, determining a third target training sample corresponding to the third target label information from a training sample set includes: randomly determining the third target label information in response to receiving a fourth target training sample from a client; and determining the third target training sample corresponding to the third target label information from the training sample set based on the third target label information.

[0146] According to embodiments of this disclosure, when the tag information of the second real client model is unknown, the third target tag information is randomly determined. Under the constraints of the server, the function of the second real client model can also be obtained.

[0147] According to embodiments of this disclosure, adjusting the model parameters of the second pseudo-client model based on the second output information and the third target label information includes: inputting the second output information and the third target label information into a second loss function to obtain a second loss value; obtaining a third gradient and a fourth gradient based on the second loss value; inputting the third gradient into the first pseudo-client module through the second server model to obtain a fifth gradient; adjusting the model parameters of the second pseudo-client module based on the fourth gradient; and adjusting the model parameters of the first pseudo-client module based on the fifth gradient.

[0148] According to embodiments of this disclosure, second output information and third target label information are input into a second loss function, and forward propagation is performed to obtain a second loss value. Based on the second loss value, backpropagation is performed to obtain a third gradient and a fourth gradient, wherein the third gradient may include the gradient of the output of the second pseudo-intermediate layer, and the fourth gradient may include the gradient of the pseudo-top model. The third gradient is input into the pseudo-bottom model through the second server model, and backpropagation is performed to obtain a fifth gradient, wherein the fifth gradient may include the gradient of the pseudo-bottom model.

[0149] According to embodiments of this disclosure, the model parameters of the pseudo-bottom model are adjusted based on the gradient of the pseudo-bottom model. Similarly, the model parameters of the pseudo-top model are adjusted based on the gradient of the pseudo-top model.

[0150] According to embodiments of this disclosure, the gradients of the pseudo-top model and the pseudo-bottom model are obtained based on the second output information and the third target label information. The model parameters of the pseudo-top model and the pseudo-bottom model are adjusted according to the gradients of the pseudo-top model and the pseudo-bottom model, respectively, thereby enabling the training of the second pseudo-client model.

[0151] Figure 5 The illustration shows a schematic diagram of the second pseudo-client model function acquisition and tag inference according to an embodiment of the present disclosure.

[0152] like Figure 5 As shown, in the case where the label information of the second real client model is unknown, the U-shaped split learning model is used as an example. This can represent the first pseudo-client module. This can characterize the second pseudo-client module. This can represent the first real client module. It can represent the second real client module. The first pseudo-client module can represent the pseudo-bottom model, and the second pseudo-client module can represent the pseudo-top model. The first real client module can represent the bottom model, and the second real client module can represent the top model. It can characterize the pseudo-top model after training. It can characterize the trained second server model. It can characterize the bottom model after training. X server The training samples, X, can represent the third objective. priv The raw data can characterize the bottom model, and the raw data can characterize the privacy data of the second real client model. server It can represent the third target label information, y priv It can represent the fourth target label information, where y server ≠y priv X priv Pred can represent the original input of the label inference process, while pred can represent the output information of the label inference process.

[0153] According to embodiments of this disclosure, the second pseudo-client model and the second server model are deployed on the server, and the second real client model is deployed on the client.

[0154] According to an embodiment of this disclosure, before training the second pseudo-client model, the second pseudo-client model is randomly initialized, and a time period is set as the training time for the second pseudo-client model.

[0155] According to an embodiment of this disclosure, the bottom model performs forward propagation based on its own raw data to obtain a fourth target training sample and fourth target label information, wherein the fourth target label information is the label information corresponding to the fourth target training sample, and the fourth target training sample may include the intermediate layer output information of the bottom model.

[0156] According to embodiments of this disclosure, when the label information of the second real client model is unknown, the intermediate layer output information is sent to the server. The real client sends the intermediate layer output obtained through privacy data to the server, but does not directly send the privacy data to the server, thus protecting data privacy and security. The server randomly determines the label information of the third target, selects training samples for the third target based on the third target label information, and inputs the third target label information into the pseudo-top model and the third target training samples into the pseudo-bottom model.

[0157] According to embodiments of this disclosure, a third target training sample is input into a pseudo-bottom model, forward propagation is performed, and the output information of a second pseudo-intermediate layer is obtained. The output information of the second pseudo-intermediate layer and the intermediate layer output information are then input into a second server model.

[0158] According to embodiments of this disclosure, the second server model obtains the third pseudo-intermediate layer output information to be sent by the second pseudo-intermediate layer based on the received second pseudo-intermediate layer output information. The third pseudo-intermediate layer output information is input into the pseudo-top model to obtain the second output information. The second output information and the third target label information are then input into the second loss function, and forward propagation is performed to obtain the second loss value. Based on the second loss value, backpropagation is performed to obtain the gradient of the second pseudo-intermediate layer output and the gradient of the pseudo-top model. Based on the gradient of the pseudo-top model, the model parameters of the pseudo-top model are adjusted to achieve training of the pseudo-top model.

[0159] According to an embodiment of this disclosure, the gradient output by the second pseudo intermediate layer is input into the pseudo bottom model through the second server model, backpropagation is performed to obtain the gradient of the pseudo bottom model, and the model parameters of the pseudo bottom model are adjusted according to the gradient of the pseudo bottom model to achieve training of the pseudo bottom model.

[0160] According to embodiments of this disclosure, the received intermediate layer output information is input into the top model via a second server model. Forward propagation is performed to obtain the loss value of the bottom model. Based on the loss value, backpropagation is performed to obtain the gradient of the top model and the gradient of the intermediate layer output. The model parameters of the top model are adjusted based on the gradient of the top model.

[0161] According to an embodiment of this disclosure, the gradient output by the intermediate layer is sent to the second server model. Backpropagation is performed based on the gradient output by the intermediate layer to obtain the gradient of the second server model. The model parameters of the second server model are adjusted based on the gradient of the second server model.

[0162] According to an embodiment of this disclosure, the gradient output by the intermediate layer is sent to the bottom model through the second server model. Backpropagation is performed based on the gradient output by the intermediate layer to obtain the gradient of the bottom model. The model parameters of the bottom model are adjusted based on the gradient of the bottom model.

[0163] According to embodiments of this disclosure, when the set time for training the second pseudo-client model is reached, model training ends, and the acquisition of the functions of the second real client model is completed. Once the second pseudo-client model has completed acquiring the functions of the second real client model, the server can independently perform inference tasks based on the second server model and the second pseudo-client model, without the participation of the first real client model. Simultaneously, label information can be inferred based on the bottom model, the second server model, and the pseudo-top model.

[0164] Figure 6 A flowchart illustrating a training method for a reconstructed model according to an embodiment of the present disclosure is shown schematically.

[0165] like Figure 6As shown, the method 600 includes operations S610 to S630.

[0166] When operating S610, the training sample of the fifth target is input into the pseudo-client model to obtain the output information of the fourth pseudo intermediate layer.

[0167] According to embodiments of this disclosure, the pseudo-client model may include a pseudo-bottom model in the first pseudo-client model and the second pseudo-client model. Both the first pseudo-client model and the pseudo-bottom model in the second pseudo-client model are models trained as described above, capable of obtaining the mapping relationship between the original input data and the feature space output by the intermediate layer.

[0168] According to embodiments of this disclosure, the fifth target training sample is obtained by the server from a training sample set on the server. The fifth target training sample may include image samples.

[0169] According to embodiments of this disclosure, using the fifth target training sample as the original input data of the pseudo-client model, the fourth pseudo intermediate layer output information of the image sample can be obtained.

[0170] During the operation of S620, the output information of the fourth pseudo intermediate layer is input into the fifth deep learning model to obtain the reconstructed sample.

[0171] According to embodiments of this disclosure, a fifth deep learning model may include a reconstructed model, which may include an inverse model, and the inverse model may include transposed convolutions and upsampling layers.

[0172] In operation S630, the model parameters of the fifth deep learning model are adjusted based on the fifth target training samples and reconstructed samples to obtain the reconstructed model.

[0173] According to embodiments of this disclosure, image samples are input into a pseudo-client model to obtain the fourth pseudo-intermediate layer output information of the image samples, which can transform high-resolution original input data into low-resolution fourth pseudo-intermediate layer output information. The fourth pseudo-intermediate layer output information is input into an inverse model to obtain reconstructed samples. The model parameters of the inverse model are adjusted so that the obtained reconstructed sample values ​​are equal to the fifth target training sample values, thus obtaining a reconstructed model.

[0174] According to embodiments of this disclosure, a reconstructed model is obtained based on the fifth target training samples and the reconstructed samples, which can be used to map the intermediate layer output back to the original input data.

[0175] Figure 7 A flowchart illustrating a data reconstruction method according to an embodiment of the present disclosure is shown schematically.

[0176] like Figure 7 As shown, the method 700 includes operations S710 to S730.

[0177] When operating the S710, the target data is input into the reconstruction model to obtain the original reconstruction data.

[0178] According to embodiments of this disclosure, the target data may include the actual intermediate layer output, and the reconstructed model is the model obtained through the above training.

[0179] When operating the S720, the original reconstruction data is input into the pseudo-client model to obtain intermediate reconstruction data.

[0180] According to embodiments of this disclosure, the pseudo-client model may include a pseudo-bottom model in the first pseudo-client model and the second pseudo-client model. Both the first pseudo-client model and the pseudo-bottom model in the second pseudo-client model are models trained as described above, capable of obtaining the mapping relationship between the original input data and the feature space output by the intermediate layer.

[0181] In operation S730, if the difference between the target data and the intermediate reconstructed data meets the preset conditions, the original reconstructed data is determined as the target reconstructed data.

[0182] According to embodiments of this disclosure, the target reconstructed data can characterize the original input data of the real client model, and the original input data can characterize the privacy data of the real client model.

[0183] According to embodiments of this disclosure, based on the reconstruction model and the pseudo-client model, when the difference between the target data and the intermediate reconstruction data meets a preset condition, the obtained original reconstruction data is determined as the target reconstruction data, which enables the reconstruction of privacy data of the real client model.

[0184] According to embodiments of this disclosure, the above data reconstruction method further includes: when it is determined that the difference between the target data and the intermediate reconstructed data does not meet a preset condition, adjusting the original reconstructed data to obtain new intermediate reconstructed data, until the difference between the target data and the new intermediate reconstructed data meets the preset condition.

[0185] According to embodiments of this disclosure, based on the reconstruction model and the pseudo-client model, when it is determined that the difference between the target data and the intermediate reconstruction data does not meet the preset conditions, the obtained original reconstruction data is adjusted, thereby enabling the reconstruction of privacy data of the real client model.

[0186] Figure 8 A schematic diagram illustrating privacy data reconstruction according to an embodiment of this disclosure is shown.

[0187] like Figure 8 As shown, f -1 It can represent the reconstructed model, X server It can represent the training samples and reconstructed samples of the fifth objective. X can characterize the trained pseudo-client model. smashed It can represent the target data, and the target data can represent the actual intermediate layer output, X rec1 It can characterize the original reconstructed data.

[0188] According to embodiments of this disclosure, a pseudo-client model and an inverse model can be used to train a reconstruction model. The reconstruction model can be used to achieve coarse-grained reconstruction of the original data, and then the pseudo-client model can be used to adjust the original reconstructed data to obtain fine-grained original data.

[0189] Figure 9 The illustration shows a schematic diagram comparing the results of a label alignment experiment according to an embodiment of the present disclosure.

[0190] like Figure 9 As shown, the horizontal axis represents the number of samples in each class, and the vertical axis represents the gap inaccuracy. In each comparison, the left side represents the random selection strategy, and the right side represents the label alignment strategy.

[0191] According to embodiments of this disclosure, in a handwritten digit classification task, with 25 samples per class in the server dataset, a random selection strategy (i.e., training without label alignment) results in an inference accuracy gap of 1.95% between the pseudo-client model and the real client model; while the gap in inference accuracy between the pseudo-client model and the real client model trained with label alignment is reduced to 1.09%. Experimental results show that label alignment can significantly improve the accuracy of the pseudo-client model.

[0192] Figure 10 The diagram illustrates a comparison of delayed start experimental results according to an embodiment of the present disclosure.

[0193] like Figure 10As shown, using the left-hand axis as the reference, the curves from top to bottom represent, respectively, the loss (Loss) of splitting the learning main line in the first training epoch, the test accuracy (Text accuracy) of splitting the learning main line in the first training epoch, and the final test accuracy (Accuracy) of the pseudo-client model starting training from a certain iteration. The lower horizontal axis represents the initial iteration number (Index of the start batch) of the pseudo-client model. The PCAT (Pseudoclientmodel attack algorithm, the pseudo-client model's acquisition of the functionality of the real client model) Acc label represents the test accuracy (Text accuracy) curve of splitting the learning main line in the first training epoch, and the Loss label represents the loss (Loss) of splitting the learning main line in the first training epoch.

[0194] According to embodiments of this disclosure, experiments were conducted on the MNIST (Mixed National Institute of Standards and Technology database, handwritten digit recognition) dataset, using a LeNet-5 (convolutional neural network for handwritten character recognition) model, with one sample per class on the server. When the pseudo-client model training started from the 100th iteration, the inference accuracy improved from 89.57% to 91.05%. However, starting too late is not advisable; otherwise, the server model and the real client model will have already begun to converge, and the pseudo-client model will lose some bootstrapping information. The server should start training the pseudo-client model when the test accuracy of the real client model begins to rise and the loss begins to decrease. Experimental results indicate that an appropriate delayed start can improve the performance of the pseudo-model.

[0195] Figure 11 A schematic diagram illustrating a model splitting strategy according to an embodiment of the present disclosure is shown.

[0196] According to embodiments of this disclosure, the acquisition of the functionality of a real client model by a pseudo-client model is achieved under three common machine learning tasks.

[0197] (1) MNIST Dataset and LeNet-5 Model: The MNIST dataset contains 70,000 handwritten digits, including 60,000 training samples and 10,000 test samples, each of which is a 28×28 grayscale image. The handwritten digit classification task was trained on MNIST using the LeNet-5 model, which includes approximately 61.5K parameters.

[0198] (2) CIFAR-10 (Computer Vision Dataset for Universal Object Recognition) Dataset and VGG16 (Visual Geometry Group Network) Model: The CIFAR-10 dataset includes 10 classes and 60,000 color images with a resolution of 32×32, of which 50,000 are training samples and 10,000 are test samples. A color image classification task was trained on CIFAR-10 using VGG16 with 14.6M parameters.

[0199] (3) Tiny ImageNet (image classification dataset) and MobileNet (separable convolutional) model: The Tiny ImageNet dataset is a subset of ImageNet, containing 200 classes and 120,000 samples, including 100,000 training samples, 10,000 validation samples, and 10,000 test samples. Image classification tasks were trained on Tiny ImageNet using MobileNet with 28.5M parameters.

[0200] According to embodiments of this disclosure, for two-way split learning, the model is divided into a server model and a client model. Due to limited computing resources on the client side, more layers are allocated to the server. For U-shaped split learning, the client's bottom and top models each have one or two layers, while the middle layers are allocated to the server.

[0201] According to embodiments of this disclosure, during data processing, the training set of each public dataset is randomly divided into public datasets X. pub and privacy dataset X priv , making X pub ∶X priv =1∶9. X priv The data is allocated to the real client model as a privacy-preserving training set for split learning. pub It is a public dataset, and the server can access it from the public dataset X. pub Or privacy dataset X priv To obtain some samples to form the training sample set X server Unless otherwise specified, X is set. server and X priv Independent and identically distributed (i.i.d.). The effectiveness of PCAT on non-independent and identically distributed (non-i.i.d.) datasets will be analyzed. During the testing phase, the original test set will be used to evaluate the accuracy of the pseudo-client model.

[0202] like Figure 11 As shown, according to an embodiment of this disclosure, in X server and X privThey are independent and identically distributed (i.e., i.i.d.) and In this scenario, the pseudo-client model acquires the functionality of the real client model. To make the real client model more complex, a model splitting strategy of split layer=2 is adopted. For the MNIST, CIFAR-10, and Tiny ImageNet datasets, the server has 5, 250, and 10 training samples for each class, respectively.

[0203] Figure 12 The diagram illustrates the accuracy and loss convergence curves of a PCAT pseudo-client model according to an embodiment of the present disclosure.

[0204] like Figure 12 As shown, taking the right side of the coordinate axis as the reference, the curves in the three graphs from top to bottom can be represented as the split learning main line Acc, PCAT Acc, Vanilla-PCAT Acc, independent training Acc, Vanilla-PCAT MSE, and PCATMSE, respectively. Iter can represent one iteration, and one Epoch can represent iterating through the entire dataset. One Epoch can include 100 Iters. The MNIST dataset converges relatively quickly, requiring only a few Epochs, so Iter is chosen.

[0205] According to embodiments of this disclosure, independent training refers to the server using only its own data X. server Train a complete model. Decomposing the learning thread refers to... priv The real client model and server model are trained on it. MSELoss can represent the distance between the real feature space and the pseudo feature space.

[0206] According to embodiments of this disclosure, Vanilla-PCAT (ordinary PCAT) can refer to fixing the final parameters of the server model, randomly initializing the pseudo-client model, and using X... server Training is performed. The difference between Vanilla-PCAT and PCAT is that PCAT uses intermediate results from a series of server models, while Vanilla-PCAT is used to compare the performance of PCAT.

[0207] According to embodiments of this disclosure, for MNIST, CIFAR-10, and TinyImageNet, the server's dataset has 5, 250, and 10 training samples per class, respectively. The splitting strategy for all models is splitlayer=2.

[0208] According to embodiments of this disclosure, when the server uses only its own data X serverWhen training a complete model independently, the accuracy on the three datasets is only 63.38%, 74.16%, and 13.62%, respectively. Using the same server dataset X... server In these cases, PCAT improved inference accuracy to 96.89%, 89.48%, and 73.46%, very close to the performance of the target client model. The experimental results show that PCAT can effectively capture the functionality of the client model.

[0209] Figure 13(a) schematically illustrates the reconstruction results of input data on the MNIST dataset according to an embodiment of the present disclosure.

[0210] As shown in Figure 13(a), the server uses data from X. priv The reconstruction results were obtained by using 5 samples per class for split layer=1 and 10 samples per class for split layer=2.

[0211] Figure 13(b) schematically illustrates the reconstruction results of input data on the CIFAR-10 dataset according to an embodiment of the present disclosure.

[0212] As shown in Figure 13(b), the server uses data from x. priv The reconstruction results were obtained by using 50 samples per class for case split layer=1 and 250 samples per class for case split layer=2.

[0213] Figure 13(c) schematically illustrates the reconstruction results of input data on the Tiny ImageNet dataset according to an embodiment of the present disclosure.

[0214] As shown in Figure 13(c), the server uses data from X. priv The reconstruction results were obtained by using 5 samples per class for case split layer=1 and 10 samples per class for case split layer=2.

[0215] According to embodiments of this disclosure, based on Figures 13(a) to 13(b) It is evident that after acquiring the functionality of the real client model, the server can reconstruct the privacy data of the real client. Experimental results show that the images reconstructed by PCAT are clear, even under complex conditions involving the real client model and the learning task, such as a real client model on CIFAR-10 and a VGG16 model with two layers; it can still reconstruct input images with fairly good clarity.

[0216] According to embodiments of this disclosure, label inference is performed using U-shaped splitting learning. As shown in Table 1, PCAT achieves high accuracy in label inference. On the MNIST dataset, the label inference accuracy is 94.42% when there are only 2 samples per class on the server, and 98.23% when there are 25 samples per class on the server.

[0217] Table 1

[0218]

[0219] Based on the training method and data reconstruction method of the above model, this disclosure also provides a training device for a first pseudo-client model. The following will combine... Figure 14 The device is described in detail.

[0220] Figure 14 A schematic block diagram of a training apparatus for a first pseudo-client model according to an embodiment of the present disclosure is shown.

[0221] like Figure 14 As shown, the training device 1400 for the first pseudo-client model in this embodiment includes a determination module 1410, a first acquisition module 1420, a second acquisition module 1430, and an adjustment module 1440.

[0222] The determining module 1410 is used to determine a first target training sample corresponding to the first target label information from the training sample set. The first target label information matches second target label information, which is the label information corresponding to the second target training sample. The second target training sample is used to train a first real client model in a second deep learning model. The second deep learning model also includes the first server model, and the first real client model is deployed on a client. In one embodiment, the determining module 1410 can be used to perform the operation S210 described above, which will not be repeated here.

[0223] The first acquisition module 1420 is used to input the first target training sample into the first pseudo-client model to obtain the output information of the first pseudo intermediate layer. In one embodiment, the first acquisition module 1420 can be used to perform the operation S220 described above, which will not be repeated here.

[0224] The second obtaining module 1430 is used to input the first pseudo intermediate layer output information into the first server model to obtain the first output information. In one embodiment, the second obtaining module 1430 can be used to perform the operation S230 described above, which will not be repeated here.

[0225] The adjustment module 1440 is used to adjust the model parameters of the first pseudo-client model according to the first output information and the first target label information. In one embodiment, the adjustment module 1440 can be used to perform the operation S240 described above, which will not be repeated here.

[0226] According to embodiments of this disclosure, the determining module 1410 includes a determining unit.

[0227] The determining unit is configured to, in response to receiving second target label information from the client, determine a first target training sample from the training sample set that corresponds to the first target label information, based on first target label information that is consistent with the second target label information.

[0228] According to embodiments of this disclosure, the adjustment module 1440 includes a first obtaining unit, a second obtaining unit, a third obtaining unit, and an adjustment unit.

[0229] The first acquisition unit is used to input the first output information and the first target label information into the first loss function to obtain the first loss value.

[0230] The second acquisition unit is used to obtain the first gradient based on the first loss value.

[0231] The third acquisition unit is used to input the first gradient into the first pseudo-client model to obtain the second gradient.

[0232] The adjustment unit is used to adjust the model parameters of the first pseudo-client model according to the second gradient.

[0233] According to embodiments of this disclosure, any plurality of modules among the determining module 1410, the first obtaining module 1420, the second obtaining module 1430, and the adjusting module 1440 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the determining module 1410, the first obtaining module 1420, the second obtaining module 1430, and the adjusting module 1440 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these methods. Alternatively, at least one of the determining module 1410, the first obtaining module 1420, the second obtaining module 1430, and the adjusting module 1440 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0234] Based on the training method and data reconstruction method of the above model, this disclosure also provides a training device for a second pseudo-client model. The following will combine... Figure 15 The device is described in detail.

[0235] Figure 15 A schematic block diagram of a training apparatus for a second pseudo-client model according to an embodiment of the present disclosure is shown.

[0236] like Figure 15 As shown, the training device 1500 for the second pseudo-client model in this embodiment includes a determination module 1510, a first acquisition module 1520, a second acquisition module 1530, a third acquisition module 1540, and an adjustment module 1550.

[0237] The determining module 1510 is used to determine a third target training sample corresponding to the third target label information from the training sample set. The third target label information corresponds to a fourth target label information, which is the label information corresponding to the fourth target training sample. The fourth target training sample is used to train the second real client model in the fourth deep learning model. The second real client model includes a first real client module and a second real client module. The fourth deep learning model also includes a second server model, which is deployed on a client. In one embodiment, the determining module 1510 can be used to perform the operation S410 described above, which will not be repeated here.

[0238] The first acquisition module 1520 is used to input the third target training sample into the first pseudo-client module to obtain the output information of the second pseudo intermediate layer. In one embodiment, the first acquisition module 1520 can be used to perform the operation S420 described above, which will not be repeated here.

[0239] The second obtaining module 1530 is used to input the second pseudo intermediate layer output information into the second server model to obtain the third pseudo intermediate layer output information. In one embodiment, the second obtaining module 1530 can be used to perform the operation S430 described above, which will not be repeated here.

[0240] The third obtaining module 1540 is used to input the third pseudo intermediate layer output information into the second pseudo client module to obtain the second output information. In one embodiment, the third obtaining module 1540 can be used to perform the operation S440 described above, which will not be repeated here.

[0241] The adjustment module 1550 is used to adjust the model parameters of the second pseudo-client model based on the second output information and the third target label information. In one embodiment, the adjustment module 1550 can be used to perform the operation S450 described above, which will not be repeated here.

[0242] According to embodiments of this disclosure, the determining module 1510 includes a first determining unit and a second determining unit.

[0243] The first determining unit is used to randomly determine the label information of the third target in response to receiving the fourth target training sample from the client.

[0244] The second determination is used to determine the third target training sample corresponding to the third target label information from the training sample set based on the third target label information.

[0245] According to embodiments of this disclosure, the adjustment module 1550 includes a first obtaining unit, a second obtaining unit, a third obtaining unit, a first adjusting unit, and a second adjusting unit.

[0246] The first acquisition unit is used to input the second output information and the third target label information into the second loss function to obtain the second loss value.

[0247] The second acquisition unit is used to obtain the third and fourth gradients based on the second loss value.

[0248] The third obtaining unit is used to input the third gradient into the first pseudo-client module through the second server model to obtain the fifth gradient.

[0249] The first adjustment unit is used to adjust the model parameters of the second pseudo-client module according to the fourth gradient.

[0250] The second adjustment unit is used to adjust the model parameters of the first pseudo-client module according to the fifth gradient.

[0251] According to embodiments of this disclosure, any plurality of modules among the determining module 1510, the first obtaining module 1520, the second obtaining module 1530, the third obtaining module 1540, and the adjusting module 1550 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the determining module 1510, the first obtaining module 1520, the second obtaining module 1530, the third obtaining module 1540, and the adjusting module 1550 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these methods. Alternatively, at least one of the determining module 1510, the first obtaining module 1520, the second obtaining module 1530, the third obtaining module 1540, and the adjusting module 1550 may be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0252] Based on the training method and data reconstruction method of the above model, this disclosure also provides a training device for reconstructing the model. The following will combine... Figure 16 The device is described in detail.

[0253] Figure 16 A schematic block diagram of a reconstruction model training apparatus according to an embodiment of the present disclosure is shown.

[0254] like Figure 16 As shown, the reconstruction model training device 1600 of this embodiment includes a first acquisition module 1610, a second acquisition module 1620, and an adjustment module 1630.

[0255] The first acquisition module 1610 is used to input the fifth target training sample into the pseudo-client model to obtain the fourth pseudo intermediate layer output information. In one embodiment, the first acquisition module 1610 can be used to perform the operation S610 described above, which will not be repeated here.

[0256] The second obtaining module 1620 is used to input the output information of the fourth pseudo intermediate layer into the fifth deep learning model to obtain reconstructed samples. In one embodiment, the second obtaining module 16200 can be used to perform the operation S620 described above, which will not be repeated here.

[0257] The adjustment module 1630 is used to adjust the model parameters of the fifth deep learning model based on the fifth training sample and the reconstructed sample to obtain the reconstructed model. In one embodiment, the adjustment module 1630 can be used to perform the operation S630 described above, which will not be repeated here.

[0258] According to embodiments of this disclosure, any plurality of modules among the first obtaining module 1610, the second obtaining module 1620, and the adjustment module 1630 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the first obtaining module 1610, the second obtaining module 1620, and the adjustment module 1630 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these. Alternatively, at least one of the first obtaining module 1610, the second obtaining module 1620, and the adjustment module 1630 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0259] Based on the training method and data reconstruction method of the above model, this disclosure also provides a data reconstruction apparatus. The following will combine... Figure 17 The device is described in detail.

[0260] Figure 17 A schematic block diagram of a data reconstruction apparatus according to an embodiment of the present disclosure is shown.

[0261] like Figure 17 As shown, the data reconstruction apparatus 1700 of this embodiment includes a first acquisition module 1710, a second acquisition module 1720, and a determination module 1730.

[0262] The first acquisition module 1710 is used to input the target data into the reconstruction model to obtain the original reconstruction data. In one embodiment, the first acquisition module 1710 can be used to perform the operation S710 described above, which will not be repeated here.

[0263] The second obtaining module 1720 is used to input the original reconstructed data into the pseudo-client model to obtain intermediate reconstructed data. In one embodiment, the second obtaining module 1720 can be used to perform the operation S720 described above, which will not be repeated here.

[0264] The determining module 1730 is used to determine the original reconstructed data as the target reconstructed data when the difference between the target data and the intermediate reconstructed data meets a preset condition. In one embodiment, the determining module 1730 can be used to perform the operation S730 described above, which will not be repeated here.

[0265] According to embodiments of this disclosure, the data reconstruction apparatus 1700 further includes an adjustment module.

[0266] The adjustment module is used to adjust the original reconstructed data to obtain new intermediate reconstructed data when the difference between the target data and the intermediate reconstructed data does not meet the preset conditions, until the difference between the target data and the new intermediate reconstructed data meets the preset conditions.

[0267] According to embodiments of this disclosure, any plurality of modules among the first obtaining module 1710, the second obtaining module 1720, and the determining module 1730 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules may be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the first obtaining module 1710, the second obtaining module 1720, and the determining module 1730 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented by any other reasonable means of integrating or packaging circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these. Alternatively, at least one of the first obtaining module 1710, the second obtaining module 1720, and the determining module 1730 may be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0268] Figure 18 A block diagram schematically illustrates an electronic device suitable for implementing a model training method and a data reconstruction method according to embodiments of the present disclosure.

[0269] like Figure 18As shown, an electronic device 1800 according to an embodiment of the present disclosure includes a processor 1801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1802 or a program loaded from a storage portion 1808 into a random access memory (RAM) 1803. The processor 1801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1801 may also include onboard memory for caching purposes. The processor 1801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0270] RAM 1803 stores various programs and data required for the operation of electronic device 1800. Processor 1801, ROM 1802, and RAM 1803 are interconnected via bus 1804. Processor 1801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 1802 and / or RAM 1803. It should be noted that the programs may also be stored in one or more memories other than ROM 1802 and RAM 1803. Processor 1801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0271] According to embodiments of this disclosure, the electronic device 1800 may further include an input / output (I / O) interface 1805, which is also connected to a bus 1804. The electronic device 1800 may also include one or more of the following components connected to the I / O interface 1805: an input section 1806 including a keyboard, mouse, etc.; an output section 1807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1808 including a hard disk, etc.; and a communication section 1809 including a network interface card such as a LAN card, modem, etc. The communication section 1809 performs communication processing via a network such as the Internet. A drive 1810 is also connected to the I / O interface 1805 as needed. A removable medium 1811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1810 as needed so that computer programs read from it can be installed into the storage section 1808 as needed.

[0272] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0273] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1802 and / or RAM 1803 and / or one or more memories other than ROM 1802 and RAM 1803 described above.

[0274] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the item recommendation method provided in the embodiments of this disclosure.

[0275] When the computer program is executed by the processor 1801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0276] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 1809, and / or installed from a removable medium 1811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0277] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1809, and / or installed from the removable medium 1811. When the computer program is executed by the processor 1801, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0278] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0279] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0280] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0281] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A data reconstruction method, comprising: inputting target data into a reconstruction model to obtain original reconstructed data; wherein the reconstruction model is trained by inputting a fifth target training sample into a pseudo-client model to obtain fourth pseudo-intermediate layer output information, inputting the fourth pseudo-intermediate layer output information into a fifth deep learning model to obtain a reconstructed sample, adjusting model parameters of the fifth deep learning model according to the fifth target training sample and the reconstructed sample, and obtaining the reconstruction model; inputting the original reconstructed data into a pseudo-client model to obtain intermediate reconstructed data; in a case where a difference between the target data and the intermediate reconstructed data meets a preset condition, determining the original reconstructed data as target reconstructed data; wherein the pseudo-client model comprises one of a first pseudo-client model and a second pseudo-client model; wherein a first deep learning model comprises a first pseudo-client model and a first server model, the first deep learning model is deployed on a server, the first pseudo-client model is trained by determining a first target training sample corresponding to first target label information from a training sample set, wherein the first target label information matches second target label information, the second target label information is label information corresponding to a second target training sample, the second target training sample is used to train a first real client model in a second deep learning model, the second deep learning model further comprises the first server model, the first real client model is deployed on a client, inputting the first target training sample into the first pseudo-client model to obtain first pseudo-intermediate layer output information, inputting the first pseudo-intermediate layer output information into the first server model to obtain first output information, and adjusting model parameters of the first pseudo-client model according to the first output information and the first target label information. The third deep learning model comprises a second pseudo-client model and a second server model, the second pseudo-client model comprises a first pseudo-client module and a second pseudo-client module, and the third deep learning model is deployed on a server; the second pseudo-client model is obtained by training through the following operations: determining a third target training sample corresponding to third target label information from a training sample set, wherein the third target label information corresponds to fourth target label information, the fourth target label information is label information corresponding to a fourth target training sample, the fourth target training sample is used for training a second real client model in a fourth deep learning model, the second real client model comprises a first real client module and a second real client module, the fourth deep learning model further comprises the second server model, and the second real client model is deployed on a client; inputting the third target training sample into the first pseudo-client module to obtain second pseudo-intermediate layer output information; inputting the second pseudo-intermediate layer output information into the second server model to obtain third pseudo-intermediate layer output information; inputting the third pseudo-intermediate layer output information into the second pseudo-client module to obtain second output information; and adjusting model parameters of the second pseudo-client model according to the second output information and the third target label information.

2. The method of claim 1, further comprising: in a case where the difference between the target data and the intermediate reconstruction data does not satisfy the preset condition, adjusting the original reconstruction data to obtain new intermediate reconstruction data, until the difference between the target data and the new intermediate reconstruction data satisfies the preset condition.

3. The method of claim 1, wherein, The first target training sample corresponding to the first target label information is determined from the training sample set according to the first target label information consistent with the second target label information in response to receiving the second target label information from the client. The model parameters of the first pseudo-client model are adjusted according to the first output information and the first target label information, comprising:

4. The method of claim 1, wherein, inputting the first output information and the first target label information into a first loss function to obtain a first loss value; obtaining a first gradient according to the first loss value; inputting the first gradient into the first pseudo-client model to obtain a second gradient; adjusting the model parameters of the first pseudo-client model according to the second gradient. The third target training sample corresponding to the third target label information is determined from the training sample set, comprising:

5. The method of claim 1, wherein, in response to receiving the fourth target training sample from the client, randomly determining the third target label information; and determining the third target training sample corresponding to the third target label information from the training sample set according to the third target label information. The model parameters of the second pseudo-client model are adjusted according to the second output information and the third target label information, comprising:

6. The method of claim 1 or 5, wherein, ​ inputting the second output information and the third target label information into a second loss function to obtain a second loss value; obtaining a third gradient and a fourth gradient according to the second loss value; inputting the third gradient into the first pseudo-client module through the second server model to obtain a fifth gradient; adjusting model parameters of the second pseudo-client module according to the fourth gradient; and adjusting model parameters of the first pseudo-client module according to the fifth gradient.

7. An electronic device, comprising: one or more processors; storage for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, enable the one or more processors to perform the method according to any one of claims 1-6.

Images