Data processing method and apparatus, device, and storage medium

Abstract

Provided are a data processing method and apparatus, a device, and a storage medium. The method comprises: receiving a first neural network model sent by each of N participants; aggregating the first neural network model sent by each participant to obtain a second neural network model; embedding identity information into the second neural network model to obtain a third neural network model, wherein the identity information comprises first identity information of the participants and second identity information of a server to which the second neural network model belongs; and sending the third neural network model to each participant.

Description

Data processing method, device, equipment and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202310946123.3, filed on July 28, 2023, entitled “Data processing method, device, equipment and storage medium,” and the entire contents of that application are incorporated herein by reference. Technical Field

[0003] The present application belongs to the field of data processing technology, and in particular relates to a data processing method, apparatus, device and storage medium. Background Art

[0004] Federated machine learning is a distributed machine learning approach that allows multiple participants (such as computing devices or nodes) to jointly train a shared global model while maintaining data privacy. Each participant trains the model locally. The trained local model data is then sent to the owner of the global model for aggregation to create the global model. The global model data is then distributed back to each participant for further training.

[0005] However, if a participant maliciously leaks the data of the local model it has trained, the current owner of the global model will have no way to prove that the stolen model data belongs to them, nor will they be able to determine which participant leaked the data. This will cause losses to the owner of the global model.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a data processing method, apparatus, device, and storage medium, which can solve the problem in related technologies that some model data involved in federated machine learning cannot be traced, resulting in loss to the owner of the global model.

[0008] In a first aspect, an embodiment of the present application provides a data processing method, which may include:

[0009] Receiving a first neural network model sent by each of N participants, where N is an integer greater than 1;

[0010] Aggregating the first neural network model sent by each participant to obtain a second neural network model;

[0011] Embedding identity information into the second neural network model to obtain a third neural network model, wherein the identity information includes the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs;

[0012] A third neural network model is sent to each participant.

[0013] In a second aspect, an embodiment of the present application provides an information identification method, which may include:

[0014] receiving a target neural network model sent by a third-party computing platform, where the target neural network model includes a fourth neural network model;

[0015] extracting a second hidden layer corresponding to a pre-stored layer identifier from the fourth neural network model, where the pre-stored layer identifier is an identifier of the hidden layer in which the identity information is embedded;

[0016] Extracting target identity information from the third feature parameter of the second hidden layer;

[0017] When the target identity information matches the pre-stored identity information, a prompt message is displayed. The pre-stored identity information includes the identity information of the target participant participating in the training of the fourth neural network model and the identity information of the service provider to which the fourth neural network model belongs. The prompt information is used to prompt the third-party computing platform to steal the service provider's neural network model, and the fourth neural network model is trained by the target participant.

[0018] In a third aspect, an embodiment of the present application provides a data processing device, including:

[0019] a receiving module, configured to receive a first neural network model sent by each of N participants, where N is an integer greater than 1;

[0020] an aggregation module, configured to aggregate the first neural network model sent by each participant to obtain a second neural network model;

[0021] An embedding module, configured to embed identity information into the second neural network model to obtain a third neural network model, wherein the identity information includes the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs;

[0022] Sending model, used to send the third neural network model to each participant.

[0023] In a fourth aspect, an embodiment of the present application provides an information identification device, including:

[0024] a receiving module, configured to receive a target neural network model sent by a third-party computing platform, wherein the target neural network model includes a fourth neural network model;

[0025] an extraction module, configured to extract, from the fourth neural network model, a second hidden layer corresponding to a pre-stored layer identifier, where the pre-stored layer identifier is an identifier of the hidden layer in which the identity information is embedded;

[0026] an extraction module, configured to extract target identity information from the third feature parameter of the second hidden layer;

[0027] The display module is used to display a prompt message when the target identity information matches the pre-stored identity information. The pre-stored identity information includes the identity information of the target participant participating in the training of the fourth neural network model and the identity information of the service provider to which the fourth neural network model belongs. The prompt information is used to prompt the third-party computing platform to steal the neural network model of the service provider and that the fourth neural network model is trained by the target participant.

[0028] In a fifth aspect, an embodiment of the present application provides a computer device, the computer device comprising: a processor and a memory storing computer program instructions;

[0029] When the processor executes the computer program instructions, it implements the data processing method shown in the first aspect or the information identification method shown in the second aspect.

[0030] In a sixth aspect, an embodiment of the present application provides a computer storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the data processing method shown in the first aspect or the information identification method shown in the second aspect is implemented.

[0031] In the seventh aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the data processing method shown in the first aspect or the information identification method shown in the second aspect.

[0032] In an eighth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the data processing method shown in the first aspect or the information identification method shown in the second aspect.

[0033] The data processing method, apparatus, device and storage medium of the embodiment of the present application can receive the first neural network model sent by each of the N participants, and aggregate the first neural network model sent by each participant to obtain a second neural network model, and then embed the identity information into the second neural network model to obtain a third neural network model, wherein the identity information includes the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs, and send the third neural network model to each participant. In this way, by embedding the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs into the second neural network model after aggregation of each participant, and sending the third neural network model with embedded identity information to each participant for the next model training, when a participant leaks the data of the local model it has trained, the leaked model data can be accurately locked as the neural network model of the service provider and the participant to whom the model data was leaked through the identity information, thereby avoiding the loss of the service provider. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] FIG1 is an architecture diagram of a data processing method provided in an embodiment of the present application;

[0036] FIG2 is a flow chart of a data processing method provided in an embodiment of the present application;

[0037] FIG3 is a schematic diagram of the training principle of a data processing method provided in an embodiment of the present application;

[0038] FIG4 is a schematic diagram of a verification process of a data processing method provided in an embodiment of the present application;

[0039] FIG5 is a flow chart of an information identification method provided in an embodiment of the present application;

[0040] FIG6 is a schematic structural diagram of a data processing device provided by an embodiment of the present application;

[0041] FIG7 is a schematic diagram of the structure of an information identification device provided by an embodiment of the present application;

[0042] FIG8 is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0043] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0044] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0045] The acquisition, storage, use, and processing of data (including but not limited to the features and information herein) in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0046] Federated machine learning is a distributed machine learning approach in which the designer and / or owner of a federated machine learning global model allows multiple participants (such as computing devices or nodes) to jointly train a shared global model while maintaining data privacy. Each participant trains the model locally, and the trained local model data is sent to the owner of the global model for aggregation to obtain the global model. The global model data is then distributed back to each participant for further training.

[0047] At this time, if one or several participants sell the local model to a third party in exchange for benefits, then there are the following disadvantages for the designer and / or owner of the global model: First, the global model trained based on Bang machine learning usually contains a large number of parameters and weights. These parameters may vary significantly in different training stages and data sets. Even if two models are similar in structure, their parameters may also be different, which makes it difficult to determine the ownership of the model. As a result, the designer and / or owner of the global model has no way to prove that they own the stolen model; second, since the global model is aggregated by multiple local models, each participant has made a certain degree of contribution to the global model. Multiple contributions can easily lead to confusion in ownership. Therefore, in the process of tracking the stolen model, it becomes very complicated to determine which part belongs to other participants; third, there is a lack of a reliable traceability mechanism.

[0048] In order to solve the problems in the related art, the embodiments of the present application provide a data processing method, an information identification method, a data processing device, an information identification device, a computer device and a storage medium.

[0049] The data processing method, information identification method, apparatus, computer equipment and storage medium of the embodiments of the present application will be described in detail below with reference to Figures 1 to 8. It should be noted that these embodiments are not intended to limit the scope of disclosure of the present application.

[0050] First, the data processing architecture provided in the embodiments of the present application is described in detail.

[0051] As shown in Figure 1 , the data processing architecture 10 may include a server 10 and N participants, where N is an integer greater than 1. The server 10 may be the designer and / or owner of the global model (i.e., the second neural network model). The aforementioned modules are described in detail below.

[0052] The server 10 can interact with N participants (for example, participant 1, participant 2, ..., participant N) for data exchange. In one example, the server 10 can plan a training plan for the second neural network model, and according to the training plan, send the training data of each sub-model in the second neural network model, i.e., the first neural network model, to each of the N participants, so that each participant can obtain the first neural network model based on its training data. In another example, the server 10 can also be used to receive the first neural network model sent by each of the N participants after training based on the training data, and aggregate the N first neural network models to obtain the second neural network model, embed the identity information into the second neural network model, and obtain the third neural network model, wherein the identity information includes the first identity information of the participant and the second identity information of the service party to which the second neural network model belongs; send the third neural network model to each participant respectively, so that each participant can use the third neural network model as training data to continue the next round of training.

[0053] The participant can be used to receive the training data sent by the server 10, train the initial neural network model based on its training data, obtain a first neural network model, and send the first neural network model to the server 10. The participant can also receive a third neural network model sent by the server 10, in which the first identity information of the participant and the second identity information of the server to which the second neural network model belongs are embedded. In this way, the third neural network model can be used as training data to continue the next round of training.

[0054] Therefore, by embedding the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs in the second neural network model after aggregation of each participant, and sending the third neural network model with embedded identity information to each participant for the next model training, when a participant leaks the data of its trained local model, the leaked model data can be accurately locked as the neural network model of the service provider and the participant who leaked the model data through the identity information, thereby avoiding losses to the service provider.

[0055] In order to better illustrate the contents in the embodiments of the present application, based on the above-mentioned data processing architecture, a data processing method and an information identification method provided in the embodiments of the present application are described below in combination with Figures 2 to 8, as shown below.

[0056] FIG2 is a flow chart of a data processing method provided in an embodiment of the present application.

[0057] As shown in FIG2 , the data processing method can be applied to the server as shown in FIG1 , and the data processing method may specifically include the following steps:

[0058] Step 210: Receive the first neural network model sent by each of the N participants, where N is an integer greater than 1; Step 220: Aggregate the first neural network model sent by each participant to obtain a second neural network model; Step 230: Embed the identity information into the second neural network model to obtain a third neural network model, where the identity information includes the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs; Step 240: Send the third neural network model to each participant.

[0059] In this way, by embedding the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs in the second neural network model after aggregation of each participant, and sending the third neural network model with embedded identity information to each participant for the next model training, when a participant leaks the data of its trained local model, the leaked model data can be accurately locked as the neural network model of the service provider and the participant who leaked the model data through the identity information, thereby avoiding losses to the service provider.

[0060] The above steps are described in detail below.

[0061] First, in step 210, each participant can train an initial neural network model (such as a convolutional neural network model) based on the training data sent by the server (or the data stored locally by the participant). Assume that there are N participants, namely {1, 2, 3, ., i, .., N}, and the i-th participant is denoted as i. In this way, when the current round of federated machine learning is round t, the server can receive the first neural network model sent by the participants {1, 2, ., i, .., N}. It can be understood that the first neural network model sent by participant 1 is The first neural network model sent by participant 2 is The same logic can be applied, so I will not elaborate on it here.

[0062] In this way, by each participant using locally stored data to train the convolutional neural network model, it can be ensured that the data does not leave the participant, thereby ensuring data privacy.

[0063] Next, in step 220, the server can aggregate the first neural network models submitted by each participant, for example, Aggregated into the second neural network model, namely

[0064] Furthermore, regarding step 230, the embodiment of the present application can embed the second identity information of the service provider to which its second neural network model belongs and the first identity information of each participant into the second neural network model without affecting the performance of the second neural network model. Here, referring to Figure 1, the first identity information and second identity information of the i-th participant among the N participants can be embedded into the second neural network model to obtain a third neural network model, namely the third neural network model-i. Then, the third neural network model-i can be sent to the i-th participant, i∈[1,N]. Based on this, for example, the first identity information and second identity information of participant 1 among the N participants are embedded into the second neural network model to obtain the third neural network model-1. Then, the third neural network model-1 can be sent to participant 1. Similarly, the first identity information and second identity information of participant 2 among the N participants are embedded into the second neural network model to obtain the third neural network model-2. Then, the third neural network model-2 can be sent to participant 2. Based on this, the specific identity information embedding process is as follows.

[0065] In one or more possible embodiments, the second neural network model includes M hidden layers, where M is a positive integer. Based on this, step 230 may specifically include:

[0066] Step 2301, extracting the first hidden layer from the M hidden layers;

[0067] Step 2302: Generate first feature parameters of the first hidden layer based on the weight parameters in the first hidden layer;

[0068] Step 2303: Generate a second characteristic parameter based on the identity information and the first characteristic parameter;

[0069] Step 2304: Replace the first feature parameter with the second feature parameter to obtain a third neural network model.

[0070] Based on this, step 2302 may specifically include:

[0071] Based on the weight parameters and the number of output channels of the convolution kernel in the second neural network model, the mean of the model weights along the output channels is calculated;

[0072] The mean is determined as the first characteristic parameter.

[0073] For example, taking the second neural network model as a convolutional neural network as an example, the weight parameter of the first hidden layer is extracted from the convolutional neural network, that is, W, and W is used as the identity information to be embedded I i The carrier, denoted as W∈R K×K×M×N , where K is the size of the convolution kernel, M is the number of input channels of the convolution kernel, and N is the number of output channels of the convolution kernel, i.e., K×K×M=D. Here, in the embodiment of the present application, the bias term b in the neural network is ignored, and the first characteristic parameter, i.e., the mean of the model weight, is calculated along the output channel, which can be obtained by the following formula (1):

[0074] in, is the first characteristic parameter, which is expressed as the weight after the identity information is embedded. It contains personal information (of course, it also includes the original function of the model). It is worth noting that the first hidden layer extracted in the embodiment of the present application is itself also a key, which is only stored by the server itself.

[0075] Then, step 2303 and step 2304 can be illustrated by the following specific steps.

[0076] In the embodiment of the present application, the first identity information includes at least one of the following: the organization code of the participant, the identity certificate of at least one first user of the participant, and the first identification information configured by the service provider for the participant; the second identity information includes at least one of the following: the organization code of the service provider, the identity certificate of at least one second user of the service provider, and the second identification information configured by the service provider. Furthermore, in one example, step 2303 may specifically include:

[0077] Step 23031, projecting the first characteristic parameter using a preset key matrix to obtain a simulated parameter of the first characteristic parameter;

[0078] Step 23032, iteratively train the imitation parameter using the first representation parameter of the identity information until the target distance value between the first representation parameter and the imitation parameter is less than or equal to the preset distance value, and then determine the imitation parameter corresponding to the target distance value as the second feature parameter.

[0079] For example, a preset key matrix P is defined, P∈R L×D The elements in the key matrix P are randomly generated and independently extracted from the standard normal distribution N(0, 1). The key matrix P is a matrix with fixed values, and its value will be fixed once the random generation is completed.

[0080] Then, by presetting the key matrix, the imitation parameter Y is defined. Y∈R L×1 , Y is the same matrix as I, both have length L, I is the first representation parameter, y j is the jth element in Y and can be obtained by the following formula (2):

[0081] Among them, Y is the first feature parameter generated by the weight parameter in the first hidden layer Projection, P It becomes a one-dimensional binary string, which is Y. However, there are still differences between Y and I, so further iteration is required.

[0082] Based on this, the first representation parameter in the embodiment of the present application can be obtained in the following manner, that is, the data processing method can further include:

[0083] The first identity information and the second identity information are encoded to obtain a first representation parameter of the identity information.

[0084] For example, the first identity information is the identity certificate of the first user (such as the ID card information of the person in charge of the participant "12345678910987") and the second identity information is the second identification information configured by the service provider (such as the date of establishment of the service provider "200203") as an example for explanation. Among them, the first user can designate a security person in charge for the participant when joining the federated learning network. Once a model leakage incident occurs, the security person in charge needs to bear legal responsibility. At this time, the identity certificate of the first user and the second identification information configured by the service provider can be encoded into a binary string, and the first representation parameter of the identity information of the participant is I i,I represents Identity, which can be encoded by encoding each number into a 4-bit binary string and then combining them. The matrix length of the first representation parameter of the participant is L. For example, the prefix "200203" represents the model ownership + ID number, so the total length is 24 times 4, which is equal to 96 bits, L=96.

[0085] Furthermore, the matrix lengths of the imitation parameter and the first representation parameter are the same and the value of the matrix length is the target length value. The first representation parameter includes the jth element. Based on this, step 23032 may specifically include:

[0086] Extract the jth parameter from the first representation parameter, j∈[1,L], where L is the target length value;

[0087] According to the length of the matrix, the distance between the jth element and the jth parameter is calculated.

[0088] For example, the loss function using binary cross entropy is defined as the distance between the jth parameter and the imitation parameter It can be obtained by formula (3):

[0089] in, Indicates the first representation parameter I i The purpose of this loss function is to calculate the jth binary bit (0 or 1). The distance from Y, the purpose of training is to minimize the loss function, that is, to minimize The distance between Y and Y, train Y closer and closer to I i , when the training is completed, I i The distance from Y is minimized. yes gradient.

[0090] In summary, step 230 can be implemented as shown in FIG3 by the following operations, that is, embedding the identity information into the second neural network model:

[0091] (from Take out the weight parameters of the first hidden layer 0 means step 0)

[0092] (layer Corresponding to a Y)

[0093] (One Y corresponds to one J, by Y 0 express)

[0094] (Stop after R rounds, R is custom)

[0095] Among them, η is the learning rate, which is used to control the step size of the descent and can be customized. In this way, the original first characteristic parameter Replaced by the second characteristic parameter

[0096] Therefore, the data processing method provided by the embodiment of the present application can realize the traceability of the neural network model by embedding identity information. Among them, the weight parameters of the first hidden layer can be extracted from the second neural network model, and the first characteristic parameters of the first hidden layer can be generated. Then, the first characteristic parameters are projected through the preset key matrix to obtain a one-dimensional imitation parameter. Next, the imitation parameter of this identity information is trained to gradually approach the real information of the first hidden layer in the second neural network model. After the training is completed, the updated parameters of this layer are put back into the neural network to replace the original parameters, thereby completing the embedding of the identity information.

[0097] Then, referring to step 240, in one or more possible embodiments, the third neural network model can be sent to the corresponding participant based on the first identity information of the participant in the third neural network.

[0098] For example, referring to FIG1 , the third neural network model with identity information embedded in it is sent to the information holder. For example, the third neural network model with the identity information of participant 1 embedded in it is sent to participant 1, the third neural network model with the identity information of participant 2 is sent to participant 1, and so on. In this way, each participant can prepare for the next round of training, i.e., round t+1, based on the third neural network model.

[0099] In addition, if an external suspicious model is found, identity information can be extracted through the following steps 2501 to 1504 to prove the ownership of the model and the leaker. Based on this, after step 240, the data processing method can also include:

[0100] Step 2501: When obtaining a fourth neural network model from a third-party computing platform, obtain a pre-stored layer identifier of a first hidden layer;

[0101] Step 2502: extracting a second hidden layer corresponding to the layer identifier from the fourth neural network model;

[0102] Step 2503, extracting target identity information from the third feature parameter of the second hidden layer;

[0103] Step 2504: When the target identity information matches the identity information, a prompt message is displayed, which is used to prompt the third-party computing platform to steal the neural network model of the service provider, and the fourth neural network model is trained by the target parameter provider.

[0104] For example, the process of extracting the identity information of the participant is the process of extracting the target identity information from the third characteristic parameter of the second hidden layer of the fourth neural network model, that is, the target identity information It can be calculated by the following formula (4):

[0105] The target identity information is the identity certificate of the first user of the participant. i The value of the j-th binary bit of , the step function is expressed as follows as formula (5):

[0106] In this way, after all binary bits are extracted, the identity credentials of the first user of the participant and the identity information of the service provider can be obtained.

[0107] Furthermore, step 250 may specifically include:

[0108] Extract the i-th element from the second identification parameter of the target identity information, i∈[1,L], where L is a target length value, and the target length value is the value of the matrix length of the first representation parameter of the identity information;

[0109] Perform binary bit conversion on the i-th element to obtain the target identity information.

[0110] In summary, the data processing method, apparatus, device and storage medium of the embodiments of the present application can receive the first neural network model sent by each of the N participants, and aggregate the first neural network model sent by each participant to obtain a second neural network model, and then embed the identity information into the second neural network model to obtain a third neural network model, wherein the identity information includes the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs, and send the third neural network model to each participant. In this way, by embedding the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs into the second neural network model after aggregation of each participant, and sending the third neural network model with embedded identity information to each participant for the next model training, when a participant leaks the data of the local model it has trained, the leaked model data can be accurately locked as the neural network model of the service provider and the participant to whom the model data was leaked through the identity information, thereby avoiding the loss of the service provider.

[0111] Furthermore, by embedding identity information in the embodiment of the present application, not only can the labeling of the neural network model be guaranteed, but the actual performance of the neural network model will not be affected.

[0112] First, the data processing method used in the embodiments of this application does not directly embed the binary string of identity information into the model. Instead, it performs a two-dimensional projection of the single-layer model parameters to form a string. Specifically, the number of weight parameters in a single convolutional layer is very large, typically ranging from hundreds of thousands to millions. In this case, multiplying these parameters with a key matrix can generate new parameter values.

[0113] Secondly, the elements of the preset key matrix are drawn from a standard normal distribution, with values ​​between 0 and 1. They are then processed through the sigmoid activation function, converting all numbers to either 0 or 1. In other words, each weight parameter in the neural network model corresponds to a bit of 0 or 1. Because the elements of the preset key matrix are small, their impact on the size of each parameter is very small, typically no more than 0.001%. For example, if a model parameter is originally 100, after calculation, it may become 100.0000001.

[0114] It's worth noting that neural network models exhibit a certain degree of robustness. This means they maintain good performance even in the face of such small parameter changes. This robustness ensures that neural networks can provide reliable predictions and analysis results across diverse tasks and datasets. In short, by projecting single-layer model parameters into two-dimensional strings and performing the corresponding computational processing, effective parameter transformation and optimization can be achieved without compromising neural network performance.

[0115] The above statement can be based on the following verification process.

[0116] The data processing method of the embodiment of the present application verifies that the third neural network model will not affect the actual performance of its second neural network model. The CIFAR-10 data set can be used for testing. The CIFAR-10 data set contains 10 categories, each category has 6,000 images, of which 50,000 are used for training and 10,000 are used for testing. The image size is 32x32, with three color channels. The verification process can include (1) embedding the second convolutional layer conv2, with a total of 3×3×64×64=36864 model weight parameters, 73728 parameters for ResNet-18 and 110592 parameters for ResNet-34. (2) embedding identity information with a length of 96 bits. (3) After 16 rounds of R bit, the embedding is completed, and information is extracted from the verification model corresponding to the third neural network model. Compared with the original information in the second neural network model, the accuracy is higher than the preset threshold. (4) The test results are shown in Figure 4.

[0117] Therefore, through the above verification process, we can draw the following conclusions: embedding identity information into the weight parameters of the second neural network model has little effect on the performance of the ResNet model. In the verification results, the model performance of the weights after embedding user information is not much different from the model performance on the original data set. In addition, in ResNet structures of different depths, the model performance is basically consistent, which shows that the degree of influence of user information on ResNet models of different depths is similar, and when there are enough parameters, the influence can completely disappear. In addition, after the user information is extracted from the model weights, it does not change compared to the original information, which proves the feasibility of the data processing method provided in the embodiment of the present application.

[0118] Furthermore, based on the data processing method shown in FIG2 and based on the same inventive concept, an embodiment of the present application also provides an information identification method, which is specifically described in conjunction with FIG5 .

[0119] FIG5 is a flowchart of an information identification method provided in an embodiment of the present application.

[0120] As shown in FIG5 , the information identification method can be applied to the server, and the information identification method can specifically include the following steps:

[0121] Step 510, receiving the target neural network model sent by the third-party computing platform, the target neural network model includes the fourth neural network model; step 520, extracting the second hidden layer corresponding to the pre-stored layer identifier from the fourth neural network model, the pre-stored layer identifier is the identifier of the hidden layer embedded with identity information; step 530, extracting the target identity information from the third characteristic parameter of the second hidden layer; step 540, when the target identity information matches the pre-stored identity information, displaying a prompt message, the pre-stored identity information includes the identity information of the target participant participating in the training of the fourth neural network model and the identity information of the service provider to which the fourth neural network model belongs, the prompt message is used to prompt the third-party computing platform that the service provider's neural network model has been stolen and that the fourth neural network model is trained by the target participant.

[0122] Therefore, by embedding the identity information of each participant and the identity information of the service provider to which the second neural network model belongs in the second neural network model after aggregation of each participant, the leaked model data can be accurately identified as the neural network model of the service provider and the participant who leaked the model data, thereby avoiding losses to the service provider.

[0123] Based on the same inventive concept, the present application also provides a data processing device, which will be described in detail with reference to FIG6 .

[0124] FIG6 is a schematic diagram of the structure of a data processing device provided in one embodiment of the present application.

[0125] In some embodiments of the present application, the data processing device shown in Figure 6 can be set at the server end of the embodiment of the present application.

[0126] As shown in FIG6 , the data processing device 60 may specifically include:

[0127] A receiving module 601 is configured to receive a first neural network model sent by each of N participants, where N is an integer greater than 1;

[0128] Aggregation module 602, configured to aggregate the first neural network model sent by each participant to obtain a second neural network model;

[0129] An embedding module 603 is configured to embed identity information into the second neural network model to obtain a third neural network model, wherein the identity information includes the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs;

[0130] Sending model 604 is used to send the third neural network model to each participant.

[0131] The data processing device 60 in the embodiment of the present application is described in detail below.

[0132] In one or more optional embodiments, the data processing device 60 in the embodiment of the present application may further include a first extraction module, a generation module and a replacement module; wherein,

[0133] A first extraction module extracts a first hidden layer from the M hidden layers when the second neural network model includes M hidden layers, where M is a positive integer;

[0134] A generating module, configured to generate a first feature parameter of the first hidden layer based on a weight parameter in the first hidden layer;

[0135] The generating module is further configured to generate a second characteristic parameter based on the identity information and the first characteristic parameter;

[0136] The replacement module is used to replace the first characteristic parameter with the second characteristic parameter to obtain a third neural network model.

[0137] In another or more optional embodiments, the data processing device 60 in the embodiment of the present application may further include a projection module and a training module; wherein,

[0138] A projection module, configured to project the first characteristic parameter using a preset key matrix to obtain a simulated parameter of the first characteristic parameter;

[0139] The training module is used to iteratively train the imitation parameter using the first representation parameter of the identity information until the target distance value between the first representation parameter and the imitation parameter is less than or equal to the preset distance value, and then determine the imitation parameter corresponding to the target distance value as the second feature parameter.

[0140] In one or more optional embodiments, the data processing device 60 in the embodiment of the present application may further include a first extraction module and a first calculation module; wherein,

[0141] A first extraction module is configured to extract a j-th parameter from the first representation parameter when the matrix lengths of the imitation parameter and the first representation parameter are the same and the matrix length is the target length value, and the first representation parameter includes the j-th element, where j∈[1,L], and L is the target length value;

[0142] The first calculation module is used to calculate the distance value between the jth parameter and the imitation parameter according to the length of the matrix through the jth element and the jth parameter.

[0143] In one or more optional embodiments, the data processing device 60 in the embodiment of the present application may further include an encoding module; wherein,

[0144] The encoding module is used to encode the first identity information and the second identity information to obtain a first representation parameter of the identity information.

[0145] In one or more optional embodiments, the data processing device 60 in the embodiment of the present application may further include a second calculation module and a determination module; wherein,

[0146] A second calculation module is used to calculate the mean of the model weights along the output channels based on the weight parameters and the number of output channels of the convolution kernel in the second neural network model;

[0147] A determination module is used to determine the mean as the first characteristic parameter.

[0148] In one or more optional embodiments, the data processing device 60 in the embodiment of the present application may further include an acquisition module, a second extraction module, a second extraction module and a display module; wherein,

[0149] an acquisition module, configured to acquire a pre-stored layer identifier of a first hidden layer when acquiring a fourth neural network model from a third-party computing platform;

[0150] A second extraction module is used to extract a second hidden layer corresponding to the layer identifier from the fourth neural network model;

[0151] a second extraction module, configured to extract target identity information from the third characteristic parameter of the second hidden layer;

[0152] The display module is used to display a prompt message when the target identity information matches the identity information. The prompt message is used to prompt the third-party computing platform to steal the neural network model of the service provider, and the fourth neural network model is trained by the target parameter provider.

[0153] In one or more optional embodiments, the data processing device 60 in the embodiment of the present application may further include a third extraction module and a conversion module; wherein,

[0154] A third extraction module is used to extract the i-th element from the second identification parameter of the target identity information, i∈[1,L], where L is a target length value, and the target length value is the value of the matrix length of the first representation parameter of the identity information;

[0155] The conversion module is used to perform binary bit conversion on the i-th element to obtain the target identity information.

[0156] In one or more optional embodiments, the first identity information includes at least one of the following: an organization code of the participant, identity credentials of at least one first user of the participant, and first identification information configured by the service provider for the participant;

[0157] The second identity information includes at least one of the following: an organization code of the service provider, identity credentials of at least one second user of the service provider, and second identification information configured by the service provider.

[0158] Therefore, the data processing device in the embodiment of the present application can receive the first neural network model sent by each of the N participants, and aggregate the first neural network model sent by each participant to obtain a second neural network model, and then embed the identity information into the second neural network model to obtain a third neural network model, the identity information including the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs, and send the third neural network model to each participant. In this way, by embedding the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs into the second neural network model after aggregation of each participant, and sending the third neural network model with embedded identity information to each participant for the next model training, when a participant leaks the data of the local model it has trained, the leaked model data can be accurately locked as the neural network model of the service provider and the participant to whom the model data was leaked through the identity information, thereby avoiding the loss of the service provider.

[0159] Based on the same inventive concept, the present application also provides an information identification device, which will be described in detail with reference to FIG7 .

[0160] FIG7 is a schematic diagram of the structure of an information identification device provided in one embodiment of the present application.

[0161] In some embodiments of the present application, the information identification device shown in Figure 7 can be set at the server end of the embodiment of the present application.

[0162] As shown in FIG7 , the information identification device 70 may specifically include:

[0163] A receiving module 701 is configured to receive a target neural network model sent by a third-party computing platform, where the target neural network model includes a fourth neural network model;

[0164] An extraction module 702 is configured to extract a second hidden layer corresponding to a pre-stored layer identifier from the fourth neural network model, where the pre-stored layer identifier is an identifier of the hidden layer in which the identity information is embedded;

[0165] An extraction module 703 is used to extract target identity information from the third characteristic parameter of the second hidden layer;

[0166] Display module 704 is used to display prompt information when the target identity information matches the pre-stored identity information. The pre-stored identity information includes the identity information of the target participant participating in the training of the fourth neural network model and the identity information of the service provider to which the fourth neural network model belongs. The prompt information is used to prompt the third-party computing platform to steal the neural network model of the service provider and that the fourth neural network model is trained by the target participant.

[0167] Based on the same inventive concept, the present application also provides a computer device, which will be described in detail with reference to FIG8 .

[0168] FIG8 is a schematic diagram of the structure of a computer device provided in one embodiment of the present application.

[0169] As shown in Figure 8, the computer device may include at least one of the following involved in the embodiments of the present application: an electronic device, a server, etc. The computer device may include a processor 801 and a memory 802 storing computer program instructions.

[0170] Specifically, the processor 801 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0171] Memory 802 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 802 may include removable or non-removable (or fixed) media. Where appropriate, memory 802 may be inside or outside the computer device. In a specific embodiment, memory 802 is a non-volatile solid-state memory. In a specific embodiment, memory 802 includes solid-state storage (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0172] The processor 801 implements any one of the data processing methods in the above embodiments by reading and executing computer program instructions stored in the memory 802 .

[0173] In one example, the computer device may further include a communication interface 803 and a bus 810. As shown in FIG8, the processor 801, the memory 802, and the communication interface 803 are connected via the bus 810 and communicate with each other.

[0174] The communication interface 803 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0175] Bus 810 comprises hardware, software or both, and the parts of flow control device are coupled to each other.For example, and not limitation, bus can comprise accelerated graphics port (AGP) or other graphics buses, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus 810 can comprise one or more buses.Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0176] The data processing device can execute the data processing method in the embodiment of the present application, thereby realizing the data processing method and apparatus described in conjunction with Figures 1 to 8.

[0177] In addition, in conjunction with the data processing methods in the above embodiments, embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data processing methods in the above embodiments is implemented.

[0178] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0179] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0180] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0181] The above is only a specific implementation method of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited to this. Any technician familiar with this technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application.

Claims

1. A data processing method, comprising: Receiving a first neural network model sent by each of N participants, where N is an integer greater than 1; Aggregating the first neural network model sent by each participant to obtain a second neural network model; Embedding identity information into the second neural network model to obtain a third neural network model, wherein the identity information includes the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs; The third neural network model is sent to each of the participants.

2. The method according to claim 1, wherein: The second neural network model includes M hidden layers, where M is a positive integer; The embedding the identity information into the second neural network model to obtain the third neural network model comprises: Extracting a first hidden layer from the M hidden layers; Based on the weight parameters in the first hidden layer, generating a first feature parameter of the first hidden layer; Based on the identity information and the first characteristic parameter, generate the second characteristic parameter; The first characteristic parameter is replaced by the second characteristic parameter to obtain the third neural network model.

3. The method according to claim 2, wherein: The generating the second characteristic parameter based on the identity information and the first characteristic parameter includes: Projecting the first characteristic parameter by a preset key matrix to obtain a replica parameter of the first characteristic parameter; The imitation parameter is iteratively trained through the first representation parameter of the identity information until the target distance value between the first representation parameter and the imitation parameter is less than or equal to the preset distance value, and the imitation parameter corresponding to the target distance value is determined as the second feature parameter.

4. The method according to claim 3, wherein: The matrix lengths of the imitation parameter and the first representation parameter are the same and the value of the matrix length is the target length value, and the first representation parameter includes a j-th element; The iterative training of the imitation parameter by using the first representation parameter of the identity information comprises: Extracting the jth parameter from the first representation parameter, j∈[1,L], where L is the target length value; According to the matrix length, the distance value between the jth parameter and the imitation parameter is calculated through the jth element and the jth parameter.

5. The method according to claim 3 or 4, wherein: The method further comprises: The first identity information and the second identity information are encoded to obtain a first representation parameter of the identity information.

6. The method according to claim 5, wherein: The step of generating a first characteristic parameter of the first hidden layer based on a weight parameter in the first hidden layer includes: Based on the weight parameter and the number of output channels of the convolution kernel in the second neural network model, calculating the mean value of the model weight along the output channel; The mean value is determined as the first characteristic parameter.

7. The method according to claim 1, wherein: The method further comprises: When obtaining the fourth neural network model of the third-party computing platform, obtaining a pre-stored layer identifier of the first hidden layer; Extracting a second hidden layer corresponding to the layer identifier from the fourth neural network model; Extracting target identity information from a third feature parameter of the second hidden layer; When the target identity information matches the identity information, a prompt message is displayed, wherein the prompt message is used to prompt the third-party computing platform to steal the neural network model of the service provider, and the fourth neural network model is trained by the target parameter provider.

8. The method according to claim 7, wherein: The extracting target identity information from the third characteristic parameter of the second hidden layer includes: Extracting the i-th element from the second identification parameter of the target identity information, i∈[1,L], L target length value, where the target length value is the value of the matrix length of the first representation parameter of the identity information; The i-th element is converted into binary bits to obtain the target identity information.

9. The method according to claim 1, wherein: The first identity information includes at least one of the following: an organization code of the participant, an identity certificate of at least one first user of the participant, and first identification information configured by the service provider for the participant; The second identity information includes at least one of the following: an organizational code of the service provider, identity credentials of at least one second user of the service provider, and second identification information configured by the service provider.

10. An information identification method, comprising: Receiving a target neural network model sent by a third-party computing platform, wherein the target neural network model includes a fourth neural network model; Extracting a second hidden layer corresponding to a pre-stored layer identifier from the fourth neural network model, wherein the pre-stored layer identifier is an identifier of a hidden layer in which identity information is embedded; Extracting target identity information from a third feature parameter of the second hidden layer; When the target identity information matches the pre-stored identity information, a prompt message is displayed, wherein the pre-stored identity information includes the identity information of the target participant participating in the training of the fourth neural network model and the identity information of the service provider to which the fourth neural network model belongs, and the prompt information is used to prompt the third-party computing platform to have stolen the neural network model of the service provider, and that the fourth neural network model is trained by the target participant.

11. A data processing device, comprising: A receiving module, used for receiving a first neural network model sent by each of N participants, where N is an integer greater than 1; An aggregation module, used for aggregating the first neural network model sent by each participant to obtain a second neural network model; An embedding module, used for embedding identity information into the second neural network model to obtain the neural network model, wherein the identity information includes the first identity information of the participant and the second identity information of the service provider to which the second neural network model belongs; A sending model is used to send the third neural network model to each of the participants.

12. An information recognition device, comprising: A receiving module, configured to receive a target neural network model sent by a third-party computing platform, wherein the target neural network model includes a fourth neural network model; An extraction module is used to extract the corresponding layer identifier from the fourth neural network model. A second hidden layer, wherein the pre-stored layer identifier is an identifier of a hidden layer in which identity information is embedded; An extraction module, used for extracting target identity information from the third characteristic parameter of the second hidden layer; A display module is used to display prompt information when the target identity information matches the pre-stored identity information, wherein the pre-stored identity information includes the identity information of the target participant participating in the training of the fourth neural network model and the identity information of the service provider to which the fourth neural network model belongs, and the prompt information is used to prompt the third-party computing platform to steal the neural network model of the service provider and that the fourth neural network model is trained by the target participant.

13. A computer device, the device comprising: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the data processing method according to any one of claims 1 to 9, or the information identification method according to claim 10.

14. A storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the data processing method according to any one of claims 1 to 9 or the information identification method according to claim 10.

Images