Distributed data processing method and apparatus based on privacy protection

Abstract

The present disclosure relates to the field of data processing, and discloses a privacy protection-based distributed data processing method and device. The method comprises: obtaining input data and output data; preprocessing the input data to obtain preprocessed input data; determining a residual difference based on the preprocessed input data; determining a language model based on the preprocessed input data, the output data and the residual difference; training the language model based on a loss function to determine initial model parameters of the language model; obtaining updated model parameters to update the language model, and processing new preprocessed input data using the updated language model to obtain an output result. The present disclosure accelerates the operation process through distributed parallel data processing technology, can significantly reduce the calculation time and resource consumption, improves the data processing efficiency, reduces the security risk of private data in the social field in the process of centralized storage and processing, prevents data leakage, and enhances the privacy protection of data.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of data processing, in particular to a privacy protection based distributed data processing method and device. BACKGROUND

[0002] Private user data is often superior to public data in quantity and quality, because private data sets usually contain more comprehensive and detailed features, and follow stricter standards in data collection process. For example, in the fields of medical, financial and social networks, private data can provide more personalized training materials.

[0003] However, the use of private data in the social field also faces great challenges. The private data in the social field in the prior art is usually stored in isolated data silos. For example, user data on a mobile device is stored on the local device, which involves a large amount of personal privacy information. This makes it an important problem to safely obtain and utilize these private data without infringing on the privacy rights of users. Centralized storage of private data in the social field not only violates the laws and regulations of data privacy protection, but also may lead to the risk of data leakage, which poses a threat to the personal safety of users. Therefore, a new privacy protection based data processing method needs to be proposed to reduce the risk of data leakage and ensure the privacy and security of user data.

[0004] In view of this, the present disclosure is proposed. SUMMARY

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a privacy protection based distributed data (for example, multimedia data) processing method and device, which accelerates the operation process through distributed parallel processing, can significantly reduce the calculation time and resource consumption, improves the data processing efficiency, and reduces the security risk of data in the social field in the process of centralized storage and processing, effectively prevents data leakage and information theft, and ensures the privacy and security of user data.

[0006] In a first aspect, the embodiments of the present disclosure provide a privacy protection based distributed data processing method, which comprises:

[0007] obtaining input data of a client and output data corresponding to the input data, wherein the input data is multimedia data including one or more of pictures, texts, voices and videos;

[0008] preprocessing the input data to obtain preprocessed input data, wherein the preprocessed input data includes preprocessed input data at time t and preprocessed input data at time t-1;

[0009] determine a residual difference based on the preprocessed input data at time t and the preprocessed input data at time t-1;

[0010] determine a language model based on the preprocessed input data, the output data corresponding to the input data, and the residual difference;

[0011] train the language model based on a loss function to determine initial model parameters of the language model;

[0012] update the initial model parameters based on the preprocessed input data and the residual difference to obtain updated model parameters;

[0013] update the language model based on the updated model parameters, and process new preprocessed input data using the updated language model to obtain an output result.

[0014] In a second aspect, embodiments of the present disclosure further provide a privacy protection based distributed data processing apparatus, which comprises:

[0015] a client module configured to obtain input data of a client and output data corresponding to the input data, pre-process the input data to obtain preprocessed input data, wherein the input data is multimedia data including one or more of pictures, texts, voices, and videos, and the preprocessed input data includes preprocessed input data at time t and preprocessed input data at time t-1;

[0016] a language model training module configured to determine a residual difference based on the preprocessed input data at time t and the preprocessed input data at time t-1, determine a language model based on the preprocessed input data, the output data corresponding to the input data, and the residual difference, and train the language model based on a loss function to determine initial model parameters of the language model;

[0017] a server module configured to update the initial model parameters based on the preprocessed input data and the residual difference to obtain updated model parameters;

[0018] a result obtaining module configured to update the language model based on the updated model parameters, and process new preprocessed input data using the updated language model to obtain an output result.

[0019] In a third aspect, embodiments of the present disclosure further provide an electronic device, which comprises one or more processors, a storage apparatus configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the privacy protection based distributed data processing method as described above.

[0020] In a fourth aspect, the embodiments of the present disclosure also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the privacy protection based distributed data processing method as described above.

[0021] The privacy protection based distributed data processing method provided by the embodiments of the present disclosure comprises: obtaining input data of a client and output data corresponding to the input data; preprocessing the input data to obtain preprocessed input data, wherein the input data is multimedia data comprising one or more of pictures, texts, voices and videos, and the preprocessed input data comprises preprocessed input data at time t and preprocessed input data at time t-1; determining a residual difference based on the preprocessed input data at time t and the preprocessed input data at time t-1; determining a language model based on the preprocessed input data, the output data corresponding to the input data and the residual difference; training the language model based on a loss function to determine initial model parameters of the language model; updating the initial model parameters based on the model parameters and the residual difference to obtain updated model parameters; updating the language model based on the updated model parameters and processing new preprocessed input data using the updated language model to obtain output results. The present disclosure reduces the security risks of private data in the social field in the process of centralized storage and processing through distributed processing technology, effectively prevents data leakage and information theft, and enhances the privacy protection of user data. At the same time, the method allows effective data collaborative processing while data is isolated, solving the problem of privacy protection of user data on local devices such as mobile devices. At the same time, the distributed processing method can process data in parallel by multiple clients, which accelerates the operation process compared with traditional centralized server processing, can significantly reduce the calculation time and resource consumption, and improves the efficiency of data processing, especially when processing large-scale private data sets. The distributed characteristics of the method make it applicable across fields, not only in the social field, but also different fields of private data can be effectively utilized under the premise of protecting privacy. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent as various embodiments of the present disclosure are described in conjunction with the following detailed description, taken in conjunction with the accompanying drawings. Throughout the drawings, like or similar reference numerals refer to like or similar elements. It should be understood that the drawings are schematic, and the sizes of the original and elements are not necessarily drawn according to the scale.

[0023] Figure 1 A flowchart of a privacy protection based distributed data processing method in an embodiment of the present disclosure.

[0024] Figure 2A structural schematic diagram of one of the distributed data processing apparatuses based on privacy protection in embodiments of the present disclosure.

[0025] Figure 3 A structural schematic diagram of one of the electronic devices in embodiments of the present disclosure. DETAILED DESCRIPTION

[0026] Embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, but rather the embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are merely for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0027] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are merely used to distinguish different apparatuses, modules or units, and are not intended to limit the order or interdependence of the functions performed by these apparatuses, modules or units.

[0028] The names of the messages or information exchanged between the plurality of apparatuses in the embodiments of the present disclosure are merely for illustrative purposes and are not intended to limit the scope of these messages or information.

[0029] In the context of how to reasonably use private data, a federated learning mechanism is introduced, so that the model can be locally trained on multiple user devices without uploading the private data of the user to the central server. The core idea of federated learning is to keep the data on the user's device and protect personal privacy by training the model locally. After the user's device trains on its local data, it only needs to send the model parameters and gradient updates such as information to the server, and the server aggregates these parameters to generate a global model without obtaining the original data of the user. Through this method, private data can be effectively used for model updating on the basis of ensuring the security of private data, thereby improving the performance of the model. Therefore, with the help of the federated learning framework, not only can the risk of privacy leakage be effectively reduced, but also the training process of the model can be accelerated on the premise of preserving data privacy, ultimately achieving higher performance.

[0030] Traditional language model training methods based on federated learning framework have some significant shortcomings. For example, high computing resource requirements, traditional federated learning framework usually requires each client to have sufficient computing resources to train large language models containing a large number of parameters. However, in many real scenarios, especially in mobile devices or resource-constrained environments, it is difficult for clients to meet such high computing requirements. This limits the wide application of large language models and makes it difficult for many potential users to participate in the training process of the model. Secondly, there is a security risk of privacy leakage. In the existing separate learning transformer framework, although most of the parameters are concentrated on the server, the client is mainly responsible for the training of the embedding layer and the output layer, but the gradient of the embedding layer is still vulnerable to attacks. This vulnerable nature allows attackers to reconstruct private data or perform reverse engineering by analyzing the data transmitted by the client using techniques such as beam search, thereby threatening the privacy and security of users. Moreover, parallel training efficiency is low. In the current federated learning training process, the server only processes one client's update request at a time. This serial processing method significantly reduces the parallelism of the training and causes low efficiency. In the context of growing data training needs, how to accelerate the training process and improve the overall efficiency of the system is a problem that needs to be solved. The gradient and loss generated by the client are calculated locally and then returned to the server, and then the server needs to integrate these updates and generate a global model. Due to the delay of this update process, the performance of the model may not reflect the changes in user needs in a timely manner, affecting the training effect. Based on the above problems, the present application aims to provide a distributed data processing method based on privacy protection.

[0031] To solve the above problems, the present application provides a distributed data processing method based on privacy protection, which allows effective data collaborative processing while ensuring data isolation, and solves the problem of privacy protection of user data on local devices such as mobile devices. At the same time, the distributed processing method can process data in parallel, improving the efficiency of data processing.

[0032] Figure 1 A flowchart of a distributed data processing method based on privacy protection in an embodiment of the present application. The method can be performed by a distributed data processing device based on privacy protection, which can be implemented in software and / or hardware, and can be configured in an electronic device. As shown in Figure 1 The method can specifically include the following steps:

[0033] S110, obtaining input data of a client and output data corresponding to the input data.

[0034] Specifically, the input data and the output data corresponding to the input data can be stored as user privacy data saved locally by the client, and the data between the clients can be shared, so that each client can locally process the privacy data and protect the privacy of the user. The client refers to a personal device of the user, such as a smartphone, a personal computer, a tablet computer and the like, and the device stores the data of the user. The input data can be multimedia data, including the nickname, the introduction, the gender, the label and the number of fans of the user in a social platform; the message sent by the user, including text, voice, picture and video; user interaction data, such as likes, comments and forwards; and a hot topic or event, such as a hot search list of a microblog, but not limited thereto, and can include any message or information that can be obtained through a social platform. The output data can be a feature encoding state of the input data. For example, assuming a medical health application, the personal health data (such as heart rate and blood pressure) of the user is stored on the smartphone of the user, and the data is the input data, and the corresponding output data can be a health suggestion of a doctor or an automatically generated health report type. The input data can be represented as: , represents a data point in the input data, represents the number of data points in the input data, and the output data corresponding to the input data can be represented as: It should be noted that the client stores the data at different time points. For example, the input data and the output data at time t can be represented as:

[0035] ,

[0036] ,

[0037] Specifically, the client locally processes the data, including data preprocessing, model training and updating, and processing of the preprocessed input data, without the need to send the data to a remote server, thereby protecting the privacy of the user. Through the allocation of the server, multiple clients can process the data in parallel, accelerating the operation process, significantly reducing the calculation time and resource consumption, and improving the data processing efficiency.

[0038] The embodiment obtains the input data and the output data stored by the local client, effectively prevents unauthorized requests for hidden states, thereby alleviating the risk of embedded gradient attacks and improving system security.

[0039] S120, pre-processing the input data to obtain pre-processed input data, wherein the pre-processed input data includes pre-processed input data at time t and pre-processed input data at time t-1.

[0040] On the basis of the above embodiment, the mean value of the data points in the input data and the variance of the data points in the input data are calculated first, and then the mean value and the variance and the logistic distribution density function are used to calculate the preprocessed input data. The expression of the preprocessed input data is:

[0041] ,

[0042] wherein, represents the preprocessed input data, represents the input data, represents the mean value of the input data , represents the variance of the input data , represents the scaling factor, represents the logistic distribution density function, represents the distribution function influence factor.

[0043] S130, based on the preprocessed input data at the t time and the preprocessed input data at the t-1 time, determine the residual difference.

[0044] Further, on the basis of the above embodiment, the preprocessed input data at the t time and the preprocessed input data at the t-1 time are obtained, and the residual difference is calculated by the preprocessed input data at the t time and the preprocessed input data at the t-1 time, the aggregation factor and the harmonic factor, wherein the expression of the residual difference is:

[0045] ,

[0046] wherein, represents the residual difference, represents the preprocessed input data at the t time, represents the preprocessed input data at the t-1 time, represents the harmonic factor, represents the first aggregation factor, represents the second aggregation factor, represents the number of clients.

[0047] S140, based on the preprocessed input data, the output data corresponding to the input data and the residual difference, determine the language model;

[0048] Optionally, the language model is constructed based on the preprocessed input data, the output data corresponding to the input data, the residual difference and the model parameters. It should be noted that the model parameters are unknown at this time, and the model needs to be continuously trained to update the model parameters subsequently, wherein the model parameters include the first model parameters and the second model parameters, and the expression of the language model is:

[0049] ,

[0050] wherein, represents output data corresponding to preprocessed input data, represents output data corresponding to input data, represents a first model parameter, represents a second model parameter, represents a trigonometric function factor, represents a phase factor, represents a transpose convolution operation, represents a model weight factor.

[0051] S150, training the language model based on the loss function to determine the initial model parameters of the language model.

[0052] Specifically, the loss function can be constructed based on the output data corresponding to the input data and the output data output by the language model (i.e., the output data corresponding to the preprocessed input data), wherein the expression of the loss function is:

[0053] ,

[0054] wherein, represents the value of the loss function, represents preprocessed input data, represents output data corresponding to input data, represents output data corresponding to preprocessed input data, represents the number of data points in the preprocessed input data .

[0055] In an optional embodiment, the language model is trained based on the loss function to determine the initial model parameters of the language model, including: optimizing the loss function so that the initial model parameters are obtained when the value of the loss function is minimized.

[0056] On the basis of the above embodiment, the loss function is optimized, including respectively taking partial derivatives of the first model parameter and the second model parameter based on the loss function, and the initial model parameters are obtained when the loss function is minimized and stable by continuously optimizing the loss function. It should be noted that the initial model parameters are unknown when the initial model parameters are taken as partial derivatives of the loss function. The known value of the initial model parameters is obtained by taking partial derivatives and continuously optimizing. The specific partial derivative formula is as follows:

[0057] ,

[0058] ,

[0059] ,

[0060] ,

[0061] The embodiment determines the initial model parameters of the language model through the loss function, and improves the efficiency of model optimization.

[0062] S160, updating the initial model parameters based on the preprocessed input data and the residual difference to obtain updated model parameters.

[0063] It can be understood that the initial model parameters are updated using the preprocessed input data and the residual difference to obtain updated model parameters. The updated model parameters can be stored on the server module, and the server encrypts the updated model parameters to reduce the risk of data leakage between the server and the client.

[0064] Specifically, the present disclosure can implement but is not limited to the following:

[0065] The server is responsible for coordinating data sharing and global updating of the model between the clients, which is not described in detail here. Through the allocation of the server, multiple clients can handle data in parallel, accelerating the operation process, which can significantly reduce the calculation time and resource consumption, and improve the data processing efficiency.

[0066] Data sharing coordination: the server does not directly store user data, but coordinates data sharing between clients in a secure manner to facilitate data processing.

[0067] Model updating: the server can receive model update requests from the clients and update the global model according to the information provided by the clients, and then distribute the updated model parameters to each client.

[0068] For example, the server can be a cloud service that does not directly access user health data, but can receive model update requests from multiple clients and update the model parameters back to each client to update the model at the client and perform data processing. In this way, each client can perform private data processing locally, while the server is responsible for coordinating and updating the global model, ensuring user privacy while also enabling effective training and updating of the model. This distributed processing method not only protects user privacy, but also improves the efficiency of data processing and the accuracy of the model.

[0069] S170, updating the language model based on the updated model parameters, and processing new preprocessed input data using the updated language model to obtain an output result.

[0070] The output result can be output data corresponding to the preprocessed data of each client, for example, the output result of the first client can be text, pictures, the output result of the second client can be voice, video, etc., and the disclosure is not limited thereto. Through the distributed processing technology, the security risk of private data in the social field in the centralized storage and processing process is reduced, data leakage and information theft are effectively prevented, and the privacy protection of user data is enhanced. At the same time, the method allows effective data collaborative processing while data is isolated, solving the privacy protection problem of user data on local devices such as mobile devices. At the same time, the distributed processing method can perform parallel processing of data by multiple clients, improving the efficiency of data processing, especially when processing large-scale private data sets.

[0071] In an optional embodiment, updating the model parameters includes updating the first model parameters and updating the first model parameters, and updating the initial model parameters based on the preprocessed input data and the residual difference to obtain the updated model parameters includes:

[0072] Based on the preprocessed input data at time t, the preprocessed input data at time t-1, the residual difference and the first model parameters, the first formula is used to determine the updated first model parameters;

[0073] Based on the preprocessed input data at time t, the preprocessed input data at time t-1, the residual difference and the second model parameters, the second formula is used to determine the updated second model parameters.

[0074] Specifically, on the basis of the above embodiment, the input data and the preprocessed input data obtained at different times and the initial model parameters obtained by optimizing the loss function are used to update the initial model parameters to obtain the updated model parameters, and then the input data and the preprocessed input data at other times and the updated model parameters are used to update the updated model parameters again using the formula to obtain new updated model parameters. This cycle of updating continues until the updated model parameters are stable, the model training is completed, and the final updated model parameters are obtained.

[0075] For example, at time t, based on the preprocessed input data at time t, the preprocessed input data at time t-1, the residual difference and the first model parameters, the first formula is used to determine the updated first model parameters; based on the preprocessed input data at time t, the preprocessed input data at time t-1, the residual difference and the second model parameters, the second formula is used to determine the updated second model parameters, wherein the expression of the first formula is:

[0076]

[0077] The expression of the second formula is:

[0078] ​ ,

[0079] wherein, denotes updating the first model parameters, denotes updating the second model parameters, denotes the difference influence factor of the first model parameters , denotes the difference influence factor of the second model parameters , denotes the chi-square density distribution function.

[0080] The first formula and the second formula in the embodiment update the initial model parameters to obtain updated model parameters, and then constantly optimize the updated model parameters by using the first formula and the second formula, so that the model parameters can be trained in parallel, thereby improving the speed and accuracy of model updating, and further improving the computing power of the model.

[0081] In an optional embodiment, the updated model parameters are retained on the server and are encrypted for ensuring the security of user privacy data.

[0082] By retaining the updated model parameters on the server and encrypting them, the embodiment significantly reduces the risk of data leakage between the server and the client. In addition, the client and the server jointly own and use the entire language model, which further enhances the overall security of the system. Unauthorized requests for hidden states are effectively prevented, thereby mitigating the risk of embedded gradient attacks and improving system security.

[0083] It should be noted that the distributed language model can be deployed to a hierarchical server architecture, which is particularly suitable for a cluster environment with multiple machines and GPUs. By initializing the sub-servers for parallel training, the central server is responsible for aggregating and distributing model updates, which can effectively improve the processing speed and avoid the limitation of server capacity. In the single-machine and multi-GPU training scenarios, the framework uses the client batch processing method to concatenate the information of the clients, thereby improving the parallel execution capability and achieving significant improvement in training efficiency.

[0084] The privacy protection based distributed data processing method provided by the embodiment comprises the following steps: obtaining input data of a client and output data corresponding to the input data; preprocessing the input data to obtain preprocessed input data, wherein the preprocessed input data comprises preprocessed input data at time t and preprocessed input data at time t-1; determining a residual difference based on the preprocessed input data at time t and the preprocessed input data at time t-1; determining a language model based on the preprocessed input data, the output data corresponding to the input data and the residual difference; training the language model based on a loss function to determine initial model parameters of the language model; updating the initial model parameters based on the preprocessed input data and the residual difference to obtain updated model parameters; updating the language model based on the updated model parameters and processing new preprocessed input data by using the updated language model to obtain an output result. The disclosure can train and update a distributed language model by using private data of a client, thereby ensuring the security of data, effectively preventing data leakage and information theft, improving the rate and accuracy of model updating, thereby effectively improving the anti-attack ability of the model, protecting the data integrity in the training process, and reducing the risk of data leakage by performing data preprocessing and post-processing only on the client.

[0085] Figure 2 The privacy protection based distributed data processing device in the embodiment of the disclosure is shown in a structural schematic diagram. As shown in the figure, the device comprises a client module 210, a language model training module 220, a server module 230 and a result acquisition module 240. Figure 2

[0086] The client module 210 is used to obtain input information and output information of a client, preprocess the input data to obtain preprocessed input data, wherein the preprocessed input data comprises preprocessed input data at time t and preprocessed input data at time t-1.

[0087] The language model training module 220 is used to determine a residual difference based on the preprocessed input data at time t and the preprocessed input data at time t-1; determine a language model based on the preprocessed input data, the output data corresponding to the input data and the residual difference; train the language model based on a loss function to determine initial model parameters of the language model;

[0088] The server module 230 is used to update the initial model parameters based on the preprocessed input data and the residual difference to obtain updated model parameters.

[0089] The result acquisition module 240 is used to update the language model based on the updated model parameters and process new preprocessed input data by using the updated language model to obtain an output result.

[0090] ​It should be noted that the result obtaining module 240 can exist in the client, can exist in a third-party platform capable of ensuring data privacy, or in some cases, can exist in a separate device, equipment, etc. capable of ensuring data privacy in the server. In addition, the type of input data is described above in the description of the method, and is not repeated here.

[0091] wherein the expression of the pre-processed input data is:

[0092] ,

[0093] wherein, represents the pre-processed input data, represents the input data, represents the mean of the input data , represents the variance of the input data , represents the scaling factor, represents the logistic distribution density function, represents the distribution function influence factor.

[0094] wherein the expression of the residual difference is:

[0095] ,

[0096] wherein, represents the residual difference, represents the pre-processed input data at time t, represents the pre-processed input data at time t-1, represents the harmonic factor, represents the first aggregation factor, represents the second aggregation factor, represents the number of clients.

[0097] In an embodiment, the initial model parameters include first model parameters and second model parameters, and the expression of the language model is:

[0098] ,

[0099] wherein, represents the output data corresponding to the pre-processed input data, represents the output data corresponding to the input data, represents the first model parameter, represents the second model parameter, represents the trigonometric function factor, represents the phase factor, represents the transposed convolution operation, denotes a model weight factor.

[0100] Optionally, the language model training module 220 is further configured to optimize the loss function, so that the initial model parameters are obtained when the value of the loss function is minimized, wherein the expression of the loss function is:

[0101] ,

[0102] wherein, denotes the value of the loss function, denotes the preprocessed input data, denotes the output data corresponding to the input data, denotes the output data corresponding to the preprocessed input data, denotes the preprocessed input data .

[0103] Optionally, the server module 230 is further configured to determine the updated first model parameters by using a first formula based on the preprocessed input data at time t, the preprocessed input data at time t-1, the residual difference and the first model parameters;

[0104] determine the updated second model parameters by using a second formula based on the preprocessed input data at time t, the preprocessed input data at time t-1, the residual difference and the second model parameters, wherein the expression of the first formula is:

[0105] ,

[0106] the expression of the second formula is:

[0107] ,

[0108] wherein, denotes the updated first model parameters, denotes the updated second model parameters, denotes the difference impact factor of the first model parameters , denotes the difference impact factor of the second model parameters , denotes the chi-square density distribution function.

[0109] Optionally, the server module 230 is further configured to retain the updated model parameters on the server and encrypt the updated model parameters to ensure the security of user privacy data.

[0110] The distributed data processing device based on privacy protection provided by the embodiment of the present disclosure can execute the steps of the distributed data processing method based on privacy protection provided by the embodiment of the method of the present disclosure. The execution steps and beneficial effects are not repeated here.

[0111] Figure 3 This is a schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 3 , which shows a structural diagram of an electronic device 500 suitable for implementing the embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0112] like Figure 3 As shown, electronic device 500 may include a processing device 501, ROM 502, RAM 503, a bus 504, an input / output (I / O) interface 505, an input device 506, an output device 507, a storage device 508, and a communication device 509. The processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501 can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage device 508 into random access memory (RAM) 503 to implement the privacy-preserving distributed multimedia data processing method described in the embodiments of the present disclosure. RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing device 501, ROM 502, and RAM 503 are interconnected via bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0113] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart, thereby implementing the distributed data processing method based on privacy protection as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0114] It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0115] The computer readable medium described above can be contained in the electronic device described above; or can exist separately and not be assembled into the electronic device. The computer readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to: obtain input data of a client and output data corresponding to the input data; pre-process the input data to obtain pre-processed input data, wherein the pre-processed input data includes pre-processed input data at time t and pre-processed input data at time t-1; determine a residual difference based on the pre-processed input data at time t and the pre-processed input data at time t-1; determine a language model based on the pre-processed input data, the output data corresponding to the input data and the residual difference; train the language model based on a loss function to determine initial model parameters of the language model; update the initial model parameters based on the pre-processed input data and the residual difference to obtain updated model parameters; update the language model based on the updated model parameters, and process new pre-processed input data using the updated language model to obtain an output result.

[0116] Optionally, when the one or more programs are executed by the electronic device, the electronic device can further perform other steps as described in the above embodiments.

[0117] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] The above description is merely illustrative of the exemplary embodiments of this disclosure and the principles of the application. It will be apparent to those skilled in the art that modifications in the preferred embodiment according to specific requirements or applications of this disclosure can be made without departing from the scope of the disclosure, which is defined by the following claims. For example, the features of the above-described embodiments can be combined with other features disclosed in this disclosure (but not limited to) having similar functions to form other embodiments.

Claims

1. A privacy-preserving distributed data processing method, characterized in that, The method comprises: obtaining input data of a client and output data corresponding to the input data, wherein the input data is multimedia data including one or more of pictures, texts, voices and videos; preprocessing the input data to obtain preprocessed input data, wherein the preprocessed input data includes preprocessed input data at time t and preprocessed input data at time t-1; determining a residual difference based on the preprocessed input data at time t and the preprocessed input data at time t-1; determining a language model based on the preprocessed input data, the output data corresponding to the input data and the residual difference; training the language model based on a loss function to determine initial model parameters of the language model; updating the initial model parameters based on the preprocessed input data and the residual difference to obtain updated model parameters; updating the language model based on the updated model parameters and processing new preprocessed input data using the updated language model to obtain output results, wherein an expression of the residual difference is: ， wherein, denotes a residual difference, denotes pre-processed input data at time t, denotes pre-processed input data at time t-1, denotes a harmonic factor, denotes a first aggregation factor, denotes a second aggregation factor, denotes a number of clients, wherein the initial model parameters include first model parameters and second model parameters, and an expression of the language model is: ， wherein, represents output data corresponding to pre-processed input data, represents output data corresponding to input data, represents a first model parameter, represents a second model parameter, represents a trigonometric function factor, represents a phase factor, represents a transpose convolution operation, represents a model weight factor.

2. The method of claim 1, wherein, an expression of the preprocessed input data is: ， wherein denotes pre-processed input data, denotes input data, denotes input data a mean value, denotes input data a variance, denotes a scaling factor, denotes a logistic distribution density function, denotes a distribution function influence factor.

3. The method of claim 1, wherein, training the language model based on a loss function to determine initial model parameters of the language model comprises: optimizing the loss function so that the initial model parameters are obtained when the value of the loss function is the minimum.

4. The method of claim 3, wherein, an expression of the loss function is: ， wherein, represents a value of a loss function, represents pre-processed input data, represents output data corresponding to the input data, represents output data corresponding to the pre-processed input data, represents pre-processed input data represents a number of data points in the data.

5. The method of claim 4, wherein, the updated model parameters include updated first model parameters and updated second model parameters, updating the initial model parameters based on the preprocessed input data and the residual difference to obtain updated model parameters comprises: determining updated first model parameters using a first formula based on the preprocessed input data at time t, the preprocessed input data at time t-1, the residual difference and the first model parameters; determining updated second model parameters using a second formula based on the preprocessed input data at time t, the preprocessed input data at time t-1, the residual difference and the second model parameters.

6. The method of claim 5, wherein, an expression of the first formula is: ， an expression of the second formula is: ， wherein, denotes updating the first model parameters, denotes updating the second model parameters, denotes the first model parameters of the difference impact factor, denotes the second model parameters of the difference impact factor, denotes the chi-square density distribution function.

7. The method of claim 1, wherein, further comprising: retaining the updated model parameters on a server and encrypting the updated model parameters to ensure the safety of user private data.

8. A privacy protection based distributed data processing apparatus, characterized by, The apparatus comprises: a client module configured to obtain input data of a client and output data corresponding to the input data, preprocess the input data to obtain preprocessed input data, wherein the input data is multimedia data including one or more of pictures, texts, voices and videos, and the preprocessed input data includes preprocessed input data at time t and preprocessed input data at time t-1. The language model training module is configured to determine a residual difference based on the preprocessed input data at the t time and the preprocessed input data at the t-1 time; determine a language model based on the preprocessed input data, the output data, and the residual difference; and train the language model based on a loss function to determine initial model parameters of the language model. The server module is configured to update the initial model parameters based on the preprocessed input data and the residual difference to obtain updated model parameters. The result obtaining module is configured to update the language model based on the updated model parameters, and process new preprocessed input data by using the updated language model to obtain an output result. The expression of the residual difference is as follows: ， wherein, denotes a residual difference, denotes pre-processed input data at time t, denotes pre-processed input data at time t-1, denotes a harmonic factor, denotes a first aggregation factor, denotes a second aggregation factor, denotes a number of clients, The initial model parameters include first model parameters and second model parameters, and the expression of the language model is as follows: ， wherein, represents output data corresponding to pre-processed input data, represents output data corresponding to input data, represents a first model parameter, represents a second model parameter, represents a trigonometric function factor, represents a phase factor, represents a transpose convolution operation, represents a model weight factor.

Images