Image processing model training method and system

Abstract

Embodiments of the present specification provide an image processing model training method and system. The image processing model training method is applied to a first training unit and includes receiving an image processing model sent by a second training unit, training the image processing model using image training data to obtain update data of each network layer in the image processing model, wherein the update data is used to update parameters of the each network layer; performing data sensitivity detection on each update data according to sensitive image data in the image training data to determine data sensitivity information corresponding to each update data; performing data desensitization on each update data using the data sensitivity information to obtain desensitized update data of the each network layer; and sending the desensitized update data of the each network layer to the second training unit for parameter updating. The embodiments avoid the problem of exposing sensitive private information in a distributed machine learning process.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of artificial intelligence, and in particular to an image processing model training method; one or more embodiments of the present specification also relate to another image processing model training method, an image processing model training system, a computing device, a computer-readable storage medium, and a computer program product. BACKGROUND

[0002] With the continuous development of artificial intelligence technology, neural network models are also widely used in various data processing scenarios. In order to solve the problems of less training data and data islands, a distributed machine learning method can be used for model training, so as to ensure data security while avoiding problems such as less training data and data islands. For example, a federated learning method can be used.

[0003] Currently, there is still a risk of sensitive data leakage in the process of model training using a distributed machine learning method, which may expose sensitive private information. Therefore, how to ensure data security and avoid sensitive data leakage in the process of model training has become a problem to be solved. SUMMARY

[0004] Therefore, embodiments of the present specification provide an image processing model training method. One or more embodiments of the present specification also relate to another image processing model training method, an image processing model training system, an image processing model training device, another image processing model training device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects in the prior art.

[0005] According to a first aspect of embodiments of the present specification, an image processing model training method is provided, applied to a first training unit, comprising:

[0006] receiving an image processing model sent by a second training unit, and performing model training on the image processing model using image training data to obtain update data of each network layer in the image processing model, wherein the update data is used to update parameters of the each network layer;

[0007] According to sensitive image data in the image training data, data sensitivity detection is performed on each update data to determine data sensitivity information corresponding to each update data;

[0008] Using the data sensitivity information, data desensitization is performed on each update data to obtain desensitized update data of each network layer;

[0009] The desensitized update data of each network layer is sent to the second training unit for parameter update.

[0010] According to a second aspect of the embodiments of the present specification, an image processing model training apparatus is provided, applied to a first training unit, comprising:

[0011] The model training module is configured to receive an image processing model sent by a second training unit, and perform model training on the image processing model by using image training data, to obtain update data of each network layer in the image processing model, wherein the update data is used for parameter updating of the each network layer;

[0012] The data sensitive detection module is configured to perform data sensitive detection on each update data according to sensitive image data in the image training data, to determine data sensitive information corresponding to the each update data;

[0013] The data desensitization module is configured to perform data desensitization on the each update data by using the data sensitive information, to obtain desensitization update data of the each network layer;

[0014] The data sending module is configured to send the desensitization update data of the each network layer to the second training unit for parameter updating.

[0015] According to a third aspect of the embodiments of the present specification, another image processing model training method is provided, applied to a second training unit, comprising:

[0016] Sending an image processing model to a plurality of first training units;

[0017] Receiving desensitization update data of each network layer in the image processing model sent by each first training unit, wherein the desensitization update data is obtained by performing data desensitization on update data of the each network layer by the first training unit, and the update data is obtained by performing model training on the image processing model by the first training unit by using image training data;

[0018] Performing data sensitive detection on each desensitization update data by using sensitive image data in the second training unit, to determine data sensitive information corresponding to each desensitization update data;

[0019] Determining target desensitization update data from the each desensitization update data based on the data sensitive information;

[0020] Performing parameter updating on the each network layer in the image processing model by using the target desensitization update data, to obtain an updated image processing model.

[0021] According to a fourth aspect of the embodiments of the present specification, another image processing model training apparatus is provided, applied to a second training unit, comprising:

[0022] a model sending module configured to send an image processing model to a plurality of first training units;

[0023] a data receiving module configured to receive desensitization update data of each network layer in the image processing model sent by each first training unit, wherein the desensitization update data is obtained by data desensitization of update data of the each network layer by the first training unit, and the update data is obtained by model training of the image processing model by the first training unit using image training data;

[0024] a data sensitivity detection module configured to perform data sensitivity detection on each desensitization update data using sensitive image data in the second training unit to determine data sensitivity information corresponding to each desensitization update data;

[0025] a data determination module configured to determine target desensitization update data from the each desensitization update data based on the data sensitivity information;

[0026] a parameter update module configured to perform parameter update on the each network layer in the image processing model using the target desensitization update data to obtain an updated image processing model.

[0027] According to a fifth aspect of the embodiments of the present specification, an image processing model training system is provided, comprising a server and a plurality of clients, wherein,

[0028] the server is configured to send an image processing model to the plurality of clients, receive desensitization update data of each network layer in the image processing model sent by each client, wherein the desensitization update data is obtained by data desensitization of update data of the each network layer by the client, and the update data is obtained by model training of the image processing model by the client using image training data, perform data sensitivity detection on each desensitization update data using sensitive image data in the server to determine data sensitivity information corresponding to each desensitization update data, determine target desensitization update data from the each desensitization update data based on the data sensitivity information, perform parameter update on the each network layer in the image processing model using the target desensitization update data to obtain an updated image processing model;

[0029] The client receives the image processing model sent by the server, and trains the image processing model using the image training data to obtain update data of each network layer in the image processing model, wherein the update data is used to update parameters of the each network layer, each update data is subjected to data sensitivity detection according to sensitive image data in the image training data, data sensitivity information corresponding to the each update data is determined, the each update data is subjected to data desensitization using the data sensitivity information, desensitized update data of the each network layer is obtained, and the desensitized update data of the each network layer is sent to the server for parameter updating.

[0030] According to a sixth aspect of an embodiment of the present specification, a computing device is provided, comprising:

[0031] a memory and a processor;

[0032] The memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, which realize the steps of any one of the above methods when executed by the processor.

[0033] According to a seventh aspect of an embodiment of the present specification, a computer readable storage medium is provided, which stores computer programs / instructions, which realize the steps of any one of the above methods when executed by the processor.

[0034] According to an eighth aspect of an embodiment of the present specification, a computer program product is provided, comprising computer programs / instructions, which realize the steps of any one of the above methods when executed by the processor.

[0035] The image processing model training method applied to the first training unit in one or more embodiments of the present specification can receive an image processing model sent by a second training unit in the process of distributed machine learning, and train the image processing model using local image training data to obtain update data of each network layer in the image processing model; in the process of sending the update data to the second training unit for parameter updating, in order to avoid the risk that the update data may leak sensitive data, each update data is subjected to data sensitivity detection according to sensitive image data in the image training data, so as to determine data sensitivity information corresponding to each update data, and each update data is subjected to data desensitization using the data sensitivity information, and then the desensitized update data is sent to the second training unit, thereby avoiding the problem that sensitive private information may be exposed in the process of distributed machine learning, ensuring data security, and reducing the risk of sensitive data leakage. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1is a flowchart of an image processing model training method provided by one embodiment of the present specification;

[0037] Figure 2 is a flowchart of an image processing model training method provided by one embodiment of the present specification;

[0038] Figure 3 is a flowchart of an image processing model training method provided by one embodiment of the present specification;

[0039] Figure 4 is a flowchart of an image processing model training method provided by one embodiment of the present specification;

[0040] Figure 5 is a flowchart of an image processing model training method provided by one embodiment of the present specification;

[0041] Figure 6 is a flowchart of an image processing model training method provided by one embodiment of the present specification;

[0042] Figure 7 is a structural block diagram of a computing device provided by one embodiment of the present specification. DETAILED DESCRIPTION

[0043] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present specification. However, the present specification can be practiced without the specific details, other than in the examples, and it is understood that the scope of the present specification is not limited to the details below.

[0044] The terminology used in one or more embodiments of the present specification is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present specification. As used in one or more embodiments of the present specification and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0045] It should be understood that, although the terms first, second, etc. can be employed in describing various information in one or more embodiments of the present specification, the information should not be limited to such terms. These terms are only used to differentiate one piece of information from another piece of information of the same type. For example, without departing from the scope of one or more embodiments of the present specification, first can also be referred to as second, and similarly, second can also be referred to as first. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "upon" or "in response to determining".

[0046] In addition, it should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of the present specification are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0047] In one or more embodiments of the present specification, a large model refers to a deep learning model with a large number of model parameters, usually containing hundreds of millions, tens of billions, hundreds of billions, thousands of billions or even tens of billions of model parameters. The large model can also be called a foundation model. Through large-scale unlabeled corpus pre-training, a pre-trained model with hundreds of millions of parameters is produced. Such a model can adapt to a wide range of downstream tasks, and the model has good generalization ability. For example, large language models (LLM) and multi-modal pre-training models.

[0048] In actual application, the large model only needs a small amount of sample to fine-tune the pre-trained model and can be applied to different tasks. The large model can be widely applied to natural language processing (NLP) and computer vision fields. Specifically, it can be applied to computer vision field tasks such as visual question answering (VQA), image captioning (IC), image generation, and natural language processing field tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the large model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0049] First, the terms involved in one or more embodiments of the present specification are explained.

[0050] Distributed machine learning: refers to a technique that utilizes multiple computing nodes for machine learning or deep learning. This distributed training technique can greatly improve the speed and efficiency of training large-scale data sets.

[0051] LDP (Local Differential Privacy): refers to a local differential privacy.

[0052] L2 norm: also known as Euclidean norm or 2-norm.

[0053] Federated learning (FL): a distributed machine learning method that aims to train models with distributed devices to achieve large-scale cross-domain data learning and inference while protecting user data privacy. Centralized machine learning models require all data to be centralized on a central server for training, but this may expose sensitive user privacy information. Federated learning avoids sending user data to a central server by training on local devices, achieving data availability invisibility.

[0054] Image Forgery Detection: a technique designed to detect and locate areas of tampering or forgery in images. With the development of digital image processing and editing technology, users can use image editing tools and generative models to edit and tamper with image content, such as image synthesis, copy and paste, deletion and modification, repair, etc. The goal of image forgery detection is to analyze the features and content of the image to identify tampered areas in the image to reveal the authenticity and integrity of the image.

[0055] Model Inversion Attacks: without further specification, the reverse attacks mentioned in this article refer to reverse attacks under the federated learning framework. In federated learning, reverse attacks refer to attacks by malicious participants to obtain original data or model parameters from the federated learning process. This attack aims to reverse engineer the federated learning model to infer private information contained in the local data. Specifically, in federated learning, data reverse attacks may occur: attackers may attempt to infer the characteristics or sensitive information of local data by analyzing the model parameters or gradient information transmitted during federated learning. This attack may result in the exposure of private data.

[0056] Differential Privacy (DP) algorithm: a classic privacy protection algorithm that introduces certain noise during data publishing to prevent attackers from inferring individual sensitive information by analyzing the output results.

[0057] With the continuous development of artificial intelligence technology, neural network models are also widely used in various data processing scenarios. For example, in the consumer finance scenario, a large number of financial credentials with user identity information are involved between financial consumers and financial institutions, such as income certificates, professional credentials, and identity cards. Due to the temptation of improper gains, some users self-drive or with the help of "proxy rights protection" to mass-produce false credentials and deliver them to multiple financial institutions. How to identify false credentials has become a common difficulty faced by the industry.

[0058] In order to solve the above problems of less training data and data island, a distributed machine learning method can be used for model training, so as to avoid the problems of less training data and data island while ensuring data security. For example, a federated learning method can be used.

[0059] As a representative technology of cross-domain learning, federated learning can realize data availability and invisibility, and is an ideal solution for joint modeling between multiple institutions. However, the potential privacy leakage risk and model accuracy degradation in federated learning are still key problems to be solved.

[0060] In view of the above potential privacy leakage risk and model accuracy degradation, the four schemes provided by the present specification,

[0061] The first scheme can realize federated learning image classification through a local differential privacy algorithm, especially for medical images with a large amount of personal privacy information. This scheme attempts to integrate local differential privacy (LDP) into the system to achieve strict privacy protection. However, this scheme only protects each sample with the same mechanism without considering the privacy preference of each client for privacy sensitive information (PSI). In this case, excessive privacy protection of non-privacy sensitive information may introduce unnecessary noise, thereby reducing model performance.

[0062] The second scheme is a decentralized federated learning framework for the image field, which is used to solve the communication and computing bottlenecks in the centralized server federated learning framework.

[0063] The third scheme is a study on backdoor attacks in federated learning, which proposes a federated learning image classification method that can defend against backdoor attacks through dimension reduction technology and clustering algorithms.

[0064] The fourth scheme inputs a local privacy region image into a local model, and uses the output of the model at a specified layer to locate the gradient region corresponding to the privacy region. Obviously, this scheme has many manual super parameters, such as the number of local privacy region image input networks, the selection of key layers, and the design of gradient threshold. In addition, the gradient positioning method proposed in the scheme requires strong expert experience, and a large amount of manual verification and adjustment optimization is required for different local models, and the generalization is poor.

[0065] Based on this, in the specification, an image processing model training method is provided, one or more embodiments of the specification also relate to another image processing model training method, an image processing model training system, an image processing model training device, another image processing model training device, a computing device, a computer readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0066] Referring to Figure 1 , Figure 1 An application diagram of an image processing model training method provided according to an embodiment of the specification is shown, based on Figure 1 It can be known that the server 102 can initialize an image processing model (i.e., a global model) and distribute it to all participating clients 104; and each client 104 uses local image data (i.e., a local data set) to train the received global model to generate model updates (i.e., gradient data); In order to avoid sensitive data leakage, the client 104 will define privacy sensitive information (Privacy Sensitive Information, PSI), and calculate the sensitivity of the gradient data to the PSI region; The gradient data is desensitized by the sensitivity, thereby obtaining desensitized gradient data; Finally, the client 104 sends the desensitized gradient data to the server 102 for federated learning, thereby realizing the protection of private data.

[0067] Referring to Figure 2 , Figure 2 A flowchart of an image processing model training method provided according to an embodiment of the specification is shown, which is applied to a first training unit and specifically includes the following steps.

[0068] Step 202: receiving an image processing model sent by a second training unit, and using image training data to train the image processing model to obtain update data of each network layer in the image processing model, wherein the update data is used to update parameters of the each network layer.

[0069] The first training unit can be understood as a unit in the image processing model training system. The first training unit can train the image processing model sent by the second training unit using the locally stored image training data, obtain the update data of each network layer in the image processing model, and feed back the desensitization update data after desensitization processing to the unit of the second training unit. The first training unit can be a server, a server cluster, a cloud server, a client, etc. In one or more embodiments provided in the specification, the first training unit can be a client. The first training unit can also be understood as an institution participating in federated learning, such as a bank institution, a financial institution, etc.

[0070] The second training unit can be understood as a unit in the image processing model training system. The second training unit can be used to send the image processing model to be trained to a plurality of first training units, and update the parameters of each network layer in the image processing model based on the desensitization update data fed back by the plurality of first training units. The second training unit can be a server, a server cluster, a server, a cloud server, etc. In one or more embodiments provided in the specification, the second training unit can be a server. The image processing model training system can be understood as a federated learning system for the image processing model.

[0071] The image training data can be understood as image data for model training of the image processing model. For example, the image data can be financial image data (such as report image, contract image), face image, building image, image of proof material (such as image of identity proof material), etc.

[0072] The image processing model can be understood as a model for processing image data. The image processing model can be a neural network model, a deep learning model, a large model, etc. For example, the image processing model can be an image classification model, an image information recognition model, a text image tampering detection model, etc. without specific limitation. The text image tampering detection model is used to perform a text image tampering detection task.

[0073] The update data can be understood as data for adjusting the parameters of each network layer in the image processing model. For example, the update data can be gradient data, loss data, etc. Adjusting the parameters of each network layer in the image processing model can be understood as adjusting the weight parameters or matrix parameters of each network layer in the image processing model.

[0074] Step 204: According to the sensitive image data in the image training data, data sensitive detection is performed on each update data to determine the data sensitive information corresponding to each update data.

[0075] The sensitive image data can be understood as sensitive data contained in the image training data. For example, the sensitive image data can be sensitive information in the image training data, such as name information, gender information, identity information, etc. For example, the sensitive image data can be a sensitive image region in the image training data, such as a face region, a seal region, a name region, etc.

[0076] The data sensitive information can be understood as information for indicating whether the update data is sensitive or the degree of sensitivity. For example, the data sensitive information can be a data sensitive label, a data sensitivity, a sensitivity, etc. of the update data.

[0077] Specifically, the image processing model training method provided by the present specification can determine sensitive image data from image training data. Then, according to the sensitive image data, the data sensitivity of each update data is detected, and the data sensitive information corresponding to each update data is determined.

[0078] The way of determining the sensitive image data from the image training data can be to determine the sensitive image region and the non-sensitive image region from the image training data, and to determine the sensitive image region as the sensitive image data.

[0079] Alternatively, the way of determining the sensitive image data from the image training data can be to determine a plurality of initial sensitive image data from the image training data, to evaluate the sensitivity of each initial sensitive image data, and to determine the corresponding sensitive level for each initial sensitive image data according to the sensitivity. For example, the sensitive levels are: first-level sensitive data, second-level sensitive data, third-level sensitive data, fourth-level sensitive data, etc.

[0080] The initial sensitive image data with the same sensitive level as the preset sensitive level (for example, third-level sensitive data and / or fourth-level sensitive data) is determined as the sensitive image data from the image training data.

[0081] The image processing model training method provided by the present specification is explained and described by taking the application of the image processing model training method in the federated learning personalized differential privacy protection scene as an example. The first training unit is the client of the federated learning, the second training unit is the server of the federated learning, the update data is the gradient data, and the sensitive image data is the privacy sensitive information (Privacy Sensitive Information, PSI).

[0082] Based on this, in the process of model training, the privacy sensitive information (Privacy Sensitive Information, PSI) needs to be defined, and the privacy preference of each client needs to be identified and marked.

[0083] Specifically, in the financial scenario, the image training data can be text image data of financial text (identity card, contract). For the image material of the certificate with the official seal, some clients (such as the bank participating in joint modeling) may consider the name and the identity card number as the most important privacy information, and therefore mark them as more sensitive privacy sensitive information (i.e. sensitive image data), while some other institutions may consider the identity card number, the official seal and the address as the most important privacy information. In specific practice, the privacy sensitive information PSI of different institutions can be accurately identified by manual or automatic program.

[0084] After determining the privacy sensitive information, the gradient data of each network layer can be subjected to data sensitivity detection by using the privacy sensitive information, so as to determine the data sensitivity (i.e. data sensitive information) corresponding to the gradient data.

[0085] It should be noted that in the process of data sensitive detection of each update data according to the sensitive image data in the image training data, the sensitivity (i.e. data sensitive information) of the gradient data to the input value (i.e. sensitive image data in the image training data) can be measured by using the Shannon value.

[0086] In one or more embodiments provided in the present specification, the update data is gradient data.

[0087] The data sensitive detection of each update data according to the sensitive image data in the image training data, and the determination of the data sensitive information corresponding to each update data, comprises:

[0088] Determining the sensitive image data contained in the image training data;

[0089] Using the sensitive image data and the partial derivative matrix corresponding to each gradient data to perform data sensitive detection on the gradient data, and determining the data sensitive information corresponding to the gradient data.

[0090] The partial derivative matrix can be understood as a matrix containing the partial derivative of the gradient data with respect to the input data (i.e. image training data), for example, the partial derivative matrix can be a Jacobian matrix.

[0091] Specifically, the method provided in the present specification can determine the sensitive image data from the image training data according to the pre-labeled sensitive identifier, wherein the sensitive identifier can be a sensitive level or a sensitive data label.

[0092] Then, the partial derivative matrix corresponding to each gradient data is calculated, and the sensitive matrix region is determined from the partial derivative matrix according to the sensitive image data, and the data sensitivity information corresponding to each gradient data is determined according to the sensitive matrix region, so as to accurately determine the sensitivity (i.e. data sensitivity information) of each gradient data, which facilitates subsequent accurate noise reduction of the gradient data.

[0093] In one or more embodiments provided in the specification, the data sensitivity detection of each gradient data is performed by using the sensitive image data and the partial derivative matrix corresponding to the gradient data, and the data sensitivity information corresponding to each gradient data is determined, including:

[0094] The partial derivative of each gradient data with respect to the image training data is calculated, and the partial derivative matrix corresponding to each gradient data is constructed based on the partial derivative;

[0095] The data region information of the sensitive image data is determined, and the matrix parameters of the sensitive image data are determined from the partial derivative matrix based on the data region information;

[0096] The data sensitivity information corresponding to each gradient data is calculated based on the matrix parameters.

[0097] In the process of sending the gradient data to the server for parameter updating by the client, in order to avoid the risk that the gradient data may leak sensitive data, the gradient data needs to be data desensitized; in the process of data desensitization, it is necessary to determine the sensitivity of each gradient data to the PSI first, that is, each client needs to calculate the sensitivity of the gradient data to the PSI region, and the specific implementation manner can include the following three steps:

[0098] Step 1, the Jacobian matrix of the gradient data corresponding to each network in the model is calculated, that is, the first-order partial derivative of the gradient with respect to the input data x, and the calculation method of the Jacobian matrix can refer to the following formula (1).

[0099]

[0100] Wherein, g l (x) represents the gradient of the input data x at the lth layer.

[0101] Step 2, different clients determine the privacy preference region PSI of different data samples, and determine the response region of the privacy sensitive region in the Jacobian matrix through the following formula (2) Positioning.

[0102]

[0103] wherein, for each sample point (i.e. image training data), its privacy preference region can be represented as w x h x c, w represents the width of the region, h represents the height of the region, and c represents the number of channels of the region.

[0104] ||·|| in the above formula F represents the L2 norm operation on the matrix elements; through this operation, the privacy sensitive region PSI can be located in the response region of the Jacobian matrix. The response region can be understood as the matrix parameter region in the Jacobian matrix corresponding to the privacy sensitive region, and the matrix parameter region contains the matrix parameters of the sensitive image data; for example, the matrix parameters can be partial derivatives.

[0105] In one or more embodiments of the present specification, the first output layer of the network (image processing model) and the input image (i.e. image training data) are consistent in length and width size, so the region positioning can be directly completed by means of the above formula (2). If the intermediate output layer of the network used and the input image size are inconsistent, size alignment can be considered to be completed by means of upsampling or downsampling before performing the operation.

[0106] Step 3, different clients calculate the gradient data corresponding to each network layer by the following formula (3), and the sensitivity of the privacy sensitive region, so as to obtain the sensitivity of each network layer:

[0107] Specifically, the client can determine the privacy sensitive region, and the response region of the Jacobian matrix of a certain network layer

[0108] Then, the matrix parameters (i.e. matrix parameters of sensitive image data) in the response region are averaged to obtain the PSI score S l (i.e. data sensitive information) corresponding to a certain network layer. The scalar value S l reflects the quantization result of the privacy preference region in the local data set of a specific client.

[0109]

[0110] By performing the above operation on each network layer, the PSI score S l of each network layer, i.e. the sensitivity, is obtained.

[0111] Based on the above steps, it can be known that in the process of gradient sensitivity scoring, the client can calculate the sensitivity of the gradient to the PSI, and the specific implementation is to use the Jacobian matrix to evaluate the sensitivity of the gradient to the input data.

[0112] Step 206: using the data sensitive information, data desensitization is performed on the update data to obtain desensitized update data of the network layer.

[0113] The data desensitization can be understood as a data noise operation or a data perturbation operation, and the desensitized update data can be understood as update data after desensitization.

[0114] In the above example, a random response mechanism can be used to perturb the gradient data locally at each client.

[0115] In one or more embodiments provided in the specification, the data sensitivity information is data sensitivity.

[0116] The data desensitization of the update data using the data sensitivity information obtains desensitized update data of the network layers, and includes:

[0117] According to the data sensitivity of the update data, a data desensitization parameter corresponding to the update data is calculated, wherein the data desensitization parameters of the update data are the same or different.

[0118] Based on the data desensitization parameter corresponding to the update data, the update data is desensitized to obtain the desensitized update data of the network layers.

[0119] The data desensitization parameter can be understood as a parameter for determining the degree of data desensitization of the update data, for example, the data desensitization parameter can be a hyperparameter, a privacy budget, etc.

[0120] It should be noted that the data desensitization parameters of the update data of the network layers can be the same or different.

[0121] Specifically, after determining the data sensitivity, the method provided in the specification can calculate different or same data desensitization parameters for the update data corresponding to different network layers according to the data sensitivity; then, the data desensitization parameters are used to perform different degrees of data desensitization operation on each update data to obtain the desensitized update data of the network layers, thereby avoiding unnecessary data desensitization operation and performance loss caused by excessive privacy protection.

[0122] In one or more embodiments provided in the specification, the data desensitization of the update data based on the data desensitization parameter corresponding to the update data obtains the desensitized update data of the network layers, and includes:

[0123] A target network layer in the network layers is determined, and a target data desensitization parameter corresponding to the target network layer is determined, wherein the target network layer is any one of the network layers.

[0124] Determine the noise intensity information corresponding to the target data desensitization parameter, and use the noise intensity information to data-noise the update data of the target network layer to obtain the desensitized update data.

[0125] Following the above example, the client needs to allocate privacy budget during the data-noise process, which means that different privacy budgets (ε l ) are allocated to each network layer according to the sensitivity score of the gradient. For details, see formula (4) below.

[0126]

[0127] Where ∈ is a hyperparameter, representing the overall privacy budget for performing local differential privacy. Generally, the smaller the ∈ l , the greater the noise intensity of differential privacy noise, the stronger the privacy protection, and the lower the data availability; on the contrary, the greater the ∈ l , the smaller the noise intensity of differential privacy noise, the weaker the privacy protection, and the higher the data availability.

[0128] Then the client needs to perform gradient clipping and noise injection.

[0129] First, the client clips the gradient to convert data points with large numerical values to data points with small numerical values, thereby limiting the maximum impact of individual data points on model updates.

[0130] Second, according to the ∈ l of each network layer, Gaussian noise is injected into the clipped gradient data to meet the allocated privacy budget and achieve LDP. The calculation formulas (5) and (6) of the steps are as follows:

[0131]

[0132] Based on the above steps, to solve the problems of insufficient privacy protection and performance degradation in existing joint modeling schemes for private images, the present scheme proposes a federated learning personalized differential privacy protection method based on protection preference analysis to solve the problems. Specifically, by allocating privacy budget layer by layer, unnecessary noise for information that is not sensitive to privacy is reduced, thereby avoiding performance loss caused by excessive privacy protection.

[0133] Step 208: Send the desensitized update data of each network layer to the second training unit for parameter update.

[0134] Following the above example, the client sends the perturbed gradient data to the server.

[0135] The image processing model training method applied to the first training unit in one or more embodiments of the present specification can receive an image processing model sent by a second training unit in the process of distributed machine learning, and use local image training data to train the image processing model to obtain update data of each network layer in the image processing model; in the process of sending the update data to the second training unit for parameter updating, in order to avoid the risk that the update data may leak sensitive data, it is necessary to detect the data sensitivity of each update data according to the sensitive image data in the image training data, so as to determine the data sensitive information corresponding to each update data, and then desensitize each update data by using the data sensitive information, and then send the desensitized update data to the second training unit, thereby avoiding the problem that sensitive private information may be exposed in the process of distributed machine learning, ensuring data security and reducing the risk of sensitive data leakage.

[0136] The following describes the embodiments of the present specification in conjunction with the accompanying Figure 3 The image processing model training method provided by the present specification is taken as an example for further description of the image processing model training method in the application of federated learning personalized differential privacy protection scene. Among them, Figure 3 The processing process flow diagram of an image processing model training method provided by one embodiment of the present specification is shown, and the image processing model training method is applied to a federated learning system (i.e. an image processing model training system), which includes a server (Server) and multiple clients (Client).

[0137] Among them, the client (Client): refers to multiple clients participating in joint learning, and each client has a local data set. The server (Server): is used to be responsible for managing the training process, including the collection, aggregation of gradients and the update and distribution of model parameters.

[0138] The image processing model training method specifically includes the following steps.

[0139] Step 302: global model initialization.

[0140] Specifically, the server can initialize a global model (i.e. an image processing model) and distribute it to all participating clients.

[0141] Step 304: local model training.

[0142] Specifically, each client uses its local data set to train the received global model to generate model updates (gradient data).

[0143] Step 306: private sensitive information (PSI) identification.

[0144] Specifically, the client identifies the PSI in the local training data and assigns a corresponding privacy preference to each PSI data point (i.e., PSI region).

[0145] The privacy preference can be understood as a label representing that a certain image region in the image training data is private data.

[0146] For example, in a financial scenario, the image training data can be text image data of financial text (identity card, contract). For a certificate image material with a seal, some clients (such as a bank participating in joint modeling) may consider the name and ID number to be the most important private information, and therefore mark them as more sensitive private sensitive information PSI, while other institutions may consider the ID number, seal, and address to be the most important private sensitive information PSI. In specific practice, the private sensitive information PSI of different institutions can be accurately identified by manual or automated procedures.

[0147] Step 308: Gradient sensitivity score.

[0148] Specifically, the client can calculate the sensitivity of the gradient to the PSI. In the method provided in this specification, the Jacobian matrix can be used to evaluate the sensitivity of the gradient to the input data; the specific implementation mode is described below:

[0149] (1) Calculate the Jacobian matrix J l (x) of the gradient data corresponding to each network in the model, i.e., the first-order partial derivative of the gradient with respect to the input data x. The calculation method of the Jacobian matrix J l (x) can be seen in the following formula (1).

[0150]

[0151] where g l (x) represents the gradient of the input data x at the lth layer; y in the above formula represents the output data of the lth network layer; w represents the weight data of the lth network layer. is the partial derivative symbol.

[0152] (2) Different clients determine the privacy preference region PSI of different data samples, and determine the response region of the privacy sensitive region in the Jacobian matrix through the following formula (2):

[0153]

[0154] where a and b represent the a-th row and b-th column of the Jacobian matrix.

[0155] ​For each sample point (i.e. image training data), its privacy preference region can be represented as w x h x c, where w represents the width of the region, h represents the height of the region, and c represents the number of channels of the region.

[0156] ||·|| in the above formula F represents the L2 norm operation on the matrix elements; through this operation, the privacy sensitive region PSI can be located in the response region of the Jacobian matrix. The response region can be understood as the matrix parameter region in the Jacobian matrix corresponding to the privacy sensitive region.

[0157] In one or more embodiments of the present specification, the lth output layer of the network (image processing model) and the input image (i.e. image training data) are consistent in length and width dimensions, so the region positioning can be directly completed by means of the above formula (2). If the intermediate output layer of the network used and the input image size are inconsistent, size alignment can be considered by means of upsampling or downsampling before performing the operation.

[0158] (3), different clients calculate the gradient data corresponding to each network layer by the following formula (3), and the sensitivity of the privacy sensitive region, so as to obtain the sensitivity S of each network layer l :

[0159] Specifically, the client can determine the privacy sensitive region, and the response region of the Jacobian matrix of a certain network layer

[0160] Then, the matrix parameters (i.e. partial derivatives) in the response region are averaged to obtain the PSI score S of a certain network layer l . The scalar value reflects the quantization result of the privacy preference region in the local data set of a particular client.

[0161]

[0162] By performing the above operation on each network layer, the PSI score S of each network layer is obtained l .

[0163] Step 310: privacy budget allocation.

[0164] Specifically, according to the sensitivity score S of the gradient data corresponding to each network layer l , the client can allocate different privacy budgets (ε l ) to each network layer, which can be seen from the following formula (4).

[0165]

[0166] wherein ∈ is a hyperparameter, and represents the overall privacy budget for performing local differential privacy.

[0167] Generally, the ∈ l The smaller the value, the greater the noise intensity of differential privacy noise, the stronger the privacy protection strength, and the data availability will also decrease; on the contrary, the ∈ l The greater the value, the smaller the noise intensity of differential privacy noise, the weaker the privacy protection strength, and the data availability will also increase.

[0168] Step 312: Gradient clipping and noise injection.

[0169] First, the client clips the gradient, which converts some data points with large numerical values to data points with numerical values conforming to the average size, thereby limiting the maximum influence of a single data point on model updates.

[0170] Second, according to the ∈ l , inject Gaussian noise into the clipped gradient data to meet the allocated privacy budget and achieve LDP. The calculation formulas (5) and (6) of the step are as follows:

[0171]

[0172]

[0173] In the above formula, C represents the gradient boundary, which is a manually set hyperparameter. During the clipping of the gradient by the client, C can be used to convert some data points with large numerical values to data points with numerical values conforming to the average size, thereby limiting the maximum influence of a single data point on model updates.

[0174] In the above formula, N() represents a Gaussian distribution; N(mean, variance). The meaning of this formula indicates that a variance with a mean of 0 and a variance of (C^2*σ l ^2) is added to the original gradient. This variance is calculated according to the set C and a subsequent allocation, so the calculation results of this variance for different layers will be different to achieve the effect of adaptive allocation.

[0175] In the above formula, The δ in the formula is a hyperparameter, which reflects the random failure probability of the differential privacy algorithm, T represents the communication round, and I represents the unit matrix. Through this value, the layered adaptive noise injection algorithm in this specification satisfies (ε, δ)-DP. Thus, the privacy protection strength of the algorithm proposed in this paper is theoretically guaranteed.

[0176] wherein, is the gradient data of the lth layer after noise injection; is the gradient data of the model after overall noise injection.

[0177] Step 314: uploading the perturbed gradient.

[0178] Specifically, the client sends the gradient data perturbed by noise to the server.

[0179] Based on the above steps, the image processing model training method provides a federated learning personalized differential privacy protection method based on protection preference analysis, aiming to solve the privacy leakage problem existing in joint learning; this method protects the privacy of the gradient uploaded by the client through a layer-by-layer local differential privacy mechanism. Specifically, first, the privacy sensitive information is defined, second, the sensitivity of the gradient to the PSI region is calculated, and finally, different privacy budgets are allocated to each network layer to achieve strict privacy protection of PSI. Through the layer-by-layer local differential privacy mechanism, different privacy budgets are allocated to each network layer according to the sensitivity information preference of the local client, reducing the noise interference on the information that is not sensitive to privacy, thereby achieving personalized differential privacy protection for clients with different preferences, and ultimately achieving an effective balance between privacy protection and model accuracy in the federated solution.

[0180] Moreover, the adaptive privacy allocation algorithm is realized by allocating different privacy budgets to each network layer, which avoids excessive expert experience and model customization, and is a more general and more accurate local privacy protection federated learning method.

[0181] Referring to Figure 4 , Figure 4 A flowchart of another image processing model training method provided according to an embodiment of the present specification is shown, which is applied to a second training unit and specifically includes the following steps.

[0182] Step 402: sending an image processing model to a plurality of first training units.

[0183] Take the application of another image processing model training method in the present specification in the federated learning personalized differential privacy protection scenario as an example to explain and describe another image processing model training method, wherein the first training unit is the client of federated learning, the second training unit is the server of federated learning, the desensitization update data is desensitization gradient data, and the sensitive image data is privacy sensitive information (Privacy Sensitive Information, PSI).

[0184] Based on this, the server provided in the present specification can initialize a global model and distribute it to all participating clients.

[0185] Step 404: receiving, by the server, desensitization update data of each network layer in the image processing model sent by each first training unit, wherein the desensitization update data is obtained by desensitizing update data of the first training unit on the network layer, and the update data is obtained by the first training unit by using image training data to train the image processing model.

[0186] In the above example, the server can collect desensitization gradient data of all clients. The specific determination method of the desensitization gradient data can refer to the above embodiments.

[0187] Step 406: using sensitive image data in the second training unit to perform data sensitive detection on each desensitization update data, and determining data sensitive information corresponding to each desensitization update data.

[0188] In one or more embodiments provided in the specification, the desensitization update data is desensitization gradient data.

[0189] The use of sensitive image data in the second training unit to perform data sensitive detection on each desensitization update data, and determine the data sensitive information corresponding to each desensitization update data, includes:

[0190] Determine the sensitive detection image data in the second training unit, and determine the sensitive image data from the sensitive detection image data;

[0191] Using the sensitive image data and the partial derivative matrix corresponding to each desensitization gradient data, the data sensitive detection is performed on each desensitization gradient data, and the data sensitive information corresponding to each desensitization gradient data is determined.

[0192] The above steps can refer to the embodiments of determining data sensitive information in the above image processing model training method, which will not be repeated here.

[0193] In one or more embodiments provided in the specification, using the sensitive image data and the partial derivative matrix corresponding to each desensitization gradient data, the data sensitive detection is performed on each desensitization gradient data, and the data sensitive information corresponding to each desensitization gradient data is determined, including:

[0194] Calculating the partial derivative of each desensitization gradient data with respect to the sensitive detection image data, and constructing the partial derivative matrix corresponding to each desensitization gradient data based on the partial derivative;

[0195] Determine the data region information of the sensitive image data, and determine the matrix parameter of the sensitive image data from the partial derivative matrix based on the data region information;

[0196] Based on the matrix parameter, the data sensitive information corresponding to each desensitization gradient data is calculated.

[0197] The above steps can refer to the embodiments of determining data sensitive information in the above image processing model training method, which will not be repeated here.

[0198] In the above example, in order to solve the problems of insufficient privacy protection and performance degradation in the existing joint modeling scheme of privacy image, the method proposes a federated learning personalized differential privacy protection method based on protection preference analysis. The method optimizes the gradient aggregation process of the global model through the parameter aggregation mechanism, reduces the noise introduced by local gradient perturbation, and improves the model performance. That is to say, the method proposes a parameter aggregation mechanism based on perturbation information distribution in the gradient aggregation step. The specific implementation is as follows:

[0199] The server collects all the gradient data of the clients (i.e. desensitization gradient data), and evaluates the sensitivity of the gradient data using a public data set. The sensitivity can be determined by the following formula (7).

[0200]

[0201] Wherein, PSI represents the calculation of the PSI sensitivity in the above embodiment, that is, the central server quantifies the privacy sensitive area information of the gradient matrix uploaded by different clients by means of shareable open source data, to complete the evaluation of the perturbation of the gradient uploaded by different clients.

[0202] Step 408: Based on the data sensitive information, determine the target desensitization update data from the desensitization update data.

[0203] In one or more embodiments provided in the specification, the data sensitive information is data sensitivity;

[0204] The data sensitive information based on the data sensitive information, the target desensitization update data is determined from the desensitization update data, including:

[0205] Determine the data sensitivity of the desensitization update data, and determine the desensitization update data with the data sensitivity less than or equal to the preset score threshold as the target desensitization update data.

[0206] Specifically, the method can weight and aggregate the gradient according to the sensitivity of the gradient, and preferentially aggregate the gradient with lower sensitivity in the local gradient. The specific determination is made by the following formula (8). The method preferentially aggregates the gradient with lower sensitivity, and reduces the influence of noise on the performance of the global model.

[0207]

[0208] Step 410: updating parameters of the network layers in the image processing model using the target desensitization update data to obtain an updated image processing model.

[0209] Continuing with the above example, the server updates the global model parameters using gradient plasticity and evaluates the model performance.

[0210] In one or more embodiments provided in the specification, after the image processing model is updated using the target desensitization update data, the method further comprises:

[0211] If the updated image processing model meets the model training stop condition, the updated image processing model is determined as the trained image processing model.

[0212] If the updated image processing model does not meet the model training stop condition, the updated image processing model is used as the image processing model to be trained, and the step of sending the image processing model to the plurality of first training units is continued until the updated image processing model meets the model training stop condition.

[0213] Continuing with the above example, the server sends the updated model parameters back to all clients for the next round of training. The steps of model training are repeated until the model reaches the predetermined performance standard or completes the set training period.

[0214] The image processing model training method applied to the second training unit in one or more embodiments of the specification can send the image processing model to a plurality of first training units in the process of distributed machine learning.

[0215] Each first training unit trains the image processing model using local image training data to obtain update data for each network layer in the image processing model. In the process of sending the update data to the second training unit for parameter updating, in order to avoid the risk that the update data may leak sensitive data, it is necessary to detect the data sensitivity of each update data according to the sensitive image data in the image training data, so as to determine the data sensitivity information corresponding to each update data, and then desensitize each update data using the data sensitivity information, and then send the desensitized update data to the second training unit.

[0216] After receiving the desensitization update data sent by the first training unit, since the desensitization update data contains interference information due to data desensitization processing, in order to avoid the interference information from affecting the performance of the image processing model, the data sensitivity of each desensitization update data can be detected by using the sensitive image data in the second training unit, the data sensitivity information corresponding to each desensitization update data is determined, and the target desensitization update data with lower sensitivity is determined from each desensitization update data according to the data sensitivity information. Then, the parameters of each network layer in the image processing model are updated by using a smaller number of target desensitization update data, thereby avoiding excessive interference information from affecting the performance of the image processing model and improving the processing efficiency of the image processing model.

[0217] The above is a schematic scheme of another image processing model training method of the embodiment. It should be noted that the technical scheme of the another image processing model training method belongs to the same concept as the technical scheme of the above-mentioned image processing model training method. The details of the technical scheme of the another image processing model training method that are not described in detail can be referred to the description of the technical scheme of the above-mentioned image processing model training method.

[0218] The following describes the another image processing model training method in combination with the accompanying Figure 5 The another image processing model training method provided in the specification is taken as an example to further illustrate the another image processing model training method in the application of federated learning personalized differential privacy protection scene. Among them, Figure 5 Fig. 1 shows a processing process flowchart of the another image processing model training method provided by an embodiment of the specification. The another image processing model training method is applied to a server of a federated learning system, and specifically includes the following steps.

[0219] Step 502: Gradient aggregation is performed on the desensitization gradient data sent by each client.

[0220] Specifically, the method proposes a parameter aggregation mechanism based on perturbation information distribution in the gradient aggregation step. In specific practice, the server collects the gradient data of all clients, and performs weighted aggregation according to the sensitivity of the gradient data, preferentially aggregates the gradients with lower sensitivity in the local gradient, and updates the global model. The specific execution steps are as follows:

[0221] (1) The mechanism uses public data sets to evaluate the sensitivity of the gradient. The specific sensitivity The sensitivity can be determined by the following formula (7).

[0222]

[0223] In the above formula, PSI represents the calculation process of the sensitivity of the client, that is, the central server quantifies the privacy-sensitive area information of the gradient matrix uploaded by different clients by means of shareable open source data, so as to evaluate the perturbation of the gradient uploaded by different clients.

[0224] It should be noted that the content of determining the sensitivity in the above formula (7) can refer to the step of calculating the sensitivity of the gradient to the PSI in the above client, which will not be repeated here.

[0225] (2) Preferentially aggregate the gradient with lower sensitivity to reduce the influence of noise on the performance of the global model.

[0226] Specifically, the local mechanism weights the gradient data according to the sensitivity of the gradient data; for example, the smaller the sensitivity, the greater the weight.

[0227] Then, the gradient is aggregated according to the weight corresponding to the gradient data, and the gradient with lower sensitivity in the local gradient is preferentially aggregated, which can be realized by the following formula (8).

[0228]

[0229] In the above formula, is the weighted gradient data.

[0230] Step 504: Update the global model.

[0231] Specifically, the server can update the global model parameters by using the gradient with lower sensitivity, and evaluate the model performance.

[0232] Step 506: Distribute the model parameters.

[0233] Specifically, the server sends the updated model parameters back to all clients for the next round of training

[0234] Step 508: Iterative training.

[0235] Specifically, steps 502 to 506 are repeated until the model reaches the predetermined performance standard or completes the set training period.

[0236] Based on the above steps, the image processing model training method provides a federated learning personalized differential privacy protection method based on protection preference analysis, aiming to solve the privacy leakage problem existing in joint learning. The method introduces a novel parameter aggregation mechanism that evaluates the sensitivity of the gradient using public datasets, prioritizes the aggregation of gradients with lower sensitivity in the local gradient, and reduces the noise introduced by the local perturbed gradient, thereby maintaining the effectiveness of the global model. Superior performance is achieved in terms of privacy and accuracy.

[0237] Corresponding to the method embodiments described above, the specification also provides image processing model training system embodiments, Figure 6 A structural schematic diagram of an image processing model training system provided by an embodiment of the specification is shown. As shown in the figure, Figure 6 The system server 602 and a plurality of clients 604 are shown, wherein,

[0238] The server 602 is configured to send an image processing model to the plurality of clients 604, receive desensitization update data for each network layer in the image processing model sent by each client 604, wherein the desensitization update data is obtained by desensitizing the update data for each network layer by the client 604, the update data is obtained by training the image processing model using image training data by the client 604, sensitive image data in the server 602 is used to detect the data sensitivity of each desensitization update data, determine the data sensitivity information corresponding to each desensitization update data, based on the data sensitivity information, determine the target desensitization update data from the desensitization update data, use the target desensitization update data to update the parameters of the image processing model, and obtain an updated image processing model.

[0239] The client 604 is configured to receive the image processing model sent by the server 602, and train the image processing model using the image training data to obtain the update data for each network layer in the image processing model, wherein the update data is used to update the parameters of each network layer, the sensitive image data in the image training data is used to detect the data sensitivity of each update data, the data sensitivity information corresponding to each update data is determined, the data sensitivity information is used to desensitize each update data, the desensitization update data for each network layer is obtained, and the desensitization update data for each network layer is sent to the server 602 for parameter update.

[0240] Wherein, the server 602 can be understood as the second training unit in the image processing model training; the client 604 can be understood as the first training unit in the image processing model training.

[0241] One or more embodiments of the specification provide an image processing model training system comprising a server and a plurality of clients. In the process of distributed machine learning, the server can send an image processing model to the plurality of clients;

[0242] Each client trains the image processing model using local image training data to obtain update data of each network layer in the image processing model. In the process of sending the update data to the server for parameter updating, in order to avoid the risk that the update data may leak sensitive data, data sensitive detection needs to be performed on each update data according to sensitive image data in the image training data, so as to determine the data sensitive information corresponding to each update data, and then data desensitization is performed on each update data using the data sensitive information, and then the desensitized update data is sent to the server;

[0243] After receiving the desensitized update data, the server needs to perform data sensitive detection on each desensitized update data using sensitive image data in the server to determine the data sensitive information corresponding to each desensitized update data, and then determine target desensitized update data with lower sensitivity from each desensitized update data according to the data sensitive information, and then use the target desensitized update data with less quantity to update the parameters of each network layer in the image processing model, so as to avoid the influence of excessive interference information on the performance of the image processing model, and improve the processing efficiency of the image processing model.

[0244] The above is a schematic scheme of an image processing model training system according to the embodiment. It should be noted that the technical scheme of the image processing model training system belongs to the same concept as the technical scheme of any one of the above image processing model training methods, and the details of the technical scheme of the image processing model training system that are not described in detail can be referred to the description of the technical scheme of any one of the above image processing model training methods.

[0245] Corresponding to the above method embodiments, the specification also provides an image processing model training device embodiment. The device is applied to a first training unit and comprises:

[0246] The model training module is configured to receive an image processing model sent by a second training unit, and train the image processing model using image training data to obtain update data of each network layer in the image processing model, wherein the update data is used to update the parameters of the network layers.

[0247] a data sensitive detection module configured to perform data sensitive detection on each update data according to sensitive image data in the image training data, and determine data sensitive information corresponding to the each update data;

[0248] a data desensitization module configured to perform data desensitization on the each update data by using the data sensitive information, and obtain desensitized update data of the each network layer;

[0249] a data sending module configured to send the desensitized update data of the each network layer to the second training unit for parameter update.

[0250] Optionally, the update data is gradient data.

[0251] The data sensitive detection module is further configured to:

[0252] determine sensitive image data contained in the image training data;

[0253] perform data sensitive detection on each gradient data by using the sensitive image data and a partial derivative matrix corresponding to the each gradient data, and determine the data sensitive information corresponding to the each gradient data.

[0254] Optionally, the data sensitive detection module is further configured to:

[0255] calculate a partial derivative of the each gradient data with respect to the image training data, and construct the partial derivative matrix corresponding to the each gradient data based on the partial derivative;

[0256] determine data region information of the sensitive image data, and determine a matrix parameter of the sensitive image data from the partial derivative matrix based on the data region information;

[0257] calculate the data sensitive information corresponding to the each gradient data based on the matrix parameter.

[0258] Optionally, the data sensitive information is data sensitivity.

[0259] The data desensitization module is further configured to:

[0260] calculate data desensitization parameters corresponding to the each update data according to the data sensitivity of the each update data, wherein the data desensitization parameters of the each update data are the same or different;

[0261] perform data desensitization on the each update data based on the data desensitization parameters corresponding to the each update data, and obtain the desensitized update data of the each network layer.

[0262] Optionally, the data desensitization module is further configured to:

[0263] determine a target network layer in the network layers, and determine a target data desensitization parameter corresponding to the target network layer, wherein the target network layer is any one of the network layers;

[0264] determine noise intensity information corresponding to the target data desensitization parameter, and perform data noise addition on the update data of the target network layer by using the noise intensity information, to obtain the desensitized update data.

[0265] In one or more embodiments of the present specification, the image processing model training device applied to the first training unit can receive the image processing model sent by the second training unit in the process of distributed machine learning, and train the image processing model by using the local image training data to obtain the update data of each network layer in the image processing model. In the process of sending the update data to the second training unit for parameter updating, in order to avoid the risk that the update data may leak sensitive data, the data sensitivity of each update data needs to be detected according to the sensitive image data in the image training data, so as to determine the data sensitivity information corresponding to each update data, and then the data desensitization is performed on each update data by using the data sensitivity information, and then the desensitized update data is sent to the second training unit, thereby avoiding the problem that sensitive private information may be exposed in the process of distributed machine learning, ensuring the data security, and reducing the risk of sensitive data leakage.

[0266] The above is a schematic scheme of the image processing model training device of the present embodiment. It should be noted that the technical scheme of the image processing model training device belongs to the same concept as the technical scheme of the above-mentioned image processing model training method, and the details of the technical scheme of the image processing model training device which are not described in detail can be referred to the description of the technical scheme of the above-mentioned image processing model training method.

[0267] Corresponding to the above method embodiment, the present specification also provides another image processing model training device embodiment, which is applied to the second training unit and includes:

[0268] The model sending module is configured to send the image processing model to a plurality of first training units.

[0269] The data receiving module is configured to receive the desensitized update data of each network layer in the image processing model sent by each first training unit, wherein the desensitized update data is obtained by the first training unit performing data desensitization on the update data of each network layer, and the update data is obtained by the first training unit training the image processing model by using image training data;

[0270] The data sensitive detection module is configured to perform data sensitive detection on each desensitization update data by using sensitive image data in the second training unit, and determine data sensitive information corresponding to each desensitization update data.

[0271] The data determination module is configured to determine target desensitization update data from the desensitization update data based on the data sensitive information.

[0272] The parameter update module is configured to perform parameter update on the network layers in the image processing model by using the target desensitization update data, and obtain an updated image processing model.

[0273] Optionally, the desensitization update data is desensitization gradient data.

[0274] The data sensitive detection module is further configured to:

[0275] determine sensitive detection image data in the second training unit, and determine the sensitive image data from the sensitive detection image data;

[0276] perform data sensitive detection on the desensitization gradient data by using the sensitive image data and a partial derivative matrix corresponding to each desensitization gradient data, and determine the data sensitive information corresponding to each desensitization gradient data.

[0277] Optionally, the data sensitive detection module is further configured to:

[0278] calculate a partial derivative of each desensitization gradient data with respect to the sensitive detection image data, and construct the partial derivative matrix corresponding to each desensitization gradient data based on the partial derivative;

[0279] determine data region information of the sensitive image data, and determine a matrix parameter of the sensitive image data from the partial derivative matrix based on the data region information;

[0280] calculate the data sensitive information corresponding to each desensitization gradient data based on the matrix parameter.

[0281] Optionally, the data sensitive information is data sensitivity.

[0282] The data determination module is further configured to:

[0283] determine the data sensitivity of each desensitization update data, and determine the desensitization update data with a data sensitivity less than or equal to a preset score threshold as the target desensitization update data.

[0284] The other image processing model training device further includes a model processing module configured to:

[0285] In a case where it is determined that the updated image processing model meets the model training stop condition, the updated image processing model is determined as the image processing model whose training is completed.

[0286] In a case where it is determined that the updated image processing model does not meet the model training stop condition, the updated image processing model is taken as the image processing model to be trained, and the step of sending the image processing model to the plurality of first training units is continuously performed until the updated image processing model meets the model training stop condition.

[0287] The image processing model training apparatus applied to the second training unit in one or more embodiments of the present specification can send the image processing model to the plurality of first training units in the process of distributed machine learning.

[0288] Each first training unit trains the image processing model by using local image training data to obtain updated data of each network layer in the image processing model. In the process of sending the updated data to the second training unit for parameter updating, in order to avoid the risk that the updated data may leak sensitive data, data sensitive detection needs to be performed on each updated data according to sensitive image data in the image training data, so as to determine data sensitive information corresponding to each updated data, and then data desensitization is performed on each updated data by using the data sensitive information, and then the desensitized updated data is sent to the second training unit.

[0289] After receiving the desensitized updated data, the second training unit needs to perform data sensitive detection on each desensitized updated data by using sensitive image data in the second training unit, determine data sensitive information corresponding to each desensitized updated data, and then determine target desensitized updated data with lower sensitivity from each desensitized updated data according to the data sensitive information, and then update parameters of each network layer in the image processing model by using a smaller number of target desensitized updated data, so as to avoid the influence of excessive interference information on the performance of the image processing model, and improve the processing efficiency of the image processing model.

[0290] The above is a schematic scheme of another image processing model training apparatus of the present embodiment. It should be noted that the technical scheme of the another image processing model training apparatus belongs to the same concept as the technical scheme of the another image processing model training method described above, and the details of the technical scheme of the another image processing model training apparatus which are not described in detail can be referred to the description of the technical scheme of the another image processing model training method.

[0291] Figure 7A structural block diagram of a computing device 700 is shown, according to one embodiment of the present specification. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected with the memory 710 through a bus 730, and a database 750 is used to save data.

[0292] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include the public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 can include one or more of any type of network interface (e.g., a network interface card (NIC)), wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, Near Field Communication (NFC).

[0293] In one embodiment of the present specification, the above-mentioned components of the computing device 700 and other components not shown in the Figure 7 may be connected with each other, for example, through a bus. It should be understood that Figure 7 The structural block diagram of the computing device shown is only for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced as needed by those skilled in the art.

[0294] The computing device 700 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 700 can also be a mobile or stationary server.

[0295] The processor 720 is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of any of the image processing model training methods described above.

[0296] Each of the embodiments described in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, the computing device embodiments are described simply because they are substantially similar to any of the image processing model training methods. The relevant parts can be referred to the description of any of the image processing model training methods.

[0297] An embodiment of the specification also provides a computer-readable storage medium storing computer programs / instructions, which, when executed by a processor, implement the steps of any of the image processing model training methods described above.

[0298] Each of the embodiments described in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, the computer-readable storage medium embodiments are described simply because they are substantially similar to any of the image processing model training methods. The relevant parts can be referred to the description of any of the image processing model training methods.

[0299] An embodiment of the specification also provides a computer program product including computer programs / instructions, which, when executed by a processor, implement the steps of any of the image processing model training methods described above.

[0300] The above is a schematic scheme of a computer program product of an embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of any of the image processing model training methods described above belong to the same concept. The technical scheme of the computer program product is not described in detail, and can be referred to the description of the technical scheme of any of the image processing model training methods.

[0301] The above describes particular embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than those described in the embodiments, and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or possible.

[0302] The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or subtractions according to the requirements of patent practice. For example, in some regions, according to patent practice, computer readable medium does not include electrical carrier signals and telecommunication signals.

[0303] It should be noted that for the foregoing method embodiments, in order to facilitate description, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the embodiments of the present specification are not limited by the order of the described actions, because according to the embodiments of the present specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary for the embodiments of the present specification.

[0304] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0305] The preferred embodiments of the present specification disclosed above are only used to help explain the present specification. The alternative embodiments do not describe all the details and do not limit the invention to the specific embodiments described. Obviously, according to the content of the embodiments of the present specification, many modifications and changes can be made. The present specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present specification, so that those skilled in the art can well understand and use the present specification. The present specification is limited only by the claims and their full scope and equivalents.

Claims

1. An image processing model training method applied to a first training unit, comprising: receiving an image processing model sent by a second training unit, and performing model training on the image processing model by using image training data to obtain update data of each network layer in the image processing model, wherein the update data is used to update parameters of the each network layer; performing data sensitivity detection on each update data according to sensitive image data in the image training data to determine data sensitivity information corresponding to each update data, comprising: calculating partial derivatives of each gradient data with respect to the image training data, and constructing a partial derivative matrix corresponding to each gradient data based on the partial derivatives; determining data region information of the sensitive image data, and determining matrix parameters of the sensitive image data from the partial derivative matrix based on the data region information; and calculating the data sensitivity information corresponding to each gradient data based on the matrix parameters, wherein the update data is gradient data; performing data desensitization on each update data by using the data sensitivity information to obtain desensitized update data of the each network layer; sending the desensitized update data of the each network layer to the second training unit for parameter updating.

2. The image processing model training method of claim 1, wherein the data sensitivity information is data sensitivity. The data desensitization on each update data by using the data sensitivity information to obtain desensitized update data of the each network layer, comprising: calculating data desensitization parameters corresponding to each update data according to the data sensitivity of each update data, wherein the data desensitization parameters of each update data are the same or different; performing data desensitization on each update data based on the data desensitization parameters corresponding to each update data to obtain the desensitized update data of the each network layer.

3. The image processing model training method of claim 2, wherein the data desensitization on each update data based on the data desensitization parameters corresponding to each update data to obtain the desensitized update data of the each network layer, comprising: determining a target network layer in the each network layer, and determining a target data desensitization parameter corresponding to the target network layer, wherein the target network layer is any one of the each network layer; determining noise intensity information corresponding to the target data desensitization parameter, and performing data noise addition on the update data of the target network layer by using the noise intensity information to obtain the desensitized update data.

4. An image processing model training method applied to a second training unit, comprising: sending an image processing model to a plurality of first training units; receiving desensitized update data of each network layer in the image processing model sent by each first training unit, wherein the desensitized update data is obtained by performing data desensitization on update data of the each network layer by the first training unit, and the update data is obtained by performing model training on the image processing model by the first training unit by using image training data; The data sensitive detection is performed on each desensitization update data by using the sensitive image data in the second training unit, the data sensitive information corresponding to each desensitization update data is determined, including: calculating the partial derivative of each desensitization gradient data with respect to the sensitive detection image data, and constructing the partial derivative matrix corresponding to each desensitization gradient data based on the partial derivative; determining the data region information of the sensitive image data, and determining the matrix parameter of the sensitive image data from the partial derivative matrix based on the data region information; calculating the data sensitive information corresponding to each desensitization gradient data based on the matrix parameter, wherein the desensitization update data is desensitization gradient data; Based on the data sensitive information, the target desensitization update data is determined from the desensitization update data; The parameter of the image processing model in the network layer is updated by using the target desensitization update data, and an updated image processing model is obtained.

5. The image processing model training method of claim 4, wherein the data sensitive information is data sensitivity. The target desensitization update data is determined from the desensitization update data based on the data sensitive information, including: The data sensitivity of each desensitization update data is determined, and the desensitization update data with a data sensitivity less than or equal to a preset score threshold is determined as the target desensitization update data.

6. The image processing model training method of claim 4, wherein after the parameter of the image processing model in the network layer is updated by using the target desensitization update data, an updated image processing model is obtained, the method further comprises: In a case where it is determined that the updated image processing model satisfies a model training stop condition, the updated image processing model is determined as a trained image processing model; In a case where it is determined that the updated image processing model does not satisfy the model training stop condition, the updated image processing model is taken as an image processing model to be trained, and the step of sending the image processing model to the plurality of first training units is continued to be performed until the updated image processing model satisfies the model training stop condition.

7. An image processing model training system, comprising a server and a plurality of clients, wherein The server is configured to send the image processing model to the plurality of clients, receive desensitization update data of each network layer in the image processing model sent by each client, wherein the desensitization update data is obtained by desensitizing the update data of each network layer by the client, the update data is obtained by training the image processing model by the client using image training data, sensitive image data in the server is used to detect the data sensitivity of each desensitization update data, determine the data sensitive information corresponding to each desensitization update data, determine the target desensitization update data from each desensitization update data based on the data sensitive information, update the parameters of each network layer in the image processing model by using the target desensitization update data, and obtain an updated image processing model, wherein the sensitive image data in the server is used to detect the data sensitivity of each desensitization update data, determine the data sensitive information corresponding to each desensitization update data, comprising: calculating the partial derivative of each desensitization gradient data with respect to the sensitive detection image data, and constructing the partial derivative matrix corresponding to each desensitization gradient data based on the partial derivative; determine the matrix parameter of the sensitive image data from the partial derivative matrix based on the data region information of the sensitive image data; based on the matrix parameter, calculate the data sensitive information corresponding to each desensitization gradient data, and the desensitization update data is desensitization gradient data. The client is configured to receive the image processing model sent by the server, and train the image processing model by using the image training data, to obtain the update data of each network layer in the image processing model, wherein the update data is used to update the parameters of each network layer, the sensitive image data in the image training data is used to detect the data sensitivity of each update data, determine the data sensitive information corresponding to each update data, desensitize each update data by using the data sensitive information, obtain the desensitization update data of each network layer, and send the desensitization update data of each network layer to the server for parameter update, wherein the sensitive image data in the image training data is used to detect the data sensitivity of each update data, determine the data sensitive information corresponding to each update data, comprising: calculating the partial derivative of each gradient data with respect to the image training data, and constructing the partial derivative matrix corresponding to each gradient data based on the partial derivative; determine the matrix parameter of the sensitive image data from the partial derivative matrix based on the data region information of the sensitive image data; based on the matrix parameter, calculate the data sensitive information corresponding to each gradient data, and the update data is gradient data.

8. A computing device comprising: a memory and a processor; The memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, and the computer programs / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 6. 9.A computer readable storage medium storing computer programs / instructions, and the computer programs / instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6. 10.A computer program product comprising computer programs / instructions, and the computer programs / instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.

Images