Method, system and apparatus for processing images

Abstract

The present disclosure relates to a processing method, system and device of an image, and relates to the technical field of computer. The processing method comprises: a client performing client perturbation on input features of a first inference model to generate first perturbation data, wherein the input features are generated according to an image to be processed; a server calculating a first result by using a second inference model according to the first perturbation data, and performing first server perturbation on the first result to generate second perturbation data, wherein the first inference model and the second inference model have the same structure; and the client calculating a processing result of the image to be processed by using the first inference model according to the second perturbation data in which the client perturbation is eliminated. The technical scheme of the present disclosure can improve inference efficiency and reduce construction difficulty.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image processing method, an image processing system, an image processing apparatus, and a non-volatile computer-readable storage medium. Background Technology

[0002] Deep privacy-preserving inference refers to privacy-preserving inference for deep learning, which is applicable when users do not want to disclose their private data, and the model does not want to disclose its model weight data.

[0003] Among related technologies, semi-honest deep privacy-preserving inference frameworks include Delphi and Gazelle. Delphi is a semi-honest privacy-preserving inference framework for convolutional neural networks that uses cryptographic techniques such as obfuscated circuits and NAS (Neural Architecture Search) deep network framework automatic learning techniques. Summary of the Invention

[0004] The inventors of this disclosure have discovered the following problems in the aforementioned related technologies: the extensive use of encryption technologies such as obfuscated circuits results in insufficient processing speed and low inference efficiency in industrial applications; moreover, the industrial construction of NAS frameworks is quite difficult.

[0005] In view of this, this disclosure proposes an image processing technology solution that can improve reasoning efficiency and reduce the difficulty of construction.

[0006] According to some embodiments of this disclosure, an image processing method is provided, comprising: a client perturbing the input features of a first inference model to generate first perturbation data, wherein the input features are generated based on the image to be processed; a server calculating a first result using a second inference model based on the first perturbation data, and performing a first server perturbation on the first result to generate second perturbation data, wherein the first inference model and the second inference model have the same structure; and the client calculating the processing result of the image to be processed using the first inference model based on the second perturbation data after the client perturbation has been eliminated.

[0007] In some embodiments, generating the first perturbation data includes: the client generating a first random number; the client using the first random number to perturb the input features to generate the first perturbation data; generating the second perturbation data includes: the server generating a second random number; the server using the second random number to perturb the first result to generate the second perturbation data.

[0008] In some embodiments, generating the first perturbation data includes: the client perturbs the input features of the current layer of the first inference model to generate the first perturbation data; generating the second perturbation data includes: the server calculates a first result based on the first perturbation data using the current layer of the second inference model; the server perturbs the first result to generate the second perturbation data; calculating the processing result of the image to be processed includes: the client calculates a second result based on the second perturbation data after client perturbation has been eliminated using the current layer of the first inference model; the client and server repeat the above steps until the client calculates the processing result of the image to be processed using the last layer of the first inference model.

[0009] In some embodiments, calculating the first result using the current layer of the second inference model includes: if the current layer of the second inference model is not the first layer, the server eliminates the second server-side perturbation of the input layer of the current layer of the second inference model in the first perturbation data; the server calculates the first result using the current layer of the second inference model based on the first perturbation data after eliminating the second server-side perturbation.

[0010] In some embodiments, the current layer of the first inference model is a linear layer, and generating the first perturbation data includes: the client dividing the input features into multiple sub-features; the client generating a first random number for each sub-feature; and the client perturbing the input features by multiplying each sub-feature by its corresponding first random number to generate the first perturbation data.

[0011] In some embodiments, the current layer of the first inference model is a linear layer, and generating the first perturbation data includes: the client generating a second random number for each sub-feature; the client perturbing the input features by multiplying each sub-feature by its corresponding first random number and then adding it to its corresponding second random number to generate the first perturbation data.

[0012] In some embodiments, the sum of the first random numbers is 0.

[0013] In some embodiments, the current layer of the second inference model is a linear layer, a nonlinear layer, or a nonparametric layer, and generating the second perturbation data includes: the server generating a third random number; the server using the product of the first result and the third random number to perform a first server perturbation on the first result to generate the second perturbation data.

[0014] In some embodiments, the current layer of the first inference model is a nonlinear layer or a nonparametric layer, and generating the first perturbation data includes: the client generating a fourth random number; the client perturbing the input features by multiplying the input features by the fourth random number, thereby generating the first perturbation data.

[0015] In some embodiments, when the current layer of the first inference model is a non-linear layer, the fourth random number is a positive number.

[0016] In some embodiments, the current layer of the first inference model is a softmax layer. Generating the first perturbation data includes: the client generating a public key and its corresponding private key; the client using the public key to homomorphically encrypt the first product of the input feature and the fourth random number to generate the first perturbation data; homomorphically encrypting the fourth random number; generating the second perturbation data includes: the server generating a third random number; the server using the public key to homomorphically add the second product of the encrypted fourth random number and the third random number and the first perturbation data to generate the second perturbation data; calculating the processing result of the image to be processed includes: the client using the private key to homomorphically decrypt the second perturbation data and then eliminating the client perturbation.

[0017] In some embodiments, homomorphically adding the second product of the encrypted fourth random number and the third random number and the first perturbation data includes: when the current layer of the second inference model is not the first layer, the server eliminates the second server perturbation of the input layer of the current layer of the second inference model in the first perturbation data; and homomorphically adding the second product of the encrypted fourth random number and the third random number and the first perturbation data after eliminating the second server perturbation.

[0018] In some embodiments, the processing method further includes: the client sending an inference request to the server, the inference request including relevant information of the first inference model; the server sending structural information of the second inference model to the client based on the relevant information; the client perturbing the input features of the first inference model includes: the client determining the input features of the current layer of the first inference model based on the structural information.

[0019] According to some other embodiments of this disclosure, an image processing method is provided, comprising: performing client-side perturbation on input features of a first inference model to generate first perturbation data, wherein the input features are generated based on an image to be processed; calculating a processing result of the image to be processed using the first inference model based on second perturbation data with client-side perturbation eliminated, wherein the second perturbation data is generated by the server performing a first server-side perturbation on the first result, and the first result is calculated by the server using the second inference model based on the first perturbation data, wherein the first inference model and the second inference model have the same structure.

[0020] In some embodiments, the second perturbation data is generated by the server using a second random number it generates to perform the first server-side perturbation on the first result. Generating the first perturbation data includes: generating a first random number; and using the first random number to perform client-side perturbation on the input features to generate the first perturbation data.

[0021] In some embodiments, the first result is calculated by the server using the current layer of the second inference model based on the first perturbation data. The second perturbation data is generated by the server performing a first server-side perturbation on the first result. Generating the first perturbation data includes: performing client-side perturbation on the input features of the current layer of the first inference model to generate the first perturbation data. Calculating the processing result of the image to be processed includes: calculating the second result using the current layer of the first inference model based on the second perturbation data after eliminating client-side perturbation. The steps of generating the first perturbation data and calculating the processing result of the image to be processed are repeated until the processing result of the image to be processed is calculated using the last layer of the first inference model.

[0022] In some embodiments, the current layer of the first inference model is a linear layer, and generating the first perturbation data includes: dividing the input features into multiple sub-features; generating a first random number for each sub-feature; and using the product of each sub-feature and its corresponding first random number to perform client-side perturbation on the input features to generate the first perturbation data.

[0023] In some embodiments, the current layer of the first inference model is a linear layer, and generating the first perturbation data includes: generating a second random number for each sub-feature; and performing client-side perturbation on the input features by multiplying each sub-feature by its corresponding first random number and then adding it to its corresponding second random number to generate the first perturbation data.

[0024] In some embodiments, the current layer of the first inference model is a nonlinear layer or a nonparametric layer, and generating the first perturbation data includes: generating a fourth random number; using the product of the input features and the fourth random number to perform client-side perturbation on the input features to generate the first perturbation data.

[0025] In some embodiments, the current layer of the first inference model is a softmax layer, and generating the first perturbation data includes: generating a public key and its corresponding private key; using the public key to homomorphically encrypt the first product of the input feature and the fourth random number to generate the first perturbation data; homomorphically encrypting the fourth random number; and calculating the processing result of the image to be processed includes: using the private key to homomorphically decrypt the second perturbation data and eliminating the client perturbation, wherein the second perturbation data is generated by the server using the public key to homomorphically add the second product of the encrypted fourth random number and the third random number and the first perturbation data, and the third random number is generated by the server.

[0026] According to some other embodiments of this disclosure, an image processing system is provided, comprising: a client, configured to perform client-side perturbation on input features of a first inference model to generate first perturbation data, wherein the input features are generated based on an image to be processed, and to calculate a processing result of the image to be processed using the first inference model based on second perturbation data after the client-side perturbation has been eliminated; and a server, configured to calculate a first result using a second inference model based on the first perturbation data, and to perform a first server-side perturbation on the first result to generate second perturbation data, wherein the first inference model and the second inference model have the same structure.

[0027] In some embodiments, the client generates a first random number; the client uses the first random number to perform client-side perturbation on the input features to generate first perturbation data; the server generates a second random number; the server uses the second random number to perform a first server-side perturbation on the first result to generate second perturbation data.

[0028] In some embodiments, the client perturbs the input features of the current layer of the first inference model to generate first perturbation data; the server calculates a first result using the current layer of the second inference model based on the first perturbation data; the server perturbs the first result to generate second perturbation data; the client calculates a second result using the current layer of the first inference model based on the second perturbation data after client perturbation has been eliminated; the client and server repeat the above steps until the client calculates the processing result of the image to be processed using the last layer of the first inference model.

[0029] In some embodiments, if the current layer of the second inference model is not the first layer, the server eliminates the second server-side perturbation of the input layer of the current layer of the second inference model in the first perturbation data; the server calculates the first result using the current layer of the second inference model based on the first perturbation data with the second server-side perturbation eliminated.

[0030] In some embodiments, the current layer of the first inference model is a linear layer, and generating the first perturbation data includes: the client dividing the input features into multiple sub-features; the client generating a first random number for each sub-feature; and the client perturbing the input features by multiplying each sub-feature by its corresponding first random number to generate the first perturbation data.

[0031] In some embodiments, the client generates a second random number for each sub-feature; the client perturbs the input features by multiplying each sub-feature by its corresponding first random number and then adding it to its corresponding second random number, thereby generating first perturbation data.

[0032] In some embodiments, the sum of the first random numbers is 0.

[0033] In some embodiments, the server generates a third random number; the server uses the product of the first result and the third random number to perform a first server-side perturbation on the first result, generating second perturbation data.

[0034] In some embodiments, the current layer of the first inference model is a nonlinear layer or a nonparametric layer, and the client generates a fourth random number; the client uses the product of the input features and the fourth random number to perturb the input features and generate the first perturbation data.

[0035] In some embodiments, when the current layer of the first inference model is a non-linear layer, the fourth random number is a positive number.

[0036] In some embodiments, the current layer of the first inference model is a softmax layer. The client generates a public key and its corresponding private key. The client uses the public key to homomorphically encrypt the first product of the input feature and the fourth random number to generate first perturbation data. The fourth random number is homomorphically encrypted. The server generates a third random number. The server uses the public key to homomorphically add the second product of the encrypted fourth random number and the third random number, as well as the first perturbation data, to generate second perturbation data. The client uses its private key to homomorphically decrypt the second perturbation data to eliminate the client perturbation.

[0037] In some embodiments, if the current layer of the second inference model is not the first layer, the server eliminates the second server-side perturbation of the input layer of the current layer of the second inference model in the first perturbation data; and performs homomorphic addition on the second product of the encrypted fourth random number and the third random number and the first perturbation data after eliminating the second server-side perturbation.

[0038] In some embodiments, the client sends an inference request to the server, the inference request including relevant information about the first inference model; the server sends structural information of the second inference model to the client based on the relevant information; the client perturbs the input features of the first inference model by: the client determining the input features of the current layer of the first inference model based on the structural information.

[0039] According to some other embodiments of this disclosure, an image processing apparatus is provided, comprising: a generation unit, configured to perform client-side perturbation on input features of a first inference model to generate first perturbation data, wherein the input features are generated based on an image to be processed; and a calculation unit, configured to calculate a processing result of the image to be processed using the first inference model based on second perturbation data after client-side perturbation has been eliminated, wherein the second perturbation data is generated by the server performing a first server-side perturbation on the first result, and the first result is calculated by the server using the second inference model based on the first perturbation data, wherein the first inference model and the second inference model have the same structure.

[0040] In some embodiments, the generation unit generates a first random number; using the first random number, the input features are perturbed by the client to generate first perturbed data.

[0041] In some embodiments, the first result is calculated by the server based on the first perturbation data using the current layer of the second inference model. The second perturbation data is generated by the server performing a first server-side perturbation on the first result. The generation unit performs client-side perturbation on the input features of the current layer of the first inference model to generate the first perturbation data. The calculation unit calculates the second result based on the second perturbation data after eliminating client-side perturbation using the current layer of the first inference model. The steps of generating the first perturbation data and calculating the processing result of the image to be processed are repeated until the processing result of the image to be processed is calculated using the last layer of the first inference model.

[0042] In some embodiments, the generation unit divides the input features into multiple sub-features; generates a first random number for each sub-feature; and uses the product of each sub-feature and its corresponding first random number to perform client-side perturbation on the input features to generate first perturbation data.

[0043] In some embodiments, the generation unit generates a second random number for each sub-feature; by multiplying each sub-feature by its corresponding first random number and then adding it to its corresponding second random number, the input features are perturbed to generate first perturbed data.

[0044] In some embodiments, the generation unit generates a fourth random number; and uses the product of the input feature and the fourth random number to perform client-side perturbation on the input feature to generate first perturbation data.

[0045] In some embodiments, the generation unit generates a public key and its corresponding private key; uses the public key to homomorphically encrypt the first product of the input feature and the fourth random number to generate first perturbation data; homomorphically encrypts the fourth random number; the calculation unit uses the private key to homomorphically decrypt the second perturbation data to eliminate client perturbation. The second perturbation data is generated by the server using the public key to homomorphically add the second product of the encrypted fourth random number and the third random number, as well as the first perturbation data. The third random number is generated by the server.

[0046] According to further embodiments of the present disclosure, an image processing apparatus is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the image processing method of any of the above embodiments based on instructions stored in the memory device.

[0047] According to further embodiments of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the image processing method of any of the above embodiments.

[0048] In the above embodiments, complex and time-consuming encryption methods such as obfuscating circuits are abandoned, and a simple perturbation method is used for encryption, which improves inference efficiency and reduces the difficulty of construction. Attached Figure Description

[0049] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.

[0050] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description, wherein:

[0051] Figure 1a Flowcharts illustrating some embodiments of the image processing methods of this disclosure;

[0052] Figure 1b Flowcharts illustrating other embodiments of the image processing methods of this disclosure;

[0053] Figure 2a Schematic diagrams illustrating some embodiments of the initialization phase of this disclosure;

[0054] Figure 2b A flowchart illustrating some embodiments of the online inference phase of this disclosure;

[0055] Figures 3a-3e Schematic diagrams illustrating some embodiments of the image processing methods of this disclosure;

[0056] Figure 4a Block diagrams illustrating some embodiments of the image processing system of this disclosure;

[0057] Figure 4b Block diagrams illustrating some embodiments of the image processing apparatus of this disclosure;

[0058] Figure 5 Block diagrams illustrating some embodiments of the image processing apparatus of this disclosure;

[0059] Figure 6 Block diagrams illustrating other embodiments of the image processing apparatus of this disclosure. Detailed Implementation

[0060] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0061] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0062] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0063] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0064] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0065] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0066] Figure 1a Flowcharts illustrating some embodiments of the image processing methods of this disclosure are shown.

[0067] like Figure 1a As shown, in step 110a, the client perturbs the input features of the first inference model to generate first perturbation data, and the input features are generated based on the image to be processed.

[0068] In some embodiments, the client sends an inference request to the server, the inference request including relevant information about the first inference model; the server sends structural information of the second inference model to the client based on the relevant information; the client determines the input features of the current layer of the first inference model based on the structural information.

[0069] For example, the client sends an inference request to the server, informing it of the relevant information about the model to be inferred (i.e., the first inference model). This information may include a unique ID agreed upon by the server and client for the first inference model, as well as the data format input to the first inference model. For instance, the first and second inference models may be CNN (Convolutional Neural Networks) models with the same structure.

[0070] Based on the client's inference request, the server loads the corresponding second inference model from the first inference model and generates a computation graph, which is then sent back to the client. For example, the computation graph only contains the network structure of the second inference model and does not reveal the weight information of the second inference model.

[0071] In some embodiments, the client generates a first random number; the client uses the first random number to perturb the input features, generating first perturbed data. For example, the client perturbs the input features of the current layer of the first inference model to generate the first perturbed data.

[0072] For example, the client generates random numbers to perturb the input features, and then sends the perturbed input features to the server.

[0073] In some embodiments, the current layer of the first inference model is a linear layer, the client divides the input features into multiple sub-features, the client generates a first random number for each sub-feature, and the client uses the product of each sub-feature and its corresponding first random number to perturb the input features to generate first perturbation data.

[0074] In some embodiments, the current layer of the first inference model is a linear layer, and the client generates a second random number for each sub-feature; the client perturbs the input features by multiplying each sub-feature by its corresponding first random number and then adding it to its corresponding second random number, generating first perturbation data. For example, the sum of the first random numbers is 0.

[0075] For example, for linear layers (such as fully connected layers, convolutional layers, etc.), on the client side, the input x of the current layer is split into N parts; each part is then added with its own random number. Then, it is sent to the server. The sum of their respective random numbers is 0, that is...

[0076] CNN models are often used in image processing. Simply multiplying by a random number only changes the image's depth and tone, which is not sufficiently secure. Therefore, splitting the image into multiple parts and perturbing them can ensure the confidentiality of the inference.

[0077] After the server receives the perturbed data x' from the client, it first removes the perturbation t' from the previous layer on the server and then performs the corresponding linear operation. After multiplying it by the newly generated random number perturbation t from the current layer, it sends it back to the client. The client then removes N copies of its own perturbation.

[0078] In some embodiments, the current layer of the first inference model is a nonlinear layer or a nonparametric layer, and the client generates a fourth random number; the client uses the product of the input features and the fourth random number to perturb the input features and generate the first perturbation data.

[0079] In some embodiments, when the current layer of the first inference model is a non-linear layer, the fourth random number is a positive number.

[0080] For example, for nonlinear layers such as ReLU (Rectified Linear Units) and PreLU (Parametric Rectified Linear Unit), the client multiplies the current layer's data with positive random numbers to create a perturbation, which is then sent to the server. The server removes the perturbation random numbers from the previous layer, performs the corresponding operation, and sends the result back to the client, which then removes the positive random number perturbation from its current layer.

[0081] For example, for non-parametric layers, such as reshape layers and skip connection (addition) layers, the client performs the following operation: multiply the layer by a random number perturbation and send it to the server; the server then removes the random number perturbation from the previous layer, performs the corresponding operation, and sends it back to the client; finally, the client removes the random number perturbation from the current layer.

[0082] In step 120a, the server calculates the first result based on the first perturbation data using the second inference model, and performs a first server-side perturbation on the first result to generate the second perturbation data. The first inference model and the second inference model have the same structure.

[0083] In some embodiments, the server generates a second random number; the server uses the second random number to perform a first server-side perturbation on the first result, generating second perturbation data. For example, the server calculates the first result using the current layer of the second inference model based on the first perturbation data; the server performs a first server-side perturbation on the first result, generating second perturbation data.

[0084] For example, if the current layer of the second inference model is a linear layer, a nonlinear layer, or a nonparametric layer, the server generates a third random number; the server uses the product of the first result and the third random number to perform a first server-side perturbation on the first result, generating a second perturbation data.

[0085] In some embodiments, if the current layer of the second inference model is not the first layer, the server eliminates the second server-side perturbation of the input layer of the current layer of the second inference model in the first perturbation data; the server calculates the first result using the current layer of the second inference model based on the first perturbation data with the second server-side perturbation eliminated.

[0086] For example, after removing the server-side perturbation from the previous layer, the server performs the calculation of the second inference model; after adding the perturbation from the current layer, the data is sent back to the client.

[0087] In step 130a, the client calculates the processing result of the image to be processed using the first inference model based on the second perturbation data that has eliminated client perturbation.

[0088] In some embodiments, the client computes a second result using the current layer of the first inference model based on second perturbation data that has eliminated client perturbations.

[0089] In some embodiments, the client perturbs the input features of the current layer of the first inference model to generate first perturbation data; the server calculates a first result using the current layer of the second inference model based on the first perturbation data; the server perturbs the first result to generate second perturbation data; the client calculates a second result using the current layer of the first inference model based on the second perturbation data after client perturbation has been eliminated; the client and server repeat the above steps until the client calculates the processing result of the image to be processed using the last layer of the first inference model.

[0090] For example, after removing client-side perturbations at the current layer, the client performs the calculations required for the current layer of the first inference model. The client then determines whether it is the last layer of the first inference model. If it is, the inference stops and the result of this layer is output. If not, the inference continues to the next layer until the last layer ends.

[0091] In some embodiments, the current layer of the first inference model is a softmax layer, the client generates a public key and its corresponding private key; the client uses the public key to perform homomorphic encryption on the first product of the input feature and the fourth random number to generate the first perturbation data; and performs homomorphic encryption on the fourth random number.

[0092] The server generates a third random number. Using its public key, the server homomorphically adds the encrypted fourth random number to the second product of the third random number and the first perturbation data, generating a second perturbation data. The client then homomorphically decrypts the second perturbation data using its private key, eliminating the client-side perturbation.

[0093] For example, if the current layer of the second inference model is not the first layer, the server eliminates the second server-side perturbation of the input layer of the current layer of the second inference model in the first perturbation data; and performs homomorphic addition on the second product of the encrypted fourth random number and the third random number and the first perturbation data after eliminating the second server-side perturbation.

[0094] For example, for nonparametric layers and softmax layers, homomorphic encryption can be used. After generating a random number r, the client generates a public key and a private key pair, perturbs the input characteristics by x×r, and then performs homomorphic encryption to generate x'. The client also encrypts the random number r separately to generate r', and sends x', r', and the public key to the server.

[0095] The server generates its own random number t, and after eliminating the perturbation t' from the previous layer, it obtains the result. The server multiplies r′ by its own random number t, and then uses the public key and... By performing homomorphic addition, we obtain:

[0096]

[0097] The server sends the sum of the two data points to the client; the client uses its private key to decrypt the result, removing the local perturbation 'r', and obtains the final result.

[0098] Based on the characteristics of the softmax layer, its output can be obtained as follows: array i represents the i-th dimension of array m. Therefore, the client can cancel out t to restore the true result of the inference model.

[0099] In some embodiments, a privacy-preserving inference protocol is configured for the deep CNN classification model. CNN classification models typically use softmax as the output layer. Based on the characteristics of softmax, complex and time-consuming encryption methods such as obfuscation circuits can be abandoned, and encryption can be performed using simple random number perturbation, thereby improving the efficiency of privacy-preserving inference.

[0100] The privacy-preserving inference protocol configured in the above embodiments is applicable to any deep convolutional neural network classification model with softmax as the output layer.

[0101] In some embodiments, the entire reasoning process is divided into two parts: an initialization phase and an online reasoning phase.

[0102] Figure 1b Flowcharts illustrating some other embodiments of the image processing methods of this disclosure are shown.

[0103] like Figure 1b As shown, in step 110b, the input features of the first inference model are perturbed by the client to generate the first perturbation data. The input features are generated based on the image to be processed.

[0104] In step 120b, based on the second perturbation data that has eliminated client-side perturbation, the processing result of the image to be processed is calculated using the first inference model. The second perturbation data is generated by the server performing a first server-side perturbation on the first result. The first result is calculated by the server using the second inference model based on the first perturbation data. The first inference model and the second inference model have the same structure.

[0105] In some embodiments, a first random number is generated; the first random number is used to perturb the input features on the client side to generate first perturbation data.

[0106] In some embodiments, the first result is calculated by the server using the current layer of the second inference model based on the first perturbation data, and the second perturbation data is generated by the server by performing a first server perturbation on the first result.

[0107] The input features of the current layer of the first inference model are perturbed by the client to generate first perturbation data; based on the second perturbation data after the client perturbation has been eliminated, the second result is calculated using the current layer of the first inference model; the steps of generating the first perturbation data and calculating the processing result of the image to be processed are repeated until the processing result of the image to be processed is calculated using the last layer of the first inference model.

[0108] In some embodiments, the input features are divided into multiple sub-features; a first random number is generated for each sub-feature; and the input features are perturbed by multiplying each sub-feature by its corresponding first random number to generate first perturbed data.

[0109] In some embodiments, a second random number is generated for each sub-feature; the input features are perturbed by multiplying each sub-feature by its corresponding first random number and then adding it to its corresponding second random number to generate first perturbed data.

[0110] In some embodiments, a fourth random number is generated; the input features are perturbed by multiplying the input features by the fourth random number to generate first perturbed data.

[0111] In some embodiments, a public key and its corresponding private key are generated; the first product of the input feature and the fourth random number is homomorphically encrypted using the public key to generate first perturbation data; the fourth random number is homomorphically encrypted; the second perturbation data is homomorphically decrypted using the private key to eliminate client perturbation. The second perturbation data is generated by the server using the public key to homomorphically add the second product of the encrypted fourth random number and the third random number, as well as the first perturbation data. The third random number is generated by the server.

[0112] Figure 2a Schematic diagrams illustrating some embodiments of the initialization phase of this disclosure are shown.

[0113] like Figure 2a As shown, during the initialization phase, the client sends an inference request to the server, informing it of the relevant information about the model to be inferred. Based on the client's requirements, the server loads the corresponding inference model and generates a computation graph, which is then sent back to the client. The computation graph only contains the network structure of the CNN model and does not reveal the model's weight information.

[0114] For example, the client randomly generates an image that matches the input shape and initializes it; the client initiates a request to initialize the server; the server receives the relevant information of the inference model sent by the client and automatically generates a computation graph from the model file; after removing the weights of the inference model from the computation graph, the server sends the computation graph to the client.

[0115] Figure 2b Flowcharts illustrating some embodiments of the online inference phase of this disclosure are shown.

[0116] like Figure 2b As shown, the online inference phase is performed layer by layer. Layers related to model weights are handled by the server, while layers unrelated to model weights are handled by either the server or the client. Regardless of the layer type, it must first be handled by the client.

[0117] The client reads the input image information, performs image preprocessing according to the method required by the inference model, and then performs inference layer by layer according to the computation graph collected from the server during the initialization phase.

[0118] The client generates random numbers to perturb the input feature x of this layer, obtaining x'. The perturbed data x' is then sent to the server for inference.

[0119] After removing the perturbations from the previous layer on the server side, the server performs the necessary calculations for its current layer. After adding the new perturbation from the current layer on the server side, the data z is obtained. This data z is then sent back to the client.

[0120] After removing client-side perturbations from the current layer in z, the client performs the necessary calculations for this layer to obtain y. The client then determines whether this is the last layer of its inference model; if so, it stops inference and outputs the current layer result y; otherwise, it continues inference to the next layer until the last layer is reached.

[0121] Figure 3aSchematic diagrams illustrating some embodiments of the image processing methods of this disclosure.

[0122] like Figure 3a As shown, for linear layers (such as fully connected layers, convolutional layers, etc.), on the client side, the input of this layer is split into N parts, and each part is added with its own random number. Then send it to the server (make sure the sum of the random numbers is 0). ).

[0123] For example, for the i-th layer, generate a set of random numbers r, which contains n random numbers rj; divide the input feature x into n sub-features xj. j Perturbation is performed separately:

[0124] x′→m j ×x j +r j

[0125]

[0126] In the above embodiments, CNNs are often used for image processing. Simply multiplying by a random number to perturb the image can only change its depth, which is not secure enough. Therefore, splitting the image into multiple parts ensures the confidentiality of the inference process.

[0127] After receiving the perturbed data x' from the client, the server removes the perturbation t' from the previous layer on the server and performs the corresponding linear operation (such as multiplying by A); it then multiplies the data by the newly generated random number perturbation t to generate data z; after processing the perturbation t, it generates d, and sends z and d back to the client. The client removes N parts of their respective perturbations to obtain y. For example, the data z is:

[0128]

[0129] b is the offset parameter, and A is the weight.

[0130] Figure 3b Schematic diagrams illustrating some embodiments of the image processing methods of this disclosure.

[0131] like Figure 3b As shown, for nonlinear layers (such as ReLU and PreLU), on the client side, the data x of this layer is multiplied by a positive random number r_pos, and after perturbation, x' is generated and sent to the server side.

[0132] After removing the perturbation random number t' from the previous layer of x' on the server side, the corresponding operation is performed, and the perturbation t of the current layer is added. This generates z.

[0133]

[0134] The z is sent back to the client, which removes the positive random number perturbation from its own layer, resulting in:

[0135]

[0136] Figures 3c-3d Schematic diagrams illustrating some embodiments of the image processing methods of this disclosure.

[0137] like Figures 3c-3d As shown, for non-parametric layers (such as reshape, skip connection (addition), etc.), the operations performed by the client are all multiplied by a random number perturbation before being sent to the server.

[0138] For example, for the i-th layer, generate a random number r; perturb the input features x1, x2, ... respectively to generate x' = [x1×r, x2×r, ...]. Send x' and i to the server.

[0139] Remove each x on the server side. j Random number perturbation t′ on the upper layer of the server side i After performing the corresponding operations, the perturbation t of the current layer is added to generate z:

[0140]

[0141] z is sent back to the client; the client removes the random number perturbation of this layer and obtains y.

[0142] For example, for the i-th layer, generate a random number r; perturb the input feature x to generate x' = x × r. Send x' and i to the server.

[0143] Remove the random number perturbation t′ from the previous layer on the server side in x'. i After performing the corresponding operations, the perturbation t of the current layer is added to generate z:

[0144]

[0145] z is sent back to the client; the client removes the random number perturbation of this layer and obtains y.

[0146] Figure 3e Schematic diagrams illustrating some embodiments of the image processing methods of this disclosure.

[0147] like Figure 3eAs shown, homomorphic encryption is used for the non-parametric layer and the softmax layer. The client first generates a random number r, and then generates a public key pair pub. k , private key k :

[0148] pub k ,pri k =HE.generate_keypairs()

[0149] Homomorphic encryption is performed on the perturbed input features x×r to generate x':

[0150] x' = HE.enc(x × r)

[0151] Generate r' by encrypting the random number r separately:

[0152] r' = HE.enc(r)

[0153] Combine x',r' and the public key pub k Send to the server; the server generates its own random number t, removes the perturbation t' from the previous layer, and obtains the result. Then; multiply r′ by the server's random number t and then... Adding homomorphisms yields z:

[0154]

[0155] z is sent to the client; the client decrypts it using its private key and removes the perturbation r from this layer to obtain z′:

[0156]

[0157] The client receives the processing result Result = softmax(z'). This is due to the characteristics of the softmax processing result. We can cancel out t to restore the true result of the model.

[0158] In the above embodiments, the characteristics of the softmax layer are utilized to eliminate all obfuscation circuits and most of the required homomorphic encryption in the privacy-preserving inference protocol for the CNN classification model. Only random number perturbations and homomorphic encryption applied to the final softmax layer are used to restore the same result as direct inference. This improves inference efficiency.

[0159] Figure 4a Block diagrams illustrating some embodiments of the image processing system of this disclosure are shown.

[0160] like Figure 4aAs shown, the image processing system 4 includes: a client 41a, used to perform client-side perturbation on the input features of a first inference model to generate first perturbation data, wherein the input features are generated based on the image to be processed, and to calculate the processing result of the image to be processed using the first inference model based on the second perturbation data after the client-side perturbation has been eliminated; and a server 42a, used to calculate a first result using a second inference model based on the first perturbation data, and to perform a first server-side perturbation on the first result to generate second perturbation data, wherein the first inference model and the second inference model have the same structure.

[0161] In some embodiments, client 41a generates a first random number; client 41a uses the first random number to perform client-side perturbation on the input features to generate first perturbation data; server 42a generates a second random number; server 42a uses the second random number to perform a first server-side perturbation on the first result to generate second perturbation data.

[0162] In some embodiments, client 41a performs client-side perturbation on the input features of the current layer of the first inference model to generate first perturbation data; server 42a calculates a first result using the current layer of the second inference model based on the first perturbation data; server 42a performs a first server-side perturbation on the first result to generate second perturbation data; client 41a calculates a second result using the current layer of the first inference model based on the second perturbation data with client-side perturbation eliminated; client 41a and server 42a repeat the above steps until client 41a calculates the processing result of the image to be processed using the last layer of the first inference model.

[0163] In some embodiments, if the current layer of the second inference model is not the first layer, the server 42a eliminates the second server-side perturbation of the input layer of the current layer of the second inference model in the first perturbation data; the server 42a calculates the first result using the current layer of the second inference model based on the first perturbation data with the second server-side perturbation eliminated.

[0164] In some embodiments, the current layer of the first inference model is a linear layer, and the client 41a divides the input features into multiple sub-features; the client 41a generates a first random number for each sub-feature; the client 41a uses the product of each sub-feature and its corresponding first random number to perform client perturbation on the input features to generate first perturbation data.

[0165] In some embodiments, client 41a generates a second random number for each sub-feature; client 41a performs client perturbation on the input features by multiplying each sub-feature by its corresponding first random number and then adding it to its corresponding second random number, thereby generating first perturbation data.

[0166] In some embodiments, the sum of the first random numbers is 0.

[0167] In some embodiments, server 42a generates a third random number; server 42a uses the product of the first result and the third random number to perform a first server-side perturbation on the first result to generate second perturbation data.

[0168] In some embodiments, the current layer of the first inference model is a nonlinear layer or a nonparametric layer, and the client 41a generates a fourth random number; the client 42a uses the product of the input features and the fourth random number to perform client perturbation on the input features and generate the first perturbation data.

[0169] In some embodiments, when the current layer of the first inference model is a non-linear layer, the fourth random number is a positive number.

[0170] In some embodiments, the current layer of the first inference model is a softmax layer. Client 41a generates a public key and its corresponding private key. Client 41a uses the public key to homomorphically encrypt the first product of the input feature and the fourth random number to generate first perturbation data. The fourth random number is then homomorphically encrypted. Server 42a generates a third random number. Server 42a uses the public key to homomorphically add the second product of the encrypted fourth random number and the third random number, as well as the first perturbation data, to generate second perturbation data. Client 41a uses the private key to homomorphically decrypt the second perturbation data and eliminates the client perturbation.

[0171] In some embodiments, when the current layer of the second inference model is not the first layer, the server 42a eliminates the second server perturbation of the input layer of the current layer of the second inference model in the first perturbation data; and performs homomorphic addition on the second product of the encrypted fourth random number and the third random number and the first perturbation data after eliminating the second server perturbation.

[0172] In some embodiments, client 41a sends an inference request to server, the inference request including relevant information of the first inference model; server 42a sends structural information of the second inference model to client 41a based on the relevant information; client 41a performs client perturbation on the input features of the first inference model, including client 41a determining the input features of the current layer of the first inference model based on the structural information.

[0173] Figure 4b Block diagrams illustrating some embodiments of the image processing apparatus of this disclosure are shown.

[0174] like Figure 4bAs shown, the image processing device 4b includes: a generation unit 41b, used to perform client-side perturbation on the input features of the first inference model to generate first perturbation data, wherein the input features are generated based on the image to be processed; and a calculation unit 42b, used to calculate the processing result of the image to be processed using the first inference model based on the second perturbation data after client-side perturbation has been eliminated, wherein the second perturbation data is generated by the server performing a first server-side perturbation on the first result, and the first result is calculated by the server using the second inference model based on the first perturbation data, wherein the first inference model and the second inference model have the same structure.

[0175] In some embodiments, the generation unit 41b generates a first random number; using the first random number, the input features are perturbed to generate first perturbation data.

[0176] In some embodiments, the first result is calculated by the server using the current layer of the second inference model based on the first perturbation data. The second perturbation data is generated by the server performing a first server-side perturbation on the first result. The generation unit 41b performs client-side perturbation on the input features of the current layer of the first inference model to generate the first perturbation data. The calculation unit 42b calculates the second result using the current layer of the first inference model based on the second perturbation data after eliminating client-side perturbation. The steps of generating the first perturbation data and calculating the processing result of the image to be processed are repeated until the processing result of the image to be processed is calculated using the last layer of the first inference model.

[0177] In some embodiments, the generation unit 41b divides the input features into multiple sub-features; generates a first random number for each sub-feature; and uses the product of each sub-feature and its corresponding first random number to perform client-side perturbation on the input features to generate first perturbation data.

[0178] In some embodiments, the generation unit 41b generates a second random number for each sub-feature; by multiplying each sub-feature by its corresponding first random number and then adding it to its corresponding second random number, the input features are perturbed to generate first perturbed data.

[0179] In some embodiments, the generation unit 41b generates a fourth random number; and uses the product of the input feature and the fourth random number to perform client-side perturbation on the input feature to generate first perturbation data.

[0180] In some embodiments, the generation unit 41b generates a public key and its corresponding private key; uses the public key to homomorphically encrypt the first product of the input feature and the fourth random number to generate first perturbation data; homomorphically encrypts the fourth random number; the calculation unit uses the private key to homomorphically decrypt the second perturbation data to eliminate client perturbation. The second perturbation data is generated by the server using the public key to homomorphically add the second product of the encrypted fourth random number and the third random number and the first perturbation data. The third random number is generated by the server.

[0181] Figure 5 Block diagrams illustrating some embodiments of the image processing apparatus of this disclosure are shown.

[0182] like Figure 5 As shown, the image processing apparatus 5 of this embodiment includes a memory 51 and a processor 52 coupled to the memory 51. The processor 52 is configured to execute an image processing method in any embodiment of this disclosure based on instructions stored in the memory 51.

[0183] The memory 51 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory stores, for example, the operating system, application programs, boot loader, database, and other programs.

[0184] Figure 6 Block diagrams illustrating other embodiments of the image processing apparatus of this disclosure.

[0185] like Figure 6 As shown, the image processing apparatus 6 of this embodiment includes a memory 610 and a processor 620 coupled to the memory 610. The processor 620 is configured to execute the image processing method of any of the foregoing embodiments based on instructions stored in the memory 610.

[0186] The memory 610 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory may store, for example, the operating system, application programs, boot loader, and other programs.

[0187] The image processing device 6 may also include an input / output interface 630, a network interface 640, and a storage interface 650. These interfaces 630, 640, and 650, as well as the memory 610 and processor 620, can be connected, for example, via a bus 660. The input / output interface 630 provides a connection interface for input / output devices such as a monitor, mouse, keyboard, touchscreen, microphone, and speakers. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0188] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] The image processing method, image processing system, image processing apparatus, and non-volatile computer-readable storage medium according to this disclosure have been described in detail above. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions disclosed herein based on the above description.

[0190] The methods and systems of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the specific order described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0191] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for processing an image, comprising: a client perturbing an input feature of a first inference model to generate first perturbation data, the input feature being generated according to an image to be processed; a server calculating a first result using a second inference model according to the first perturbation data, and perturbing the first result to generate second perturbation data, the first inference model and the second inference model having the same structure; the client eliminating the client perturbation in the second perturbation data, and calculating a processing result of the image to be processed using the first inference model according to the second perturbation data from which the client perturbation is eliminated, wherein the generating first perturbation data comprises the client perturbing an input feature of one layer of the first inference model to generate the first perturbation data; the generating second perturbation data comprises the server calculating the first result using one layer of the second inference model according to the first perturbation data, and perturbing the first result to generate the second perturbation data; the calculating a processing result of the image to be processed using the first inference model comprises: the client calculating a second result using the one layer of the first inference model according to the second perturbation data from which the client perturbation is eliminated, the client and the server repeat the above steps until the client calculates the processing result of the image to be processed using a last layer of the first inference model; the calculating the first result using one layer of the second inference model comprises: in a case where the one layer of the second inference model is not a first layer, the server eliminating a second server perturbation of an input layer of the one layer of the second inference model in the first perturbation data; the server calculating the first result using the one layer of the second inference model according to the first perturbation data from which the second server perturbation is eliminated.

2. The method according to claim 1, wherein: the generating first perturbation data comprises: the client generating a first random number; the client perturbing the input feature using the first random number to generate the first perturbation data; the generating second perturbation data comprises: the server generating a second random number; the server perturbing the first result using the second random number to generate the second perturbation data.

3. The treatment method of claim 1, wherein, the one layer of the first inference model is a linear layer, and the generating the first perturbation data comprises: the client dividing the input feature into a plurality of sub-features; the client generating a first random number for each sub-feature; the client perturbing the input feature using a product of each sub-feature and the corresponding first random number to generate the first perturbation data.

4. The treatment method of claim 3, wherein, the one layer of the first inference model is a linear layer, and the generating the first perturbation data comprises: the client generating a second random number for each sub-feature; The client generates the first perturbation data by performing the client perturbation on the input feature by multiplying each sub-feature with its corresponding first random number and adding the result to its corresponding second random number.

5. The treatment method of claim 3, wherein, The sum of the first random numbers is 0.

6. The treatment method of claim 1, wherein, The one layer of the first inference model is a linear layer, a nonlinear layer, or a non-parametric layer, and the generating the second perturbation data comprises: The server generates a third random number; The server performs the first server perturbation on the first result by using the product of the first result and the third random number to generate the second perturbation data.

7. The treatment method of claim 1, wherein, The one layer of the first inference model is a nonlinear layer or a non-parametric layer, and the generating the first perturbation data comprises: The client generates a fourth random number; The client performs the client perturbation on the input feature by using the product of the input feature and the fourth random number to generate the first perturbation data.

8. The treatment method of claim 7, wherein, In the case where the one layer of the first inference model is a nonlinear layer, the fourth random number is a positive number.

9. The treatment method of claim 8, wherein, The one layer of the first inference model is a softmax layer, and the generating the first perturbation data comprises: The client generates a public key and a corresponding private key; The client homomorphically encrypts the first product of the input feature and the fourth random number by using the public key to generate the first perturbation data; The fourth random number is homomorphically encrypted; The generating the second perturbation data comprises: The server generates a third random number; The server homomorphically adds the second product of the encrypted fourth random number and the third random number and the first perturbation data by using the public key to generate the second perturbation data; The calculating the processing result of the to-be-processed image comprises: The client homomorphically decrypts the second perturbation data by using the private key and then eliminates the client perturbation.

10. The treatment method of claim 9, wherein, The homomorphically adding the second product of the encrypted fourth random number and the third random number and the first perturbation data comprises: In the case where the one layer of the second inference model is not the first layer, the server eliminates the second server perturbation of the input layer of the one layer of the second inference model in the first perturbation data; The homomorphically adding the second product of the encrypted fourth random number and the third random number and the first perturbation data from which the second server perturbation is eliminated.

11. The processing method of any one of claims 1-10, further comprising: The client sends an inference request to the server, the inference request comprising related information of the first inference model; The server sends structure information of the second inference model to the client according to the related information; The client performing the client perturbation on the input feature of the first inference model comprises: The client determines the input feature of the one layer of the first inference model according to the structure information.

12. An image processing method performed by a client, comprising: The input feature of the first inference model is disturbed by a client to generate first disturbance data, wherein the input feature is generated according to a to-be-processed image; The second disturbance data is eliminated from the client disturbance, and a processing result of the to-be-processed image is calculated by using the first inference model according to the second disturbance data from which the client disturbance is eliminated, wherein the second disturbance data is generated by a server from a first result by first server disturbance, and the first result is calculated by the server from the first disturbance data by using a second inference model, and the first inference model and the second inference model have the same structure, The first disturbance data is generated by disturbing the input feature of one layer of the first inference model. The processing result of the to-be-processed image is calculated by using the first inference model, The second result is calculated by using the one layer of the first inference model according to the second disturbance data from which the client disturbance is eliminated, The steps of generating the first disturbance data and calculating the processing result of the to-be-processed image by using the first inference model are repeated until the processing result of the to-be-processed image is calculated by using the last layer of the first inference model. The processing method further comprises: sending an inference request to the server, wherein the inference request comprises related information of the first inference model, receiving structure information of the second inference model sent by the server according to the related information, determining the input feature of the one layer of the first inference model according to the structure information.

13. The treatment method of claim 12, wherein, The second disturbance data is generated by the server from the first result by first server disturbance using a second random number generated by the server, and the first disturbance data is generated by disturbing the input feature by using the first random number. generating a first random number, disturbing the input feature by using the first random number to generate the first disturbance data.

14. The treatment method of claim 12, wherein, The one layer of the first inference model is a linear layer, and the first disturbance data is generated by disturbing the input feature by using the product of each sub-feature and the corresponding first random number. The one layer of the first inference model is a linear layer, and the first disturbance data is generated by disturbing the input feature by using the product of each sub-feature and the corresponding first random number. The one layer of the first inference model is a linear layer, and the first disturbance data is generated by disturbing the input feature by using the product of each sub-feature and the corresponding first random number. The one layer of the first inference model is a non-linear layer or a non-parameter layer, and the first disturbance data is generated by disturbing the input feature by using the product of the input feature and the fourth random number.

15. The treatment method of claim 12, wherein, The one layer of the first inference model is a softmax layer, and the first disturbance data is generated by disturbing the input feature by using the product of the input feature and the fourth random number. generating a public key and a corresponding private key, ​ 16. The treatment method of claim 12, wherein, ​ ​ ​ 17. The treatment method of claim 16, wherein, ​ ​ encrypt the first product of the input feature and the fourth random number homomorphically using the public key to generate the first perturbation data; encrypt the fourth random number homomorphically; the calculating the processing result of the to-be-processed image comprises: eliminate the client perturbation after homomorphic decryption of the second perturbation data by using the private key, the second perturbation data is generated by the server by homomorphic addition of the first perturbation data and a second product of the encrypted fourth random number and a third random number using the public key, and the third random number is generated by the server.

18. An image processing system, comprising: a client configured to perform client perturbation on input features of a first inference model to generate first perturbation data, the input features being generated according to a to-be-processed image, eliminate second perturbation data from the client perturbation, and calculate a processing result of the to-be-processed image by using the first inference model according to the second perturbation data from which the client perturbation is eliminated; a server configured to calculate a first result by using a second inference model according to the first perturbation data, and perform first server perturbation on the first result to generate second perturbation data, the first inference model and the second inference model having the same structure, wherein: the client is configured to perform the client perturbation on input features of one layer of the first inference model to generate the first perturbation data; the server is configured to calculate the first result by using one layer of the second inference model according to the first perturbation data, and perform the first server perturbation on the first result to generate the second perturbation data; the client is further configured to calculate a second result by using the one layer of the first inference model according to the second perturbation data from which the client perturbation is eliminated, and the client and the server are configured to repeat the above steps until the client calculates the processing result of the to-be-processed image by using a last layer of the first inference model; the server is further configured to eliminate second server perturbation of an input layer of the one layer of the second inference model in the first perturbation data in a case where the one layer of the second inference model is not a first layer, and calculate the first result by using the one layer of the second inference model according to the first perturbation data from which the second server perturbation is eliminated.

19. An image processing apparatus located at a client, comprising: a generating unit configured to perform client perturbation on input features of a first inference model to generate first perturbation data, the input features being generated according to a to-be-processed image; a calculating unit configured to calculate a processing result of the to-be-processed image by using the first inference model according to second perturbation data from which the client perturbation is eliminated, the second perturbation data being generated by a server by performing first server perturbation on a first result, the first result being calculated by the server by using a second inference model according to the first perturbation data, the first inference model and the second inference model having the same structure, The generating unit is configured to generate the first perturbation data by performing the client perturbation on input features of one layer of the first inference model; The computing unit is configured to calculate a second result by using the one layer of the first inference model according to the second perturbation data in which the client perturbation is eliminated, and repeat the steps of generating the first perturbation data and calculating the processing result of the to-be-processed image by using the first inference model until a processing result of the to-be-processed image is calculated by using a last layer of the first inference model; The processing device is further configured to: send an inference request to the server, the inference request including related information of the first inference model, receive structure information of the second inference model sent by the server according to the related information, determine input features of the one layer of the first inference model according to the structure information.

20. An image processing device, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute an image processing method according to any one of claims 1-11 or an image processing method according to any one of claims 12-17 based on instructions stored in the memory.

21. A non-volatile computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements an image processing method according to any one of claims 1-11 or an image processing method according to any one of claims 12-17.

Images