Image processing method, device, equipment and storage medium based on privacy protection

Abstract

One or more embodiments of the present specification provide a privacy-protection-based image processing method, apparatus, device, and storage medium, wherein the method includes: first acquiring a first image containing user privacy information, determining a privacy protection level of the first image based on a user operation, and then performing an operation corresponding to the privacy protection level on the first image based on the privacy protection level, wherein the performed operation includes one of the following: using the first image as an image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as an image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as an image to be uploaded, and finally uploading the image to be uploaded to a server; wherein the reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model.

Description

Technical Field

[0001] This document relates to the field of image processing, and in particular to an image processing method, apparatus, device and storage medium based on privacy protection. Background Art

[0002] In recent years, biometric identification methods based on user biometrics, such as facial recognition and fingerprint recognition, have become widely used. These methods can be used to identify users in areas such as payment and travel. User devices can pre-capture images containing the user's biometrics and send them to a server for storage, allowing the server to perform biometric identification based on the stored images. However, after the user device captures the image, during the process of transmitting it to the server, there is a risk that the image could be stolen by a third party, leading to the leakage of user privacy. Therefore, it is necessary to provide a technical solution to ensure the security of user privacy. Summary of the Invention

[0003] The purpose of one or more embodiments of this specification is to provide an image processing method based on privacy protection, comprising: obtaining a first image containing user privacy information. Determining the privacy protection level of the first image based on a user operation. Based on the privacy protection level, performing an operation corresponding to the privacy protection level on the first image, the performed operation including one of the following: using the first image as an image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as an image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as an image to be uploaded. Uploading the image to be uploaded to a server. The reversible privacy processing and the irreversible privacy processing are implemented using the same machine learning model.

[0004] One or more embodiments of this specification aim to provide a privacy-preserving image processing method, comprising: obtaining a first image containing user private information; performing reversible privacy processing on the first image to obtain a second image; or performing irreversible privacy processing on the first image to obtain a third image. The reversible privacy processing and the irreversible privacy processing are implemented using the same machine learning model. The second image or the third image is used to identify the user.

[0005] The purpose of one or more embodiments of the present specification is to provide an image processing device based on privacy protection, comprising: a first image acquisition unit, which acquires a first image containing user privacy information. The privacy protection level of the first image is determined according to a user operation. A first operation execution unit, which performs an operation corresponding to the privacy protection level on the first image according to the privacy protection level, wherein the executed operation includes one of the following: using the first image as the image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded. An image uploading unit, which uploads the image to be uploaded to a server. The reversible privacy processing and the irreversible privacy processing are implemented by the same machine learning model.

[0006] One or more embodiments of this specification aim to provide a privacy-preserving image processing device, comprising: a second image acquisition unit configured to acquire a first image containing user private information; a second operation execution unit configured to perform reversible privacy processing on the first image to obtain a second image, or to perform irreversible privacy processing on the first image to obtain a third image. The reversible privacy processing and the irreversible privacy processing are implemented using the same machine learning model. The second image or the third image is used to identify the user.

[0007] The purpose of one or more embodiments of the present specification is to provide an image processing device based on privacy protection, the device comprising: a processor; and a memory arranged to store computer-executable instructions. When the computer-executable instructions are executed, the processor is caused to: obtain a first image containing user privacy information. Determine the privacy protection level of the first image according to the user operation. According to the privacy protection level, perform an operation corresponding to the privacy protection level on the first image, the performed operation comprising one of the following: using the first image as the image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded. Upload the image to be uploaded to a server. Wherein, the reversible privacy processing and the irreversible privacy processing are implemented by the same machine learning model.

[0008] The purpose of one or more embodiments of the present specification is to provide a storage medium for storing computer-executable instructions, which, when executed by a processor, implement the following method: obtaining a first image containing user privacy information. Determining the privacy protection level of the first image based on a user operation. Based on the privacy protection level, performing an operation corresponding to the privacy protection level on the first image, the performed operation including one of the following: using the first image as an image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as an image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as an image to be uploaded. Uploading the image to be uploaded to a server. The reversible privacy processing and the irreversible privacy processing are implemented by the same machine learning model.

[0009] One or more embodiments of this specification are intended to provide a privacy-preserving image processing device, comprising: a processor; and a memory configured to store computer-executable instructions. When executed, the computer-executable instructions cause the processor to: obtain a first image containing user private information; perform reversible privacy processing on the first image to obtain a second image; or perform irreversible privacy processing on the first image to obtain a third image. The reversible privacy processing and the irreversible privacy processing are implemented using the same machine learning model. The second image or the third image is used to identify the user.

[0010] One or more embodiments of this specification are intended to provide a storage medium for storing computer-executable instructions that, when executed by a processor, implement the following method: obtaining a first image containing user private information; performing reversible privacy processing on the first image to obtain a second image; or performing irreversible privacy processing on the first image to obtain a third image. The reversible privacy processing and the irreversible privacy processing are implemented using the same machine learning model. The second image or the third image is used to identify the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in one or more of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 A flowchart of an image processing method based on privacy protection provided in one embodiment of this specification;

[0013] Figure 2 A schematic diagram of the structure of a machine learning model provided in one embodiment of this specification;

[0014] Figure 3 A schematic diagram of the training process of a machine learning model provided in one embodiment of this specification;

[0015] Figure 4 A schematic diagram of the structure of a machine learning model in training provided in one embodiment of this specification;

[0016] Figure 5 This is a schematic diagram of image privacy protection level settings provided in one embodiment of this specification;

[0017] Figure 6 A schematic diagram of setting a default image privacy protection level provided in an embodiment of this specification;

[0018] Figure 7 A flowchart of an image processing method based on privacy protection provided in another embodiment of this specification;

[0019] Figure 8 A flowchart of an image processing method based on privacy protection provided in another embodiment of this specification;

[0020] Figure 9 A schematic diagram of the structure of an image processing device based on privacy protection provided in one embodiment of this specification;

[0021] Figure 10 A schematic diagram of the structure of an image processing device based on privacy protection provided in one embodiment of this specification;

[0022] Figure 11 This is a structural diagram of an image processing device provided in one embodiment of this specification. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solutions in one or more of the present specification, the technical solutions in one or more of the present specification will be clearly and completely described below in conjunction with the drawings in one or more of the present specification. Obviously, the described embodiments are only one or more partial embodiments of the present specification, not all of the embodiments. Based on one or more of the embodiments in the present specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document. It should be noted that, in the absence of conflict, one or more of the embodiments in the present specification and the features in the embodiments can be combined with each other. One or more embodiments of the present specification will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0024] One or more embodiments of this specification provide a privacy-protected image processing method, apparatus, device, and storage medium, which can ensure the security of user privacy through image processing when a user device sends an image involving user privacy to a server.

[0025] Figure 1 This is a flowchart of a privacy protection-based image processing method provided in one embodiment of this specification. The method can be applied to and executed by a user device, which can be a computer, tablet computer, mobile phone, etc. Figure 1 As shown, the method includes the following steps:

[0026] Step S102: obtaining a first image containing user privacy information, and determining a privacy protection level of the first image according to a user operation;

[0027] Step S104: performing an operation corresponding to the privacy protection level on the first image according to the privacy protection level, the performed operation including one of the following: using the first image as the image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded;

[0028] Step S106: Upload the image to be uploaded to the server; wherein, the reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model.

[0029] In this embodiment, for images involving user privacy, before the images are uploaded to the server, the privacy protection level of the images can be determined based on user operations, and operations corresponding to the privacy protection level can be performed on the images, thereby protecting the privacy of the images and ensuring the security of user privacy.

[0030] In step S102, the user device obtains a first image containing user privacy information. In one scenario, the first image is an image captured by a camera of the user device and containing the user's biometric information. Accordingly, the user privacy information includes the user's biometric information, which includes at least one of fingerprint information, facial information, and pupil information. In another scenario, the first image is an image generated by the user device based on a privacy-sensitive application. The first image includes the user's privacy information. Examples of privacy-sensitive applications include memos, notepads, account books, and financial applications. The user privacy information includes, but is not limited to, the user's private content, such as account books, diaries, notes, and bills. In yet another scenario, the first image is an image captured by the user device, downloaded, received from another device, or generated based on an application. The user has assigned a tag to the first image. Based on the tag's content, if the tag's content indicates that the first image involves user privacy, the user device determines that the first image includes user privacy information. Examples of the tag's content include "private" and "secret."

[0031] In the above-mentioned step S102, the user device determines the privacy protection level of the first image based on the user operation. In one case, the user device prompts the user to set a privacy protection level for the first image, such as displaying three options of low, medium, and high to the user, and determines the privacy protection level of the first image based on the user's selection operation. In another case, when the user sets a label for the first image, the user device determines the privacy protection level of the first image based on the content of the label set by the user for the first image. If the label content includes a predefined first-category keyword, the privacy protection level of the first image is determined to be a medium level. If the label content includes a predefined second-category keyword, the privacy protection level of the first image is determined to be a high level. If the label content does not include the first and second-category keywords, the privacy protection level of the first image is determined to be a low level. The specific process of obtaining the first image and determining the privacy protection level of the first image will be described in detail later.

[0032] In the above-mentioned step S104, based on the privacy protection level of the first image, an operation corresponding to the privacy protection level is performed on the first image, wherein the privacy protection level of the first image includes three levels: low, medium, and high. The performed operation includes one of the following: if the privacy protection level is low, the first image is used as the image to be uploaded; if the privacy protection level is medium, reversible privacy processing is performed on the first image to obtain a second image, and the second image is used as the image to be uploaded; if the privacy protection level is high, irreversible privacy processing is performed on the first image to obtain a third image, and the third image is used as the image to be uploaded.

[0033] Specifically, when the privacy protection level of the first image is low, no additional processing is performed on the first image, and the first image is directly used as the image to be uploaded; when the privacy protection level of the first image is medium, reversible privacy processing is performed on the first image to obtain a second image, and the second image is used as the image to be uploaded; when the privacy protection level of the first image is high, irreversible privacy processing is performed on the first image to obtain a third image, and the third image is used as the image to be uploaded.

[0034] Reversible privacy processing can be achieved through reversible privacy encryption, and irreversible privacy processing can be achieved through irreversible privacy encryption. Reversible privacy encryption uses reversible operations to protect the privacy of images. The protected image is visually unrecognizable, but the original image can be restored through decryption. It can be used to restore the scene after a public opinion incident occurs. Its disadvantage is that it has poor security and can be easily restored to the original image through brute force. Irreversible privacy encryption is just the opposite. It uses irreversible operations to protect the privacy of images. The protected image is visually unrecognizable and the original image cannot be restored. It has better security, but has the disadvantage of being unable to obtain evidence after a public opinion incident occurs.

[0035] In certain scenarios, different users have different privacy protection requirements. For example, some users don't mind the leakage of their biometric information, while others believe that biometric information should be strictly protected. To address this situation, the image processing method in the above embodiment can determine the privacy protection level of the first image based on user operations and perform corresponding privacy protection operations. This balances user preferences and privacy protection, meeting the different privacy protection needs of different users and achieving a compromise between security and public opinion screening that meets user expectations.

[0036] In the above step S106, the user device uploads the image to be uploaded obtained in step S104 to the server, and the server can use the received image to perform various operations such as identity recognition and user authentication.

[0037] In this embodiment, the reversible privacy processing and the irreversible privacy processing in step S104 are implemented through the same machine learning model, which can be a neural network model. The following focuses on the relevant content of the machine learning model.

[0038] Figure 2 This is a schematic diagram of the structure of the machine learning model provided in one embodiment of this specification, such as Figure 2As shown, the machine learning model includes: a reversible privacy processing unit. In the above process, reversible privacy processing is performed on the first image to obtain the second image, specifically: the first image is input into the reversible privacy processing unit, and the reversible privacy processing is performed on the first image by the reversible privacy processing unit to obtain the second image. Specifically, the reversible privacy processing unit includes a reversible encoder, which can be a neural network model based on the UNET model structure. The training method of the reversible privacy processing unit will be introduced later. The first image is input into the reversible privacy processing unit, and the reversible privacy processing unit encrypts the first image by reversible encoding to obtain the second image.

[0039] like Figure 2 As shown, the machine learning model also includes: a nonlinear transformation unit, and the output of the reversible privacy processing unit is also connected to the nonlinear transformation unit. Based on this, in the above process, irreversible privacy processing is performed on the first image to obtain a third image, specifically: the first image is input into the reversible privacy processing unit, the first image is reversibly privacy-processed by the reversible privacy processing unit to obtain a second image, the second image is input into the nonlinear transformation unit, and the second image is nonlinearly transformed by the nonlinear transformation unit to obtain a third image. Specifically, the nonlinear transformation unit can be an irreversible enhancement encoder including multiple convolutional layers. When irreversibly processing the first image, the first image is first input into the reversible privacy processing unit for processing to obtain a second image, and then the second image is input into the nonlinear transformation unit for processing to obtain a third image. The nonlinear transformation unit can perform a nonlinear transformation operation on the second image to enhance the irreversibility of the third image.

[0040] In this embodiment, it is possible to Figure 2 The same model shown in the figure performs reversible privacy processing or irreversible privacy processing on the first image. The same model can be used to implement two image privacy protection operations, simplifying the complexity of image processing and the difficulty of model training.

[0041] The following introduces Figure 2 The specific training process of the machine learning model in . Figure 3 This is a diagram of the training process of the machine learning model provided in one embodiment of this specification, such as Figure 3 As shown, the above-mentioned reversible privacy processing unit and nonlinear transformation unit are trained as follows:

[0042] Step S302: obtaining a first training sample image and a preset first training constraint condition, and preliminarily training a reversible privacy processing unit using the first training sample image and the first training constraint condition;

[0043] Step S304: After the initial training of the reversible privacy processing unit is completed, a second training sample image and a preset second training constraint condition are obtained;

[0044] Step S306: Optimize the reversible privacy processing unit and train the nonlinear transformation unit using the second training sample image and the second training constraint condition.

[0045] In step S302, a first training sample image is obtained. The first training sample image can be an image containing biometric information obtained on the Internet through crawling or other means, such as a face image, pupil image, etc. Then, a preset first training constraint condition is obtained. The first training constraint condition includes but is not limited to the loss function and activation function used when training the reversible privacy processing unit. Then, the reversible privacy processing unit is preliminarily trained using the first training sample image and the first training constraint condition. The reversible privacy processing unit obtained through preliminary training is not the reversible privacy processing unit to be used in the end. After preliminary training, the reversible privacy processing unit needs to be optimized later.

[0046] In step S304, after the initial training of the reversible privacy processing unit is completed, a second training sample image and a second preset training constraint are obtained. The second training sample image can also be an image containing biometric information obtained online through crawling or other means. The second training sample image is preferably different from the first training sample image. The second training constraint includes, but is not limited to, the loss function and activation function used when optimizing the reversible privacy processing unit and training the nonlinear transformation unit.

[0047] In step S306, the reversible privacy processing unit is optimized and the nonlinear transformation unit is trained using the second training sample image and the second training constraint. In other words, when using the second training sample image and the second training constraint, the nonlinear transformation unit and the reversible privacy processing unit are trained simultaneously, thereby optimizing the reversible privacy processing unit.

[0048] During the above training process, the reversible privacy processing unit is preliminarily trained using the first training sample image and the first training constraint condition, specifically:

[0049] (a1) obtaining a first training sample image, and inputting the first training sample image into a reversible privacy processing unit for processing to obtain a first processed image;

[0050] (a2) decoding the first processed image using a preset decoder to obtain a first reconstructed original image;

[0051] (a3) Preliminarily training the reversible privacy processing unit based on a first distance between the first training sample image and the first processed image, a second distance between the first training sample image and the first reconstructed original image, and a first training constraint.

[0052] The reversible privacy processing unit can be a neural network model based on the UNET model structure. During the initial training, a first training sample image is obtained and input into the reversible privacy processing unit for processing to obtain a first processed image. The first processed image theoretically undergoes a certain reversible encryption process and has a certain gap with the first training sample image. Then, the first processed image is decoded and restored by a preset decoder to obtain a first reconstructed image. The decoder can also be a neural network model based on the UNET model structure. The decoder is not pre-trained and can be trained synchronously with the reversible privacy processing unit in the above (a1)-(a3) process. After the first processed image is input into the decoder, the decoder theoretically decodes and restores the first processed image to obtain a first reconstructed image. Since the reversible privacy processing unit performs reversible encryption on the first training sample image, the first reconstructed image should have a small gap with the first training sample image and be relatively similar.

[0053] Since, after both the decoder and the reversible privacy processing unit are trained, the first training sample image should have a certain distance from the first processed image, resulting in a significant visual difference, while the first training sample image should have a smaller distance from the first reconstructed image, resulting in a smaller visual difference, the reversible privacy processing unit is initially trained based on the first distance between the first training sample image and the first processed image, the second distance between the first training sample image and the first reconstructed original image, and the first training constraint. In this embodiment, the distance between the two images can be calculated using any method, such as the Euclidean distance, and this embodiment does not limit this.

[0054] Considering that when the decoder and the reversible privacy processing unit are trained, the first training sample image should have a certain gap with the first processed image, and the visual difference is large, while the first training sample image has a small gap with the first reconstructed image, and the visual difference is small, therefore, the first training constraint condition may include: the first distance is greater than the corresponding first distance threshold, and the second distance is less than the corresponding second distance threshold. In specific implementation, since there can be multiple first training sample images, and the first training sample image, the first processed image and the first reconstructed original image have different visual differences, the first training sample image and the first processed image have different visual differences. Figure 1There are multiple first processed images and first reconstructed original images. In this case, the average distance between the multiple first training sample images and the multiple first processed images can be used as the first distance, and the average distance between the multiple first training sample images and the multiple first reconstructed original images can be used as the second distance. The reversible privacy processing unit and the above-mentioned preset decoder are trained using the first training constraint: the first distance is greater than the corresponding first distance threshold, and the second distance is less than the corresponding second distance threshold. This can be considered as preliminary training of the reversible privacy processing unit and the decoder. The first training constraint can be laid out in the model as a loss function during model training.

[0055] During the above training process, the second training sample image and the second training constraint condition are used to optimize the reversible privacy processing unit and train the nonlinear transformation unit, specifically:

[0056] (a4) obtaining a second training sample image, and inputting the second training sample image into the reversible privacy processing unit for processing to obtain a second processed image;

[0057] (a5) inputting the second processed image into a nonlinear transformation unit for processing to obtain a third processed image;

[0058] (a6) decoding the second processed image using a preset decoder to obtain a second reconstructed original image, and decoding the third processed image using a decoder to obtain a third reconstructed original image;

[0059] (a7) Based on the third distance between the second training sample image and the second processed image, the fourth distance between the second training sample image and the second reconstructed original image, the fifth distance between the second training sample image and the third processed image, the sixth distance between the second training sample image and the third reconstructed original image, and the second training constraint, optimize the reversible privacy processing unit and train the nonlinear transformation unit.

[0060] After the initial training of the reversible privacy processing unit and the decoder is completed, a second training sample image, different from the first training sample image, is obtained and input into the reversible privacy processing unit for processing to obtain a second processed image. Theoretically, the second processed image undergoes certain reversible encryption processing and differs from the second training sample image by a certain amount. Next, the second processed image is input into the nonlinear transformation unit for nonlinear transformation processing to obtain a third processed image. The nonlinear transformation unit is used to add irreversible processing operations to the second processed image to ensure the irreversibility of the third processed image. Therefore, the third processed image should also differ from the second training sample image by a certain amount. In theory, the better the training of the nonlinear transformation unit, the more irreversible operations are added, and the stronger the irreversibility of the third processed image.

[0061] Next, the second processed image is decoded and restored by a preset decoder to obtain a second reconstructed original image, and the third processed image is decoded and restored by a decoder to obtain a third reconstructed original image. The decoder has been preliminarily trained during the preliminary training of the reversible privacy processing unit and can be further optimized during the training of the nonlinear transformation unit. After the second processed image is input to the decoder, the decoder theoretically decodes and restores the second processed image to obtain a second reconstructed image. Since the reversible privacy processing unit performs reversible encryption on the second training sample image, the second reconstructed image should be relatively similar to the second training sample image. Conversely, after the third processed image is input to the decoder, the decoder theoretically decodes and restores the third processed image to obtain a third reconstructed image. Since the nonlinear transformation unit performs irreversible processing on the second processed image, the third reconstructed image should be relatively different from the second training sample image.

[0062] Since the decoder, reversible privacy processing unit and nonlinear transformation unit are all trained, the second training sample image should have a certain gap with the second processed image, and the visual difference is large. The second training sample image and the second reconstructed image have a small gap and are visually similar. The second training sample image should have a certain gap with the third processed image, and the visual difference is large. The second training sample image and the third reconstructed image have a large visual gap. Therefore, the second training constraint conditions may include: the above-mentioned third distance is greater than the corresponding third distance threshold, the above-mentioned fourth distance is less than the corresponding fourth distance threshold, the above-mentioned fifth distance is greater than the corresponding fifth distance threshold, and the above-mentioned sixth distance is greater than the corresponding sixth distance threshold.

[0063] In a specific implementation, since there can be multiple second training sample images, and the second training sample image, the second processed image, the second reconstructed original image, the third processed image, and the third reconstructed original image are Figure 1One-to-one correspondence, so there are also multiple second processed images, second reconstructed original images, third processed images, and third reconstructed original images. In this case, the average distance between the multiple second training sample images and the multiple second processed images can be used as the third distance, the average distance between the multiple second training sample images and the multiple second reconstructed original images can be used as the fourth distance, the average distance between the multiple second training sample images and the multiple third processed images can be used as the fifth distance, and the average distance between the multiple second training sample images and the multiple third reconstructed original images can be used as the sixth distance. Using the second training constraint condition: the third distance is greater than the corresponding third distance threshold, the fourth distance is less than the corresponding fourth distance threshold, the fifth distance is greater than the corresponding fifth distance threshold, and the sixth distance is greater than the corresponding sixth distance threshold, the reversible privacy processing unit and the above-mentioned preset decoder are optimized and the nonlinear transformation unit is trained so that the reversible privacy processing unit, the decoder, and the nonlinear transformation unit are all trained. Among them, the second training constraint condition can be arranged in the model as a loss function in model training.

[0064] Figure 4 This is a schematic diagram of the structure of the machine learning model in training provided in an embodiment of this specification, such as Figure 4 As shown, the machine learning model under training includes not only a reversible privacy processing unit and a nonlinear transformation unit, but also a preset decoder. The preset decoder can be connected to the outputs of the reversible privacy processing unit and the nonlinear transformation unit to decode and restore the first processed image and the second processed image. The concept of training the machine learning model in this embodiment is as follows: Phase 1, reversible training: preliminary training of the reversible privacy processing unit and decoder; Phase 2, irreversible training: optimization of the reversible privacy processing unit and decoder and training of the nonlinear transformation unit.

[0065] In phase one, the reversible privacy processing unit and decoder are initially trained through the above-mentioned actions (a1)-(a3). During training, the first loss function, the second loss function, and the above-mentioned first training constraint are used for training. The first loss function is used to calculate the above-mentioned first distance, and the second loss function is used to calculate the above-mentioned second distance. When the values of the loss functions, i.e., the above-mentioned first and second distances, satisfy the above-mentioned first training constraint and converge, the training is determined to be complete. Then, phase two training is initiated. Through the above-mentioned actions (a4)-(a7), the reversible privacy processing unit and decoder are optimized and adjusted, and the nonlinear transformation unit is trained. During training, the first loss function, the second loss function, the third loss function, the fourth loss function, and the fourth loss function are used to calculate the above-mentioned sixth distance. When the values of the loss functions, i.e., the above-mentioned third, fourth, fifth, and sixth distances, satisfy the above-mentioned second training constraint and converge, the training of the reversible privacy processing unit, the decoder, and the nonlinear transformation unit is determined to be complete. Among them, the first loss function and the third loss function can be called privacy constraint functions, which are used to limit the difference between the images before and after processing. The second loss function can be called a reversible loss function, which is used to limit the ability to restore the image after reversible processing. The fourth loss function can be called an irreversible loss function, which is used to limit the ability to restore the image after irreversible processing. The distances between images mentioned in the above embodiments can all be Euclidean distances.

[0066] In one embodiment, the model training process is completed on the server side. After training is complete, the reversible privacy processing unit and the nonlinear transformation unit are deployed on the user device, and the decoder does not need to be deployed on the user device. Since irreversible image processing requires the joint implementation of the reversible privacy processing unit and the nonlinear transformation unit, when reversible privacy processing is performed on the first image, it is processed by the reversible privacy unit, and when irreversible privacy processing is performed on the first image, it is performed by the irreversible privacy unit and the nonlinear transformation unit.

[0067] In one embodiment, in the above step S102, the privacy protection level of the first image is determined according to the user operation, specifically: prompting the user to set the privacy protection level of the first image, and determining the privacy protection level of the first image according to the privacy protection level setting operation performed by the user on the first image.

[0068] Figure 5 This is a schematic diagram of image privacy protection level settings provided in an embodiment of this specification, such as Figure 5As shown, the user terminal can pop up a dialog box through interface interaction, prompting the user to set the privacy protection level for the first image, and providing three levels: low, medium, and high for the user to select. Based on the privacy protection level selection operation performed by the user on the first image, the level selected by the user is used as the privacy protection level of the first image. In other embodiments, the user device can also inquire about the privacy protection level of the first image via voice, and determine the privacy protection level of the first image based on the user's response to the voice operation.

[0069] When the user selects a low-level protection level, the first image is directly uploaded. When the user selects a medium-level protection level, a reversible privacy-preserving process is performed on the first image to obtain the uploaded image. When the user selects a high-level protection level, an irreversible privacy-preserving process is performed on the first image to obtain the uploaded image. Users can click the details button for each of the three levels to view detailed information.

[0070] In another embodiment, in the above step S102, the privacy protection level of the first image is determined based on the user operation, specifically by obtaining a label setting operation performed by the user on the first image, and determining the privacy protection level of the first image based on the label setting operation. In the case where the user has set a label for the first image, the label content set by the user is determined based on the user's label setting operation, and the privacy protection level of the first image is determined based on the label content. For example, when the label content includes predefined first-category keywords, the privacy protection level of the first image is determined to be a medium level. If the label content includes predefined second-category keywords, the privacy protection level of the first image is determined to be a high level. If the label content does not include first-category and second-category keywords, the privacy protection level of the first image is determined to be a low level.

[0071] In this embodiment, the privacy protection level of the first image can be automatically determined based on the user's tag setting operation on the first image without the user's awareness, without the user having to specifically perform the privacy protection level setting operation, thereby improving the user experience.

[0072] In one embodiment, after determining the privacy protection level of the first image, the method further includes: prompting the user to set a default image privacy protection level; and setting the privacy protection level of the first image to the default image privacy protection level according to the default level setting operation performed by the user.

[0073] Figure 6 This is a schematic diagram of setting the default image privacy protection level provided in an embodiment of this specification, such as Figure 6As shown, after determining the privacy protection level of the first image, the user terminal also prompts the user whether to set the privacy protection level of the first image to the default image privacy protection level. If the user checks yes, the user device sets the privacy protection level of the first image to the default image privacy protection level based on the default level setting operation performed by the user. Of course, the user terminal can also ask the user whether to set the privacy protection level of the first image to the default image privacy protection level through voice inquiry. If the user answers yes, the user device sets the privacy protection level of the first image to the default image privacy protection level based on the user's response.

[0074] In one embodiment, in the above step S102, obtaining the first image containing the user's private information is specifically: obtaining the first image containing the user's private information captured by a camera; wherein the user's private information includes the user's biometric information.

[0075] In one embodiment, after acquiring a first image captured by a camera and containing user privacy information, the process further includes: detecting whether the user's biometric information in the first image meets preset feature requirements; if so, determining the privacy protection level of the first image based on user operations. Preset feature requirements include, but are not limited to, clarity requirements, resolution requirements, and pixel size requirements. For example, the process detects whether the facial portion of the first image meets preset resolution requirements; if so, determining the privacy protection level of the first image based on user operations.

[0076] Figure 7 A flowchart of an image processing method based on privacy protection is provided in another embodiment of this specification, such as Figure 7 As shown, the process can be executed by the user equipment, including:

[0077] Step S702: obtaining a first image captured by a camera and containing user privacy information; wherein the user privacy information includes the user's biometric information;

[0078] Step S704, detecting whether the user's biometric information in the first image meets preset feature requirements;

[0079] Step S706: After determining that the conditions are met, prompt the user to set a privacy protection level for the first image;

[0080] Step S708, determining the privacy protection level of the first image according to the privacy protection level setting operation performed by the user on the first image;

[0081] Step S710, prompting the user to set a default image privacy protection level;

[0082] Step S712: setting the privacy protection level of the first image to a default image privacy protection level according to the default level setting operation performed by the user;

[0083] Step S714: Based on the privacy protection level of the first image, an operation corresponding to the privacy protection level is performed on the first image. The performed operation includes one of the following: using the first image as the image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded;

[0084] Step S716: Upload the image to be uploaded to the server. The reversible privacy protection and irreversible privacy protection are implemented using the same machine learning model. After the image to be uploaded is uploaded to the server, it is used to identify the user.

[0085] Figure 7 If it is detected that the biometric information of the user in the first image does not meet the preset feature requirements, the user is prompted to retake the first image.

[0086] The method described above can be applied to a facial recognition system in a user's device. The facial recognition system captures a user's facial image as a first image and implements the aforementioned process to perform privacy processing before uploading the facial image to a server. After the image is uploaded to the server, the server can perform operations such as user identification and authentication based on the image.

[0087] The image processing method provided in the above embodiment can realize reversible and irreversible privacy processing of images through a single model. During model training, the reversible and irreversible processing parts can be trained simultaneously, which simplifies the model structure and improves the training efficiency of the model.

[0088] Figure 8 A flowchart of an image processing method based on privacy protection is provided in another embodiment of this specification, such as Figure 8 As shown, the process can be executed by a user device or a server, including:

[0089] Step S802, obtaining a first image containing user privacy information;

[0090] Step S804: performing reversible privacy processing on the first image to obtain a second image, or performing irreversible privacy processing on the first image to obtain a third image;

[0091] Among them, reversible privacy processing and irreversible privacy processing are implemented through the same machine learning model; the second image or the third image is used to identify the user.

[0092] Figure 8 The method and Figure 1-Figure 7 The methods in are similar, so we will not explain them one by one here. Figure 8 The method in

[15] can achieve reversible or irreversible privacy processing of images through the same model, simplifying the model structure and improving the image processing efficiency, achieving the technical effect of protecting user privacy.

[0093] Figure 9 This is a structural diagram of an image processing device based on privacy protection provided in an embodiment of this specification, such as Figure 9 As shown, the device includes:

[0094] A first image acquisition unit 91, which acquires a first image containing user privacy information and determines a privacy protection level of the first image according to a user operation;

[0095] a first operation execution unit 92, configured to execute an operation corresponding to the privacy protection level on the first image according to the privacy protection level, wherein the executed operation includes one of the following: using the first image as an image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as an image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as an image to be uploaded;

[0096] The image uploading unit 93 uploads the image to be uploaded to the server; wherein the reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model.

[0097] Optionally, the machine learning model includes a reversible privacy processing unit: the first operation execution unit, which: inputs the first image into the reversible privacy processing unit, performs reversible privacy processing on the first image through the reversible privacy processing unit, and obtains the second image.

[0098] Optionally, the machine learning model also includes a nonlinear transformation unit: the first operation execution unit, which: inputs the first image into the reversible privacy processing unit, performs reversible privacy processing on the first image through the reversible privacy processing unit to obtain the second image; inputs the second image into the nonlinear transformation unit, performs nonlinear transformation on the second image through the nonlinear transformation unit to obtain the third image.

[0099] Optionally, the device also includes a model training unit, which: obtains a first training sample image and a preset first training constraint condition, and uses the first training sample image and the first training constraint condition to preliminarily train the reversible privacy processing unit; after the preliminary training of the reversible privacy processing unit is completed, obtains a second training sample image and a preset second training constraint condition; uses the second training sample image and the second training constraint condition to optimize the reversible privacy processing unit and train the nonlinear transformation unit.

[0100] Optionally, the model training unit obtains a first training sample image, inputs the first training sample image into the reversible privacy processing unit for processing, and obtains a first processed image; decodes the first processed image through a preset decoder to obtain a first reconstructed original image; and preliminarily trains the reversible privacy processing unit based on a first distance between the first training sample image and the first processed image, a second distance between the first training sample image and the first reconstructed original image, and the first training constraint condition.

[0101] Optionally, the model training unit obtains a second training sample image, inputs the second training sample image into the reversible privacy processing unit for processing to obtain a second processed image; inputs the second processed image into the nonlinear transformation unit for processing to obtain a third processed image; decodes the second processed image through a preset decoder to obtain a second reconstructed original image, and decodes the third processed image through the decoder to obtain a third reconstructed original image; based on the third distance between the second training sample image and the second processed image, the fourth distance between the second training sample image and the second reconstructed original image, the fifth distance between the second training sample image and the third processed image, the sixth distance between the second training sample image and the third reconstructed original image, and the second training constraint condition, optimizes the reversible privacy processing unit and trains the nonlinear transformation unit.

[0102] Optionally, the first training constraint condition includes: the first distance is greater than a corresponding first distance threshold, and the second distance is less than a corresponding second distance threshold.

[0103] Optionally, the second training constraint includes: the third distance is greater than the corresponding third distance threshold, the fourth distance is less than the corresponding fourth distance threshold, the fifth distance is greater than the corresponding fifth distance threshold, and the sixth distance is greater than the corresponding sixth distance threshold.

[0104] Optionally, the first image acquisition unit prompts the user to set a privacy protection level of the first image; and determines the privacy protection level of the first image according to a privacy protection level setting operation performed by the user on the first image.

[0105] Optionally, the first image acquisition unit acquires a label setting operation performed by a user on the first image; and determines the privacy protection level of the first image according to the label setting operation.

[0106] Optionally, the method further includes: a default setting unit, which prompts the user to set a default image privacy protection level; and sets the privacy protection level of the first image to the default image privacy protection level according to the default level setting operation performed by the user.

[0107] Optionally, the first image acquisition unit acquires a first image captured by a camera and containing user privacy information; wherein the user privacy information includes the user's biometric information.

[0108] Optionally, it further includes a detection unit, which detects whether the biometric information of the user in the first image meets the preset feature requirements; after determining that it meets the requirements, performs an operation of determining the privacy protection level of the first image according to the user operation.

[0109] Optionally, after the image to be uploaded is uploaded to the server, the image to be uploaded is used to identify the user.

[0110] The image processing device provided in this embodiment can realize the aforementioned Figure 1-Figure 7 The various processes of the image processing method in and achieve the same effect and function are not repeated here.

[0111] Figure 10 This is a structural diagram of an image processing device based on privacy protection provided in an embodiment of this specification, such as Figure 10 As shown, the device includes:

[0112] A second image acquisition unit 1001, which acquires a first image containing user privacy information;

[0113] A second operation execution unit 1002 is configured to perform reversible privacy processing on the first image to obtain a second image, or to perform irreversible privacy processing on the first image to obtain a third image;

[0114] The reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model; the second image or the third image is used to identify the user.

[0115] The image processing device provided in this embodiment can realize the aforementioned Figure 8The various processes of the image processing method in and achieve the same effect and function are not repeated here.

[0116] One or more embodiments of this specification further provide an image processing device based on privacy protection, which is used to execute the above-mentioned image processing method. Figure 11 This is a structural diagram of an image processing device provided in one embodiment of this specification, such as Figure 11 As shown, the image processing device may vary significantly due to different configurations or performances, and may include one or more processors 1101 and memory 1102, and the memory 1102 may store one or more applications or data. The memory 1102 may be a temporary storage or a persistent storage. The application stored in the memory 1102 may include one or more modules (not shown in the figure), each of which may include a series of computer-executable instructions in the image processing device. Furthermore, the processor 1101 may be configured to communicate with the memory 1102 to execute the series of computer-executable instructions in the memory 1102 on the image processing device. The image processing device may also include one or more power supplies 1103, one or more wired or wireless network interfaces 1104, one or more input / output interfaces 1105, one or more keyboards 1106, etc.

[0117] In a specific embodiment, the image processing device includes a processor and a memory arranged to store computer-executable instructions, wherein when the computer-executable instructions are executed, the processor implements the following process:

[0118] Acquire a first image containing user privacy information, and determine a privacy protection level of the first image according to a user operation;

[0119] performing an operation corresponding to the privacy protection level on the first image according to the privacy protection level, the performed operation including one of the following: using the first image as an image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded;

[0120] The image to be uploaded is uploaded to a server; wherein the reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model.

[0121] The image processing device provided in this embodiment can realize the aforementioned Figure 1-Figure 7 The various processes of the image processing method in and achieve the same effect and function are not repeated here.

[0122] In another specific embodiment, the image processing device includes a processor and a memory arranged to store computer-executable instructions, wherein when the computer-executable instructions are executed, the processor implements the following process:

[0123] Acquire a first image containing user private information;

[0124] performing reversible privacy processing on the first image to obtain a second image, or performing irreversible privacy processing on the first image to obtain a third image;

[0125] The reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model; the second image or the third image is used to identify the user.

[0126] The image processing device provided in this embodiment can realize the aforementioned Figure 8 The various processes of the image processing method in and achieve the same effect and function are not repeated here.

[0127] Furthermore, one or more embodiments of this specification further provide a storage medium for storing computer-executable instructions. In a specific embodiment, the storage medium may be a USB flash drive, an optical disk, a hard disk, etc. The computer-executable instructions stored in the storage medium, when executed by a processor, can implement the following process:

[0128] Acquire a first image containing user privacy information, and determine a privacy protection level of the first image according to a user operation;

[0129] performing an operation corresponding to the privacy protection level on the first image according to the privacy protection level, the performed operation including one of the following: using the first image as an image to be uploaded; performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded;

[0130] The image to be uploaded is uploaded to a server; wherein the reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model.

[0131] When the computer executable instructions in the storage medium provided in this embodiment are executed, the aforementioned Figure 1-Figure 7 The various processes of the image processing method in and achieve the same effect and function are not repeated here.

[0132] In another specific embodiment, the storage medium may be a USB flash drive, an optical disk, a hard disk, etc., and the computer executable instructions stored in the storage medium, when executed by the processor, can implement the following process:

[0133] Acquire a first image containing user private information;

[0134] performing reversible privacy processing on the first image to obtain a second image, or performing irreversible privacy processing on the first image to obtain a third image;

[0135] The reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model; the second image or the third image is used to identify the user.

[0136] When the computer executable instructions in the storage medium provided in this embodiment are executed, the aforementioned Figure 8 The various processes of the image processing method in and achieve the same effect and function are not repeated here.

[0137] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0138] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures such as diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0139] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the means for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the means for implementing various functions can be considered as both a software module implementing the method and a structure within the hardware component.

[0140] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0141] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing one or more of the present descriptions, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0142] It will be understood by those skilled in the art that one or more embodiments of this specification may be provided as methods, systems, or computer program products. Thus, one or more of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] One or more of the present specification is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to one or more embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0146] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0147] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0148] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0149] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0150] Those skilled in the art will appreciate that one or more embodiments of this specification may be provided as methods, systems, or computer program products. Thus, one or more of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] One or more of the present disclosures may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. One or more of the present disclosures may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0152] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0153] The foregoing description is merely an example of one or more embodiments of this specification and is not intended to limit this specification. Persons skilled in the art will readily appreciate that this specification may be modified and varied in various ways. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this specification shall be included within the scope of the claims of this specification.

Claims

1. A privacy-preserving image processing method, comprising: Acquire a first image containing user privacy information, and determine a privacy protection level of the first image according to a user operation; According to the privacy protection level, performing an operation corresponding to the privacy protection level on the first image, the performed operation comprising one of the following: using the first image as an image to be uploaded; Performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded; The image to be uploaded is uploaded to a server; wherein the reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model.

2. The method according to claim 1, wherein The machine learning model includes a reversible privacy processing unit: performing reversible privacy processing on the first image to obtain a second image, including: The first image is input into the reversible privacy processing unit, and the reversible privacy processing unit performs reversible privacy processing on the first image to obtain the second image.

3. The method according to claim 2, wherein: The machine learning model further includes a nonlinear transformation unit: performing irreversible privacy processing on the first image to obtain a third image, including: Inputting the first image into the reversible privacy processing unit, and performing reversible privacy processing on the first image by the reversible privacy processing unit to obtain the second image; The second image is input to the nonlinear transformation unit, and the nonlinear transformation unit performs nonlinear transformation on the second image to obtain the third image.

4. The method according to claim 3, wherein: The reversible privacy processing unit and the nonlinear transformation unit are trained in the following manner: Obtaining a first training sample image and a preset first training constraint condition, and preliminarily training the reversible privacy processing unit using the first training sample image and the first training constraint condition; After the initial training of the reversible privacy processing unit is completed, obtaining a second training sample image and a preset second training constraint condition; The reversible privacy processing unit is optimized and the nonlinear transformation unit is trained using the second training sample image and the second training constraint condition.

5. The method according to claim 4, wherein Preliminarily training the reversible privacy processing unit using the first training sample image and the first training constraint condition includes: Acquire a first training sample image, and input the first training sample image into the reversible privacy processing unit for processing to obtain a first processed image; Decoding the first processed image using a preset decoder to obtain a first reconstructed original image; The reversible privacy processing unit is preliminarily trained based on a first distance between the first training sample image and the first processed image, a second distance between the first training sample image and the first reconstructed original image, and the first training constraint condition.

6. The method according to claim 4, wherein: Optimizing the reversible privacy processing unit and training the nonlinear transformation unit using the second training sample image and the second training constraint condition, including: Acquire a second training sample image, and input the second training sample image into the reversible privacy processing unit for processing to obtain a second processed image; inputting the second processed image into the nonlinear transformation unit for processing to obtain a third processed image; Decoding the second processed image through a preset decoder to obtain a second reconstructed original image, and decoding the third processed image through the decoder to obtain a third reconstructed original image; Based on the third distance between the second training sample image and the second processed image, the fourth distance between the second training sample image and the second reconstructed original image, the fifth distance between the second training sample image and the third processed image, the sixth distance between the second training sample image and the third reconstructed original image, and the second training constraint condition, the reversible privacy processing unit is optimized and the nonlinear transformation unit is trained.

7. The method according to claim 5, wherein: The first training constraint condition includes: the first distance is greater than a corresponding first distance threshold, and the second distance is less than a corresponding second distance threshold.

8. The method according to claim 6, wherein: The second training constraint condition includes: the third distance is greater than the corresponding third distance threshold, the fourth distance is less than the corresponding fourth distance threshold, the fifth distance is greater than the corresponding fifth distance threshold, and the sixth distance is greater than the corresponding sixth distance threshold.

9. The method according to claim 1, wherein Determining the privacy protection level of the first image according to the user operation includes: prompting the user to set a privacy protection level for the first image; The privacy protection level of the first image is determined according to a privacy protection level setting operation performed by a user on the first image.

10. The method according to claim 1, wherein Determining the privacy protection level of the first image according to the user operation includes: Acquire a tag setting operation performed by the user on the first image; According to the tag setting operation, a privacy protection level of the first image is determined.

11. The method according to claim 9 or 10, wherein: After determining the privacy protection level of the first image, the method further includes: Prompt users to set the default image privacy protection level; According to a default level setting operation performed by the user, the privacy protection level of the first image is set to a default image privacy protection level.

12. The method according to claim 1, wherein Obtaining a first image containing user private information, including: A first image captured by a camera and containing user privacy information is obtained; wherein the user privacy information includes user biometric information.

13. The method according to claim 12, wherein: After obtaining the first image containing the user's private information captured by the camera, the method further includes: detecting whether the biometric information of the user in the first image meets preset feature requirements; After it is determined that the conditions are met, an operation of determining the privacy protection level of the first image according to a user operation is performed.

14. The method according to claim 1, wherein After the image to be uploaded is uploaded to the server, the image to be uploaded is used to identify the user.

15. A privacy-preserving image processing method, comprising: Acquire a first image containing user private information; determining a privacy protection level of the first image according to a user operation, and performing reversible privacy processing on the first image to obtain a second image, or performing irreversible privacy processing on the first image to obtain a third image, according to the privacy protection level; The reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model; the second image or the third image is used to identify the user.

16. An image processing device based on privacy protection, comprising: a first image acquisition unit, configured to acquire a first image containing user privacy information and determine a privacy protection level of the first image according to a user operation; a first operation execution unit, configured to execute an operation corresponding to the privacy protection level on the first image according to the privacy protection level, wherein the executed operation includes one of the following: using the first image as an image to be uploaded; Performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded; An image uploading unit uploads the image to be uploaded to a server; wherein the reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model.

17. The device according to claim 16, wherein The machine learning model includes a reversible privacy processing unit: the first operation execution unit, which: The first image is input into the reversible privacy processing unit, and the reversible privacy processing unit performs reversible privacy processing on the first image to obtain the second image.

18. The device according to claim 17, wherein The machine learning model further includes a nonlinear transformation unit: the first operation execution unit, which: Inputting the first image into the reversible privacy processing unit, and performing reversible privacy processing on the first image by the reversible privacy processing unit to obtain the second image; The second image is input to the nonlinear transformation unit, and the nonlinear transformation unit performs nonlinear transformation on the second image to obtain the third image.

19. The device according to claim 18, wherein Also included is a model training unit, which: Obtaining a first training sample image and a preset first training constraint condition, and preliminarily training the reversible privacy processing unit using the first training sample image and the first training constraint condition; After the initial training of the reversible privacy processing unit is completed, obtaining a second training sample image and a preset second training constraint condition; The reversible privacy processing unit is optimized and the nonlinear transformation unit is trained using the second training sample image and the second training constraint condition.

20. An image processing device based on privacy protection, comprising: a second image acquisition unit, which acquires a first image containing user privacy information; a second operation execution unit, configured to determine a privacy protection level of the first image according to a user operation, and perform reversible privacy processing on the first image to obtain a second image, or perform irreversible privacy processing on the first image to obtain a third image, according to the privacy protection level; The reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model; the second image or the third image is used to identify the user.

21. An image processing device based on privacy protection, the device comprising: processor; as well as a memory arranged to store computer-executable instructions which, when executed, cause the processor to: Acquire a first image containing user privacy information, and determine a privacy protection level of the first image according to a user operation; According to the privacy protection level, performing an operation corresponding to the privacy protection level on the first image, the performed operation comprising one of the following: using the first image as an image to be uploaded; Performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded; The image to be uploaded is uploaded to a server; wherein the reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model.

22. A storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the following method: Acquire a first image containing user privacy information, and determine a privacy protection level of the first image according to a user operation; According to the privacy protection level, performing an operation corresponding to the privacy protection level on the first image, the performed operation comprising one of the following: using the first image as an image to be uploaded; Performing reversible privacy processing on the first image to obtain a second image, and using the second image as the image to be uploaded; performing irreversible privacy processing on the first image to obtain a third image, and using the third image as the image to be uploaded; The image to be uploaded is uploaded to a server; wherein the reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model.

23. An image processing device based on privacy protection, the device comprising: processor; as well as a memory arranged to store computer-executable instructions which, when executed, cause the processor to: Acquire a first image containing user private information; determining a privacy protection level of the first image according to a user operation, and performing reversible privacy processing on the first image to obtain a second image, or performing irreversible privacy processing on the first image to obtain a third image, according to the privacy protection level; The reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model; the second image or the third image is used to identify the user.

24. A storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the following method: Acquire a first image containing user private information; determining a privacy protection level of the first image according to a user operation, and performing reversible privacy processing on the first image to obtain a second image, or performing irreversible privacy processing on the first image to obtain a third image, according to the privacy protection level; in, The reversible privacy processing and the irreversible privacy processing are implemented through the same machine learning model; the second image or the third image is used to identify the user.

Images