Multi-entity image privacy protection method, device and equipment

Abstract

The embodiments of this specification disclose a multi-entity image privacy protection method, apparatus, and device. The method includes: obtaining entity images corresponding to multiple entity categories, and performing single-entity desensitization on the entity images to obtain privacy images as training data; using the training data to train a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images; determining a target entity category and a support set of a small sample subset containing multiple different entity categories; and performing small sample learning on the multi-entity privacy protection model for the target entity category based on the support set.

Description

Technical Field

[0001] This specification relates to the field of Internet security technology, and in particular to a method, apparatus, and device for protecting the privacy of multi-entity images. Background Art

[0002] In recent years, computer vision technology has seen widespread industrial application alongside the development of artificial intelligence. These applications have become part of people's lives and production. Examples include self-service checkout at supermarkets, facial recognition at high-speed rail stations, intelligent beautification during live broadcasts, and intelligent assisted driving. However, these applications of computer vision technology all involve the collection, transmission, calculation, and storage of private information contained in images. These processes can potentially lead to the leakage of private information and, in turn, damage to users' personal information and property. Therefore, desensitizing entities containing private information in images has become a crucial challenge for computer vision applications.

[0003] Considered privacy protection schemes can be roughly divided into two categories. The first category is privacy protection schemes based on data encryption. These schemes use traditional encryption functions to encrypt the original image. Before performing calculations, the encrypted data is decrypted to obtain the original data, and then calculations are performed on the original data. These schemes are applicable to all entities (such as faces, goods, and vehicles in the image), but their security capabilities are relatively limited and can be easily compromised by attackers. Once the encryption function is compromised, all encrypted data will be leaked. The second category is privacy protection methods based on deep learning. These schemes use deep learning models to desensitize the original image to obtain a desensitized image. The desensitized image is then desensitized using a corresponding desensitization model to obtain a reconstructed original image. These schemes have better security capabilities, but a single model can often only desensitize and desensitize one entity. Once privacy protection is required for multiple entities, multiple models need to be trained, which makes the training and maintenance costs of the entire privacy protection system increasingly high as the number of entities increases.

[0004] Based on this, there is a need for a solution that can provide privacy protection for multi-entity images more efficiently and at a lower cost. Summary of the Invention

[0005] One or more embodiments of this specification provide a multi-entity image privacy protection method, apparatus, device, and storage medium to solve the following technical problem: a solution is needed that can provide privacy protection for multi-entity images more efficiently and at a lower cost.

[0006] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0007] One or more embodiments of this specification provide a multi-entity image privacy protection method, including:

[0008] For multiple different entity categories, obtain entity images corresponding to each entity category, and private images obtained by performing single-entity desensitization on the entity images as training data;

[0009] Using the training data, training a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images;

[0010] Determine the target entity category and the support set of a small subset of samples containing multiple different entity categories;

[0011] According to the support set, a small sample learning is performed on the multi-entity privacy protection model for the target entity category.

[0012] One or more embodiments of this specification provide a multi-entity image privacy protection device, including:

[0013] The single entity preparation module obtains entity images corresponding to each entity category and performs single entity desensitization on the entity images to obtain private images as training data.

[0014] A multi-entity training module, using the training data to train a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images;

[0015] A small sample preparation module determines the target entity category and the support set of a small sample subset containing multiple different entity categories;

[0016] The multi-entity fine-tuning module performs small-sample learning on the multi-entity privacy protection model for the target entity category according to the support set.

[0017] One or more embodiments of this specification provide a multi-entity image privacy protection device, including:

[0018] at least one processor; and,

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0021] For multiple different entity categories, obtain entity images corresponding to each entity category, and private images obtained by performing single-entity desensitization on the entity images as training data;

[0022] Using the training data, training a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images;

[0023] Determine the target entity category and the support set of a small subset of samples containing multiple different entity categories;

[0024] According to the support set, a small sample learning is performed on the multi-entity privacy protection model for the target entity category.

[0025] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to:

[0026] For multiple different entity categories, obtain entity images corresponding to each entity category, and private images obtained by performing single-entity desensitization on the entity images as training data;

[0027] Using the training data, training a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images;

[0028] Determine the target entity category and the support set of a small subset of samples containing multiple different entity categories;

[0029] According to the support set, a small sample learning is performed on the multi-entity privacy protection model for the target entity category.

[0030] At least one of the above-mentioned technical solutions adopted in one or more embodiments of this specification can achieve the following beneficial effects: it can utilize the rough training results corresponding to single-entity images of multiple different entity categories to assist in training a unified multi-entity privacy protection model for multiple entity images with temporarily acceptable effects, thereby eliminating the need to spend a lot of costs to accurately and continuously iterate each single-entity privacy protection model. If the multi-entity privacy protection model is actually deployed, the multi-entity privacy protection model can be targetedly learned in small samples on the deployment end based on the target entity category focused on in the application scenario and the closest small sample subset matched by the existing support set, thereby quickly and efficiently iterating a dynamic, locally sufficiently accurate, personalized multi-entity privacy protection model. It can also reduce the burden on the cloud and avoid the continuous drag of model training and maintenance costs by newly added entity categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0032] Figure 1 A simplified conceptual diagram of a multi-entity image privacy protection solution provided for one or more embodiments of this specification;

[0033] Figure 2 A flowchart of a multi-entity image privacy protection method provided in one or more embodiments of this specification;

[0034] Figure 3 In an application scenario provided by one or more embodiments of this specification, Figure 2 A detailed flow chart of the method;

[0035] Figure 4 A schematic diagram of the structure of a multi-entity image privacy protection device provided in one or more embodiments of this specification;

[0036] Figure 5 A schematic diagram of the structure of a multi-entity image privacy protection device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0037] The embodiments of this specification provide a multi-entity image privacy protection method, apparatus, device, and storage medium.

[0038] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0039] In this solution, the entity refers to one or more objects containing private information that appear in the image, such as the user's face, the merchant's products, the background of the live broadcast, the license plate of a private car, the recipient information on the express box, etc.

[0040] In order to reduce the training and maintenance costs of deep learning models under multi-entity privacy protection, a multi-entity image privacy protection scheme based on model conversion and small-sample learning iteration is proposed. In summary, a cold-start multi-entity privacy protection model is first trained in the cloud based on multiple single-entity privacy protection models so that it can be deployed on the deployment end. Then, small-sample learning is performed according to the target entity type on the deployment end, and rapid iteration is performed to obtain a privacy protection model that can be applied to multiple entities simultaneously.

[0041] Figure 1 This is a simplified diagram of the concept of a multi-entity image privacy protection solution provided in one or more embodiments of this specification. The concept mainly includes the following four parts, which are briefly introduced as examples.

[0042] The first part involves cold-start privacy-preserving model training. Based on single-entity privacy protection, a multi-entity privacy-preserving model is trained in the cloud. This model can protect the privacy of multiple entities, but its performance for each entity may be average.

[0043] The second part involves training the entity classification model. The model is trained in the cloud and distributed to the deployment end (e.g., mobile phones and tablets) to discover locally more prominent or popular entity categories as target entity categories.

[0044] The third part is the support set distribution. The cloud selects the support sets of various entities and distributes them to the deployment end for direct use or as a seed set for further processing.

[0045] Part 4: Small-sample rapid iteration and privacy protection. The deployment end dynamically perceives the target entity category and performs small-sample rapid model iteration and updates to obtain a personalized multi-entity privacy-preserving model.

[0046] Based on this concept, the following is further described in detail.

[0047] Figure 2 A flowchart of a multi-entity image privacy protection method provided for one or more embodiments of this specification. The method can be applied to the model training and use processes in different business fields to improve the privacy of model input data. These business fields include, for example, image processing, financial business, e-commerce business, instant messaging business, gaming business, official business, etc. The process can be executed by a computing device in the corresponding field (typically a model training server in the cloud), and certain input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.

[0048] The multi-entity image here represents an image usage scenario in which there are multiple entities of different types that require privacy protection. These different types of entities can appear in the same image (for example, faces and license plates appearing simultaneously in a traffic road image), or they can appear separately in a batch of different images (for example, during live streaming, the host promotes different types of goods in sequence according to the time period, with only one type of goods appearing at the same time, resulting in successive images of the goods). This solution has a particularly outstanding advantage in the former case, as it can more accurately perceive and distinguish different types of entities in the same image, while some previously attempted models are more likely to confuse these entities or even miss them.

[0049] Figure 2 The process in may include the following steps:

[0050] S202: For a plurality of different entity categories, entity images corresponding to each entity category and private images obtained by performing single-entity desensitization on the entity images are obtained as training data.

[0051] The granularity of entity classification depends on the actual scenario requirements. For example, for commodities, under coarse-grained requirements, all commodities may be classified into the same entity category, while other entity categories may include people and natural objects in the environment. Under fine-grained requirements, household appliances, food, furniture, etc. may be classified into different entity categories. Of course, further subdivision is possible if necessary.

[0052] In one or more embodiments of this specification, the location area of ​​the entity and the entity category to which it belongs are annotated in the entity image, and these annotations are used to train single-entity privacy protection. To improve the generalization of the model, as many entity images of different types as possible are collected for processing. If only one type of entity appears in a single entity image, the requirements for annotating its location area can be relaxed, and it may not even be necessary to annotate the location area. Therefore, it is possible to consider collecting as many entity images as possible as training data to help more focused learning of the characteristics of individual entity categories.

[0053] Desensitization, for example, involves modifying, cropping, and masking pixels in corresponding areas of an image to prevent the exposure of sensitive information reflected by these pixels. By desensitizing the entity to be protected in the image, the privacy of the entity and its corresponding user is protected. In this case, if the entity in the image can be identified as accurately as possible and a high degree of desensitization is performed, basic privacy protection needs are met. However, in some practical applications, the sensitive information reflected by the entity in the image may be required to continue normal business. In such cases, after desensitization, the ability to restore the private image more accurately and in a timely manner is also required.

[0054] S204: Using the training data, training a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images.

[0055] The above training data can be used to directly train a multi-entity privacy-preserving model. However, this approach is relatively inefficient and the model converges slowly.

[0056] Based on this, a two-step approach can be considered: first, train a corresponding single-entity privacy-preserving model for each entity category; second, use the auxiliary supervision of each single-entity privacy-preserving model to transform and train a multi-entity privacy-preserving model. This approach allows the multi-entity privacy-preserving model to extract and condense the capabilities of each single-entity privacy-preserving model. This is not only efficient, but also allows at least part of the training data in S205 to be generated by the single-entity privacy-preserving model, reducing the burden of manually creating training data.

[0057] In one or more embodiments of the present specification, the same image containing entities of multiple different entity types is input into a multi-entity privacy protection model, which can intelligently desensitize multiple entities in the image at one time, and can also restore the desensitized state to the original state more accurately when necessary.

[0058] S206: Determine the target entity category and determine the support set of the small sample subset containing multiple different entity categories.

[0059] In one or more embodiments of this specification, when a single-entity privacy-preserving model is first trained, its performance is allowed to be in a general state. This is because when training a multi-entity privacy-preserving model, it can also provide complementary guidance for the optimization of each single-entity privacy-preserving model. Furthermore, for a multi-entity privacy-preserving model, the performance is also allowed to be in a general state because it will be subsequently quickly and locally optimized through small sample learning.

[0060] Based on this approach, the newly trained multi-entity privacy-preserving model is used as a general, relatively crude cold-start model. The cloud handles this workload, which is minimal due to its low requirements. The cloud then distributes this cold-start model to business terminals (the aforementioned deployment terminals) with multi-entity privacy protection requirements, performing personalized tuning based on small sample sizes. This allows business terminals to achieve a more tailored model with minimal additional burden.

[0061] In one or more embodiments of this specification, the target entity category reflects the personalized needs of a corresponding service terminal. The target entity category needs to be determined for the service terminal based on the actual conditions of different service terminals.

[0062] For a specific business terminal, most of the entities in its images tend to be concentrated in a small number of entity categories. For example, cash registers for facial payment primarily capture images of faces, primarily involving the face entity category. Traffic cameras primarily capture images of vehicles, primarily involving the vehicle and license plate entity categories. The entity categories primarily involved in these business terminals can be identified as target entity categories.

[0063] In addition to these situations, for certain business terminals, some rarely occurring and unexpected entity types may also become target entity categories. For example, for access control devices, the entity types depicted in the images captured by the living creatures are mostly ordinary visitors. However, entity types such as VIP visitors are less common, but the probability is not zero. VIP visitors may be more privacy-sensitive. In this case, VIP visitors can be identified as target entity categories.

[0064] In one or more embodiments of this specification, considering that general business terminals do not have the ability to prepare training data on their own due to factors such as non-standardized data or difficulty in determining labels, the cloud pre-prepares a support set of small sample subsets containing multiple different entity categories, and then distributes at least part of the support set to each business terminal for self-tuning. Each small sample subset contains a small number of images belonging to a corresponding entity category and their labels. The labels here can be, for example, a desensitized image used as a standard reference, a restored reconstructed image, or the original image itself.

[0065] The timing of supporting set distribution is flexible. Simply distributing the entire support set upfront is relatively straightforward. Each service terminal can then freely determine the target entity category. This is particularly suitable for situations where the target entity category of a single service terminal frequently changes. If the target entity category of some service terminals can be predetermined or is relatively fixed, only the portion of the support set relevant to the target entity category can be distributed. This reduces both the transmission burden and the storage burden on the service terminal.

[0066] S208: Performing small sample learning on the multi-entity privacy protection model for the target entity category according to the support set.

[0067] For a single business endpoint, there are only a few target entity categories. Based on the support set, we attempt to identify a small sample that successfully matches the target entity category (e.g., directly belonging to the target entity category or belonging to another similar entity category). Through small sample learning, we can quickly fine-tune the multi-entity privacy-preserving model, improving its effectiveness for the target entity category while remaining average for other entity categories. This fine-tuning can be performed dynamically, ensuring efficient alignment of the model with the current business endpoint at a minimal cost, precisely meeting individual needs.

[0068] In one or more embodiments of this specification, after deployment on a business terminal, and even after small-sample learning, the multi-entity privacy protection model can be used in actual business operations. For example, a business terminal receives an original image to be protected, which may contain multiple entities for which privacy protection is required. Using the multi-entity privacy protection model, the protected entities or entities contained in the protected area are desensitized to obtain a private image corresponding to the original image. When required for normal business operations, the private image is then restored and reconstructed for normal business use. Throughout this process, the image is kept desensitized as much as possible.

[0069] In order to cater to user needs more flexibly, the user can be allowed to specify (for example, manually circle or select through options, etc.) the entity or area to be protected in the original image to be protected in advance, and the model will only desensitize the object specified by the user.

[0070] pass Figure 2The method can use the rough training results corresponding to single-entity images of multiple entity categories to assist in training a unified multi-entity privacy protection model for multi-entity images with temporarily acceptable results, thereby eliminating the need to spend a lot of cost to accurately and continuously iterate each single-entity privacy protection model. If the multi-entity privacy protection model is actually deployed, the deployment end can perform targeted small-sample learning on the multi-entity privacy protection model based on the target entity category focused on in the application scenario and the closest small-sample subset matched by the existing support set, thereby quickly and efficiently iterating a dynamic, locally, sufficiently accurate, personalized multi-entity privacy protection model. It can also reduce the burden on the cloud and avoid the continuous increase in model training and maintenance costs due to the addition of new entity categories.

[0071] based on Figure 2 This specification also provides some specific implementation plans and extension plans of the method, which will be described below.

[0072] In one or more embodiments of this specification, the training process of multiple models involves training single-entity privacy-preserving models, cloud-based training of multi-entity privacy-preserving models, and on-device training, as well as supplementary training of entity classification models. These models have different intended uses. To better achieve these uses, this solution cleverly configures the loss function for each model, which also helps to more efficiently control model training convergence. The following describes the structure and loss function of these models as examples.

[0073] As mentioned earlier, a single-entity privacy protection model can be trained for each entity type to prepare training samples for subsequent training and to assist in supervising the multi-entity privacy protection model.

[0074] For a single-entity privacy protection model, it must at least desensitize a single entity in the image. To better protect privacy, the higher the degree of desensitization, the better. For this purpose, we define a desensitization loss. To distinguish it from similar concepts, we call it the first desensitization loss (for the same reason, we will follow this naming convention later, so I won't go into detail here). Furthermore, normal business operations may require the use of private content. Therefore, it may be necessary to restore the desensitized private image to attempt to reconstruct the original image. The closer the reconstructed image is to the original image, the better. For this purpose, we define the first reconstruction loss.

[0075] Based on this idea, a first loss function is defined that includes a first desensitization loss and / or a first reconstruction loss. For each entity category, a corresponding single-entity privacy-preserving model is constructed and trained based on the first loss function. As shown in the previous analysis, a larger first desensitization loss is preferred, while a smaller first reconstruction loss is preferred. These two goals are somewhat contradictory, as the first loss function can be used to create a state in which the two losses compete with each other, enabling adversarial training of the single-entity privacy-preserving model.

[0076] Using the trained single-entity privacy protection model, single-entity desensitization is performed on the entity images corresponding to its corresponding entity category to obtain private images, which serve as training data for the multi-entity privacy protection model.

[0077] In one or more embodiments of this specification, a second desensitization loss and a second reconstruction loss are similarly defined for the multi-entity privacy-preserving model. In addition, a model conversion loss from a single entity to a multi-entity model is also defined. This model conversion loss reflects the difference in processing between the two models for the same entity in the same image (which may only occupy a small area of ​​the image). The smaller this processing difference, the better. In this way, during training, the multi-entity privacy-preserving model can quickly extract and condense the capabilities of each single-entity privacy-preserving model through the constraints of this model conversion loss.

[0078] Based on this idea, a second loss function is determined for the multi-entity privacy protection model, which includes the second desensitization loss, the second reconstruction loss, and the model conversion loss from single entity to multi-entity. According to the second loss function, the multi-entity privacy protection model is trained adversarially.

[0079] Training a multi-entity privacy-preserving model presents another risk. It requires comprehensive capabilities for handling different entity types, a fusion of the sub-capabilities for handling single entity types. However, during training, the constraints imposed by different single entity types may differ from their actual effectiveness, and the discrepancy can even be significant. Some may become dominant single entity types, while others, while not necessarily dominant, may exhibit high responsiveness (meaning that even small, local inputs significantly affect the model's inference results). This small number of single entity types can bias the training process of the multi-entity privacy-preserving model, trapping it in a local optimal solution. This can lead to very uneven performance of the multi-entity privacy-preserving model for different entity types, and even have a negative effect on some entity types.

[0080] To address the issue in the previous paragraph, this solution aims to intentionally make certain uniformity or generalization adjustments to the multi-entity privacy protection model for different entity types during the training process. This adjustment approach is not conducive to processing multiple entities of different types in the same image. However, under the overall concept of this solution, it is possible to compensate for this disadvantage. Subsequently, on the business terminal, it is necessary to reflect on the path of small-sample learning for the target entity type.

[0081] Therefore, a more sophisticated training scheme with better matching losses is further provided, including: adding an additional entity classifier that outputs entity classification probabilities to the multi-entity privacy protection model, and defining an additional entity type decoupling loss in the above-mentioned second loss function, for example, to force the classifier output probability to tend to the average (to a certain extent, forcibly generalize or homogenize different entity types). The smaller the entity type decoupling loss, the better, so that the model's capabilities on each entity type can tend to be averaged during desensitization. This helps to better serve as a cold start model when the specific protection type is unknown.

[0082] In one or more embodiments of the present specification, when performing small-sample learning on a multi-entity privacy protection model for a target entity category, the loss function is adjusted accordingly to focus on the verification of small samples, thereby controlling the model to converge quickly towards the optimization direction.

[0083] For example, a third loss function is defined for the multi-entity privacy-preserving model, which includes a third desensitization loss, a third reconstruction loss, and a support set verification loss. Based on this third loss function and a small subset of samples from the support set that successfully match the target entity category, the multi-entity privacy-preserving model is adversarially trained. The support set verification loss reflects the gap between the processing of the small sample and its label; the smaller this loss, the better.

[0084] It should be noted that multiple losses are defined in the above loss function. According to the actual needs of efficiency and accuracy, some losses can be traded off, which will affect the effect of the model, but will not hinder the realization of the overall concept of this solution.

[0085] In addition, for ease of understanding, the above models are introduced separately, and they can be trained modularly. It is also possible to integrate these models at least partially for a more advanced end-to-end training. In this case, it is also possible to consider integrating some loss functions as well, which will help improve overall training efficiency.

[0086] In one or more embodiments of the present specification, the trained multi-entity privacy protection model is sent to the business terminal, and the business terminal decides how to tune the small sample. Specifically, for example, the entity category corresponding to the original image used for multi-entity privacy protection model processing on the business terminal is dynamically detected, and according to the detection results, the target entity category is dynamically determined among a plurality of different entity categories. For example, the detection data is periodically counted, and a part of the entity categories that appear relatively more or more frequently in the period are determined as the target entity category. In order to better perform detection, an entity classification model is pre-built and trained to detect to which entity category the entity in the original image on the business terminal belongs.

[0087] When learning with small samples, accuracy is crucial. The cloud provides support sets to business terminals, and the cloud can pre-collect support sets that cover as many entity categories as possible. However, this presents challenges in practical applications. First, achieving a complete set of entity categories is inherently challenging. Second, the more comprehensive the entity categories, the greater the burden on the business terminal when distributing the support sets.

[0088] Based on this, this solution aims to meet the diverse needs of different business terminals with a reasonably sized support set. There are two scenarios: The first is when the target entity category of the business terminal happens to be one of the multiple entity categories pre-prepared by the cloud. In this case, a small sample subset belonging to the target entity category can be directly found in the support set. The second is when the target entity category is determined to be not among the multiple entity categories but is a newly added entity category. This situation is a major obstacle to achieving the goal of a "reasonably sized support set." The following provides a solution.

[0089] Under the first solution, the newly added target entity categories (for example, image samples on a business terminal) are compared for similarity among the entity categories in the support set, and the small sample subset of the entity category with the highest similarity is used as the small sample subset that successfully matches the target entity category. When comparing similarities, for example, comparisons are made from dimensions such as the semantics of the entity category name, entity outline, and entity texture. This method is low-cost, but the accuracy is not stable enough. Based on this, a second solution is also provided.

[0090] Under the second solution, the image sample on the business terminal (representing the newly added entity category) and the small sample (representing the entity category of the existing small sample) of the entity category specified in the support set (for example, giving priority to similar ones, or traversing and trying and then revising according to the subsequent results) are respectively used as a gradient state endpoint, assuming that one is called the first gradient state endpoint and the other is called the second gradient state endpoint; then, based on the image sample and the small sample, the image gradient processing is performed from the first gradient state endpoint to the second gradient state endpoint to obtain multiple intermediate state images between the first gradient state endpoint and the second gradient state endpoint; then, a part of the intermediate state images (for example, relatively centered or slightly close to one side of the image sample) is selected as the small sample of the newly added entity category, that is, as the small sample subset that successfully matches the target entity category. In this way, a large number of small samples that did not originally exist can be reasonably and more efficiently derived based on a relatively small support set.

[0091] The above describes the present solution in detail. Based on these descriptions, more intuitively, one or more embodiments of this specification further provide an application scenario. Figure 2 A detailed process of the method in order to help better understand this solution, the detailed process is as follows Figure 3 shown.

[0092] Figure 3 The process in may include the following steps:

[0093] S302: Collect as many types of entity images as possible and mark the entity categories and location areas of the entities.

[0094] Assume that the number of entity categories involved in the collected entity images is N.

[0095] S304: Using the labeled data, for each type of entity, a single-entity privacy protection model is adversarially trained based on the first desensitization loss and the first reconstruction loss.

[0096] Then a total of N single-entity privacy protection models are obtained.

[0097] The single-entity privacy protection model, for example, includes a first desensitizing encoder that inputs an original image and outputs a corresponding private image, and a first anti-desensitizing decoder that inputs the corresponding private image and outputs a reconstructed original image.

[0098] The first loss function of the single-entity privacy protection model is, for example, formulated as follows:

[0099]

[0100] Among them, I represents the area where the corresponding entity is located in the image, subscript ori represents original, subscript pri represents desensitized privacy protection, and subscript rec represents reconstruction; Loss single represents the total loss of the single-entity privacy protection model. The first part of the total loss (i.e. ) represents the first desensitization loss, that is, the difference between the regions where the corresponding entities are located in the images before and after desensitization. The second part of the total loss (i.e. ) represents the first reconstruction loss, which is the difference between the region where the corresponding entity is located in the image before desensitization and the reconstructed image. The loss can be calculated for the region where the corresponding entity is located, and not for other regions.

[0101] Adversarial training is performed with the training objectives of maximizing the first desensitization loss, minimizing the first reconstruction loss, and minimizing the total loss until the model converges, thereby obtaining N single-entity privacy protection models.

[0102] S306: Based on the second desensitization loss, the second reconstruction loss, the model conversion loss from single entity to multi-entity, and the entity type decoupling loss, capabilities are extracted and concentrated from each single-entity privacy protection model, and adversarial training is performed to obtain a multi-entity privacy protection model.

[0103] The multi-entity privacy protection model, for example, includes a second desensitizing encoder that inputs an original image and outputs a corresponding private image, and a second anti-desensitizing decoder that inputs the corresponding private image and outputs a reconstructed original image. It also includes an entity classifier that inputs the intermediate layer features of the second desensitizing encoder (for example, taking the output of the middle layer as a multi-channel feature map) and outputs the entity classification probability. The entity classifier is, for example, a fully connected layer.

[0104] The second loss function of the multi-entity privacy protection model adopts the following formula:

[0105]

[0106] Among them, p represents the entity classification probability output by the entity classifier, the subscript pri represents desensitization privacy protection (on the multi-entity privacy protection model), and the subscript pri single Indicates the desensitized privacy protection on the corresponding single-entity privacy protection model. The subscript pred represents the classification prediction result of the entity classifier for the corresponding entity. The subscript avg represents the average classification prediction result of the entity classifier for each entity. Loss multi represents the total loss of the multi-entity privacy protection model. The first part of the total loss (i.e. ) represents the second desensitization loss, that is, the difference between the regions where the corresponding entities are located in the images before and after desensitization. The second part of the total loss (i.e. ) represents the second reconstruction loss, that is, the difference between the image before desensitization and the area where the corresponding entity is located in the reconstructed image. The third part of the total loss (i.e. ) represents the model conversion loss from single entity to multi-entity, that is, the difference in processing of the corresponding entities in the image by the single-entity privacy protection model and the multi-entity privacy protection model. The fourth part of the total loss (i.e. ) represents the entity type decoupling loss.

[0107] Adversarial training is performed with the training objectives of maximizing the second desensitization loss, minimizing the second reconstruction loss, model conversion loss and entity type decoupling loss, and minimizing the total loss until the model converges, thus obtaining a multi-entity privacy protection model.

[0108] At this point, the preparation of the most important cold start model in the cloud has been completed.

[0109] S308: Training an entity classification model for performing entity classification on multiple entity types.

[0110] For multiple (N) different entity categories, a binary classification loss function is defined for each entity category. According to each binary classification loss function, the entity classification model is trained. The input of the entity classification model is the original image to be classified, and the output is the binary classification result corresponding to each entity category (for example, N attributes are output, each attribute corresponds to an entity, and the first attribute is 1, which means that the first entity exists in the image).

[0111] For example, the ShuffleNetV2x0.25 model is used. Training is performed based on this model and the binary classification loss function until the model converges.

[0112] S310: Constructing a support set containing a small sample subset of each entity type for multiple entity types.

[0113] For N entity categories, for each entity category, a small number (e.g., 5) of original images and corresponding privacy-preserving images are randomly selected as the corresponding small sample subset.

[0114] S302 to S310 are executed in the cloud.

[0115] S312: The cloud sends the entity classification model, the support set, and the multi-entity privacy protection model as a cold start model to the business terminal.

[0116] S314: On the service terminal, the entity classification model is used to periodically perform entity classification on the original images within a certain time period, the entity categories that appear within the certain time period are recorded, and the target entity category is determined based on the records.

[0117] S316: On the business terminal, a small sample subset that successfully matches the target entity category is determined based on the support set, and a small sample learning is performed on the multi-entity privacy protection model based on the third desensitization loss, the third reconstruction loss, and the support set verification loss.

[0118] The model structure is the same as that of the cold start model, and its third loss function adopts the following formula:

[0119]

[0120] Among them, the subscript pri support Indicates the processing result for the support set, the subscript pri support-gt Indicates the label corresponding to the processing result; Loss multi ' represents the total loss of the multi-entity privacy-preserving model being tuned. The first part of the total loss (i.e. ) represents the third desensitization loss, and the second part of the total loss (i.e. ) represents the third reconstruction loss, the third part of the total loss (i.e. ) represents the support set validation loss.

[0121] Adversarial training is performed with the training objectives of maximizing the third desensitization loss, minimizing the third reconstruction loss and support set verification loss, and minimizing the total loss until the model converges, thus obtaining the tuned multi-entity privacy protection model.

[0122] S318: Desensitizing the entity to be protected or the entity contained in the area to be protected using the tuned multi-entity privacy protection model to obtain a private image corresponding to the original image to be protected.

[0123] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figure 4 、 Figure 5 shown.

[0124] Figure 4 A schematic diagram of a multi-entity image privacy protection device provided in one or more embodiments of this specification, the device comprising:

[0125] The single entity preparation module 402 obtains entity images corresponding to each entity category and private images obtained by performing single entity desensitization on the entity images as training data.

[0126] A multi-entity training module 404 trains a multi-entity privacy protection model using the training data, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images;

[0127] A small sample preparation module 406 determines a target entity category and a support set of a small sample subset containing multiple different entity categories;

[0128] The multi-entity fine-tuning module 408 performs small-sample learning on the multi-entity privacy protection model according to the support set and for the target entity category.

[0129] Optionally, the single entity preparation module 402 defines a first loss function including a first desensitization loss and a first reconstruction loss;

[0130] For each entity category, construct a corresponding single-entity privacy protection model, and perform adversarial training on the single-entity privacy protection model based on the first loss function;

[0131] Using the single-entity privacy protection model, single-entity desensitization is performed on the entity images corresponding to the corresponding entity categories to obtain private images as training data;

[0132] The single-entity privacy protection model includes a first desensitizing encoder that inputs an original image and outputs a corresponding private image, and a first anti-desensitizing decoder that inputs the corresponding private image and outputs a reconstructed original image.

[0133] Optionally, the multi-entity privacy protection model includes a second desensitizing encoder that inputs an original image and outputs a corresponding private image, and a second anti-desensitizing decoder that inputs the corresponding private image and outputs a reconstructed original image;

[0134] The multi-entity training module 404 determines a second loss function defined for the multi-entity privacy protection model, comprising a second desensitization loss, a second reconstruction loss, and a model conversion loss from a single entity to a multi-entity model;

[0135] According to the second loss function, adversarial training is performed on the multi-entity privacy protection model.

[0136] Optionally, the multi-entity privacy protection model further includes an entity classifier that inputs the intermediate layer features of the second desensitizing encoder and outputs entity classification probabilities;

[0137] The multi-entity training module 404 determines the second loss function further including entity type decoupling loss;

[0138] The multi-entity privacy protection model is adversarially trained with the adversarial training objectives of maximizing the second desensitization loss, minimizing the second reconstruction loss, the model conversion loss from a single entity to multiple entities, and the entity type decoupling loss.

[0139] Optionally, the small sample preparation module 406 determines a corresponding small sample subset for each entity category for a plurality of different entity categories;

[0140] A support set including each of the small sample subsets is determined, so as to dynamically determine a target entity category among the multiple different entity categories and / or newly added entity categories.

[0141] Optionally, the small sample preparation module 406 sends the multi-entity privacy protection model to a service terminal so as to perform the small sample learning on the service terminal;

[0142] Dynamically detecting an entity category corresponding to an original image processed by the multi-entity privacy protection model on the service terminal;

[0143] According to the detection result, the target entity category is dynamically determined among the multiple different entity categories.

[0144] Optionally, the small sample preparation module 406 pre-defines a binary classification loss function for each of the multiple entity categories;

[0145] Training an entity classification model according to each of the binary classification loss functions, wherein the input of the entity classification model is the original image to be classified as an entity, and the output is the binary classification result corresponding to each of the entity categories;

[0146] The entity classification model is used to dynamically detect entity categories corresponding to original images processed by the multi-entity privacy protection model on the service terminal.

[0147] Optionally, the business terminal includes a cash register device.

[0148] Optionally, the multi-entity fine-tuning module 408 determines a third loss function defined for the multi-entity privacy protection model, comprising a third desensitization loss, a third reconstruction loss, and a support set verification loss;

[0149] The multi-entity privacy protection model is adversarially trained based on the third loss function and a small subset of samples of the support set that successfully matches the target entity category.

[0150] Optionally, the multi-entity fine-tuning module 408 determines that the target entity category is a newly added entity category before performing small-sample learning on the multi-entity privacy protection model;

[0151] The image sample representing the newly added entity category on the deployment end and the small sample of the entity category specified in the support set are respectively used as a gradient state endpoint;

[0152] performing image gradient processing from one gradient state endpoint to another gradient state endpoint according to the image sample and the small sample, to obtain a plurality of intermediate state images between the two gradient state endpoints and a second gradient state endpoint;

[0153] A portion of the intermediate state images is selected as a small sample subset that successfully matches the target entity category.

[0154] Optionally, the multi-entity fine-tuning module 408 determines that the target entity category is a newly added entity category before performing small-sample learning on the multi-entity privacy protection model;

[0155] The newly added target entity category is compared for similarity with each entity category in the support set, and the small sample subset of the entity category with the highest similarity is used as the small sample subset that successfully matches the target entity category.

[0156] Optionally, the device further comprises:

[0157] The multi-entity reasoning module 410 receives an original image to be protected after performing small sample learning on the multi-entity privacy protection model;

[0158] Determining an entity to be protected or an area to be protected specified by a user in the original image to be protected;

[0159] The multi-entity privacy protection model is used to perform desensitization on the entity to be protected or the entity contained in the area to be protected, so as to obtain a private image corresponding to the original image to be protected.

[0160] Figure 5 This is a schematic diagram of the structure of a multi-entity image privacy protection device provided in one or more embodiments of this specification, the device comprising:

[0161] at least one processor; and,

[0162] a memory communicatively connected to the at least one processor; wherein,

[0163] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0164] For multiple different entity categories, obtain entity images corresponding to each entity category, and private images obtained by performing single-entity desensitization on the entity images as training data;

[0165] Using the training data, training a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images;

[0166] Determine the target entity category and the support set of a small subset of samples containing multiple different entity categories;

[0167] According to the support set, a small sample learning is performed on the multi-entity privacy protection model for the target entity category.

[0168] The processor and the memory may communicate with each other via a bus, and the device may further include an input / output interface for communicating with other devices.

[0169] Based on the same idea, one or more embodiments of this specification further provide a non-volatile computer storage medium corresponding to the above method, storing computer-executable instructions, wherein the computer-executable instructions are configured to:

[0170] For multiple different entity categories, obtain entity images corresponding to each entity category, and private images obtained by performing single-entity desensitization on the entity images as training data;

[0171] Using the training data, training a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images;

[0172] Determine the target entity category and the support set of a small subset of samples containing multiple different entity categories;

[0173] According to the support set, a small sample learning is performed on the multi-entity privacy protection model for the target entity category.

[0174] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0175] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0176] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0177] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0178] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0180] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0182] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0183] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0184] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0185] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0186] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A multi-entity image privacy protection method, comprising: For multiple different entity categories, obtain entity images corresponding to each entity category, and private images obtained by performing single-entity desensitization on the entity images as training data; Using the training data, training a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images; Determine the target entity category and the support set of a small subset of samples containing multiple different entity categories; performing small-sample learning on the multi-entity privacy protection model for the target entity category according to the support set; The obtaining of a private image obtained by performing single-entity desensitization on the entity image as training data specifically includes: defining a first loss function including a first desensitization loss and a first reconstruction loss; constructing a corresponding single-entity privacy protection model for each entity category, and performing adversarial training on the single-entity privacy protection model based on the first loss function; using the single-entity privacy protection model, performing single-entity desensitization on the entity image corresponding to the corresponding entity category to obtain a private image as training data; wherein the single-entity privacy protection model includes a first desensitization encoder that inputs an original image and outputs a corresponding private image, and a first anti-desensitization decoder that inputs the corresponding private image and outputs a reconstructed original image.

2. The method of claim 1, wherein the multi-entity privacy protection model comprises a second desensitizing encoder that inputs an original image and outputs a corresponding private image, and a second anti-desensitizing decoder that inputs the corresponding private image and outputs a reconstructed original image; The training of the multi-entity privacy protection model using the training data specifically includes: Determining a second loss function defined for the multi-entity privacy protection model, the second loss function including a second desensitization loss, a second reconstruction loss, and a model conversion loss from a single entity to a multi-entity model; According to the second loss function, adversarial training is performed on the multi-entity privacy protection model.

3. The method of claim 2, wherein the multi-entity privacy protection model further comprises an entity classifier that inputs the intermediate layer features of the second desensitizing encoder and outputs entity classification probabilities; The performing adversarial training on the multi-entity privacy protection model according to the second loss function specifically includes: Determining the second loss function further comprising an entity type decoupling loss; The multi-entity privacy protection model is adversarially trained with the adversarial training objectives of maximizing the second desensitization loss, minimizing the second reconstruction loss, the model conversion loss from a single entity to multiple entities, and the entity type decoupling loss.

4. The method of claim 1, wherein determining the target entity category and determining the support set of a small sample subset containing multiple different entity categories specifically comprises: For a plurality of different entity categories, a corresponding small sample subset is determined for each entity category; A support set including each of the small sample subsets is determined, so as to dynamically determine a target entity category among the multiple different entity categories and / or newly added entity categories.

5. The method according to claim 4, wherein determining the target entity category comprises: Sending the multi-entity privacy protection model to a service terminal so as to perform the small sample learning on the service terminal; Dynamically detecting an entity category corresponding to an original image processed by the multi-entity privacy protection model on the service terminal; According to the detection result, the target entity category is dynamically determined among the multiple different entity categories.

6. The method according to claim 5, wherein the dynamically detecting the entity category corresponding to the original image used for processing by the multi-entity privacy protection model on the service terminal specifically comprises: Predefine a binary classification loss function for each of the multiple different entity categories; Training an entity classification model according to each of the binary classification loss functions, wherein the input of the entity classification model is the original image to be classified as an entity, and the output is the binary classification result corresponding to each of the entity categories; The entity classification model is used to dynamically detect entity categories corresponding to original images processed by the multi-entity privacy protection model on the service terminal.

7. The method according to claim 5 or 6, wherein the business terminal comprises a cash register device.

8. The method according to claim 1, wherein the performing small-sample learning on the multi-entity privacy-preserving model for the target entity category based on the support set specifically comprises: Determining a third loss function defined for the multi-entity privacy protection model, the third loss function comprising a third desensitization loss, a third reconstruction loss, and a support set verification loss; The multi-entity privacy protection model is adversarially trained based on the third loss function and a small subset of samples of the support set that successfully matches the target entity category.

9. The method according to claim 1 or 8, wherein before performing small-sample learning on the multi-entity privacy protection model, the method further comprises: Determining that the target entity category is a newly added entity category; The image sample representing the newly added entity category on the deployment end and the small sample of the entity category specified in the support set are respectively used as a gradient state endpoint; performing image gradient processing from one gradient state endpoint to another gradient state endpoint according to the image sample and the small sample, to obtain a plurality of intermediate state images between the two gradient state endpoints and a second gradient state endpoint; A portion of the intermediate state images is selected as a small sample subset that successfully matches the target entity category.

10. The method according to claim 1 or 8, wherein before performing small-sample learning on the multi-entity privacy protection model, the method further comprises: Determining that the target entity category is a newly added entity category; The newly added target entity category is compared for similarity with each entity category in the support set, and the small sample subset of the entity category with the highest similarity is used as the small sample subset that successfully matches the target entity category.

11. The method according to claim 1, after performing small-sample learning on the multi-entity privacy protection model, the method further comprises: receiving an original image to be protected; Determining an entity to be protected or an area to be protected specified by a user in the original image to be protected; The multi-entity privacy protection model is used to perform desensitization on the entity to be protected or the entity contained in the area to be protected, so as to obtain a private image corresponding to the original image to be protected.

12. A multi-entity image privacy protection device, comprising: The single entity preparation module obtains entity images corresponding to each entity category and performs single entity desensitization on the entity images to obtain private images as training data. A multi-entity training module, using the training data to train a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images; A small sample preparation module determines the target entity category and the support set of a small sample subset containing multiple different entity categories; A multi-entity fine-tuning module, which performs small-sample learning on the multi-entity privacy-preserving model for the target entity category based on the support set; The single-entity preparation module defines a first loss function including a first desensitization loss and a first reconstruction loss; for each entity category, a corresponding single-entity privacy protection model is constructed, and adversarial training is performed on the single-entity privacy protection model based on the first loss function; using the single-entity privacy protection model, single-entity desensitization is performed on the entity image corresponding to the corresponding entity category to obtain a private image as training data; wherein the single-entity privacy protection model includes a first desensitization encoder that inputs an original image and outputs a corresponding private image, and a first anti-desensitization decoder that inputs the corresponding private image and outputs a reconstructed original image.

13. The apparatus of claim 12, wherein the multi-entity privacy protection model comprises a second desensitizing encoder that inputs an original image and outputs a corresponding private image, and a second anti-desensitizing decoder that inputs the corresponding private image and outputs a reconstructed original image; The multi-entity training module determines a second loss function defined for the multi-entity privacy protection model, comprising a second desensitization loss, a second reconstruction loss, and a model conversion loss from a single entity to a multi-entity model; According to the second loss function, adversarial training is performed on the multi-entity privacy protection model.

14. The apparatus of claim 13, wherein the multi-entity privacy protection model further comprises an entity classifier that inputs the intermediate layer features of the second desensitizing encoder and outputs entity classification probabilities; The multi-entity training module determines the second loss function further including entity type decoupling loss; The multi-entity privacy protection model is adversarially trained with the adversarial training objectives of maximizing the second desensitization loss, minimizing the second reconstruction loss, the model conversion loss from a single entity to multiple entities, and the entity type decoupling loss.

15. The apparatus according to claim 12, wherein the small sample preparation module determines a corresponding small sample subset for each entity category according to a plurality of different entity categories; A support set including each of the small sample subsets is determined, so as to dynamically determine a target entity category among the multiple different entity categories and / or newly added entity categories.

16. The apparatus according to claim 15, wherein the small sample preparation module sends the multi-entity privacy protection model to a service terminal so as to perform the small sample learning on the service terminal; Dynamically detecting an entity category corresponding to an original image processed by the multi-entity privacy protection model on the service terminal; According to the detection result, the target entity category is dynamically determined among the multiple different entity categories.

17. The apparatus according to claim 16, wherein the small sample preparation module pre-defines a binary classification loss function for each of the multiple different entity categories; Training an entity classification model according to each of the binary classification loss functions, wherein the input of the entity classification model is the original image to be classified as an entity, and the output is the binary classification result corresponding to each of the entity categories; The entity classification model is used to dynamically detect entity categories corresponding to original images processed by the multi-entity privacy protection model on the service terminal.

18. The apparatus according to claim 16 or 17, wherein the business terminal comprises a cash register device.

19. The apparatus of claim 12, wherein the multi-entity fine-tuning module determines a third loss function defined for the multi-entity privacy protection model, the third loss function comprising a third desensitization loss, a third reconstruction loss, and a support set verification loss; The multi-entity privacy protection model is adversarially trained based on the third loss function and a small subset of samples of the support set that successfully matches the target entity category.

20. The apparatus according to claim 12 or 19, wherein the multi-entity fine-tuning module determines that the target entity category is a newly added entity category before performing small-sample learning on the multi-entity privacy-preserving model; The image sample representing the newly added entity category on the deployment end and the small sample of the entity category specified in the support set are respectively used as a gradient state endpoint; performing image gradient processing from one gradient state endpoint to another gradient state endpoint according to the image sample and the small sample, to obtain a plurality of intermediate state images between the two gradient state endpoints and a second gradient state endpoint; A portion of the intermediate state images is selected as a small sample subset that successfully matches the target entity category.

21. The apparatus according to claim 12 or 19, wherein the multi-entity fine-tuning module determines that the target entity category is a newly added entity category before performing small-sample learning on the multi-entity privacy-preserving model; The newly added target entity category is compared for similarity with each entity category in the support set, and the small sample subset of the entity category with the highest similarity is used as the small sample subset that successfully matches the target entity category.

22. The apparatus of claim 12, further comprising: a multi-entity reasoning module, which receives an original image to be protected after performing small-sample learning on the multi-entity privacy protection model; Determining an entity to be protected or an area to be protected specified by a user in the original image to be protected; The multi-entity privacy protection model is used to perform desensitization on the entity to be protected or the entity contained in the area to be protected, so as to obtain a private image corresponding to the original image to be protected.

23. A multi-entity image privacy protection device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: For multiple different entity categories, obtain entity images corresponding to each entity category, and private images obtained by performing single-entity desensitization on the entity images as training data; Using the training data, training a multi-entity privacy protection model, so that the multi-entity privacy protection model can perform multi-entity desensitization on multi-entity images; Determine the target entity category and the support set of a small subset of samples containing multiple different entity categories; performing small-sample learning on the multi-entity privacy protection model for the target entity category according to the support set; The obtaining of a private image obtained by performing single-entity desensitization on the entity image as training data specifically includes: defining a first loss function including a first desensitization loss and a first reconstruction loss; constructing a corresponding single-entity privacy protection model for each entity category, and performing adversarial training on the single-entity privacy protection model based on the first loss function; using the single-entity privacy protection model, performing single-entity desensitization on the entity image corresponding to the corresponding entity category to obtain a private image as training data; wherein the single-entity privacy protection model includes a first desensitization encoder that inputs an original image and outputs a corresponding private image, and a first anti-desensitization decoder that inputs the corresponding private image and outputs a reconstructed original image.

Images