Privacy-Preserving Image Distribution

Abstract

Some embodiments can distribute data (such as videos, photos, recordings, etc.) to multiple users in a manner that protects the privacy of the corresponding users. Some embodiments utilize homomorphic encryption and proxy re-encryption techniques to manipulate the corresponding data such that selected portions thereof are revealed based on the identity of the user currently accessing the corresponding data.

Description

[0001] Related Applications

[0002] This application claims the benefit of the filing date of U.S. Provisional Patent Application No. 62 / 705,604, filed Jul. 7, 2020, titled “Privacy-Preserving Surveillance Systems and Methods,” the entire content of which is incorporated herein by reference. BACKGROUND OF THE INVENTION

[0003] The present invention relates to image processing, and more particularly, to distributing images to multiple users in a manner that employs cryptographic manipulation to protect the privacy of selected users.

[0004] Recent advances in imaging technology and artificial intelligence have led to an explosion in digital surveillance. Video surveillance in public places is commonly used by the police for crime prevention. Surveillance cameras are also increasingly being used in private homes, stores, offices, and schools. Data collected by the cameras is typically further processed to extract various features, such as license plates or the identities of people appearing in specific images.

[0005] Accordingly, there is a significant practical need to develop privacy-preserving video surveillance systems and methods. SUMMARY OF THE INVENTION

[0006] According to one aspect, a method of distributing customized privacy-preserving images to multiple users includes performing encrypted-domain image segmentation of a source image by at least one hardware processor of a privacy management server in response to receiving the encrypted source image decrypted with an available management key to produce a plurality of user-specific encrypted private images. A selected private image includes an area of the source image selected to display a private item of a selected user among the plurality of users, and another private image includes another area of the source image selected to display a private item of another user among the plurality of users. The method further includes performing an encrypted-domain key change procedure by at least one hardware processor of the privacy management server in response to the image segmentation to produce a plurality of user-specific re-encrypted images. A selected re-encrypted image includes the result of transforming the selected private image from being decryptable with the available management key to being decryptable with a private key of the selected user, and another re-encrypted image includes the result of transforming another private image from being decryptable with the available management key to being decryptable with a private key of another user. The method further includes transmitting the plurality of user-specific re-encrypted images by at least one hardware processor of the privacy management server to an image distribution server for further distribution to client devices configured to reconstruct a user-specific plaintext version of the source image.

[0007] According to another aspect, a computer system includes a privacy management server configured to perform encrypted domain image segmentation of the source image to generate a plurality of user-specific private images in response to receiving an encrypted source image decrypted with an available management key. A selected private image includes an area of the source image selected to show a private item of a selected user among the plurality of users, and another private image includes another area of the source image selected to show a private item of another user among the plurality of users. The privacy management server is further configured to perform an encrypted domain key change procedure in response to the image segmentation to generate a plurality of user-specific re-encrypted images. A selected re-encrypted image includes the result of transforming the selected private image from being decrypted with the available management key to being decrypted with a private key of the selected user, and another re-encrypted image includes the result of transforming another private image from being decrypted with the available management key to being decrypted with a private key of another user. The privacy management server is further configured to transmit the plurality of user-specific re-encrypted images to an image distribution server for further distribution to client devices configured to reconstruct a user-specific plaintext version of the source image.

[0008] According to another aspect, a non-transitory computer-readable medium stores instructions that, when executed by at least one hardware processor of a privacy management server, cause the privacy management server to perform encrypted domain image segmentation of the source image to generate a plurality of user-specific private images in response to receiving an encrypted source image decrypted with an available management key. A selected private image includes an area of the source image selected to show a private item of a selected user among the plurality of users, and another private image includes another area of the source image selected to show a private item of another user among the plurality of users. The instructions further cause the privacy management server to perform an encrypted domain key change procedure in response to the image segmentation to generate a set of user-specific re-encrypted images. A selected re-encrypted image includes the result of transforming the selected private image from being decrypted with the available management key to being decrypted with a private key of the selected user, and another re-encrypted image includes the result of transforming another private image from being decrypted with the available management key to being decrypted with a private key of another user. The instructions further cause the privacy management server to transmit the plurality of user-specific re-encrypted images to an image distribution server for further distribution to client devices configured to reconstruct a user-specific plaintext version of the source image. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above aspects and advantages of the present invention will be better understood after reading the following detailed description and referring to the drawings, in which:

[0010] Figure 1 Shows an exemplary privacy protection monitoring system according to some embodiments of the present invention.

[0011] Figure 2 Shows exemplary components of an input sensor according to some embodiments of the present invention.

[0012] Figure 3 Shows exemplary components of a client device according to some embodiments of the present invention.

[0013] Figure 4 Shows exemplary components of an image distribution server according to some embodiments of the present invention.

[0014] Figure 5 Shows exemplary components of a privacy management server according to some embodiments of the present invention.

[0015] Figure 6 Shows an exemplary source image according to some embodiments of the present invention.

[0016] Figure 7 Describes exemplary common images included in the source image according to some embodiments of the present invention Figure 6 that are included in the source image.

[0017] Figure 8 Shows exemplary private images included in the source image according to some embodiments of the present invention Figure 6 that are included in.

[0018] Figure 9 Shows an exemplary user mask according to some embodiments of the present invention.

[0019] Figure 10 Shows an exemplary data exchange performed to establish a privacy protection monitoring system according to some embodiments of the present invention.

[0020] Figure 11 Shows an exemplary data exchange performed during the operation of a privacy protection monitoring system according to some embodiments of the present invention.

[0021] Figure 12 Shows an exemplary sequence of steps performed by a privacy management server in Figure 11 the embodiments described in.

[0022] Figure 13 Shows an exemplary data exchange performed in an alternative embodiment of the present invention.

[0023] Figure 14 Shows an alternative exemplary sequence of steps implemented by a privacy management server in Figure 13 the embodiments described in.

[0024] Figure 15 Describes an exemplary sequence of steps performed by an image distribution server according to some embodiments of the present invention.

[0025] Figure 16 Shows an exemplary sequence of steps for the exchange between an overview client device and a distribution server according to some embodiments of the present invention.

[0026] Figure 17-A Shows an exemplary reconstructed image that can be used for a selected client device according to some embodiments of the present invention.

[0027] Figure 17-B Shows another exemplary reconstructed image that can be used for another client device according to some embodiments of the present invention.

[0028] Figure 18 Shows exemplary data exchange in an embodiment of the present invention configured to perform a selected task in a privacy-protected manner.

[0029] Figure 19 Illustrates an exemplary hardware configuration of a computing device configured to implement operations according to some embodiments of the present invention. Detailed Description

[0030] In the following description, it should be understood that all narrative connections between structures can be direct operational connections or indirect operational connections through intermediate structures. A set of elements includes one or more elements. Any narrative of an element is understood to refer to at least one element. A plurality of elements includes at least two elements. Unless otherwise specified, any use of "or" refers to non-exclusive or. Unless otherwise required, any described method steps are not necessarily performed in a specific recited order. A first element (e.g., data) derived from a second element encompasses a first element that is equal to the second element and a first element that is produced by processing the second element and optionally other data. Making a determination or decision based on a parameter encompasses making a determination or decision based on the parameter and optionally based on other data. Unless otherwise specified, an indicator of some quantity / data can be the quantity / data itself or an indicator different from the quantity / data itself. A computer program is a sequence of processor instructions for performing a task. The computer programs described in some embodiments of the present invention can be independent software entities or sub-entities (e.g., subroutines, libraries) of other computer programs. The term 'database' is used herein to denote any structured collection of data. Performing an encryption domain program / operation herein means performing the corresponding program / operation in the encryption domain, i.e., directly on encrypted inputs to produce encrypted outputs in a manner that does not involve decrypting the inputs. An encryption domain program is different from a program that decrypts the input and then encrypts the output of the corresponding program. In other words, an entity that performs an encryption domain program / operation on an encrypted item need not know the plaintext version of the corresponding item. Computer-readable media encompasses non-transitory media (e.g., magnetic, optical, and semiconductor storage media (e.g., hard disk drives, optical discs, flash memories, DRAMs)) and communication links (e.g., conductive cables and fiber optic links). According to some embodiments, the present invention particularly provides a computer system including hardware (e.g., one or more processors) programmed to execute the methods described herein and computer-readable media encoded instructions for performing the methods described herein.

[0031] The following description illustrates embodiments of the present invention by way of example and not by way of limitation.

[0032] Figure 1 FIG. 10 shows an exemplary privacy protection monitoring system 10 according to some embodiments of the present invention. The term "monitoring" is used herein only to clarify the present disclosure by focusing on a specific exemplary use case and does not mean being limited to typical monitoring activities such as crime prevention. Although the following description will focus on video monitoring examples, the disclosed systems and methods can be applicable to other applications, such as protecting privacy and / or ensuring confidentiality during collaboration between multiple parties processing the same document, thereby preventing cyberbullying via online messaging, etc.

[0033] System 10 particularly includes input sensor 14, distribution server 30, privacy management server 40, and a plurality of client devices 12a to c, all of which are communicatively coupled via a network 15 that may include the Internet.

[0034] Sensor 14 (such as a camera, microphone, etc.) is configured to obtain signals (such as encoded images and / or sounds), which are further manipulated and transformed as described below. In a video surveillance example, sensor 14 may include a camera positioned to obtain images of public spaces such as a campus, a market square, etc. Thus, sensor 14 may include hardware and / or software components for obtaining signals (such as charge-coupled device (CCD) light sensors), computer-readable media for storing the obtained signals, and components for transmitting the corresponding signals (such as physical layer communication hardware, encoders, antennas, etc.). Figure 2 Other exemplary components of input sensor 14 are shown, which may include a dedicated software module according to some embodiments of the present invention. Cryptographic engine 16 encrypts the obtained image / sound recording. Communication module 18 further transmits the resulting encrypted signal to privacy management server 40 and / or image distribution server 30, as detailed below.

[0035] In some embodiments, cryptographic engine 16 encrypts data according to a homomorphic encryption scheme. Homomorphic encryption is a particular type of encryption that allows performing certain calculations such as addition and / or multiplication of encrypted data, where decrypting the result of such calculations yields an output identical to that obtained by applying the corresponding calculation to the plaintext version of the same data. In other words, if Enc(p)=c represents a homomorphic encryption operation, where p represents a plaintext message and c represents its corresponding ciphertext, Dec(c)=p represents a homomorphic decryption operation for recovering the corresponding plaintext message from its ciphertext, and Eval(F,{c1,...,c k})=C represents a homomorphic evaluation procedure for generating ciphertext C by applying function F to a set of ciphertexts c i then:

[0036] Dec(C)=F(p1,...,p k ),[1]

[0037] where pi = Dec(ci), i = 1,…,k. In formal mathematical language, the encryption and decryption procedures of a homomorphic encryption scheme can be considered homomorphic between the plaintext space and the ciphertext space.

[0038] There are several homomorphic encryption schemes / cryptosystems known in the art. Schemes that maintain homomorphic properties over any combination of addition and multiplication are commonly referred to as fully homomorphic. Examples include the Gentry-Sahai-Waters (GSW) scheme, among others. Other schemes / algorithms are homomorphic only over a certain type of operation, e.g., only addition in the case of the Paillier scheme and only multiplication in the case of the Rivest-Shamir-Adelman (RSA) scheme. Such schemes are referred to in the art as partially homomorphic. In contrast, cryptosystems that do not have the above homomorphic properties are considered non-homomorphic in this document. Examples of non-homomorphic cryptosystems include the Advanced Encryption Standard (AES) used in some Transport Layer Security (TLS) communication protocols.

[0039] Client devices 12a to c generally represent any end-user electronic device for accessing and / or processing (e.g., visualizing, playing back, etc.) data provided by input sensor 14, such as a personal computer, smartphone, television, etc. In Figure 3 some of the embodiments described, client device 12 may execute a monitoring software application 22, which is configured to perform a user authentication exchange (e.g., a login procedure) with distribution server 30 and then display the reconstructed image to the user. Data reconstruction engine 24 is configured to reconstruct an image from a set of plaintext public images and a set of encrypted private images, as described below. Client cryptographic engine 26 is configured to decrypt the received encrypted private images. In some embodiments, engine 26 implements a homomorphic decryption algorithm.

[0040] Each of distribution server 30 and privacy management server 40 generally represents a set of interconnected computer systems that may or may not be physically close to each other. Exemplary components of servers 30 and 40 are shown in Figure 4 and 5 respectively. In some embodiments, such components represent computer programs (software) executed on at least one hardware processor. Not all of the illustrated components need to be executed on the same hardware processor or physical machine. Those skilled in the art will understand that in alternative embodiments, some of the illustrated components may be implemented in dedicated hardware (e.g., application-specific integrated circuits (ASICs) and / or field-programmable gate arrays (FPGAs)), firmware, or a combination thereof.

[0041] In some embodiments, the distribution server 30 manages a monitoring service that includes, for example, communication with client devices 12a to c for user registration and / or authentication and distribution of selectively encrypted data to each client device. Without loss of generality, the server 30 may be referred to herein as an image distribution server, i.e., a server configured to distribute images (e.g., videos) to clients. Those skilled in the art will appreciate that depending on the actual embodiment and use case, the server 30 may distribute other types of data, such as audio, electronic documents, etc. The user manager component 32 may manage a set of users and / or account data (user names, passwords, various service protocol parameters, etc.) and provide a user interface for user registration and account management.

[0042] The access manager component 38 may selectively store data to and / or retrieve data from the data repository 20 and selectively forward this data to each client device 12a to c based on the identity of the user currently authenticated on the corresponding client device. The access manager 38 may include a web server or the like.

[0043] The key manager 34 may initiate and / or execute key generation and exchange procedures with the client devices 12a to c and the privacy management server 40. The key manager 34 may further generate a set of proxy re-encryption tokens and selectively associate each such token with a registered user of the monitoring service and / or the client devices 12a to c. More details regarding such processes are given below.

[0044] The managed password engine 36 may be configured to perform data encryption and / or decryption operations, as further described below. The engine 36 may implement a version of a homomorphic encryption / decryption algorithm.

[0045] In some embodiments, the data repository 20 may include a computer-readable storage medium of a database configured to store private and public data. Public data may include any data accessible to all users, such as plaintext (i.e., unencrypted) images. Private data may be accessible and / or decrypted only by selected users. Examples of private data include user-specific and synthetic proxy re-encrypted images, as shown below. This data may be indexed according to users to enable selective insertion and retrieval. The index may take any form known in the art.

[0046] In some embodiments, the privacy management server 40 ( Figure 5)Provide services such as passwords and automatically detecting private / confidential items in the data provided by the input sensor 14. The re-encryption engine 46 of the server 40 is configured to perform a key exchange procedure on the encrypted data, as shown in more detail below. The key exchange procedure herein refers to a procedure of transforming ciphertext from being decryptable using one key to being decryptable using another key. An example of the key exchange procedure is known in the art as proxy re-encryption, which allows entity Z to change the ciphertext encrypted under the public key of entity X in view of some information about another entity Y, thereby making it decryptable by entity Y. In other words, entity Y can decrypt the ciphertext encrypted under the public key of X using its own key, but only after the corresponding ciphertext has been proxy re-encrypted by entity Z using a re-encryption key specific to entity Y (also known as a re-encryption token in the art). Translating this general scheme into Figure 1 In some embodiments of the privacy management server 40, the exemplary participants described in proxy re-encrypt the data encrypted with the public key of the distribution server 30 so that the corresponding data can be decrypted by selected users of the client devices 12a to c. The proxy re-encryption procedure employs a re-encryption token specific to the corresponding user and / or device, such as a token generated based on the public encryption key of the corresponding application / device.

[0047] In some embodiments, the re-encryption engine 46 operates in the encryption domain, i.e., performs the corresponding key exchange procedure without decrypting the input. To achieve key exchange in the encryption domain, some embodiments of the engine 46 implement a proxy re-encryption algorithm that is compatible with the homomorphic encryption / decryption algorithms implemented by the client devices 12a to c, the distribution server 30, and / or the input sensor 14. Such algorithms are beyond the scope of this description; several such examples are known in the field of cryptography, such as the PALISADE codebase available at https: / / gitlab.com / palisade / palisade-development.

[0048] A set of item detectors 42 can be configured to determine whether the input data received from the sensor 14 (such as frames captured by a surveillance camera) contains a representation of a private / confidential item associated with a selected user. Exemplary private items include a person, face, or some other body part, logo / trademark, license plate, bank card, personal ID (such as a driver's license, passport), handwritten text, and a person's signature, etc. In embodiments configured to operate with sound, exemplary private items can include any item that allows a person to be identified, such as any voice quality, such as timbre, creak, pitch, rhythm, intonation, etc. In embodiments configured to process text documents and / or electronic messages, exemplary private items include written names, addresses, financial information (such as credit card numbers), etc. Other examples include text written by a selected author, text written on a selected topic, and text written or conveyed in a selected style or expressing a selected emotion.

[0049] A private item can be user-specific. For example, in a campus surveillance use case, each parent can define their own child as a private item, so the corresponding child can only be visible to the corresponding parent. In some embodiments, multiple users can share a private item and / or a single user can have multiple private items. In one such instance, all members of a specific user group (e.g., parents of third-grade children) can see the faces of their children's peers, but other users cannot.

[0050] Figure 6 Shows an exemplary source image 70 received from a surveillance camera (input sensor 14), the image 70 showing an exemplary private / confidential item including a person (e.g., children 72a to b), a face 72c, and a specific object 72d. Figures 7 - 8 Shows exemplary public and private images contained in the exemplary source image 70. In some embodiments, the private image includes a representation (e.g., a digital array) of the private / confidential item. In Figure 8 the instance of, the private image 76a includes a region of the source image showing Figure 6 the private item 72a in. Accordingly, the public image 74 ( Figure 7 ) can include another region of the source image 70 that does not show any private items. For example, the public image 74 can show all non-private content of the source image 70. The exemplary public image contains the background of a scene (landscape, building, trees, courtyard, sky, etc.). In some embodiments, the public image 74 and / or the private image 76a are represented as digital arrays having the same size as the source image 70.

[0051] The item detector 42 can be constructed using any method known in the art. For example, the exemplary item detector 42 can include an artificial intelligence (AI) system 43a, such as a set of artificial neural networks pre-trained to recognize examples of the corresponding private items within the source image. The exemplary AI system 43a includes a face recognition module and an image segmentation module, among others. The structure and training of such item detectors are beyond the scope of this description; several architectures and training strategies are known in the art.

[0052] In an image processing embodiment, the exemplary item detector 42 can receive the source image 70 and output a user mask indicating the region of the source image that shows a representation of the private item (e.g., the region of the source image showing the face of a specific person). Figure 9 Shows an exemplary user mask 80a associated with Figure 6 the private item 72a of. The exemplary user mask is characterized by a subset of the pixels of the image belonging to the corresponding private item. Another exemplary user mask includes all pixels located inside a continuous region of the source image 70, the region showing the private item. For example, in Figure 9In the embodiments described, this region can be defined as the interior of a polygon (e.g., convex hull, bounding box, etc.) that encloses an image of a private item. A convenient computer-readable encoding of the user mask includes a digital sparse array that has the same size as the source image 70 and in which all elements except those corresponding to the pixels of the mask are zero. Multiple user masks can be associated with a single user (i.e., a single re-encryption token, see below). Some user masks can overlap.

[0053] In some embodiments, the detector 42 operates in the encrypted domain, i.e., without decrypting the source image. To enable this encrypted domain operation, the AI system 43a (e.g., a neural network implementing face recognition) can be deliberately structured to be compatible with a homomorphic encryption scheme. For example, the detector 42 can receive a homomorphically encrypted source image and, in response, output a homomorphically encrypted user mask that is encrypted using the same encryption key as that used to encrypt the source image. Several such AI systems have been described in the art. An example includes CryptoNets described by N. Dowlin et al., "CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy" (Proceedings of the 33rd International Conference on Machine Learning, New York, NY, 2016, JMLR: W&CP Vol. 48). In one such example, the AI system 43a includes a neural network in which selected layers are equivalent to a predetermined degree polynomial and in which a typical non-linear activation function such as the rectified linear unit (ReLU) is replaced with a polynomial approximation.

[0054] In some embodiments, the AI system 43a is pre-trained by the AI training system 11 (e.g., a machine learning algorithm executed on a processor) using training data provided by or otherwise indicated by each user. In one such example, after the registration service, each user can provide a sample representation of the corresponding user's confidential item, such as an image of a face or a voice sample of a person. Some embodiments can then train the AI system 43a to recognize the representation of the corresponding private item within the data stream received from the input sensor 14. A related example is training face recognition software on the target faces provided by each user. The training produces a set of optimized detector parameter values 45a that are transmitted to the item detector 42. In a neural network embodiment, exemplary parameters 45a include a set of synaptic weights and neuron biases, etc.

[0055] Figure 10Shows an exemplary exchange that is executed to initialize / establish a privacy - protected monitoring service according to some embodiments of the present invention. In the illustrated example, the distribution server 30 implements a key - generation procedure to generate a pair of homomorphic encryption keys (herein regarded as management keys) specific to the distribution server 30 and transmits the public key 52 of the pair to the input sensor 14 for encrypting the acquired signals / images. The server 30 further participates in a key - generation and / or exchange protocol with the client device 12 (generally representative of Figure 1 any one of the client devices 12a to 12c in

[0056] Figure 11 and 13 illustrates the data exchanges implemented in two exemplary embodiments of a privacy - protected monitoring system. For clarity, the following description will focus on video monitoring, i.e., the relevant source data includes image data. Those skilled in the art will understand that the methods described herein can be applied to other applications where the relevant data includes the encoding of sound (e.g., voice recordings), text, etc.

[0057] Figure 12 and 14 respectively show alternative sequences of steps implemented by the privacy management server 40 in the embodiments described by Figure 11 and 13 Accordingly, Figure 15 shows the exemplary steps implemented by the image distribution server 30.

[0058] In some embodiments, the data acquired by input sensor 14 is encoded as a plaintext image I, e.g., comprising an array of digits, where each digit represents the intensity of the corresponding image at a distinct location / pixel. Some images may have multiple channels (e.g., red, green, and blue); in such embodiments, each channel may be represented by a separate array. The image I is then encrypted by sensor cryptography engine 16 according to a public management key 52 to produce an encrypted data stream 60 that is transmitted to privacy management server 40. The stream 60 may include, for example, a set of encrypted source images:

[0059]

[0060] where Enc(x,k) generally represents the encryption of quantity x using key k, and k p admin represents the public management key 52. An asterisk (*) is always used to indicate an encrypted quantity. In a video surveillance embodiment, each encrypted source image I* may correspond to a distinct frame and may be tagged with an associated timestamp indicating the moment the corresponding frame was acquired.

[0061] In response to receiving the data stream 60, for each encrypted source image I*, in step 204 ( Figure 12 ), the server 40 may apply item detector 42 to determine whether the corresponding image includes private data (i.e., an image of an item that is considered private by some users). If so, some embodiments of detector 42 return a set of user masks that identify regions of the source image that exhibit various private items (see Figure 9 for exemplary mask 80a). Additionally, such masks are indexed according to the users who have declared the corresponding item as private. Some embodiments may further determine a set of public masks that include regions of the current frame that contain only public data. In an exemplary embodiment, the public masks are determined by inverting all user masks and superimposing the results. In another embodiment, item detector 42 may be trained to return a set of public masks as well as user masks.

[0062] However, since the privacy management server 40 does not possess the secret management key and thus cannot decrypt the source image I*, some embodiments of item detector 42 operate in the encrypted domain (i.e., directly on the encrypted data) and produce an encrypted output (i.e., the user masks are also encrypted). Thus, in some embodiments, although item detector 42 is executed on server 40, the server 40 does not know the content of the source image and which regions of the source image contain private items (if any).

[0063] In some embodiments, a set of steps 206 to 208 ( Figure 12)Execute the encrypted domain image segmentation program to extract a set of encrypted public and private images from the current source image I* according to the output of the item detector 42. Each private image may include the (encrypted) content of the current source image located within a different user mask. In some embodiments, the encrypted private image associated with user mask i may be determined according to the pixel-by-pixel multiplication of the encrypted source image and the encrypted mask i:

[0064]

[0065] where M* i represents the encrypted user mask i returned by the item detector 42:

[0066]

[0067] and where M i represents the unencrypted / plaintext user mask i.

[0068] The circled dot operator herein represents pixel-by-pixel multiplication:

[0069]

[0070] where {xy} index the positions / pixels within the source image and the user mask respectively. Pixel-by-pixel multiplication is applied to images / arrays of the same size.

[0071] Meanwhile, the encrypted public image of the current frame ( Figure 11 item 62 in) can be calculated according to the element-by-element multiplication of the encrypted source image and the encrypted public mask:

[0072] I *PUBLIC = I * ⊙ M *PUBLIC , [6]

[0073] where M *PUBLIC represents the encrypted public mask generated by the item detector 42:

[0074]

[0075] where M PUBLIC represents the corresponding unencrypted / plaintext public mask.

[0076] In some embodiments, at step 210, the privacy management server 40 may employ a re-encryption engine 46 to proxy re-encrypt the private image determined as seen above (e.g., formula [2]) according to the re-encryption token associated with the corresponding user / mask i to produce an individual user-specific re-encrypted private image 66 ( Figure 11) which is then transmitted to the image distribution server 30. This proxy re-encryption ensures that the corresponding private image can only be deciphered by the holder of the decryption key associated with the user / mask i. In some embodiments, the re-encrypted private image 66 is marked with an indicator of the corresponding user to enable the server 30 to selectively insert the image 66 into the data repository 20 and / or retrieve the image 66 from the data repository 20. Another sequence of steps 212 to 214 transmits the encrypted public image 62 and the re-encrypted private image 66 to the server 60 for further distribution to the client devices 12a to c.

[0077] In Figures 13 - 14 an alternative embodiment illustrated in, in step 230, the server 40 may transmit the encrypted user mask 64 to the image distribution server 30 for decryption and, in response, receive the decrypted user mask 65 from the server 30. In some embodiments, the decrypted mask 65 includes a plaintext version of the encrypted user mask determined by the item detector 42:

[0078]

[0079] where Dec(x,k) generally represents decrypting the quantity x using the key k, and where k s admin represents the secret key held by the image distribution server 30. In such embodiments, even though the privacy management server 40 can clearly see whether the source image shows a private item and which regions of the source image show the private item, since the server 40 cannot decrypt any regions of the corresponding source image I*, privacy is still protected.

[0080] Next, step 234 may extract the private image by copying the pixels of the encrypted frames located within each decrypted user mask 65. In some embodiments, this may mean determining the encrypted private image associated with mask i as:

[0081]

[0082] Additionally, step 236 may employ a re-encryption engine 46 to proxy re-encrypt each such private image with a re-encryption token of the user associated with the corresponding mask i to produce individual re-encrypted private images. Next, in step 238, in some embodiments, a synthetic re-encrypted private image 67 may be calculated based on the plurality of individual re-encrypted private images determined in step 236. In some embodiments, the synthetic image 67 includes a single image assembled from multiple private images in a mosaic manner, where each individual re-encrypted private image occupies a region of the synthetic image corresponding to the corresponding user mask M i of the synthetic image. Calculating the synthetic private image may be facilitated by zero-padding each proxy re-encrypted private image to the size of the source image. Then, the synthetic re-encrypted private image 67 may be calculated according to the following formula:

[0083]

[0084] where ReEnc(x, t) generally represents proxy re - encryption of ciphertext x using token t, and t i represents the re - encryption token associated with user / mask i. The circled plus operator represents pixel - by - pixel addition in this text:

[0085]

[0086] where {x, y} index positions / pixels within exemplary images I1 and I2 respectively. Pixel - by - pixel addition is applied to images of the same size.

[0087] Next, the computed synthetic re - encrypted private image 67 can be transmitted to the image distribution server in step 240. In an alternative embodiment, the privacy management server 40 can compute individual proxy re - encrypted private images and transmit the corresponding images to the distribution server 30. Accordingly, the server 30 can determine the synthetic image 67 from the received individual re - encrypted images using, for example, equation

[10] .

[0088] Meanwhile ( Figure 14 step 226 in), the privacy management server 40 can compute the encrypted public image 62 as shown above (e.g., equation [6]). Alternatively, the image 62 can be determined based on the plaintext public mask:

[0089] I *PUBLIC = I * ⊙M PUBLIC ,

[12]

[0090] where M PUBLIC is received from the distribution server 30. In yet another embodiment, M i can be computed by inverting all the plaintext user masks M PUBLIC received from the server 30 and superimposing the results. In any of these cases, the image 62 is encrypted with the management key by virtue of the fact that the server 40 performs image segmentation in the encrypted domain (i.e., without decrypting the source image). In other words, the server 40 does not know the plaintext content of the public image 62. In step 228, the encrypted public image 62 is transmitted to the server 30 for decryption and further distribution to the clients.

[0091] Figure 15Illustrate the exemplary operation of the image distribution server 30 according to some embodiments of the present invention. In the sequence of steps 252 to 254, the server 30 may wait for communication from the privacy management server 40. When such communication includes an encrypted user mask (step 256 feedback is), the image distribution server 30 may use the cryptographic engine 36 to decrypt the corresponding mask according to its secret management key (such as the above formula [6]) and transmit the decrypted mask to the privacy management server 40.

[0092] When the communication includes the encrypted public image 62, the server 40 may decrypt it to generate the decrypted public image 63:

[0093]

[0094] And save the image 63 to the data repository 20. The decrypted public image 63 may be marked with a timestamp, frame number, or another indicator that associates the image 63 with the source image from which the image 63 was extracted.

[0095] When the communication received from the server 40 includes the re-encrypted private image (specific to user / mask i or synthesis depending on whether the server 40 follows process Figure 12 Or 14), the image distribution server 30 may insert the corresponding private image into the data repository 20. The re-encrypted private image may also be marked according to the timestamp and / or label that associates the corresponding image with the corresponding source image. The private image may also be marked to indicate its association with a specific user and / or mask.

[0096] Figure 16 Show additional exemplary steps performed by the image distribution server 30 with respect to the client device 12 of any of the client devices 12a to c in the general representation Figure 1 In step 280, the client device 12 may implement a user authentication procedure to identify the current user of the device 12 according to the distribution server 30. Step 280 may implement any user authentication protocol known in the art (such as password, two-factor, biometric, etc.). In step 282, the device 12 may then transmit a query to the server 30 to, for example, indicate a request to view images captured from a specific surveillance camera and within a specific time frame. In response, in the sequence of steps 284 to 290, the image distribution server 30 may selectively retrieve a set of public and private images from the data repository 20 according to the query and transmit the corresponding images to the client device 12. Depending on whether the privacy management server 40 is based on Figure 12For the flowchart operations described in FIGS. 13 or 14, the private images may respectively include individual re-encrypted private images 66 or synthetic re-encrypted private images 67. For example, such transactions may be implemented via a web interface. In an alternative embodiment, the image distribution server 30 may open a dedicated connection (e.g., a VPN tunnel) with the client device 20 and transmit the public and private images via the respective connection.

[0097] Those skilled in the art will understand that although the public image has been decrypted to plaintext before distribution, step 288 does not necessarily include transmitting the corresponding public image in plaintext. Instead, step 288 may include re-encrypting the transmitted public image, for example as part of the transmission via TLS / HTTPS. However, this encryption has no effect on the image reconstruction at the client device; in a TLS / HTTPS transaction, the receiving client device can always decrypt the payload.

[0098] In response to receiving the public and private images, in step 292, the client device 12 may use the client password engine 26 ( Figure 3 ) to decrypt the corresponding private image using the secret key associated with the corresponding user of the client device 12. Next, in step 294, the data reconstruction engine 24 may calculate the reconstructed image based on the decrypted private image and further based on the decrypted public image 63 received from the image distribution server 30. For example, the reconstructed image may be calculated by pixel-by-pixel addition of the decrypted public image 63 and the decrypted private image:

[0099]

[0100] or

[0101]

[0102] where R i represents the reconstructed image seen by user i, and k s i represents the secret key of user i. When the source image includes private data of multiple users, formula

[14] may not calculate the entire reconstructed image, in the sense that the region of the reconstructed image corresponding to the user mask M j corresponding to a user belonging to a user other than the current user i of the client device 12 may be empty. To obtain a complete reconstructed image, some embodiments may fill the missing regions with pseudo data (e.g., zeros, random noise, random colors, etc.).

[0103] In cases where the masks associated with different users may overlap, for example when some information may be relevant to multiple users (e.g., relevant to members of a selected group) while other information is private to each user, it may be preferable to reconstruct the frame according to formula

[14] . Another example of this scenario may occur in an automatic image segmentation system configured to produce multi-label classifications.

[0104] In an embodiment where a reconstructed image is computed from a synthetic private image, the reconstructed image R i is complete, but the secret key k held by user i s i can only decrypt the private data of the corresponding user. Thus, the regions of the reconstructed image corresponding to the user masks M j of other users will show scrambled images. This effect is illustrated in Figure 17-A to B, Figure 17-A to B showing how two different users see the reconstruction of the same source image. Figure 17-A shows the reconstructed image seen by user A, who claims item 72a ( Figure 6 ) as private. User A will see the image of private item 72a but will not be able to see the images of the private items of other users, such as the images of items 72b to c to d (see Figure 6 ). Figure 17-B shows the reconstructed image seen by another user B whose item 72b is private. User B can see the image of item 72b but not the private items 72a and 72c to d.

[0105] In embodiments where the item detector 42 only produces non-overlapping user masks and / or where different users do not share private information, it may be preferable to reconstruct the frame R according to formula

[15] i , i.e., to reconstruct the frame R from the synthetic encrypted private image i . Otherwise, the regions of the reconstructed image covered by mask overlap cannot be deciphered by any individual user and thus may appear scrambled. Operating with synthetic private images can further save computational resources because it allows the same encrypted private data (i.e., one synthetic private image) to be sent to all users instead of storing, indexing, and selectively delivering individual private images to each user. In such embodiments, the server 40 can directly insert the private and public images into the data repository 20 without further involvement of the distribution server 30. A disadvantage of embodiments using synthetic re-encrypted private images is that they ensure a relatively lower level of privacy compared to embodiments using individual private images because, when computing the private images, the server 40 operates with decrypted / plaintext masks. In other words, although the server 40 does not know the content of the private images, it knows, for example, whether the source image includes private items and it also knows the approximate location of the corresponding private items via the corresponding plaintext masks.

[0106] Figure 18 illustrates the enhancement of the privacy protection monitoring system according to some embodiments of the present invention. In some embodiments, the privacy management server 40 ( Figure 5) is further provided with an image task module 44, which is configured to perform specific tasks according to the encrypted data stream 60 received from the input sensor 14. An example of an image processing task includes event detection (determining whether an image or a sequence of images indicates the occurrence of a specific event). In a traffic monitoring embodiment, the task module can automatically determine whether the source image indicates an accident, a traffic jam, etc. Other exemplary tasks include counting people in an image and determining whether the count exceeds a predetermined threshold. Yet another exemplary task generally includes any image classification / labeling task, such as determining whether an image shows a specific type of object (e.g., a weapon, a personal ID, a bank card, a license plate, etc.). Those skilled in the art will understand that although the above examples relate to image processing, this aspect is not meant to be limiting and some embodiments may be adapted to process other types of data such as sound files, text documents, etc.

[0107] In some embodiments, the task module 44 ( Figure 5 ) includes an AI system 43b that is pre-trained to perform the corresponding tasks. Several such examples are known in the field of computer vision; their architectures and training are beyond the scope of the present disclosure. The AI system 43b can be pre-trained by the AI training system 11. In this sense, the system 11 can, for example, determine a set of optimized task module parameter values 45b (such as synaptic weights, etc.) via a machine learning process and use the values 45b to instantiate a runtime example of the image task module 44.

[0108] The task module 44 can operate in the encrypted domain, i.e., without decrypting the source data. In such embodiments, the module 44 can input an encrypted image and produce an encrypted output including the result of performing the corresponding task, and the corresponding output is encrypted with a public management key associated with the distribution server 30 encrypted. For example, the output of the task module 44 can include an encrypted version of a decision or a label (e.g., yes / no according to whether the data stream 60 indicates the occurrence of a specific event). Since the module 44 executes in the encrypted domain, the privacy management server 40 does not know the task result.

[0109] In some embodiments, the output of the task module 44 is processed by the engine 46 ( Figure 5)Using the re-encryption tokens of the selected user to perform proxy re-encryption to generate a re-encryption task result 86, the re-encryption task result 86 is sent to the distribution server 30 for delivery to a predetermined notification device 13 (such as the smartphone of the selected user). In some embodiments, the notification device 13 may also receive the decrypted public image 63, such that in addition to being notified that the corresponding event or situation has occurred, the corresponding user can also view the public available image data. For example, the principal (or security personnel) may receive a notification that there is a fight taking place in the school building, but when this information is considered private, he / she cannot see who is actually involved in the fight. At the same time, since the task result 86 can only be deciphered by the selected notification device, all users other than the principal will not know that there is a fight. In addition, the owner / operator of the server 40 also does not know about such events.

[0110] Some embodiments are further enhanced by adding a superuser who may be allowed to view all private information contained in the source image. This superuser may represent an authoritative figure, such as a principal, a representative of the company's human resources department, etc. After establishing the monitoring service, the image distribution server 30 may create a pair of keys and a set of re-encryption tokens associated with the superuser. In one such exemplary embodiment, in response to determining the user mask and extracting the private image, the privacy management server 40 may use the superuser's re-encryption tokens to proxy re-encrypt the extracted private images associated with all users, thus creating a synthetic private image that can only be accessed by the superuser. The corresponding re-encrypted private data is then sent to the image distribution server 30 and further can be accessed by the superuser together with the decrypted public image 63. The superuser may decrypt the corresponding re-encrypted private image and thus completely reconstruct the source image based on the public image 63 and the decrypted synthetic private image. At the same time, users who do not possess the superuser's private encryption key cannot view the private data belonging to another user.

[0111] Figure 19 Exemplary computer system 90 configured to perform some of the methods described herein. The computer system 90 may represent any of the client devices 12a to c, as well as the image distribution server 30 and the privacy management server 40. The illustrated hardware configuration is that of a personal computer; the configurations of other computing devices (such as mobile phones and servers) may vary slightly from Figure 19 that shown. The processor 92 includes physical devices (such as a microprocessor, a multi-core integrated circuit formed on a semiconductor substrate) configured to perform computational and / or logical operations with a set of signals and / or data. Such signals or data may be encoded in the form of processor instructions (such as machine code) and delivered to the processor 92. The processor 92 may include a central processing unit (CPU) and / or an array of graphics processing units (GPUs).

[0112] The memory unit 93 may include volatile computer-readable media (such as dynamic random access memory (DRAM)) that stores data and / or instruction codes accessed or generated by the processor 92 during the implementation of operations. The input device 94 may include a computer keyboard, mouse, touchpad, microphone, etc., which include corresponding hardware interfaces and / or adapters that allow a user to introduce data and / or instructions into the computer system 90. The output device 95 may include a display device (such as a monitor) and speakers, etc., as well as a hardware interface / adapter (such as a graphics card) to enable the corresponding computing device to transmit data to the user. In some embodiments, the input and output devices 94 to 95 share common hardware (such as a touch screen). The storage device 96 includes computer-readable media to be able to non-volatilely store, read, and write software instructions and / or data. Exemplary storage devices include magnetic disks and optical disks and flash memory devices, as well as removable media, such as CD and / or DVD disks and drives. The network adapter 97 includes mechanical, electrical, and signaling circuitry for transmitting data through a physical link coupled to an electronic communication network (such as Figure 1 the network 15 in) and / or other devices / computer systems. The adapter 97 may be further configured to transmit and / or receive data using various communication protocols.

[0113] The controller hub 98 generally represents multiple systems, peripheral devices, and / or chipset buses and / or all other circuitry that enables communication between the processor 92 and the remaining hardware components of the computer system 90. For example, the controller hub 98 may include a memory controller, an input / output (I / O) controller, and an interrupt controller. Depending on the hardware manufacturer, some such controllers may be incorporated into a single integrated circuit and / or integrated with the processor 92. In another example, the controller hub 98 may include a north bridge that connects the processor 92 to the memory 93 and / or a south bridge that connects the processor 92 to the devices 94, 95, 96, and 97.

[0114] The exemplary systems and methods described herein allow distributing data (such as videos, photos, recordings, digital documents, etc.) to multiple users in a way that protects the privacy of the corresponding users. Some embodiments employ homomorphic encryption and proxy re-encryption techniques to manipulate the corresponding data such that selected portions thereof are revealed according to the identity of the user currently accessing the data.

[0115] One exemplary application of some embodiments includes video surveillance, where distributing the data includes an image stream received from a surveillance camera. Some embodiments employ image recognition techniques to determine whether an image contains items considered confidential by a selected user (such as a specific person or face, a specific license plate, etc.), and manipulate and selectively encrypt the corresponding image such that only the corresponding user can see the confidential items. At the same time, other users may be granted access to another version of the same image, where the confidential items are obscured (such as hidden, cropped, scrambled, etc.).

[0116] In some embodiments, a selected user (e.g., a parent) may designate some children as private items. Images of the campus captured by the camera may be distributed to multiple users. However, the images distributed to the principal and the parents of the children deemed private will show the faces of the corresponding children, while in the images distributed to all other users, the faces may be obscured or scrambled.

[0117] The applications of some embodiments are not limited to monitoring. In another example, a camera records a demonstration of a product or prototype. Subsequently, the images are transmitted to multiple remote users, e.g., in a video conferencing format. However, different users may receive different versions of the same image. For example, users who have signed a confidentiality agreement may be shown the corresponding product or prototype, while in the images distributed to other users, the corresponding item may be obscured / scrambled.

[0118] The nature of the items deemed private / confidential may vary widely between embodiments. The artificial intelligence system 43 ( Figure 5 ) can be trained to identify any such type of private item within the source image. Subsequently, some users will see the images of the corresponding item, while other users will not.

[0119] Many conventional video surveillance systems use encryption to prevent unauthorized access to the acquired images. Some such systems also add automatic image recognition and / or image segmentation functionality. However, conventional surveillance systems first decrypt the source image to prepare for image recognition. For example, a conventional computer system performing image analysis typically also has a key for decrypting the source image. In contrast, by utilizing homomorphic encryption, some embodiments of the present invention perform automatic item detection / mask construction directly in the encrypted domain, i.e., without first decrypting the source image. Specifically, the privacy management server described herein does not even have a key for decrypting the source data. Thus, in embodiments of the present invention, the computer system performing image recognition and / or segmentation does not know the content of the image being analyzed, which generally enhances the privacy of the system users.

[0120] The use of homomorphic encryption by some embodiments of the present invention also allows decoupling of the user management / image distribution activities from the image analysis activities. In Figure 1In the exemplary privacy - protected video surveillance system described, servers 30 and 40 can be owned and operated by separate entities. In an exemplary use - case scenario demonstrating the advantages of some embodiments of the present invention, Company A owns and operates input sensor 14 and distribution server 30 and outsources the image - processing service (i.e., the service provided by server 40) to another Company B. Sensor 14 can collect images from an office building, and Company A may want to automatically detect events such as abnormal office activities, presence of unknown persons, etc. to determine the attendance of certain office events, determine when certain employees arrive or leave work, etc. Company B can provide such services in a privacy - protected manner because server 40 has no access to unencrypted data and further lacks the information to decrypt the incoming source data. Instead, image segmentation and / or other task execution are implemented in the encrypted domain, and the results of such operations can only be decrypted by a computer system (such as server 30, selected client devices 12a - c) operated by a representative of Company A. Privacy is further enhanced by the fact that in Figure 11 and 13 the embodiments described, distribution server 30 has no access to the source data itself, but is limited to accessing its "public part", i.e., the part of the source image that does not show private / confidential items.

[0121] The applications of some embodiments are not limited to image processing / video surveillance, but can be applicable to the processing of sound files, documents, and electronic messages, etc. In one such exemplary embodiment, the voice of a target person can be selected as a private item. For example, the source data of a recording can be processed as shown herein, i.e., it can be divided into a private part and a public part, where the private part can consist of segments of the source recording that include the words of the target person. Then, the private part can be proxy - re - encrypted with tokens corresponding to a selected subset of users. When the corresponding recording is reconstructed, the selected users can hear the target person speaking, while other users cannot. Another exemplary embodiment can distort / scramble the utterances of certain words (such as curse words, selected names, etc.).

[0122] In an exemplary document or message - processing embodiment, private items can include certain names, addresses, phone numbers, credit card or bank account numbers, etc. In some embodiments, private items can include entire sections of a document, such as a specific chapter, a section with a specific author, a section that mentions a specific topic. In yet another exemplary embodiment, private items can include parts of a conversation (such as an electronic message exchange) that indicate a specific emotion. The item detector 42 can use a set of rules or a pre - trained artificial - intelligence system to automatically identify such private items in an encrypted source document. Using the selective proxy - re - encryption technique shown herein, the same document can then be distributed to multiple users in such a way that selected users can see the corresponding private items in plain text, while other users cannot.

[0123] Those skilled in the art will appreciate that the above embodiments can be modified in many ways without departing from the scope of the present invention. Accordingly, the scope of the present invention is to be determined by the appended claims and their legal equivalents.

Claims

1. A method for distributing customized privacy-protected images to multiple users, the method comprising using at least one hardware processor of a privacy management server: Upon receiving an encrypted source image decrypted with an available management key, performing encrypted-domain image segmentation of the source image to generate multiple user-specific encrypted private images, wherein: A selected private image includes an area of the source image that is selected to show a private item of a selected user among the multiple users, Another private image includes another area of the source image that is selected to show a private item of another user among the multiple users, and wherein determining the selected private image includes: Using a pre-trained item detector to calculate a mask that identifies the area of the source image showing the private item of the selected user, the mask being calculated in the encrypted domain and decryptable with the management key, and Determining the selected private image based on a pixel-by-pixel multiplication of the encrypted source image and the mask; Upon the image segmentation, performing an encrypted-domain key change procedure to generate multiple user-specific re-encrypted images, wherein: A selected re-encrypted image includes the result of transforming the selected private image from being decryptable with the management key to being decryptable with the private key of the selected user, and Another re-encrypted image includes the result of transforming the other private image from being decryptable with the management key to being decryptable with the private key of the other user; and Transmitting the multiple user-specific re-encrypted images to an image distribution server for further distribution to client devices configured to reconstruct a user-specific plaintext version of the source image.

2. The method according to claim 1, wherein: Performing the encrypted-domain image segmentation of the source image further includes determining an encrypted public image, the encrypted public image including an area of the source image that is selected to show neither the private item of the selected user nor the private item of the other user; wherein the method further includes using at least one hardware processor of the privacy management server to transmit the encrypted public image to the image distribution server for decryption and further distribution to client devices; and wherein the client devices are configured to further reconstruct a user-specific plaintext version of the source image based on the decrypted public image received from the image distribution server.

3. The method according to claim 1, further comprising using at least one hardware processor of the image distribution server: Upon receiving the selected re-encrypted image, selecting a device from multiple client devices based on whether the selected device is operated by the selected user; and Upon selecting the client device, transmitting the selected re-encrypted image to the selected device.

4. The method according to claim 3, further comprising using at least one hardware processor of the image distribution server: Upon receiving the other re-encrypted image, selecting the other device from the multiple client devices based on whether the other device is operated by the other user; and Upon selecting the other device, transmitting the other re-encrypted image to the other device.

5. The method according to claim 1, wherein the encrypted source image is encrypted according to a homomorphic encryption scheme.

6. The method according to claim 1, wherein the plaintext version of the source image reconstructed by the client device operated by the selected user displays the private items of the selected user and masks the private items of the other user.

7. The method according to claim 1, wherein the private items include items selected from the group consisting of a person and a human face.

8. The method according to claim 1, wherein the private item includes a bank card.

9. The method according to claim 1, wherein the private item includes a trademark.

10. A computer system, comprising a privacy management server configured to: In response to receiving an encrypted source image decrypted with an available management key, perform encrypted domain image segmentation of the source image to generate a plurality of user-specific private images, wherein: The selected private image includes an area of the source image selected to display the private items of the selected user among the plurality of users, Another private image includes another area of the source image selected to display the private items of another user among the plurality of users, and wherein determining the selected private image includes: Using a pre-trained item detector to calculate a mask that identifies the area of the source image that displays the private items of the selected user, the mask being calculated in the encrypted domain and decryptable with the management key, and Determining the selected private image according to the pixel-by-pixel multiplication of the encrypted source image and the mask; In response to the image segmentation, perform an encrypted domain key change procedure to generate a plurality of user-specific re-encrypted images, wherein: The selected re-encrypted image includes the result of transforming the selected private image from being decryptable with the management key to being decryptable with the private key of the selected user, and Another re-encrypted image includes the result of transforming the other private image from being decryptable with the management key to being decryptable with the private key of the other user; and Transmit the plurality of user-specific re-encrypted images to an image distribution server for further distribution to client devices configured to reconstruct the user-specific plaintext version of the source image.

11. The computer system according to claim 10, wherein: Performing the encrypted domain image segmentation of the source image further includes determining an encrypted public image, the encrypted public image including an area of the source image selected to neither display the private items of the selected user nor display the private items of the other user; wherein the privacy management server is further configured to transmit the encrypted public image to the image distribution server for decryption and further distribution to client devices; and wherein the client device is configured to further reconstruct the user-specific plaintext version of the source image according to the decrypted public image received from the image distribution server.

12. The computer system according to claim 10, further comprising the image distribution server and wherein the image distribution server is further configured to: In response to receiving the selected re-encrypted image, select a device from a plurality of client devices based on whether the selected device is operated by the selected user; and In response to selecting the client device, transmit the selected re-encrypted image to the selected device.

13. The computer system according to claim 12, wherein the image distribution server is further configured to: In response to receiving the other re-encrypted image, select the other device from the plurality of client devices based on whether the other device is operated by the other user; and In response to selecting the other device, transmit the other re-encrypted image to the other device.

14. The computer system according to claim 10, wherein the encrypted source image is encrypted according to a homomorphic encryption scheme.

15. The computer system according to claim 10, wherein the plaintext version of the source image reconstructed by the client device operated by the selected user displays the private items of the selected user and masks the private items of the other user.

16. The computer system according to claim 10, wherein the private items include items selected from the group consisting of a person and a face.

17. The computer system according to claim 10, wherein the private item includes a bank card.

18. The computer system according to claim 10, wherein the private item includes a trademark.

19. A non-transitory computer-readable medium storing instructions that, when executed by at least one hardware processor of a privacy management server, cause the privacy management server to: In response to receiving an encrypted source image decryptable with an available management key, perform encrypted-domain image segmentation of the source image to generate a plurality of user-specific private images, wherein: The selected private image includes the area of the source image selected to display the private items of the selected user among the plurality of users, The other private image includes another area of the source image selected to display the private items of the other user among the plurality of users, and wherein determining the selected private image includes: Using a pre-trained item detector to calculate a mask identifying the area of the source image that displays the private items of the selected user, the mask being calculated in the encrypted domain and decryptable with the management key, and Determining the selected private image based on the pixel-by-pixel multiplication of the encrypted source image and the mask; In response to the image segmentation, perform an encrypted-domain key change procedure to generate a set of user-specific re-encrypted images, wherein: The selected re-encrypted image includes the result of transforming the selected private image from being decryptable with the management key to being decryptable with the private key of the selected user, and The other re-encrypted image includes the result of transforming the other private image from being decryptable with the management key to being decryptable with the private key of the other user; and Transmit the plurality of user-specific re-encrypted images to an image distribution server for further distribution to client devices configured to reconstruct the user-specific plaintext version of the source image.

Images