Electronic device, computer-implemented method, and system for anonymising facial data of a person depicted in image data

Abstract

The invention relates to an electronic device for anonymising facial data of a person depicted in image data, the electronic device comprising: - a processor configured to receive as an input at least one data element representing image data of the person, wherein the image data comprises facial data of the person and a unique identifier associated with the person, wherein the processor is further configured to determine a data value of the unique identifier and is configured to compare the determined data value to a predefined data value, wherein the processor is further configured to mask the facial data of the person in response to a match between the predefined data value and the determined data value, and wherein the processor is configured to output a modified data element comprising the masked facial data. The invention also relates to a computer-implemented method, and a related system for anonymising facial data of a person depicted in image data.

Description

Electronic device, computer-implemented method, and system for anonymising facial data of a person depicted in image dataTechnical domain

[0001] The present invention concerns an electronic device, a computer- implemented method, and a system for anonymising facial data of a person depicted in image data.Background art

[0002] With today's ease of taking pictures in almost every event, the question of data privacy and the right to be anonymous is becoming prevalent.

[0003] Organizers of public or private events are often willing to take and publish pictures of said event for promotion or information purpose. In order to meet data privacy standards, it is usually the case that the participation to the event is conditioned to accepting an agreement to be photographed and that the photograph can be later published.

[0004] In the eventuality that someone does not agree to be photographed, it is difficult to guarantee that no picture of that person will be taken as she may appear in the background of some photograph.

[0005] It is therefore desirable to find ways of ensuring data privacy without relying on the simple acceptance / refusal of an agreement.Short disclosure of the invention

[0006] An aim of the present invention is the provision of an electronic device, a computer-implemented method, and a system all with the iobjective of overcoming the shortcomings and limitations of the state of the art.

[0007] The electronic device, the computer-implemented method, and / or the system can be used anonymising facial data of a person depicted in image data simply and efficiently, thereby empowering the privacy of individuals. However, the electronic device, the computer-implemented method, and / or the system might be used alternatively or in addition to masking any personally identifiable information of a person depicted in image data.

[0008] According to a first aspect of the present invention, an electronic device involving the features recited in claim 1 is disclosed. Further features and embodiments of the electronic device of the present invention are described in the dependent claims.

[0009] The electronic device for anonymising facial data of a person depicted in image data, the electronic device comprises:- a processor configured to receive as an input at least one data element representing image data of the person, wherein the image data comprises facial data of the person and a unique identifier associated with the person, wherein the processor is further configured to determine a data value of the unique identifier and is configured to compare the determined data value to a predefined data value, wherein the processor is further configured to mask the facial data of the person in response to a match between the predefined data value and the determined data value, and wherein the processor is configured to output a modified data element comprising the masked facial data. The term image data can also comprise video data, whereby a video can be a collection of a sequence of images.

[0010] The electronic device can be a computer device, such as a personal computer, a smartphone, tablet computer, virtual reality headset, augmented reality glasses, digital camera (including video camera), and the like.

[0011] In a first embodiment of the first aspect, the facial data can correspond to a portion of the image data.

[0012] In a second embodiment of the first aspect, the unique identifier can correspond to a portion of the image data different from the facial data.

[0013] In a third embodiment of the first aspect, the unique identifier can be provided in the form of a machine readable pattern depicted in the portion of the image data.

[0014] In a further embodiment of the first aspect, the unique identifier can correspond to the facial data.

[0015] In a different embodiment of the first aspect, the data value can be determined from the facial data in the form of a feature vector.

[0016] In another embodiment of the first aspect, the processor can comprise an image processor for processing and manipulating the image data, preferably adapted for determining the data value and masking the facial data.

[0017] In a further embodiment of the first aspect, the device can further comprise a memory, and wherein the processor can be configured to output the modified data element to said memory for storage.

[0018] Even though the individual embodiments of the first aspect cover different aspects of the invention, some or all embodiments can be combined when it is useful and feasible from a technical standpoint.

[0019] In a second aspect of the invention, a computer-implemented method is disclosed.The computer-implemented method for anonymising facial data of aperson depicted in image data including the steps of:- providing at least one data element representing image data of the person, wherein the image data comprises facial data of the person and a unique identifier associated with the person;- determining a data value from the unique identifier associated with the person;- comparing the determined data value with a predefined data value;- masking facial data of the person if the determined data value matches the predefined data value;- forming a modified data element comprising the image data of the person and the masked facial data of the said person.

[0020] In a first embodiment of the second aspect, the facial data can be provided in the form of a plurality of pixel elements, wherein the step of masking can comprise the step of changing a size, a colour, and / or a shape of the plurality of pixel elements of the facial data.

[0021] In a second embodiment of the second aspect, the method can comprise the step of generating the data element by imaging the person before the data element is inputted.

[0022] In a third embodiment of the second aspect, the unique identifier can be depicted in the image data in the form of a machine readable pattern.

[0023] In a further embodiment of the second aspect, the method can comprise the step of attaching the unique identifier to a body of the person, before the data element is generated. The unique identifier might not necessarily be attached directly or closely to the person's body. It can also be attached to or part of the person's clothes.

[0024] In a different embodiment of the second aspect, the unique identifier can be depicted in the facial data, and wherein the data value can be determined from the facial data in form of a feature vector.

[0025] In another embodiment of the second aspect, the method can comprise the steps of registering the predefined data value in a database and reading the predefined data value registered from the database before comparing the determined data value with the predefined data value.

[0026] In a further embodiment of the second aspect, the method can comprise the steps of encrypting the predefined data value registered using an encryption key and comparing the data value determined with the encrypted predefined data value.

[0027] In another embodiment of the second aspect, the method can comprise the further steps of:- inputting to a processor the modified data element and inputting a key by a user;- validating the authenticity of the key and in case the key is found authentic:- unmasking the facial data of the person;- outputting from the processor a second modified data element comprising the image data of the person and the unmasked facial data of the said person.

[0028] Even though the individual embodiments of the second aspect cover different aspects of the invention, some or all embodiments can be combined when it is useful and feasible from a technical standpoint.

[0029] In a third aspect of the present invention, a system is disclosed. The system for anonymising facial data of a person depicted in image data, the system comprising:- a person;- a unique identifier worn by the person;- an imaging device for imaging the person and the unique identifier worn by the person, wherein the imaging device is arranged to store image data of the person in a memory;- an electronic device of the first aspect (including all embodiments of acombination thereof), configured to receive as an input at least one data element representing the image data stored in the memory of the imaging device.

[0030] The imaging device can be a digital camera functioning as an instrument capable of recording, storing and / or transmitting visual images that can be utilised to take images from a person.

[0031] In an embodiment of the third aspect, the processor of the electronic device can be configured to process the steps of the computer- implemented method of the second aspect (including all embodiments of a combination thereof).

[0032] The invention is also defined by the example embodiments as included herein.

[0033] Even though all aspects and embodiments disclosed herein before relating to facial data, the first, second, and third aspects can alternatively or in addition related to anonymising personally identifiable information of a person depicted in image data. A person's identity can also be inferred from personal characteristics in the image data, such as but not limited to hair, accessories, clothing, name on a badge, etc. This personal information of a person represented in image data can be anonymised with the aid of the three aspects (including any embodiment or a combination thereof) mentioned. For this purpose, instead of the facial data, the personal information or characteristics depicted in image data are evaluated and masked accordingly.Examples of embodiments of the present invention

[0034] In the following, the term "EgoToken" can refer to a physical unique identifier worn by a person, such as an attendee of a conference, or a different unique identifier associated to the said person, which will become apparent in the following. The EgoToken can take any visual form, such as a logo, a string of character, a dedicated picture, a bar code, a QR- code or any type of glyph.

[0035] At the registration of an event, people can decide if they want to get an EgoToken in order to gain control if they are visible or not in published event photography and / or videography.

[0036] In an embodiment illustrated in Fig. 1, an EgoToken can be printed on the badge of the event and linked to registered attendees.

[0037] In another embodiment illustrated in Fig. 2, it is also possible for it to be a standalone, physical badge that one can wear on their clothes. On the left of Fig. 2, a first side of the physical badge is depicted. This side may typically contain a name, a function, a position, a company name, a photograph, etc. The EgoToken is printed this first side. The other side of the badge, illustrated on the right of Fig. 2, may contain additional information and / or terms and conditions regulating the use of EgoToken.

[0038] As schematically illustrated in Fig. 4, a picture of several persons was taken, at least one person in the picture being wearing a unique identifier as defined above. The picture contains both facial data about the persons and the unique identifier. The picture is then anonymized using the described computer-implemented method.

[0039] In another embodiment, if there is legal ground or consent given by the attendee, the EgoToken can directly be generated digitally from an image or face print, as illustrated in Fig. 3 (designed by Freepik), of thatperson, meaning, that there is no physical embodiment of the EgoToken, but instead a digital feature vector representation of that person. To ensure that this feature vector cannot be used to reverse engineer the original identity, hashing can be applied so that one can only do forward way validation.

[0040] Once images are processed and edited, the event organiser / publishing entity can take the images that should be published and run it through an anonymisation algorithm. The algorithm is able to detect the token and then anonymise the person wearing it, naturally, pixelated, blurred or any other form of anonymisation. This same algorithm can also be used live onsite, for example as part of a photo app, to allow people to post anonymised content from the event directly to social media or any other platform.

[0041] The steps of the computer-implemented method described herein are illustrated in Fig. 5. As a first step, at least one data element representing image data of a person is provided as an input to a processor, i.e. a computing unit. The image data comprises facial data of the person and a unique identifier associated with the person.

[0042] The unique identifier may be directly recognized from the image data as visible element, or complementary or alternatively, reconstructed from the image data.

[0043] Then, a data value is determined from the unique identifier associated with the person. This data value retrieved from the unique identifier is then compared with a predefined value. If the determined data value matches the predefined value, an anonymization step is executed, meaning that the facial data is anonymized, as explained above, by masking, blurring, pixelization, or any other picture anonymization technique.

[0044] It the determined data value doesn't match the predefined value, no anonymization is performed for this particular facial data.

[0045] Finally, a modified data element is generated wherein the facial data is masked or not, depending on the match between the determined and predefined data value.

[0046] If an EgoToken is occluded, cannot be parsed with high accuracy or for any other reason is not detectable, additional measures can be taken to avoid false negatives. Here, the algorithm can cross reference a feature vector representation of the target person with people in the same media dataset of the event, in order to check if the same person is associated with an EgoToken in any other picture. In case of a positive match, the target person is anonymised.

[0047] Furthermore, each EgoToken is unique, therefore, images / videos taken of a person can be linked to their EgoToken if desired or required. This enables also searching a large dataset of media for content that is associated with the EgoToken. The uniqueness of an EgoToken comes from its association to an attendee during registration, through an embedded unique visual identifier that can be parsed or due to the unique characteristics of the feature vector embodiment.

[0048] The uniqueness of the EgoToken allows attendees to gain postevent access to all the images or videos that were taken of them, with the potential to give later consent for publication. Such access to media associated with a person corresponding to an EgoToken may, but must not require some form of authentication step or is only possible through restricted access via a registered account.

[0049] One embodiment of authentication is a 2FA supporting device that is linked with the EgoToken at the registration. Alternatively, an EgoToken could also comprise of an embedded chip, e.g. NFC or similar technology, in case of any physical embodiment. This way, authenticationof EgoToken ownership post-event is possible via scanning the chip with any capable reader device.

[0050] Lastly, the EgoToken can but must not be reusable i.e. used for the same purpose at multiple events.

[0051] Figure 6 schematically illustrates the system of the present invention. A picture of a person wearing a unique identifier is taken using an imaging device. Image data from this picture is stored on a memory. Finally, an electronic device as described above receives as an input the data element representing the image data stored in the memory of the imaging device and performs the steps of the computer-implemented method described above. As output, a picture is generated wherein a facial data of a first person, wearing its unique identifier has been anonymized and facial data of a second person wearing its unique identifier has not been anonymized. The facial data of the third person is left unchanged as she's not wearing any unique identifier.

[0052] As explained above, the anonymization of the facial data is determined by comparing a determined data value (determined by the unique identifier) with a predefined data value (e.g. previously indicated by the person). This allows to discriminate between persons willing to be anonymized and persons agreeing to appear on the photographs.

Claims

Claims1. An electronic device for anonymising facial data of a person depicted in image data, the electronic device comprising:- a processor configured to receive as an input at least one data element representing image data of the person, wherein the image data comprises facial data of the person and a unique identifier associated with the person, wherein the processor is further configured to determine a data value of the unique identifier and is configured to compare the determined data value to a predefined data value, wherein the processor is further configured to mask the facial data of the person in response to a match between the predefined data value and the determined data value, and wherein the processor is configured to output a modified data element comprising the masked facial data.

2. The electronic device of claim 1, wherein the facial data corresponds to a portion of the image data.

3. The electronic device of claim 1 or 2, wherein the unique identifier corresponds to a portion of the image data different from the facial data.

4. The electronic device of any one of the claims 1 to 3, wherein the unique identifier is provided in form of a machine readable pattern depicted in the portion of the image data.

5. The electronic device of claim 1 or 2, wherein the unique identifier corresponds to the facial data.

6. The electronic device of claim 5, wherein the data value is determined from the facial data in form of a feature vector.

7. The electronic device of any one of the claims 1 to 6, wherein the processor comprises an image processor for processing and manipulatingthe image data, preferably adapted for determining the data value and masking the facial data.

8. The electronic device of any one of the claims 1 to 7 further comprising a memory and wherein the processor is configured to output the modified data element to said memory for storage.

9. A computer-implemented method for anonymising facial data of a person depicted in image data including the steps of:- providing at least one data element representing image data of the person, wherein the image data comprises facial data of the person and a unique identifier associated with the person;- determining a data value from the unique identifier associated with the person;- comparing the determined data value with a predefined data value;- masking facial data of the person if the determined data value matches the predefined data value;- forming a modified data element comprising the image data of the person and the masked facial data of the said person.

10. The computer-implemented method of claim 9, wherein the facial data is provided in the form of a plurality of pixel elements, wherein the step of masking comprises the step of changing a size, a colour, and / or a shape of the plurality of pixel elements of the facial data.11.The computer-implemented method of claim 9 or 10, comprising the step of generating the data element by imaging the person before the data element is inputted.

12. The computer-implemented method of claim 11, wherein the unique identifier is depicted in the image data in form of a machine readable pattern.

13. The computer-implemented method of claim 12, comprising the step of attaching the unique identifier to a body of the person, before the data element is generated.

14. The computer-implemented method of any one of the claims 9 to 11, wherein the unique identifier is depicted in the facial data, and wherein the data value is determined from the facial data in form of a feature vector.

15. The computer-implemented method of any one of the claims 9 to 14, comprising the steps of registering the predefined data value in a database and reading the predefined data value registered from the database before comparing the determined data value with the predefined data value.

16. The computer-implemented method of claim 15, further comprising the steps of encrypting the predefined data value registered using an encryption key and comparing the data value determined with the encrypted predefined data value.

17. The computer-implemented method of any one of the claims 9 to 16, further comprising the steps of:- inputting to a processor the modified data element and inputting a key by a user;- validating the authenticity of the key and in case the key is found authentic:- unmasking the facial data of the person;- outputting from the processor a second modified data element comprising the image data of the person and the unmasked facial data of the said person.

18. System for anonymising facial data of a person depicted in image data, the system comprising:- a person;- a unique identifier worn by the person;- an imaging device for imaging the person and the unique identifier worn by the person, wherein the imaging device is arranged to store image data of the person in a memory;- an electronic device of any one of the claims 1 to 8, configured to receive as an input at least one data element representing the image data stored in the memory of the imaging device.

19. System of claim 18, wherein the processor of the electronic device is configured to process the steps of the computer-implemented method of any one of the claims 9 to 17.

Images