Systems and methods for device content privacy

Abstract

A system for providing privacy measures for a computing device includes a memory, a display configured to display images and / or data that may be subject to privacy concerns, an image sensor configured to provide image data of a user operating the computing device and image data of an environment surrounding the user within a field of view of the image sensor, a processor to receive the image data, segment the image data to isolate facial features of the user, generate a user face vector representative of characteristics of the segmented facial image data, and compare the user face vector to face vector data associated with authorized device users stored in the memory, and a privacy controller module configured to determine a privacy threat alert based on the comparison of the face vectors.

Description

[Technical field]

[0001] The present invention relates to privacy applications for electronic devices, and more particularly to software applications for maintaining content privacy on electronic devices.

[0002] The disclosed systems and methods relate to computer hardware and computer software running on the computer hardware, computer-based systems, and computer-based methods for maintaining the privacy of computer users while using computer-based data processing and communications devices. The disclosed techniques have applications in the fields of data processing, portable computing, computer-based communications, computer security, and maintaining data privacy.

[0003] The present invention has been developed primarily for use in methods and systems for protecting private electronic content displayed on a user's electronic device, including a mobile phone, tablet or computer system, and will be described hereinafter with reference to this application, although it will be understood that the invention is not limited to this particular field of use. [Background technology]

[0004] The discussion of background art throughout this disclosure should in no way be taken as an admission that such background art is prior art or that such background art is widely known or forms part of the common general knowledge in the field in Australia or throughout the world at the priority date of this application.

[0005] All documents, including patents or patent applications, cited in this specification are incorporated herein by reference and should be read and considered as part of this specification by the reader. It is for the sake of brevity only that documents, documents, patent applications or patents cited in this specification are not repeated in this specification.

[0006] No admission is made that any reference or document cited herein is prior art. The discussion of the cited references states what their authors assert, and the applicants reserve the right to challenge the accuracy and pertinence of the cited references. Although a number of prior art documents may be cited herein, it should be expressly understood that such citation is not an admission that any of these documents form part of the general knowledge in the art in Australia or anywhere else at the priority date of this application.

[0007] Commercially available desktop and portable computers and computer-controlled devices, such as laptop computers, netbooks, tablet computers, personal digital assistants ("PDAs"), and smartphones (generally referred to herein as "devices" or "apparatus"), fail to adequately maintain privacy of information displayed to a user while the device is in use. While an authorized user is viewing information, unauthorized persons can view and even record information from nearby locations, such as over the shoulder of the device user, an act commonly referred to as "shoulder surfing." As the use of portable computing devices in public places increases, displaying information in a manner that permits unauthorized viewing in public, semi-public, and even restricted locations becomes increasingly problematic. For example, a patient's medical records displayed on a screen in an examination room may be visible to people sitting in a nearby waiting room or to maintenance personnel working in the examination room. An email may be generated in an airport lobby announcing that a public company has won a large contract, which may be viewed by another passenger waiting nearby, who may disseminate this confidential information without intending it to be made public. There are many ways in which unauthorized viewing of displayed data can cause damage or loss. Restricting the viewing of sensitive data to times and places where privacy is assured is not a practical solution to this problem, given the pace of modern business and life in general, and the ever-increasing capabilities of portable computing devices. Some means of detecting, limiting, or preventing disclosure to others while still allowing authorized users to view the information is required.

[0008] Prior art for protecting displayed data includes software commonly referred to as screensavers. Originally created to prevent damage to cathode ray tube ("CRT") monitors, where a persistently displayed image can be "burned" and remain permanently displayed on the CRT's phosphors, these programs also have some utility in preventing unauthorized viewing of on-screen data or even use of the computer. If there is no user input to the computer (such as keyboard input or pointing device activity) for a set period of time (usually one to fifteen minutes), the screensaver activates and replaces the displayed information with a non-static display such as a slideshow of images, the output of a graphics generating program, or scrolling messages. When input such as keystrokes or mouse activity resumes, the screensaver becomes inactive and reverts to the previous display of information. Some screensavers require that re-authentication, such as the entry of a password, be performed before the screensaver becomes inactive and reverts to the previous display. However, while screensavers can to some extent restrict access to data displayed when the user is not using the computer, they have some significant limitations with respect to protecting the privacy of on-screen data. First, screensavers do not protect data privacy while a user is actively working. Second, there is a delay between when a user stops working and perhaps walks away from the computer and when the screensaver activates. Third, anyone can prevent the screensaver from activating after an authorized user leaves the area by providing input to the computer, such as by moving a mouse or pressing a key on a keyboard, thus gaining extra time to read the display.

[0009] Another prior art for privacy protection is the "privacy filter." A "privacy filter" is a physical device that can be added to the front of a display to reduce the viewing angle of the display, limiting or completely blocking viewing at certain angles. Such privacy filters also have significant limitations because they cannot prevent unauthorized viewing from directly behind the user, and the effect of reducing the viewing angle from above or below may be less than the effect of reducing the viewing angle from the side. The limited effect is particularly noticeable for monitors that can be rotated between "portrait" and "landscape." In addition, privacy filters can reduce the brightness of the display by 40% or more, change the contrast of the display, or distort the image on the display, so some users, especially those with some degree of visual impairment, do not like to use them. In addition, privacy filters are generally removable, allowing users to disable their protection and violate security policies without such interference being detected.

[0010] Each of the above-mentioned prior art techniques for protecting the display of information on a computer from authorized viewing also suffers from their inherent "all-or-nothing" scope, i.e., either one must apply to the entire screen.

[0011] Moreover, the above-described methods and systems are typically applied in a static or modal fashion. In other words, the user must perform some command or other intentional action to change the degree of security to the extent that such a change in degree is possible. For example, users often want to view data displays and sensitive information, possibly without their knowledge, not only in relatively secure locations such as the office, but also at home, coffee shops, airports, and other insecure environments where unauthorized individuals and devices can view the user's display. However, users often forget to adjust security settings to account for the loss of privacy when moving from the office to public spaces, risking both intentional and inadvertent security breaches. It would therefore be useful to have some way to automatically adjust security levels depending on the computer and the user's environment.

[0012] The above-mentioned needs are addressed by the present invention using the exemplary embodiments disclosed herein, as will be appreciated by those skilled in the art. Summary of the Invention

[0013] It is an object of the present invention to overcome or ameliorate at least one or more of the disadvantages of the prior art, or to provide a useful alternative.

[0014] According to a first aspect of the present invention, a system for providing privacy measures to a computing device is provided. The system may comprise a memory. The system may further comprise a display configured to display images and / or data that may be subject to privacy concerns. The system may further comprise at least one image sensor configured to provide image data of a person including a user operating the computing device and image data of an environment surrounding the user within an image sensor field of view. The system may further comprise at least one processor. The processor may be configured to receive the image data and segment the image data to isolate facial features of the user and generate a user face vector representative of characteristics of the segmented face image data. The processor may further be configured to compare the user face vector to face vector data associated with one or more authorized device users stored in the memory. The system may further comprise a privacy controller module configured to determine a privacy threat alert based on the comparison of the face vectors. Upon receiving a privacy threat alert from the privacy controller, the processor may be configured to modify a display of the electronic device to preserve the privacy of the displayed images and / or data.

[0015] According to a particular arrangement of a first aspect, there is provided a system for providing privacy measures to a computing device, the system comprising: Memory, a display configured to display images and / or data that may be subject to privacy concerns; and at least one image sensor configured to provide image data of a person, including a user operating a computing device, and image data of an environment surrounding the user within a field of view of the image sensor; at least one processor, the at least one processor comprising: receiving image data, segmenting the image data to isolate facial features of a user, and generating a user face vector representative of characteristics of the segmented facial image data; at least one processor that compares the user face vector to face vector data associated with one or more authorized device users stored in a memory; a privacy controller module configured to determine a privacy threat alert based on a comparison of the facial vectors; Upon receiving a privacy threat alert from the privacy controller, the processor is configured to modify a display of the electronic device to preserve privacy of the displayed images and / or data.

[0016] The memory may be configured to store the face vector together with the face recognition data. The face recognition data may include a face vector having face vector data.

[0017] The processor may be configured to further segment the image data to identify a person or object within the field of view of the image sensor. The processor may be configured to further segment the image data to form further segmented image data and to identify from the further segmented image data either a further person or object within the field of view of the image sensor. The object may be an image and / or video recording device.

[0018] The privacy controller may be further configured to determine a privacy threat based on the segmented image data, and, during use, when a privacy threat is identified, the privacy controller causes the processor to modify the display of the electronic device to preserve the privacy of data previously displayed on the display.

[0019] The display modification may include displaying an image or message that at least partially obscures the image or data on the display. The display modification may include blacking out the display or placing the display in an "off" state.

[0020] According to a second aspect of the present invention, a computer program product for privacy of an electronic device is provided. The computer program product may comprise a non-transitory computer readable storage medium having computer readable program code portions stored therein. The computer readable program code portions may include a first portion configured to receive image data from an image sensor. The computer readable program code portions may further include a second portion configured to segment the image data to isolate facial features of persons including a user and generate a user face vector representative of characteristics of the segmented face image data. The computer readable program code portions may further include a third portion configured to compare the user face vector to face vector data of known authorized users having access to the electronic device, the known authorized users face vector data may be stored in the memory. The computer readable program code portions may further include a fourth portion configured to modify a display of the electronic device to obscure or remove images or data displayed on a display of the electronic device.

[0021] According to a particular arrangement of the second aspect of the present invention, there is provided a computer program product for privacy in an electronic device, the computer program product comprising a non-transitory computer readable storage medium having computer readable program code portions stored therein, the computer readable program code portions comprising: a first portion configured to receive image data from an image sensor; a second portion configured to segment the image data to isolate facial features of persons including the user and generate a user face vector representative of characteristics of the segmented facial image data; a third part configured to compare the user face vector with face vector data of known users authorized to access the electronic device; and a fourth portion configured to modify a display of the electronic device such that the image or data is not displayed on the display of the electronic device.

[0022] The computer program product is a fifth portion configured to further segment the image data to form further segmented image data to identify either a further person or object within the field of view of the image sensor; and a sixth portion configured to determine a privacy threat based on the segmented image data, and configured such that, in use, when a privacy threat is identified, the sixth portion engages the fourth portion to alter a display of the electronic device to preserve privacy of data previously displayed on the display. The object may include an image and / or video recording device.

[0023] The display modification may include displaying an image or message that at least partially obscures the image or data on the display. The display modification may include blacking out the display or placing the display in an "off" state.

[0024] According to a third aspect of the present invention, a method of providing privacy measures for a computing device is provided. The method may include providing a display configured to display images and / or data that may be subject to privacy concerns. The method may further include providing at least one image sensor configured to provide image data of a person including a user operating the computing device and image data of an environment surrounding the user within a field of view of the image sensor. The method may further include providing at least one processor configured to receive the image data, segmenting the image data to isolate facial features of the user and generating a user face vector representative of characteristics of the segmented face image data, and comparing the user face vector to face vector data associated with one or more authorized device users stored in a memory. The method may further include providing a privacy controller module configured to determine a privacy threat alert based on the comparison of the face vectors. Upon receiving a privacy threat alert from the privacy controller, the processor may be configured to modify a display of the electronic device to preserve privacy of the displayed images and / or data.

[0025] According to a particular arrangement of a third aspect, there is provided a method for providing privacy measures to a computing device, the method comprising: providing a display configured to display images and / or data that may be subject to privacy concerns; providing at least one image sensor configured to provide image data of a person including a user operating a computing device and image data of an environment surrounding the user within a field of view of the image sensor; providing at least one processor, the at least one processor comprising: Receive image data; Segmenting the image data to isolate facial features of the user and generating a face vector representative of characteristics of the segmented facial image data; configured to compare the face vector with face vector data associated with one or more authorized device users stored in a memory; providing a privacy controller module configured to determine a privacy threat alert based on the comparison of the face vectors; Upon receiving a privacy threat alert from the privacy controller, the processor is configured to modify a display of the electronic device to preserve privacy of the displayed images and / or data.

[0026] The method may further include further segmenting the image data using a processor to form further segmented image data to identify either further persons and / or objects within the field of view of the image sensor.

[0027] The device may be an image and / or video recording device.

[0028] The privacy controller may be further configured to determine a privacy threat based on the segmented image data, and in use, when a privacy threat is identified, the processor may be instructed to modify the display of the electronic device to preserve the privacy of data previously displayed on the display.

[0029] The display modification may include displaying an image or message that at least partially obscures the image or data on the display. The display modification may include blacking out the display or placing the display in an "off" state.

[0030] According to a fourth aspect of the present invention, there is provided a computer program product having a computer readable storage medium having recorded thereon a computer program for providing privacy measures on a computing device. The computing device may include at least one processor. The computing device may further include a memory. The computing device may further include at least one image sensor. The computing device may further include a display. The computer program product may include computer program code means for displaying images and / or data that may be subject to privacy concerns. The computer program product may further include computer program code means for receiving image data from the at least one image sensor, the image data including image data of a person including a user operating the computing device and image data of an environment surrounding the user within a field of view of the image sensor. The computer program product may further include computer program code means for segmenting the image data to isolate facial features of the user and generating a user face vector representative of characteristics of the segmented face image data. The computer program product may further include computer program code means for comparing the user face vector to face vector data associated with one or more authorized device users stored in the memory. The computer program product may further include computer program code means for providing a privacy controller module configured to determine a privacy threat alert based on the facial vector comparison. The computer program product may further include computer program code means for configuring the processor to modify a display of the electronic device upon receiving a privacy threat alert from the privacy controller to preserve privacy of the displayed images and / or data.

[0031] According to a particular arrangement of a fourth aspect, there is provided a computer program product having a computer readable storage medium having recorded thereon a computer program for providing privacy measures to a computing device, the computing device comprising: At least one processor; Memory, at least one image sensor; A display; The computer program product is computer program code means for displaying images and / or data that may be subject to privacy concerns; computer program code means for receiving image data from at least one image sensor, the image data including image data of a person including a user operating the computing device and image data of an environment surrounding the user within a field of view of the image sensor; computer program code means for segmenting the image data to isolate facial features of a user and generating a user face vector representative of characteristics of the segmented facial image data; computer program code means for comparing the user's facial vector to facial vector data stored in a memory and associated with one or more authorized device users; computer program code means for providing a privacy controller module configured to determine a privacy threat alert based on a comparison of face vectors; computer program code means for configuring a processor to modify a display of the electronic device to preserve privacy of displayed images and / or data upon receiving a privacy threat alert from the privacy controller; Equipped with.

[0032] The computer program product may further include computer program code means for further segmenting the image data using the processor to form further segmented image data to identify either a further person or object within the field of view of the image sensor.

[0033] The object may be an image and / or video recording device.

[0034] One embodiment provides a computer program product for performing the methods described herein.

[0035] One embodiment provides a non-transferable carrier medium for carrying computer executable code that, when executed on a processor, causes the processor to perform a method described herein.

[0036] One embodiment provides a system configured to perform the methods described herein. [Brief description of the drawings]

[0037] Notwithstanding other forms which may fall within the scope of the invention, preferred / preferred embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: [Figure 1] 1 illustrates a computing device on which various embodiments described herein may be implemented in accordance with embodiments of the present invention. [Diagram 2] 1 illustrates an embodiment of the invention described herein in use. [Diagram 3] FIG. 2 illustrates generally the operation of a privacy controller of a computing device as disclosed herein. [Figure 4] FIG. 1 is a block diagram illustrating an exemplary computer system upon which an embodiment may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] In the drawings, like structures are referenced with like numerals throughout the several views. The drawings illustrated are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.

[0039] [Definition] The following definitions are provided as general definitions and should in no way limit the scope of the present invention solely to these terms, but are presented for a better understanding of the following description.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs. Furthermore, it will be understood that the terms used herein should be interpreted as having a meaning consistent with the meaning in the context of the present specification and related art, and should not be interpreted in an idealized or overly formal sense unless expressly so defined herein. For the purposes of the present invention, additional terms are defined below. Furthermore, all definitions defined and used herein should be understood to take precedence over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms, unless there is a question about the meaning of a particular term, in which case the general dictionary definition and / or common usage of the term shall take precedence.

[0041] For purposes of the present invention, the following terms are defined below.

[0042] The articles "a" and "an" are used herein to refer to one or to more than one (i.e., to at least one) of the grammatical object of the article. By way of example, "an element" refers to one element or to more than one element.

[0043] In this specification, the term "about" is used to refer to an amount that varies by at most 30%, preferably at most 20%, and more preferably at most 10% relative to a reference amount. The use of the word "about" to modify a numerical value merely expressly indicates that the numerical value should not be interpreted as an exact value.

[0044] Throughout this specification, unless the context requires otherwise, the word "comprising" is understood to mean the inclusion of a stated step or element or group of steps or elements but not to the exclusion of other steps or elements or group of steps or elements.

[0045] As used herein, any of the terms "comprises" or "includes" or "comprising" are also open terms meaning to include at least the elements / features that follow the term, but not to exclude others. Thus, "comprises" is synonymous with and means "comprises."

[0046] In the claims, as well as in the summary above and the description below, all transitional phrases such as "comprising," "including," "having," "including," and the like, are understood to be open-ended, i.e., to mean "including, but not limited to." The transitional phrases "consisting of" and "consisting essentially of" alone shall be closed or semi-closed transitional phrases, respectively.

[0047] The term "real-time" means displaying data without any intentional delay, taking into account the processing limitations of the system and the time required to accurately measure the data, e.g., "displaying real-time data." Similarly, a process occurring "in real-time" means that the process operates without any intentional delay, or occurs contemporaneously (or nearly contemporaneously) with some action occurring.

[0048] The term "near real-time" refers to obtaining data without any intentional delay ("real-time") or as close to real-time as practically possible (i.e., with slight but minimal delay, whether intentional or not, within the constraints and processing limitations of the systems for obtaining, recording or transmitting the data), e.g., "obtaining data in real time or near real time."

[0049] Although any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention, the preferred methods and materials are described. It will be understood that the methods, devices, and systems described herein can be implemented in a variety of ways and for a variety of purposes. The description herein is illustrative only.

[0050] As used herein, the term "exemplary" is used in the sense of providing an example as opposed to indicating a quality. That is, an "exemplary embodiment" is an embodiment provided as an example, as opposed to necessarily being an embodiment of exemplary quality, e.g., serving as a desirable model or representing the best of its kind.

[0051] The various methods or processes outlined herein may be coded as software executable on one or more processors employing any one of a variety of operating systems or platforms. Moreover, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine code or intermediate code that runs on a framework or virtual machine.

[0052] In this regard, the various inventive concepts may be encoded as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., a computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memories, circuitry in field programmable gate arrays or other semiconductor devices, or other non-transitory or tangible computer storage media) and may include one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the invention described above. A computer-readable storage medium or media is transportable such that the program or programs stored thereon can be loaded onto one or more different computers or other processors to implement the various aspects of the invention described above.

[0053] The terms "program" or "software" are used herein in a generic sense to refer to any type of computer code or set of computer-executable instructions that can be employed to program a computer or other processor to perform various aspects of the embodiments as described above. Furthermore, according to one aspect, it should be understood that one or more computer programs that, when executed, perform the methods of the present invention need not reside on a single computer or processor, but may be distributed in a modular manner among a number of different computers or processors to perform various aspects of the present invention.

[0054] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0055] Also, the data structures may be stored in the computer-readable storage medium in any suitable form. For ease of illustration, the data structures may be shown as having fields that are related via location within the data structure. Such relationships may be achieved by allocating storage for the fields with locations within the computer-readable storage medium that similarly convey the relationship between the fields. However, any suitable mechanism may be used to establish relationships between information within fields of the data structures, including the use of pointers, tags, or other mechanisms that establish relationships between data elements.

[0056] Also, various inventive concepts may be embodied as one or more methods, examples of which are provided. The acts performed as part of a method may be ordered in any suitable manner. Thus, while acts are shown as sequential in an exemplary embodiment, embodiments may be constructed in which acts are performed in an order different from that shown, which may include performing some acts simultaneously.

[0057] The term "and / or" as used herein and in the claims should be understood to mean "either or both" of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with "and / or" should be interpreted similarly, i.e., to mean "one or more" of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the "and / or" clause, whether related to the specifically identified elements or not. Thus, as a non-limiting example, a reference to "A and / or B," when used in conjunction with open-ended language such as "comprising," may refer in one embodiment to only A (optionally including elements other than B), in another embodiment to only B (optionally including elements other than A), in yet another embodiment to both A and B (optionally including other elements), etc.

[0058] As used herein and in the claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" is to be interpreted as being inclusive, i.e., including at least one, including more than one, and optionally including additional items not in the list. Only terms clearly indicating the contrary, such as "only any one of," "exactly any one of," or, when used in the claims, "consisting of," refer to the inclusion of exactly one element of a number or list of elements. In general, the term "or" as used herein shall be interpreted as indicating exclusive alternatives (i.e., "either one or the other, but not both") only when preceded by a term of exclusivity, such as "any," "any one of," "only any one of," or "exactly any one of." When used in the claims, "consisting essentially of" shall have its ordinary meaning as used in the field of patent law.

[0059] As used in this specification and claims, the phrase "at least one," in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more elements in the list of elements, but not necessarily including at least one of each and every element specifically listed in the list of elements, and not excluding any combination of elements in the list of elements. This definition also allows for elements other than those specifically identified in the list of elements to which the phrase "at least one" refers, may optionally be present, whether or not related to the specifically identified elements. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B" or, equivalently, "at least one of A and / or B") can refer in one embodiment to at least one, optionally more than one, A, with no B (and optionally including elements other than B); in another embodiment to at least one, optionally including one or more B, with no A (and optionally including elements other than A); in yet another embodiment to at least one, optionally including one or more A, and at least one, optionally including one or more B (and optionally including other elements); etc.

[0060] In this specification, when method steps are described in a sequence, the sequence does not necessarily imply chronological execution of the steps in that sequence, unless there is another logical way to interpret the sequence.

[0061] Furthermore, where features or aspects of the invention are described in terms of a Markush group, those skilled in the art will recognize that the invention is thereby also described in terms of any individual element or subgroup of elements of the Markush group.

[0062] [Detailed Description] (Privacy Controller) Disclosed herein are systems and methods for maintaining a user's privacy while using a personal computing device, particularly a mobile or portable computing device, including a smartphone, tablet, or laptop.

[0063] Portable computing devices ubiquitously provide multiple camera devices, including at least one forward-facing camera device, such that the user of the device is within the field of view of the forward-facing camera while the phone is in use.

[0064] Facial recognition systems are a technology capable of matching human faces from digital images or video frames against a database of faces and are typically employed to authenticate users through identity verification services, working by pinpointing facial features from a given image.

[0065] Since its inception, facial recognition systems have been widely used in smartphones and other forms of technology, such as robotics, in recent times. Facial recognition systems are classified as biometrics, since computerized facial recognition involves measuring human physiological characteristics. As a biometric technology, facial recognition systems are less accurate than iris and fingerprint recognition, but they are widely adopted due to their contactless process and ease of integration with forward-facing cameras found in portable computing devices such as smartphones and tablets. Facial recognition systems have been deployed in advanced human-computer interaction, video surveillance, and automated image indexing.

[0066] Facial recognition systems are employed today by governments and private companies around the world. Their effectiveness has varied, and some systems have previously been phased out due to ineffectiveness. The use of facial recognition systems has also generated controversy, with claims that they violate citizens' privacy, commonly result in false identification, encourage gender norms and racial profiling, and do not protect sensitive biometric data. However, their accuracy and usefulness are improving, and facial recognition can often be utilized as a means to identify the true user or owner of a portable computing device in order to unlock the device for use by an authorized user.

[0067] Although humans can effortlessly recognize faces, facial recognition is a challenging pattern recognition problem in computing. A facial recognition system attempts to identify human faces based on their two-dimensional images, which vary in appearance in three dimensions with lighting and facial expressions. To accomplish this computational task, a facial recognition system typically performs four steps. First, face detection is used to segment the face from the image background. The segmented face image is then aligned, taking into account the face's pose, image size, and photographic characteristics such as lighting and grayscale. The goal of the alignment process is to allow accurate localization of facial features in the third step, facial feature extraction. Features such as eyes, nose, and mouth are pinpointed and measured in the image to represent the face, and a feature map or vector (user face vector) is generated that incorporates the measured facial features. The user face vector is then matched against a database of faces in the fourth step. In the example of user authentication, the user face vector is matched against a database of authorized users stored in the memory of the computing device. If the measured user face vector matches the stored face vector, the user access to the computing device is granted.

[0068] FIG. 1 illustrates a schematic of an electronic device 100 according to aspects of the invention described herein. In one embodiment, the electronic device 100 is a mobile computing device, such as, for example, a smartphone, tablet or laptop computing device, and includes a processor 110 in communication with a memory 120. The processor 110 may be a central processing unit and / or a graphics processing unit. The memory 120 is a combination of flash memory and random access memory. The memory 120 stores a privacy controller 130 for performing operations of the invention. The privacy controller 130 may include executable instructions for accessing a server (not shown) that coordinates the operations disclosed herein. Alternatively, the privacy controller 130 includes executable instructions that coordinate some of the operations disclosed herein, and the server performs other operations. The device 100 may further include an optional gaze direction controller 140.

[0069] The processor 110 is also connected to an image sensor 101, which may be what is known as a digital image sensor, such as a charge-coupled device. The image sensor captures visual media that is recorded by the processor 110 and presented on the display 103. Images captured by the digital image sensor 101 may also be stored in the memory 120 of the device 100.

[0070] The touch controller 105 is connected to the display 103 and the processor 110. The touch controller 105 is responsive to haptic signals applied to the display 103.

[0071] In one embodiment, the privacy controller 130 monitors signals from the image sensor 101. If suspicious activity is observed by the image sensor 101, the privacy controller turns off the display 103 or causes a message to appear on the screen that obscures the image currently being displayed on the display 103.

[0072] The electronic device 100 may further include other components typically associated with a smartphone, such as a radio signal processor 107 for providing connectivity to a wireless network. A power control circuit 109 and a global positioning system processor 111 may also be utilized. While many of the components of FIG. 1 are known in the art, new functionality is provided by the privacy controller 130, optionally operating in conjunction with a server (not shown) configured to assume one or more functions of the privacy controller 130, as discussed herein.

[0073] In an embodiment of the disclosed system and method, the privacy controller 130 of the computing device 100 continuously monitors the user environment of the portable computing device while it is in use by the user. In a preferred embodiment of the portable computing device, the privacy controller runs at a system layer that is not immediately visible to the user of the device, nor to unauthorized users attempting to access the device without authorization.

[0074] Preferably, the privacy module is continuously used by the device processor 110 as a background service to ensure that only authorized users of the portable computing device 100 are allowed to operate the device or view information displayed on the display device 103.

[0075] (User authentication) In a first function of the privacy controller embedded within a software application installed on the device 100 described herein, when the device 100 is powered on or woken up from a low power state and the privacy protection software module is in an active or running state, e.g., as a background process running on the device 100, the central processor 110 of the device 100 activates the device's forward-facing camera module 101, records a two-dimensional image of a user attempting to access the device 100, and generates a user face vector based on the user's facial features detected in the recorded image. The user face vector is passed to the privacy controller 130, which authenticates the user face vector by matching it with predefined face vectors of authorized users of the device 10 stored in the device memory 120. If the generated user face vector is authenticated with sufficient confidence against the stored authorized face vector, the privacy controller passes a flag to the processor 110 that the user is authorized to use the device, which allows the processor to activate the device display 103.

[0076] After the initial user authentication, a particular application of the privacy controller may continue to monitor the real-time image data from the forward-facing camera module to keep track of the authorized user's face remaining within the image data. In a particular arrangement, the privacy controller 130 monitors the image data to ensure that the authenticated face remains within the field of view of the camera module 101 once authenticated, rather than periodically repeating the user authentication procedure. The privacy controller 130 may be configured to simply monitor for the presence or absence of a face (assumed to be that of the authenticated user) after the initial authentication, or to track the face of a previously authenticated user within the field of view of the forward-facing camera module to monitor the uninterrupted presence of the authenticated user's face within the field of view of the camera module. If the privacy controller 130 detects that the tracked face in the image data has moved out of the field of view of the camera module, the privacy controller 130 may revoke the previously granted authentication. Thus, when a face is detected again within the field of view of the camera module 101 (which may of course be the face of a previously authenticated user), the privacy controller 130 is triggered to authenticate the credentials of the user now within the field of view of the camera module. Thus, the above process is repeated to authenticate the user currently within the field of view of the camera module: a new user face vector is generated for the user currently within the field of view, and the newly generated user face vector is compared to pre-defined face vectors of authorized users of device 10 stored in device memory 120. Provided that the user face vector of the newly identified user / face within the field of view of the camera module matches the stored face vector data of the authenticated user, access is granted to view the screen of the device and the display device is not changed.However, if the newly detected user's facial vector does not match the facial vector data of an authenticated user stored in the device memory, a threat signal is raised, continued access to view the display device is revoked, and the privacy controller 130 operates to modify the display 103 to prevent unauthorized viewing of content displayed on the device display 103.

[0077] In a further arrangement, the privacy controller 130 may be executed automatically whenever an authorized user of the device 10 opens a particular application, particularly an application that is suspected to contain potentially sensitive information. For example, the privacy controller 130 may be executed automatically by the device 100 when an authorized user opens a photo gallery software application or a messaging application, e.g., an email application such as Gmail, Microsoft™ Outlook, a short message service application (SMS), or a social networking application such as Facebook™, Twitter™, WhatsApp™, etc. When the user opens a software application on the portable computing device 100, the user may be presented with a prompt as to whether the privacy controller 130 should also be executed to protect against possible privacy threats when using the software application, or indeed the user may be given the opportunity to disable the privacy controller 130 if the privacy controller 130 is executed automatically upon opening the selected software application.

[0078] In certain embodiments, depending on the particular hardware modules available on the portable computing device, the privacy controller 130 may utilize three-dimensional images obtained from the image sensor 101 and an optional depth-sensing module. Additionally, the privacy controller may require the user to be located within a particular distance from a known object, such as, for example, a wall, to minimize the chance of a potential unauthorized person or image recording device being within the field of view of the image sensor 101. In certain embodiments, utilizing a depth-sensing image system and / or three-dimensional images, the user may be required to be located within a predefined distance from the image sensor 101 and a known object (e.g., a wall) behind the user.

[0079] If the generated user's face vector does not match the stored face vector of a predefined authorized user, the privacy controller 130 sends a failure flag to the processor 110 that the detected user is not authorized to access the device. In this case, the processor 110 may display a message on the device display 130 to inform the detected user that he is not authorized to access the device, or alternatively, the privacy controller 130 may instruct the processor 110 to turn off the display 103 so that no images or information are displayed to the user. In this case, when the display screen 103 is blacked out or switched "off", the display may be configured to display an image on a portion of the display screen configured to receive touch input via the touch controller 105, which may prompt the user to select an alternative authentication method, if applicable, upon the occurrence of, for example, a false negative recognition process.

[0080] In the event of a positive authentication in which the user has been properly authenticated as an authorized user of the device 100, the privacy controller 130 may send a success flag to the processor 110, which then provides an input to turn the display 130 into normal operation, i.e., "on," to display images and information to the authenticated user.

[0081] (Display screen privacy protection) While an authenticated user is operating the device 100, the privacy controller 130 is preferably active in the background and continuously monitors image data from the forward-facing image sensor device 101 to monitor for potential violations of the privacy of the user operating the device 100.

[0082] While active, the privacy controller 130 provides at least two tiers of privacy protection that operate in real-time or near real-time. In the first tier of protection, the privacy controller 130 operates as follows.

[0083] First, facial recognition and object detection Al modules run in real time, processing images captured through the phone's front camera to verify that the user is an authorized person and that there are no devices with the camera pointed at the screen (iOS and Android®).

[0084] Second, the privacy controller 130 provides protection against screenshots or screen recordings of the display screen 130 to prevent users from intentionally capturing on-screen content, as described below.

[0085] The privacy controller 130 may further include a third background real-time process that determines whether the user is located in a location that essentially minimizes the risk of an observer looking over the shoulder at the display 103, for example, if the user is sitting with their back to a wall. If the image sensor 101 includes the capability of depth detection of elements within the image sensor field of view 102, the privacy controller may be configured to determine whether the user is sitting within a predefined distance from a wall that is considered a safe area, i.e., an area where it would be difficult for an unauthorized person to stand either directly behind the user or within an angular range that allows viewing of the display 103.

[0086] The privacy controller 130 is configured to monitor image data 104 from the image sensor 101 in real time at least when the device 100 is being used for a particular application where privacy of information displayed on the display screen 102 is likely to be important to the authenticated user 150. Examples of applications for which the privacy controller may be configured to run by default include an image or video viewing application (e.g., photo gallery, etc.), a messaging application (e.g., email, social media, etc.), or a document viewer application (e.g., PDF viewer). The privacy controller may be configured by the user to run for a user-specific application when in use with the device 100.

[0087] The privacy controller is configured to detect all devices and facial recognition in real-time or near real-time within the field of view 102 of the forward-facing image sensor 101 when an authenticated user 150 is using or viewing a particular application on the portable computing device 100, including a messaging or chat application, or an image or video viewing application such as a gallery. When a user first opens such an application while the privacy controller is running on the device 100 as a background application service, the privacy controller verifies the user's identity, as described above. Upon successful user authentication, the privacy controller 130 remains active as a background application service, continuously monitoring the image data recorded by the image sensor 101 in real-time or near real-time, and when a pixel change in the image data is detected, the privacy controller checks the image data and returns a privacy flag to the processor 110 if a privacy threat is detected, as described below.

[0088] For example, as shown in Figure 2, device 100 is operated by authenticated user 150. The privacy controller 130 of the example device 100 allows the authenticated user 150 to use the device 100, so the display 103 is set to an active state to allow the user 150 to operate the device 100 normally. The forward-facing image sensor 101 has a field of view defined by dotted line 102. The authenticated user 150 is positioned with their face within the sensor field of view 102 to allow the privacy controller 130 to periodically generate a user face vector for the face of user 150, if desired, for ongoing verification of user authentication.

[0089] The privacy controller 130 also continuously monitors images received from within the sensor field of view 102 to detect potential privacy concerns. For example, a person 160 may enter the field of view 102 of the sensor 101. In a first embodiment of the privacy control method of the device of the present disclosure, the privacy controller 130 may consider the appearance of an additional person within the field of view 102 as a potential privacy threat. In a further embodiment, the privacy controller 130 may further include or be in communication with the gaze direction controller 140 of the device 100. The gaze direction controller 140 may be activated by the privacy controller 130 upon detection of an additional person within the sensor field of view 102. The gaze direction controller 140 may segment the image of the person 160 and analyze the segmented image to detect whether the face of the user 160 is observed (in which case the person 160 is generally looking in the direction of the device 100). In this case, the gaze direction controller 140 may alert the privacy controller 130 of the potential privacy threat. In further embodiments, the gaze direction controller 140 may further analyze the face of the user 160 to determine the actual gaze direction 161 of the user 160, in this case determining whether the user 160 is looking directly at the display 103 of the device 100. If the gaze direction controller 140 determines that the user 160 is looking "over the shoulder" of the authenticated user 150 at the display 130, the gaze direction controller 140 may alert the privacy controller 130 of a potential privacy threat.In a graduated privacy threat alert system, the degree of threat to the user's 150's privacy violated by an unauthorized person 160 viewing the display 130 may be graded between a low threat in the case of a person detected within the field of view 130 but with their back turned to the device 100 (the face of the person 160 is not within the field of view), a medium threat in the case of a person 160 generally facing the device 100 (perhaps indicating that they are viewing the device display 103 in their peripheral vision or only casually glancing at it), or a high threat in the case of a person looking directly at the display 103 of the device 100 as determined by the gaze direction controller 140.

[0090] The privacy controller 130 may further include an artificial intelligence (AI) module that can learn the image signatures of image and / or video recording devices in certain arrangements, where the AI ​​module is utilized, for example, to supplement an existing database of such recording devices, such that the privacy controller is not limited to only recognizing image signatures of known image recording devices from a known collection of such devices stored in an image database accessible to the privacy controller 130. The use of a database alone is limited in that it can only recognize image recording devices that were known on the particular date when the database was compiled or last updated. Rather, the addition of an artificial intelligence module connected to the privacy controller 130 can enable the privacy controller 130 to adapt to new image and / or video recording devices, thereby mitigating the need for periodic database updates that may burden users of the device 100 to continually download and install such updates. The need for periodic database updates is further not ideal, as it provides no protection to users against new image recording devices that are exposed after the last database update, and the users are unaware that such new devices exist, thus creating an unacceptable deficiency in the efficacy of the privacy controller 130 to protect users' privacy from new recording devices.

[0091] In a particular arrangement for implementation of the privacy controller 130 on an Apple™ brand portable computing device 100, such as an iPhone, iPad, or laptop computer, the user authentication procedure may utilize the TrueDepth three-dimensional camera system available on these devices to perform facial identification of the user and, for the advanced screen privacy features of the present disclosure, to determine the distance at which the user is located from background elements, such as walls, identified in the field of view 102 of the camera device 101. In Android™ brand portable computing devices, such as mobile phones and tablet terminals, a depth-sensitive camera system may not be available, so the facial recognition procedure is implemented using two-dimensional image data from the camera 101. However, modern portable computing devices are equipped with multiple camera systems, such as wide fields of view and zoom capabilities. Thus, the systems and methods of the present disclosure utilize image data from multiple image sensors 101, which may be used to infer depth information in the combined image data from the multiple image sensors 101.

[0092] In further embodiments, the privacy controller 130 may continuously monitor images received from the sensors 101 in the sensor field of view 102 to detect whether there are further privacy concerns, for example, by detecting the appearance of a camera or image recording device 170, such as a smartphone or tablet device, in the field of view 102 that may be used by an unauthorized person 160 to record images of the display 130 of the device 100. If an image recording device 170 is detected in the field of view 102, the privacy controller 130 may immediately flag the image recording device 170 as a high level privacy threat.

[0093] If a medium or high level privacy threat is detected by the privacy controller 130, a privacy alert flag is passed to the processor 101. Upon receiving a privacy threat alert from the privacy controller 130, the processor 101 may be configured to take action to protect the privacy of the information or image displayed on the display 103. For example, the processor 110 may display a message on the device display 130 to notify the authorized user of a possible privacy violation, or alternatively, the privacy controller 130 may instruct the processor 110 to turn off the display 103. Preferably, when a privacy alert message 180 is displayed on the display 130 of the device 100 for the user 150, the message may identify whether a person 160 or an image recording device 170 has been detected within the field of view 102 of the image sensor 101 and / or identify a potential privacy violation grade (i.e., low-medium threat alert or high threat alert) that prompts the user 150 to check their surroundings for such potential privacy threat violation. Preferably, when processor 104 displays privacy threat alert message 180 on display 103, the displayed message may include a predefined area 181 of the displayed image that accepts touch input via touch controller 105 by authenticated user 150 to confirm a detected potential threat or to override the message if no actual threat is apparent. Preferably, the displayed threat alert message 180 covers most or all of the display area of ​​display 103, thereby at least partially, or preferably mostly, obscuring the image or information displayed on display 103, such that at least provisional privacy protection is provided by message 180.

[0094] In an alternative embodiment, actions performed by the processor 101 upon receipt of a potential privacy threat alert from the privacy controller 103 resulting from the continuous real-time monitoring of the field of view 102 of the image sensor 101 may include placing the display 103 in a blacked-out or "off" state such that images or data displayed on the display 103 prior to receipt of the threat alert are no longer displayed and thus are not visible to either the authenticated user 150 or additional persons 160 or image recording device 170 detected within the field of view 102. As discussed above, if the screen is blacked out due to a potential privacy threat, the processor 101 may be configured to display an image on a portion of the display screen 103 configured to receive touch input via the touch controller 105, whereby the authenticated user 150 is prompted to confirm the privacy threat or indicate that such a threat is not apparent, and the display 103 is returned to an "on" state such that the user 150 can continue to view the images and / or data content on the display 103 normally. If the user 150 determines that a potential threat is apparent, the display 103 is preferably blacked out or switched to an "off" state. The display 103 may be turned off for a predetermined period of time to allow the user 150 to move to a safer location before the display 103 is turned on again. In some embodiments, the processor display 103 may be configured to display a countdown timer indicating a predetermined period of time until the display 103 is turned on again to allow the user 150 time to move to a safer location or otherwise secure his or her position for the privacy of the display 103.Alternatively, the processor 101 may be configured to display an image or message on the display 103 adapted to receive touch input from the user 150 via the touch controller 105 to confirm that a privacy threat is no longer evident and to switch the display to an "on" state so that the authenticated user 150 can again proceed with normal operations.

[0095] In certain arrangements of the systems and methods of the present disclosure, the AI ​​functionality utilized by the privacy controller as described above may be provided by existing AI infrastructure, such as TensorFlow™, or Google ML Kit, available as software-as-a-service applications for providing AI and machine learning capabilities to third-party software applications. In certain arrangements, a first AI function, such as face recognition, is provided by a first machine learning service (e.g., Google ML Kit) and a second AI function is provided by a second machine learning service (e.g., TensorFlow). In this manner, multiple AI services can be provided simultaneously, with both subroutines operating in real time, without placing a significant burden on a single machine learning service. Multiple AI services can be provided simultaneously, without placing a significant burden on a single machine learning resource that may adversely affect the real-time or near real-time operation of the privacy controller 130.

[0096] 3 provides a schematic diagram of the operation of the privacy controller 130. Image data 104 representative of the field of view 102 of the image sensor 101 is received by the privacy controller 130 and delivered to two separate subroutines. The first subroutine comprises a trained AI model 310 configured for detection of image recording devices in the field of view image data 104.

[0097] The image data 104 is received by a first machine learning controller 311 and compared to an AI model 310 to determine whether an image recording device (e.g., a camera) 313 is detected in the field of view 102 of the image data. If an image recording device is detected, the display 103 of the device 100 is configured to disable 330 the display screen 103, which may include hiding or masking images and / or data displayed on the display 103. This may include turning off the display 103 as described above, or optionally displaying a message on the display 103 to indicate to the user 150 that a potential privacy threat has been detected in the form of an image recording device.

[0098] The image data 104 is further forwarded to a further subroutine for providing user authentication functionality to the device 100 to ensure that only authorized users 150 are allowed to view the display 103. The further subroutine includes a trained AI model 320 constructed with face identification vector data corresponding to one or more authorized users 150 who are authorized to use the portable computing device 100. In a first embodiment, the image data 104 is provided to an image detection routine 321 where it is analyzed and manipulated (e.g., to detect and crop the image to isolate the detected face) to provide face image data including isolated faces detected within the field of view 102 of the image sensor 101. The face image date is provided to a second machine learning controller 323 and compared to the AI ​​model 310 to determine 325 whether the detected face in the image data 104 belongs to an authorized user 150 who has permission to use the device 100. If facial data of an unauthorized user of the device 100 is detected, the display 103 of the device 100 is configured to disable 330 the display screen 103, which may include hiding or masking the images and data displayed thereon, as described above. Both the image recording subroutine and the facial recognition subroutine are preferably executed continuously in real time, which allows the privacy controller 130 to identify potential privacy threats from the image recording device 170 or the unauthorized user 160, and also enables 340 the display screen 103 to allow viewing access to the display 103 if the following conditions are met in a comparison 335 between the two subroutines: The determination result 313 is that the image recording device 170 has not been detected. Authorized users 150 (and only authorized users, i.e. not unauthorized users 160 ) are detected within the field of view 102 of the image sensor 101 and can view the display 103 .

[0099] As mentioned above, the privacy controller 130 is configured to run as a background service of the portable computing device 100 to provide continuous real-time or near real-time monitoring of potential privacy threats to unauthorized observers viewing or recording images and / or data displayed on the display 103. In a particular arrangement of the privacy protection device 100, the privacy controller 130 is configured to monitor the field of view 102 of the image sensor 101, among other things, whenever an authorized user is viewing potentially sensitive content, such as image data within an image gallery application on the computing device 100 or a messaging service on the computing device 100, such as, for example, an email application, a social media application (e.g., Facebook™, Instagram™, etc.), an encrypted messaging service (e.g., Signal™ or Telegram™) or a document viewing application (e.g., PDF viewer). In further embodiments of the privacy controller system and method of the present disclosure, the privacy controller may further provide additional privacy detection for taking, storing, or distributing screenshot images of images or data displayed on the display 103 of the device 100 including the privacy controller 130.

[0100] For example, the privacy controller 130 may prevent a user with access to the device 100 from taking screenshots of the display 103 in a device-specific manner and from recording video captures of the display screen 103 of the device 100. Thus, the functionality of the privacy controller 103 may be broadly applicable, such that any user may take screenshots or video recordings of the display 103, even for a short time, regardless of whether that user is an authorized user 150 or an unauthorized person 160 who may have gained physical access to the device 100. Of course, the privacy controller 130 may also preferably include an override function, whereby the privacy controller 130 may be configured to provide a prompt to a user adapted to receive touch input via the touch controller 105 for an authorized user to override the screenshot blocking or display recording process. In a preferred arrangement, if the user attempts to override such functionality, the privacy controller 130 is configured to authenticate the user as an authorized user of the device 100, for example via a facial recognition procedure as described above.

[0101] Modifications and variations as would be apparent to one skilled in the art are deemed to be within the scope of the present invention. The present invention is not limited in scope by the specific embodiments described herein. These embodiments are for illustrative purposes only. Functionally equivalent products, articles and methods are clearly within the scope of the invention described herein. It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

[0102] Exemplary embodiments are provided so that the present disclosure will be thorough and fully convey the scope of the present disclosure to those skilled in the art. Numerous specific details are described, such as examples of specific components, devices, and methods, to provide a thorough understanding of the embodiments of the present disclosure. It will be apparent to those skilled in the art that specific details need not be employed, that the exemplary embodiments can be embodied in many different forms, and neither should be construed to limit the scope of the present disclosure. In some exemplary embodiments, well-known processes, well-known device structures, and well-known technologies are not described in detail.

[0103] Such variations are not to be considered as departures from the present disclosure, and all such modifications are intended to be included within the scope of the present disclosure.

[0104] The method steps, processes, and operations described herein should not be construed as necessarily requiring their performance in the particular order discussed or illustrated, unless specifically identified as an order of performance, and it should be understood that additional or alternative steps may be employed.

[0105] References to positional descriptions and spatially relative terms such as "inside," "outside," "lower," "below," "upper," "above," etc. should be taken in the context of the embodiments depicted in the figures and should not be taken as limiting the invention to a literal interpretation of the terms, but rather as would be understood by one of ordinary skill in the art.

[0106] In this specification, terms such as first, second, third, etc. may be used to describe various elements, components, regions, layers, and / or sections, but these elements, components, regions, layers, and / or sections should not be limited by these terms. These terms may only be used to distinguish one element, component, region, layer, or section from another region, layer, or section. As used herein, terms such as "first," "second," and other numerical terms do not imply a sequence or order unless clearly indicated by the context. Thus, a first element, component, region, layer, or section described below may be referred to as a second element, component, region, layer, or section without departing from the teachings of the exemplary embodiments.

[0107] When an element is referred to as "resting," "engaged," "connected," or "coupled" to another element / layer, it will be understood that it may be directly resting, engaged, connected, or coupled to the other element / layer, or there may be intervening elements / layers. Other words used to describe the relationship between elements / layers should be interpreted similarly (e.g., "between," "adjacent"). As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0108] The terms used herein are for the purpose of describing particular exemplary embodiments only and are not intended to be limiting. As used herein, the singular forms "a," "an," and "the" may be intended to include the plural forms unless the context clearly indicates otherwise. The terms "comprise," "comprises," "comprising," "including," "having," or variations thereof, are inclusive and thus specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0109] (Example - Hardware Overview) According to one embodiment, the techniques described herein are implemented by at least one computing device. The techniques may be implemented in whole or in part using a combination of at least one server computer and / or other computing devices connected using a network, such as a packet data network. The computing device may be mechanically wired to perform the techniques, or may include at least one digital electronic device, such as an application specific integrated circuit (ASIC) or field programmable gate array (FPGA), persistently programmed to perform the techniques, or may include at least one general-purpose hardware processor programmed to perform the techniques according to program instructions in firmware, memory, other storage, or a combination thereof. Such computing devices may also combine custom programming and custom hardwired logic, ASICs, or FPGAs to accomplish the techniques described. The computing device may be a server computer, a workstation, a personal computer, a portable computer system, a handheld device, a mobile computing device, a wearable, body-worn or embedded device, a smartphone, a smart appliance, an internetworking device, an autonomous or semi-autonomous device such as a robot or an unmanned ground or aerial vehicle, any other electronic device incorporating hardwired and / or program logic to implement the described techniques, one or more virtual computing machines or instances in a data center, and / or a network of server computers and / or personal computers.

[0110] Figure 4 is a block diagram illustrating an example of a computer system on which an embodiment may be implemented. In the example of Figure 4, a computer system 400 and instructions for implementing the disclosed techniques in hardware, software, or a combination of hardware and software are represented generally as boxes and circles at a level of detail similar to that commonly used by those of ordinary skill in the art to which this disclosure pertains to communicate computer architecture and computer system implementations.

[0111] Computer system 400 includes an input / output (I / O) subsystem 402, which may include a bus and / or other communication mechanisms for communicating information and / or instructions between components of computer system 400 via electronic signal paths. I / O subsystem 402 may include an I / O controller, a memory controller, and at least one I / O port. The electronic signal paths are represented diagrammatically in the drawings, for example as lines, one-way arrows, or two-way arrows.

[0112] At least one hardware processor 404 is coupled to the I / O subsystem 402 for processing information and instructions. The hardware processor 404 may include, for example, a general-purpose microprocessor or microcontroller and / or an embedded system or special purpose microprocessor such as a graphics processing unit (GPU) or digital signal processor or ARM processor. The processor 404 may be comprised of an integrated arithmetic logic unit (ALU) or may be coupled to a separate ALU.

[0113] Computer system 400 includes one or more units of memory 406, such as main memory, coupled to I / O subsystem 402 for electronically digitally storing data and instructions executed by processor 404. Memory 406 may include volatile memory, such as various forms of random access memory (RAM) or other dynamic storage devices. Memory 406 may also be used to store temporary variables or other intermediate information during execution of instructions executed by processor 404. Such instructions, when stored on a non-transitory computer-readable storage medium accessible to processor 404, may make computer system 400 into a special-purpose machine customized to perform the operations specified in the instructions.

[0114] The computer system 400 further includes a non-volatile memory, such as a read only memory (ROM) 408 or other static storage device coupled to the I / O subsystem 402 for storing information and instructions for the processor 404. The ROM 408 may include various forms of programmable ROM (PROM), such as erasable programmable read only memory (EPROM) or electrically erasable programmable read only memory (EEPROM). A unit of persistent storage 410 may include various forms of non-volatile RAM (NVRAM), such as FLASH memory, or solid-state storage, such as CD-ROM or DVD-ROM, magnetic disks or optical disks, and may be coupled to the I / O subsystem 402 for storing information and instructions. The storage 410 is an example of a non-transitory computer-readable storage medium that, when executed by the processor 404, may be used to store instructions and data that cause computer-implemented methods to perform the techniques of the present disclosure.

[0115] The instructions in memory 406, ROM 408, or storage 410 may be comprised of one or more sets of instructions organized as modules, methods, objects, functions, routines, or calls. The instructions may be configured as one or more computer programs, operating system services, or application programs, including mobile apps. The instructions may include operating systems and / or system software, one or more libraries supporting multimedia, programming, or other functionality, data protocol instructions or stacks for implementing TCP / IP, HTTP, or other communications protocols, file formatting instructions for parsing or rendering files coded using HTML, XML, JPEG, MPEG, or PNG, user interface instructions for rendering or interpreting commands in a graphical user interface (GUI), command line interface or text user interface, application software such as an office suite, internet access applications, design and manufacturing applications, graphics applications, audio applications, software engineering applications, educational applications, games, or miscellaneous applications. The instructions may implement a web server, a web application server, or a web client. The instructions may be configured as a presentation layer, an application layer, and a data storage layer, such as a relational database system using Structured Query Language (SQL) or a relational database system without SQL, an object store, a graph database, a flat file system, or other data storage.

[0116] The computer system 400 may be coupled to at least one output device 412 via the I / O subsystem 402. In one embodiment, the output device 412 is a digital computer display. Examples of displays that may be used in various embodiments include a touch screen display or a light emitting diode (LED) display or a liquid crystal display (LCD) or an electronic paper display. The computer system 400 may include other types of output devices 412 instead of or in addition to a display device. Examples of other output devices 412 include a printer, a ticket printer, a plotter, a projector, a sound or video card, a speaker, a buzzer or piezoelectric device or other audible device, a lamp or LED or LCD indicator, a tactile device, an actuator or servo.

[0117] At least one input device 414 is coupled to the I / O subsystem 402 for communicating signals, data, command selections or gestures to the processor 404. Examples of input device 414 include touch screens, microphones, still and video digital cameras, alphanumeric and other keys, keypads, keyboards, graphics tablets, image scanners, joysticks, clocks, switches, buttons, dials, slides, and / or various sensors such as force sensors, motion sensors, thermal sensors, accelerometers, gyroscopes, inertial measurement unit (IMU) sensors, and / or various transceivers such as wireless, radio frequency (RF) or infrared (IR) transceivers such as cellular or Wi-Fi, global positioning system (GPS) transceivers, etc.

[0118] Another type of input device is a control device 416, which can perform cursor control or other automatic control functions such as navigation in a graphical interface on a display screen instead of or in addition to input functions. The control device 416 may be a touchpad, mouse, trackball, or cursor direction keys for communicating directional information and command selections to the processor 404 and controlling cursor movement on the display 412. The input device may have at least two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allow the device to specify a position in a plane. Another type of input device is a wired, wireless, or optical control device, such as a joystick, wand, console, steering wheel, pedals, gear shift mechanism, or other type of control device. The input device 414 may include a combination of multiple different input devices, such as a video camera or a depth sensor.

[0119] In another embodiment, computer system 400 may constitute an Internet of Things (IoT) device in which one or more of output device(s) 412, input device(s) 414, and control device(s) 416 are omitted. Or, in such an embodiment, input device(s) 414 may consist of one or more cameras, motion detectors, thermometers, microphones, earthquake detectors, other sensors or detectors, measurement devices or encoders, and output device(s) 412 may consist of special purpose displays, such as single row LED or LCD displays, one or more indicators, display panels, meters, valves, solenoids, actuators or servos.

[0120] If computer system 400 is a mobile computing device, input device 414 may include a Global Positioning System (GPS) receiver coupled to a GPS module capable of triangulating to multiple GPS satellites to determine and generate geographic location or location data, such as latitude and longitude values ​​for the physical location of computer system 400. Output device 412 may include hardware, software, firmware, and interfaces for generating, alone or in combination with other application-specific data, location reporting packets, notifications, pulse or heartbeat signals, or other periodic data transmissions identifying the location of computer system 400, to host 424 or server 430.

[0121] The computer system 400 may implement the techniques described herein using customized hardwired logic, at least one ASIC or FPGA, firmware and / or program instructions or logic that, when loaded and used on or executed in conjunction with the computer system, cause or program the computer system to operate as a special purpose machine. According to one embodiment, the techniques of the present disclosure are executed by the computer system 400 in response to the processor 404 executing at least one sequence of at least one instruction contained in the main memory 406. Such instructions may be read into the main memory 406 from another storage medium, such as the storage 410. Execution of the sequences of instructions contained in the main memory 406 causes the processor 404 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0122] The term "storage media" as used herein refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a specific manner. Such storage media may consist of non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage 410. Volatile media include dynamic memory, such as memory 406. Common forms of storage media include, for example, hard disks, solid state drives, flash drives, magnetic data storage media, any optical or physical data storage media, memory chips, etc.

[0123] Storage media is distinct from, but may be used in conjunction with, transmission media, which media involves transferring information between storage devices. For example, transmission media include coaxial cables, copper wire, and fiber optics, including the wires that comprise a bus in I / O subsystem 402. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0124] Various forms of media may be involved in carrying at least one sequence of at least one instruction to the processor 404 for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions using a modem over a communications link, such as fiber optic, coaxial cable, or a telephone line. A modem or router local to the computer system 400 may receive the data on the communications link and convert the data into a form readable by the computer system 400. For example, a receiver such as a radio frequency antenna or infrared detector may receive the data carried in a radio or optical signal and appropriate circuitry may provide the data to the I / O subsystem 402, such as placing the data on a bus. The I / O subsystem 402 conveys the data to memory 406, from which the processor 404 retrieves and executes the instructions. The instructions received by memory 406 are optionally stored in storage 410 either before or after execution by the processor 404.

[0125] Computer system 400 also includes a communication interface 418 coupled to bus 402. Communication interface 418 provides a bidirectional data communication coupling to a network link 420 that is directly or indirectly connected to at least one communication network, such as a network 422 or a public or private cloud on the Internet. For example, communication interface 418 may be an Ethernet networking interface, an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem for providing a data communication connection to a corresponding type of communication line, such as an Ethernet cable or any type of metallic cable, or an optical fiber line or a telephone line. Network 422 broadly represents a local area network (LAN), a wide area network (WAN), a campus network, an Internet network, or any combination thereof. Communication interface 418 may be configured as a LAN card providing a data communication connection to a compatible LAN, or a cellular radiotelephone interface wired to transmit and receive cellular data according to a cellular radiotelephone wireless networking standard, or a satellite radio interface wired to transmit and receive digital data according to a satellite wireless networking standard. In any such implementation, communication interface 418 sends and receives electrical, electromagnetic or optical signals over signal paths that carry digital data streams representing various types of information.

[0126] Network link 420 typically provides electrical, electromagnetic or optical data communication directly or through at least one network to other data devices using, for example, satellite, cellular, Wi-Fi or BLUETOOTH technology. For example, network link 420 may provide a connection through network 422 to a host computer 424.

[0127] Further, network link 420 may provide connectivity to other computing devices through network 422 or through internetworking devices and / or computers operated by an Internet Service Provider (ISP) 426. ISP 426 provides data communication services through a worldwide packet data communication network represented as Internet 428. A server computer 430 may be coupled to Internet 428. Server 430 broadly represents any computer, data center, virtual machine, virtual computing instance, or computer running a containerized program system such as DOCKER or KUBERNETES, with or without a hypervisor. Server 430 may represent an electronic digital service implemented using multiple computers or instances and accessed and used by sending web service requests, uniform resource locator (URL) strings with parameters in an HTTP payload, API calls, app service calls, or other service calls. Computer system 400 and server 430 may form elements of a distributed computing system including other computers, processing clusters, server farms, or other organizations of computers collaborating to perform a task or run an application or service. The server 430 may be comprised of one or more sets of instructions organized as modules, methods, objects, functions, routines, or calls. The instructions may be organized as one or more computer programs, operating system services, or application programs, including mobile apps.The instructions may include operating system and / or system software, one or more libraries for supporting multimedia, programming, or other functionality, data protocol instructions or stacks for implementing TCP / IP, HTTP, or other communications protocols, file formatting instructions for parsing or rendering files coded using HTML, XML, JPEG, MPEG, or PNG, user interface instructions for rendering or interpreting commands for a graphical user interface (GUI), a command line interface, or a text user interface, application software such as an office suite, Internet access applications, design and manufacturing applications, graphics applications, audio applications, software engineering applications, educational applications, games, or miscellaneous applications, etc. Server 430 may include a web application server that hosts a presentation layer, an application layer, and a data storage layer, such as a relational database system using Structured Query Language (SQL) or not, an object store, a graph database, a flat file system, or other data storage.

[0128] Computer system 400 can send messages and receive data and instructions, including program code, through the network(s), network link 420 and communication interface 418. In the Internet example, a server 430 might transmit a requested code for an application program through the Internet 428, ISP 426, local network 422 and communication interface 418. The received code may be executed by processor 404 as it is received, and / or stored in storage 410, or other non-volatile storage for later execution.

[0129] Execution of instructions as described in this section may implement a process consisting of program code and its current activity in the form of an instance of a running computer program. Depending on the operating system (OS), a process may consist of multiple threads of execution that execute instructions simultaneously. In this context, a computer program is a passive collection of instructions, and a process may be the actual execution of those instructions. For example, opening multiple instances of the same program often means that multiple processes are running. Multitasking may be implemented to allow multiple processes to share the processor 404. Although each processor 404 or processor core executes a single task at a time, the computer system 400 may be programmed to implement multitasking so that each processor can switch between running tasks without waiting for each task to finish. In one embodiment, the switch may be performed when a task performs an I / O operation, when a task indicates that it is available to switch, or upon a hardware interrupt. Time sharing may be implemented to enable fast response of interactive user applications by performing context switches rapidly to make it appear as if multiple processes are running simultaneously. In one embodiment, for security and reliability, the operating system may discourage direct communication between independent processes and provide a tightly mediated and controlled inter-process communication facility.

[0130] The term "cloud computing" is generally used herein to describe a computing model that enables on-demand access to a shared pool of computing resources, such as computer networks, servers, software applications, and services, enabling rapid provisioning and release of resources with minimal administrative effort or service provider interaction.

[0131] Cloud computing environments (sometimes just called cloud environments or clouds) can be implemented in a variety of ways to best suit different requirements. For example, in a public cloud environment, the underlying computing infrastructure is owned by an organization that offers its cloud services to other organizations or the general public. In contrast, a private cloud environment is generally intended for use only by or within a single organization. A community cloud is intended to be shared by several organizations within a community, while a hybrid cloud comprises two or more types of clouds (e.g. private, community, public) joined by data and application portability.

[0132] In general, the cloud computing model provides some of the responsibilities traditionally provided by an organization's information technology department as a service tier within a cloud environment for consumption by consumers (either internal or external to the organization, depending on the public / private nature of the cloud). The exact definition of the components and functions provided by each cloud service tier varies depending on the specific implementation, but some common examples include: Software as a Service (SaaS) allows consumers to consume software applications running on a cloud infrastructure, while the SaaS provider manages or controls the underlying cloud infrastructure and applications. Platform as a Service (PaaS) allows consumers to develop, deploy, and otherwise control their own applications using software programming languages ​​and development tools supported by the PaaS provider, while the PaaS provider manages or controls other aspects of the cloud environment (i.e., everything below the runtime execution environment). Consumers can deploy and run any software application and provision processing, storage, network, and other basic computing resources, while the PaaS provider manages or controls the underlying physical cloud infrastructure (everything below the operating system layer). Database as a Service (DBaaS) allows consumers to use database servers or database management systems running on the cloud infrastructure, while the DBaaS provider manages or controls the underlying cloud infrastructure, applications, and servers, including one or more database servers.

[0133] (Embodiment) Throughout this specification, reference to an "embodiment" or "arrangement" means that a particular feature, structure, or characteristic described in connection with the embodiment / arrangement is included in at least one embodiment / arrangement of the invention. Thus, the appearances of the phrase "in one embodiment / arrangement" or "in an embodiment / arrangement" in various places throughout this specification are not necessarily all referring to the same embodiment / arrangement, but rather indicate that they may. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments / arrangements.

[0134] Similarly, in the above description of exemplary embodiments / arrangements of the invention, it will be understood that various features of the invention may be grouped together in a single embodiment / arrangement, figure, or description thereof for the purpose of streamlining the disclosure and aiding in understanding one or more of the various inventive aspects. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment / arrangement. Thus, the claims following the detailed description are expressly incorporated herein, with each claim standing on its own as a separate embodiment / arrangement of the invention.

[0135] Furthermore, some embodiments / arrangements described herein may include some features included in other embodiments / arrangements but not other features, but combinations of features of different embodiments / arrangements are meant to be within the scope of the present invention and form different embodiments / arrangements, as would be understood by one of ordinary skill in the art. For example, in the following claims, any of the claimed embodiments / arrangements may be used in any combination.

[0136] (Specific details) In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the present invention may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this specification.

[0137] (term) In describing the preferred embodiment of the invention illustrated in the drawings, specific terminology will be relied upon for clarity. However, the invention is not intended to be limited to the specific terminology so selected, and it will be understood that each specific term includes all technical equivalents that operate in a similar manner to accomplish a similar technical purpose. Terms such as "front", "rear", "radial", "periphery", "upper", "lower", etc. are used as convenient terms to provide points of reference and are not to be construed as limiting terms.

[0138] (Examples of different objects) As used herein, unless otherwise specified, the use of ordinal adjectives "first," "second," "third," etc. to describe a common object is intended merely to indicate that different instances of a similar object are being referred to, and is not intended to imply that the objects so described must be in a predetermined order, whether in time, space, precedence, or in any other manner.

[0139] ("Having" and "including") In the claims which follow and in the preceding description of the invention, unless the context requires otherwise by express language or necessary implication, the word "comprises" or similar words such as "comprises" or "comprising" are used in the inclusive sense, i.e., to specify the presence of stated features but are not used to exclude the presence or addition of further features in various embodiments of the invention.

[0140] As used herein, any one of the terms "including" or "which includes" or "that includes" is also an open term meaning "including at least" an element / feature but not excluding others. Thus, "including" is synonymous with "comprises" and means "comprises."

[0141] (Scope of the invention) While what is believed to be a preferred arrangement of the invention has thus been described, those skilled in the art will recognize that other and further modifications may be made without departing from the spirit of the invention. Functions may be added or deleted from the block diagrams, operations may be interchanged between functional blocks, and steps may be added or deleted to methods described within the scope of the invention.

[0142] Although the invention has been described with reference to specific embodiments, those skilled in the art will appreciate that the invention can be embodied in many other forms. [Industrial Applicability]

[0143] As is apparent from the above, the described arrangements are applicable to the mobile device industry, and in particular to methods and systems for delivering digital media via mobile devices.

[0144] It will be appreciated that the systems and methods described / illustrated above at least substantially provide a system and method for enhancing the privacy of data displayed on a display of a portable electronic device.

[0145] The systems and methods described herein and / or illustrated in the drawings are presented for illustrative purposes only and are not intended to limit the scope of the invention. Unless otherwise indicated, individual aspects and components of the privacy controller described herein may be modified and thus substituted with known equivalents or unknown substitutes that may be developed in the future or that may prove to be acceptable substitutes in the future. Moreover, because the range of potential applications is large and the privacy controller system and method of the present invention are intended to be adaptable to many such variations, the privacy controller system and method may be modified for various applications while remaining within the scope and spirit of the claimed invention.

Claims

1. 1. A system for providing privacy measures to a computing device, comprising: Memory, a display configured to display images and / or data that may be subject to privacy concerns; At least one image sensor configured to provide image data of a person, including a user operating a computing device, and image data of an environment surrounding the user within a field of view of the image sensor; At least one processor, the at least one processor comprising: receiving the image data, segmenting the image data to isolate facial features of the user, and generating a user face vector representative of characteristics of the segmented facial image data; the at least one processor for comparing the user face vector to face vector data associated with one or more authorized device users stored in the memory; and a privacy controller module configured to determine a privacy threat alert based on the comparison of the face vectors. Upon receiving a privacy threat alert from the privacy controller, the processor is configured to modify a display of the electronic device to preserve privacy of the displayed images and / or data. system.

2. The system of claim 1 , wherein the memory is configured to store facial recognition data.

3. The system of claim 2 , wherein the facial recognition data includes facial vector data.

4. 4. The system of claim 1 , wherein the processor is configured to further segment the image data to form further segmented image data and to identify further people and / or objects within a field of view of the image sensor from the further segmented image data.

5. The system of claim 4 , wherein the object comprises an image and / or video recording device.

6. The system of claim 4 or 5, wherein the privacy controller is configured to determine a privacy threat based on the further segmented image data and, during use, when a privacy threat is identified, to cause the processor to modify the display of the electronic device so as to preserve the privacy of the data previously displayed on the display.

7. The system of claim 1 , wherein the alteration of the display comprises displaying an image or message that at least partially obscures the display of an image or data.

8. The system of claim 1 , wherein the change in display comprises blacking out the display or turning the display to an "off" state.

9. 1. A computer program product for privacy in an electronic device, the computer program product comprising a non-transitory computer readable storage medium having computer readable program code portions stored therein, the computer readable program code portions comprising: a first portion configured to receive image data from an image sensor; a second portion configured to segment the image data to isolate facial features of persons including a user and generate a user face vector representative of characteristics of the segmented facial image data; a third part configured to compare the user face vector with face vector data of known authorized users having access to the electronic device, the face vector data of such known authorized users being stored in a memory; and a fourth portion configured to modify a display of the electronic device to obscure or remove images or data displayed on a display of the electronic device; A computer program product comprising:

10. a fifth portion configured to further segment the image data to form further segmented image data to identify additional people or objects within a field of view of the image sensor; a sixth portion configured to, in use, when a privacy threat is identified, engage said fourth portion to cause a display of said electronic device to be altered to preserve the privacy of said data previously displayed on said display, and to determine a privacy threat based on said further segmented image data; 10. The computer program product of claim 9, further comprising:

11. the object comprises an image and / or video recording device; 11. A computer program product according to claim 10.

12. modifying the display includes displaying an image or message that at least partially obscures the display of an image or data. A computer program product according to any one of claims 9 to 11.

13. 13. The computer program product of claim 9, wherein the change in display comprises blacking out the display or turning the display to an "off" state.

14. 1. A method for providing privacy measures to a computing device, comprising: providing a display configured to display images and / or data that may be subject to privacy concerns; providing at least one image sensor configured to provide image data of a person, including a user operating a computing device, and image data of an environment surrounding said user within a field of view of the image sensor; providing at least one processor, the at least one processor comprising: Receive image data; segmenting the image data to isolate facial features of the user and generating a user face vector representative of characteristics of the segmented facial image data; configured to compare the user face vector with face vector data associated with one or more authorized device users stored in a memory; providing a privacy controller module configured to determine a privacy threat alert based on the comparison of the face vectors; Upon receiving a privacy threat alert from the privacy controller, the processor is configured to modify a display of the electronic device to preserve privacy of the displayed images and / or data. method.

15. 15. The method of claim 14, further comprising further segmenting the image data with a processor to form further segmented image data to identify either further people and / or objects within the field of view of the image sensor.

16. The method of claim 15, wherein the object is an image and / or video recording device.

17. 17. The method of any one of claims 14 to 16, wherein the privacy controller is configured to determine a privacy threat based on the further segmented image data and, in use, when a privacy threat is identified, cause the processor to modify the display of the electronic device so as to preserve the privacy of the data previously displayed on the display.

18. 18. A method according to any one of claims 14 to 17, wherein modifying the display comprises displaying an image or message that at least partially obscures the image or data on the display.

19. 19. The method of any one of claims 14 to 18, wherein the change in display comprises blacking out the display or turning the display to an "off" state.

20. 1. A computer program product having a computer readable storage medium having recorded thereon a computer program for providing privacy measures to a computing device, the computing device comprising: At least one processor; Memory, At least one image sensor; A display; The computer program product comprises: computer program code means for displaying images and / or data that may be subject to privacy concerns; computer program code means for receiving image data from the at least one image sensor, the image data including image data of a person including a user operating the computing device and image data of an environment surrounding the user within a field of view of the image sensor; computer program code means for segmenting the image data to isolate facial features of a user and generating a user face vector representative of characteristics of the segmented facial image data; computer program code means for comparing said user face vector to face vector data stored in said memory and associated with one or more authorized device users; computer program code means for providing a privacy controller module configured to determine a privacy threat alert based on the comparison of the facial vectors; computer program code means for configuring the processor to, upon receiving a privacy threat alert from the privacy controller, modify a display of the electronic device to preserve privacy of the displayed images and / or data; A computer program product comprising:

21. computer program code means for further segmenting the image data using the processor to form further segmented image data to identify either a further person or object within a field of view of the image sensor.

21. A computer program product according to claim 20.

22. The method of claim 15, wherein the object is an image and / or video recording device.

Images