Method and system for foveated privacy on a screen of a display device

The method employs eye-tracking to create a clear lens around the authorized user's gaze point, applying obfuscating masks to peripheral content, addressing inefficiencies and noticeability in existing systems by ensuring discreet and effective privacy protection with reduced computational costs.

WO2024231557A1PCT designated stage expired Publication Date: 2024-11-14TOBII TECH AB

Patent Information

Application Number
PCT/EP2024/062971
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2024-05-10
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing systems for protecting screen content privacy are inefficient, noticeable, and computationally costly, often disrupting the authorized user's experience and failing to effectively prevent unauthorized access.

Method used

A computer-implemented method and system that uses eye-tracking to create a clear lens around the authorized user's gaze point, applying an obfuscating mask to the peripheral screen content while maintaining unaltered visibility for the user, using techniques like character replacement, blur, and grid segmentation to ensure privacy without noticeable artifacts.

Benefits of technology

The solution provides discreet and efficient privacy protection, minimizing computational costs and visual disruption to the authorized user, while effectively preventing unauthorized access to screen content.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2024062971_14112024_PF_FP_ABST
    Figure EP2024062971_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The present invention relates to a computer-implemented method for securing privacy of data, or content, visual output, or rendered, on a screen (1250) of a display device (1200) for an authorized user (1300). It comprises the steps of detecting a gaze point (g) of the authorized user on the screen (1250) by an eye-tracking system (1400); creating at least a clear lens (1260) of a predetermined size and shape at a location of the screen (1250) corresponding to the detected gaze point (g) of the authorized user (1300), leaving data output in the clear lens (1260) visually unaltered and thus intelligible for the authorized user; creating an obfuscating mask configured to visually alter the data output on the screen (1250), making the data unintelligible; and applying the obfuscating mask to at least a portion of the screen (1250) of the display device (1200) outside of the clear lens (1260), to make data visually output on the portion of the screen unintelligible to a non-authorized user.
Need to check novelty before this filing date? Find Prior Art

Description

METHOD AND SYSTEM FOR FOVEATED PRIVACY ON A SCREEN OF A DISPLAY DEVICETechnical Field

[0001] The present invention relates to a computer-implemented method for automatically creating and applying an obfuscating mask to at least a portion of a screen of a display device, to make content rendered on such portion of the screen unintelligible to a non-authorized user, or onlooker, while preserving the ability of an authorized, or rightful or intended, user to effectively visualize content in total privacy.Background Art

[0002] Several systems to prevent privacy invasion by people looking over the shoulder of an authorized electronic display device user are known in the prior art, some of them also employing face recognition and / or eye tracking. Hardware solutions such as physical screen filters to be installed on top of screens of display devices are also known, though cumbersome and useful only under limited work environment conditions.

[0003] For instance, US 20220245288 A1 discloses obfuscating content falling within a peripheral view of an end user; moving a clear lens to the gaze point of such end user; scrambling text in the peripheral view by replacing letters; scrambling the peripheral view by applying a blur function.

[0004] However, none of the prior art systems provides to ensure privacy while making the application of an obfuscation mask to screen content unobtrusive and discreet, so that the authorized user is virtually unaffected by the obfuscation when enjoying the screen content. Blurring the peripheral screen content as such, for instance, is detectable by the end user, both authorized and non-authorized, oftentimes with a distracting and / or annoying result for the authorized user. Such implementations are also not favorable, not only since a non-authorized user may easily detect and discern the difference between obfuscated and non-obfuscated content of the screen, which jeopardizes the privacy, but also because of the visually noticeable movement artifacts of the changing screen content when the user’s gazepoint changes, which results in relative movements of the obfuscated content on the screen.

[0005] In the following, the wording “peripheral screen content” is meant to substantially designate, and correspond to, the screen output rendered, or displayed, outside of or in close proximity to an area on the screen containing a detected gaze point of an authorized user on the screen of a display device.

[0006] While the known systems attempt to effectively lock out onlookers, or unauthorized users, by preventing the risk that they capture, in part or in total, the content of a screen, such attempts are usually not efficient in terms of computational costs and more importantly, are perceived as annoying or uncomfortable by the authorized users. Hence, there is a need to reduce these computational costs in order to keep latency and resource allocation low.

[0007] Therefore, there is a need for a computer-implemented method, and a corresponding system, allowing to keep privacy of screen content on a display device for an authorized user in a targeted way, by a data-efficient, swift manipulation of peripheral screen content, in way that is both secure and less detectable / perceived to a peripheral vision of the authorized user, thus not disrupting the authorized user’s overall visual experience.Disclosure of the Invention

[0008] According to the invention a computer-implemented method for securing privacy of data, or content, that is visually output, or rendered, on a screen of a display device for an authorized user is provided, as it is defined by the features of independent claim 1 , and by a corresponding system configured to carry out such method. Preferred embodiments are subject of the dependent claims.

[0009] In particular, the invention relates to a computer-implemented method for securing privacy of data which are visually output on a screen of a display device for an authorized user, based on a detection by an eye-tracking system of a gaze point of at least the authorized user on the screen.

[0010] The present invention also relates to a computing system, designed to carry out the above computer-implemented method, wherein the computing system comprises a computing device including one or more processors; one or more input elements; a memory; and one or more computer programs stored in the memory including instructions for implementing the method.

[0011] The present invention further relates to a non-transitory computer-readable storage medium, storing one or more computer programs configured to be executed by one or more processors of the computing device with one or more input elements, the one or more programs including instructions for carrying out the above-mentioned computer implemented method.

[0012] In the context of the present invention, the term “clear lens” is meant to substantially designate, and correspond to, a region / area of a screen with a defined shape and size containing and / or being around a detected gaze point of the authorized user, such that the displayed content with this region is unaltered and legible for the user.

[0013] In the context of the present invention, the term “gaze point” refers to an estimated location on a screen, or monitor, of a display device that a user, or an observer, is looking at. Such a location may result from an intersection between an eye gaze direction, e.g., taking into account left and right sight line in combination or separately, and the plane of the screen. In general, each detected gaze point is an individual record of a user’s gaze on a screen at a given moment, represented by a sample of data collected by an eye tracking system. As will be apparent to a skilled person, the term “eye-tracking system” includes both hardware, such as an image sensor with one or more illuminators, or alternatively, a built-in or external / peripheral camera which many computer screens are equipped with nowadays, commonly referred to as a web-camera, and a control unit / processor; and a software, or computer-implemented methods run on the processor of the system, for measuring eye position, eye movements, etc. to detect zones on which a user is gazing or showing a particular interest in, at a specific time.

[0014] The current invention attempts to achieve an efficient foveated privacy, understood as rendering privacy for a region on which a user is looking at, thusensuring that the authorized user enjoys a clear view of the screen content currently gazed at, while eliminating the risk of sharing screen content information with an onlooker, only the part of the screen where the authorized user is in the moment looking at is made intelligible, preferably exclusively to the same identified authorized user, whereas the rest of the screen is obfuscated as illustrated in the following.

[0015] The method according to the present invention comprises a step of creating an obfuscating mask configured to visually alter the data output on the screen, making the data unintelligible. The mask can be advantageously created at the display device comprising the screen on which the data is output, when the display device is provided with computing means such as one or more processors. More particularly, an obfuscating mask according to the present invention is applied to at least a portion of the screen outside of a detected gaze point of the authorized user, while a clear lens is created at the detected gaze point of the authorized user, so that the screen of the display device shows unaltered, i.e., unobfuscated content for the authorized user at his or her gaze point.

[0016] Once created, the obfuscating mask is applied to at least a portion of the screen of the display device. The portion of the screen to be provided with the obfuscating mask can be set to a screen portion falling outside of a threshold distance from the detected gaze point of the authorized user, that is an area of the screen, away from the center of gaze or gaze point by a pre-set distance, which renders peripheral screen content for the authorized user’s peripheral vision.

[0017] Thus, the data, or content, visually output on such portion of the screen, currently not at the center of attention of the authorized user, is made unintelligible. This is particularly important that a non-authorized onlooker or occasional bystander would not be able to comprehend the content he / she is not authorized to view. A breach of data privacy by a viewer other than the authorized user is therefore prevented.

[0018] The advantages of the current invention will be apparent to a skilled person, including the effect of reduced visual difference between obfuscated and nonobfuscated display content without substantial movement artifacts of the displayed content, performed at a reduced computational cost.

[0019] The method according to the present invention creates, or keeps, a clear lens, or clear window, within the above threshold distance from the detected gaze point of the authorized user, leaving data output within the clear lens visually unaltered, i.e. , not manipulated by application of the above obfuscation mask, and thus intelligible for the authorized user.

[0020] Accordingly, the screen content that fits within the clear lens - directed by, and positioned at, the authorized users’ gaze - will be intelligible to the authorized user, and the screen content that does not fit within the clear lens, will remain unintelligible. It should be noted that the screen content within the clear lens may also be intelligible to other users. However, since a second user, different from the authorized user, whether further authorized or non-authorized, does not know a priori where the gaze of the authorized user is directed to, such intelligible content on the screen is not easily discerned by the second user.

[0021] The method according to the present invention optimizes the above obfuscation mask so that the application thereof is discreet enough not to disturb the peripheral view of the authorized user, to the point of being substantially unnoticeable to the authorized user. Such changes to the visibility of the contents are not directly noticeable to non-authorized users in a similar fashion as a non-disturbed view for the authorized user, because the provision of the transparent clear lens is not directly discernable by any user. Additionally, the effective non-visibility of the discreetly shown content is not detected by the non-authorized user, which provides enhanced privacy for the authorized user. In fact, while human peripheral vision has very poor visual acuity and sensitivity to colors, it is extremely sensitive to movements and changes in high-contrast areas (such as in screen portions displaying text) and changes in luminance.

[0022] At the same time, the method according to the present invention achieves to divert attention of an onlooker from the unaltered data shown in the clear lens around the gaze point of the authorized user, or at least not offer pointers to the existence thereof. In this way, the onlooker may not directly notice the position of the clear lens on the screen and hence, may not comprehend the content of unaltered data viewed by the authorized user. Hence, the current invention offers advantages over the knownmethods that usually contain movement artifacts leading non-authorized user to easily notice the change on the screen due to the constantly changing screen content.

[0023] Moreover, the method and the system according to the present invention optimize reaction times upon detection of an authorized user’s gaze (and / or of an onlooker’s gaze), considering the extreme speeds of saccades (typically 20-40 milliseconds). The latency of the present system in creating and applying the inventive obfuscation mask is therefore minimized.

[0024] In some embodiments of the invention, different types of original content output in respective screen portions, or areas, of the display device can be identified, for instance by techniques like optical character recognition and machine learning or machine learning powered image segmentation, to divide displayed text document into coherent and semantically meaningful segments. .

[0025] Thus, different obfuscation methods or degrees of displayed content manipulation can be applied to these identified screen areas with different original content.

[0026] Examples of types of content relevant to detect comprise but are not limited to any combination of: images; videos; text; spreadsheets; graphs; games; advertisements.

[0027] Different variants of obfuscation can be applied to different types of content, in order to reduce noticeability of the authorized user and to make the obfuscation more effective towards the unauthorized user. For instance, character-based obfuscation could be used for areas of the screen that contain mainly text; a blur-based obfuscation could be used for areas that contain images; and some other obfuscation methods could be applied for video, graphs, games etc. It will be understood by a skilled person that a “character” refers to written letters, symbols, shape representations or numbers irrespective of the language.

[0028] When applying an indiscriminately strong manipulation, such as a blur function, to manipulate content according to prior art techniques, the result is noticeable to the user which can be perceived as annoying to the human vision. It should be noted that the screen content within the clear lens may also be perceived byother users. However, since a second user, different from the authorized user, whether further authorized or non-authorized, does not know a priori where the gaze of the authorized user is directed to and further is not able to discern a transparent clear lens on the screen, such intelligible content on the screen is not easily discerned by the second user. If, on the other hand, a manipulation function that is too weak is applied, the obfuscation for an unauthorized viewer becomes insufficient.

[0029] According to some embodiments of the present invention, a degree of manipulation, such as blurring, to obtain a robust obfuscation is made dependent on the nature of the original content output on the screen. For instance, large font text may require a stronger manipulation to obfuscate than small font text; and images typically require a stronger manipulation than text. Hence, by semantically decoding the type of data shown and the contents as above described, the present invention allows to apply optimal types of manipulation functions to different types of screen content.

[0030] Alternatively, in other embodiments of the present invention, a non-semantic obfuscation can be created, to make implementation easier as well as to reduce computational load. According to a non-semantic approach, the system does not need to semantically interpret and categorize the underlying display content.

[0031] One first example of non-semantic mask obfuscation according to the present invention is curtain segmentation. In curtain segmentation, the screen of the display device is segmented into vertical or horizontal “curtains”. Each curtain can be X pixels wide. The relative obfuscation mask can then be created by manipulating each or every 2nd or 3rd etc. curtain in a way that aims at maintaining “visual similarity” to the human peripheral vision system, while still creating a strong obfuscation to the unauthorized viewer. In an alternative implementation of the current invention, a combination of both vertical and horizontal curtains can be applied in various steps, for example applying first a horizontal curtain to the original content and consequently, applying a vertical curtain on the obfuscated content. One skilled in the art would understand the various modifications within this method and all the possible permutations are not discussed here for brevity reasons. One example of such manipulation is to “mirror” the inner contents of the curtains that should be manipulated. Another example of such manipulation is to blur the contents of the manipulated curtains.

[0032] A second example of non-semantic mask obfuscation according to the present invention is grid segmentation. In grid segmentation, the screen of the display device is segmented into a grid of rectangles of X by Y pixels. The corresponding obfuscation mask is created by manipulating each or every 2nd, or 3rd or so rectangle, for instance with a horizontal, or vertical mirror function or a blur function.

[0033] Incidentally, it may prove advantageous that the position of both curtains and grid rectangles in the above examples are static, since this may further help reduce the noticeability to the peripheral view of the authorized user.

[0034] In some embodiments of the present invention, rather than employing standard blurring functions and / or random scrambling of text as proposed by prior art methods, the obfuscation mask may be created by replacing all or some letters of recognized text in the visually output content with other letters or graphical elements that have similar appearance.

[0035] According to the above approach, an algorithm embedded in the method according to the present invention may implement the substitution of some or all letters, for example by substituting “c” with “o”; “n” with “u”; “t” with “I”; “i” with “I”; “T” with “F”; “b” with “d”; “e” with “c”, or similar. An analogous operation may be executed with respect to numbers. A variant of this is to replace some or all letters with other graphical elements that are not even regular letters or numbers, but still visually resemble letters or numbers. By way of example, an “A” can be replaced with a “A”.

[0036] To create such a character-replacement obfuscation mask, the system executes some or all of the below steps:- determine that something is text or symbol;- determine which specific character or integer, the letter or symbol is;- determine what font the letter or symbol is displayed in;- determine the size of the letter or symbol;- determine a suitable replacement letter or symbol or graphical element;- insert the replacement letter or symbol or graphical element in the obfuscation mask, instead of the original letter symbol, in the same position as the original letter or symbol.

[0037] The first four steps above can be done using optical character recognition or machine learning. A suitable replacement letter or symbol or graphical element may be determined based on look-up tables or databases or some alternative logic functions.

[0038] By way of example, an original sentence reciting the text:may be transformed, by application of an obfuscation mask as described, into

[0039] A character-replacement obfuscation mask as described above guarantees extreme discreteness to the peripheral vision of the authorized user. This advantageously has the added benefit of making the clear lens almost impossible for the unauthorized viewer to detect, since it is very difficult to determine where the clear lens is when the entire content resembles regular text.

[0040] As a less computationally heavy alternative to the specific embodiment of character-replacement obfuscation mask above exemplified, a character mirroring obfuscation mask can be created with analogous results. In fact, as an alternative, the method according to the present invention can replace each, or every single, 2nd, 3rd or so, letter / number or symbol, with a mirror-image of that same letter / number or symbol.

[0041] This has the benefit of still being a visually similar and efficient obfuscation, while removing the need for the software to semantically understand each underlying letter. As such, the software does not need to understand that an “a” is an “a”, just that it is a “character”, which for instance can be done by optical character recognition, or machine learning, or a custom algorithm detecting heaps of high-contrast pixels lumped together.

[0042] By way of example, an original sentence reciting the text:once obfuscated according to the above procedure may appear on a screen as

[0043] In some embodiments of the present invention, a blur function component employed for creation of an obfuscation mask according to the present invention is ameliorated with respect to prior art blur functions in that it achieves not bringing about changes in the luminance of the visually output image, therefore reducing noticeability to the authorized user. In fact, traditional blur functions change the luminance of the area being blurred and adversely affect a user’s peripheral vision which is sensitive to luminance changes.

[0044] In some embodiments wherein the obfuscation mask is obtained by blurring, the present invention solves several drawbacks of various types of known blurring methods linked to a concurrent, undesirable change in luminosity of the screen displayed content or data.

[0045] In fact, different types of blurs are created by convolving images using a kernel moving left to right and top to bottom. A blurring function takes the average of all pixels under the kernel area and replaces the central element. As such, the kernel needs to have an odd number of elements / row and column, otherwise it will not have a true center. The larger the kernel, the more “blurred” an image becomes.

[0046] So called average blur makes use of uniformly weighted kernels, e.g., all pixels are valued equally when performing the convolution. The resulting blur is not especially natural.

[0047] Among other blurring methods which are per se known, a Gaussian blur technique - a form of blur where weights are applied in such a way that neighborhood pixels that are closer to the center contribute more to the average - has been found as the blurring method which yields the most natural blur for text content on a screen, in studies related to the present invention.

[0048] It has been verified in connection with the present invention that an obfuscation mask obtained via application of blurring functions achieves surprisingly good, natural results when specifically, a Gaussian blur technique is combined with an amount of gamma correction that is adjusted based on the level of contrast detected in the content, or data, visually output on the screen. Thus, the Gaussian blur will not result in a change in luminosity to screen content, thanks to a compensating gamma correction, made dependent on contrast in the content on the screen, which mitigates the luminosity changes.

[0049] Gamma correction is a non-linear operation to encode and decode luminance, used for instance in photography to adjust the luminosity of images. The relation between camera and light is linear, e.g., in a camera that will register photons in a linear way, 2 times the photons is registered as 2 times brighter; whereas the human eye perceives this in a nonlinear way, such that 2 times the photons is perceived as just slightly brighter.

[0050] The synergistic effect of Gaussian blur and contrast level-adjusted gamma correction makes the blurring operation according to the present invention luminanceinvariant, and therefore the manipulation is advantageously undiscernible to the peripheral view of an authorized user. The implementation of the present special blurring advantageously considers different types of content contrast when adjusting the relative gamma correction, as, e.g., luminosity changes more when applying blur to high-contrast content (such as black text on white background) than to low-contrast content (such as images with a smaller range of colors). Consequently, black text on a white background, that is content with high contrast, will need a proportionally higher amount of positive gamma correction applied, not to become darker following blurring application; than it is the case for instance for an image or website with different hues of blues.

[0051] The above Gaussian blur and contrast level-adjusted gamma correction solution can advantageously be combined with embodiments of the present invention making use of the above described semantic-obfuscation approach and / or of contentdependent manipulation factor (that is, customization of different types of manipulation functions to different types of screen content), wherein the underlying screen contentis evaluated to determine what degree of Gaussian blurring is optimally applied, and what degree of gamma correction is optimally added to maintain luminance.

[0052] In some embodiments of the present invention, any of the obfuscation techniques as described above can be modulated so that the obfuscation mask is created partially and / or intermittently.

[0053] By creating the obfuscation mask partially or intermittently - as opposed to continuously and at the same refresh rate as the eye tracker system detecting the user gaze - use of computational resources can be reduced.

[0054] Accordingly, the creation of the obfuscation mask could either be done at a regular interval, or could be triggered by certain events, such as when the underlying content of the screen of the display device is changed, or when the authorized user initiates certain actions, such as for instance scrolling or movement of a window.

[0055] The obfuscation mask may be therefore created at a significantly lower framerate than the refresh rate of the eye tracker or the display. Typically, an eye tracker has a refresh rate of 30 - 200 times per second, and a computer display typically has a refresh rate of 60 - 240 times per second. However, for most computer work, the contents of the screen tend to change much slower than such frequencies, so for most use-cases the obfuscation mask may be created at, for instance, 1 - 5 times per second. This saves significant compute resources.

[0056] Alternatively, the creation of the obfuscation mask may be triggered by certain events, such as when the underlying display content is changed, or when the user initiates certain actions, such as scrolling or moving a window. Usually while scrolling, the obfuscation mask may update too slowly. One option to address this problem is to increase obfuscation mask refresh rate during scrolling actions. Another option is to deactivate the obfuscation altogether during scrolling, since the screen is very difficult to comprehend for an unauthorized user anyway during that time.

[0057] In some other embodiments, the update of the obfuscation mask may be triggered independently for different parts of the display based on certain criteria. For instance, if only a certain region of the underlying display content is changed, the obfuscation mask could be updated for only that region, while the remaining part of theobfuscation mask may stay intact. Another variant of this is to only update the part of the obfuscation mask that relates to the currently active window on a regular basis.

[0058] A gaze directed clear lens as introduced above can be carried out in several ways according to the present invention.

[0059] In some embodiments, a gaze point threshold is implemented that controls the movement of the clear lens, limiting its movement to when needed based on substantial displacement of the authorized user’s gaze. Thus, the number and disturbance of artifacts due to movement of the clear lens are effectively reduced or eliminated.

[0060] In fact, a continuously moving lens results in an unnecessarily high amount of change in the visual rendering. Each lens movement can potentially be perceived as a movement artifact in the parafoveal or peripheral vision of the user.

[0061] By detecting a new gaze point of the authorized user, following a displacement thereof on the screen, and considering the radius of the clear lens in relation to a preset gaze point threshold, the method according to the present invention can determine whether the clear lens is to be moved or not.

[0062] Thus, for any gaze point registered by the eye tracking system within a preset gaze point threshold on the screen (or a domain or domain area thereof), the method according to the present invention may instruct no change in the position of the clear lens created for the benefit of the authorized user. Whereas, if gaze point is registered by the eye tracking system outside of the domain area of the pre-set gaze point threshold, the clear lens is instructed to move to that position. The pre-set gaze point threshold may be a radius or an axis of a domain area having is a function of the lens shape and size. The semi-static clear lens created according to this approach stays put as long as a gaze point is registered inside the pre-set gaze point threshold. As soon as a gaze point is registered outside the edge of the domain defined by the pre-set gaze point threshold, the clear lens is moved to the newly registered gaze point position. Various filters can be applied to the gaze point data before applying the above logic, for instance to filter out noise artifacts in the underlying gaze data.

[0063] In some embodiments, compatible with substantially all of the above already described, the shape of the clear lens created for intelligibility of the output content on the screen to the authorized user can be changed based on user activity and / or user’s eye movement pattern. Detection of a new, different eye movement pattern and / or user activity, by the eye tracking system or by some other input means, e.g., associated to the screen of the display device, can trigger a change in the shape and / or in the behavior of the created clear lens. By way of example, if reading behavior is detected, the method according to the present invention can instruct that the lens be made horizontally elongated and thinner; and / or made it longer to the right than to the left in case a language is used which reads from left to right, otherwise the opposite if the text language is to be read right to left. If, on the other hand, it is detected that that user is not currently reading, the method can instruct that the lens be made substantially circular and larger.

[0064] In some embodiments, also compatible with substantially all the above already described embodiments, the method according to the present invention can be configured to make a prediction of where the next gaze point of the authorized user is going to be. This achieves displaying a predicted clear lens at that predicted position, thus alleviating problems linked to latency in the system and reducing noticeability.

[0065] In fact, a common underlying problem to hardware and software, including eye tracking hardware and software, is that they are affected by latency to varying degrees. Some residual latency in the process of detecting eye movement being made, a gaze point being registered, and in the process of the clear lens moving to the proper position on the screen is inherent to such technology, however advanced. This issue is even exacerbated by lower performing hardware. If there is a high latency from the moment in time when the authorized user moves their eyes to when the display is updated with the clear lens, the authorized user will perceive an annoying “flicker” of the screen of the display device.

[0066] Making a prediction of where the next gaze point of the authorized user is according to the present invention may be implemented in several ways. Any combination of each of the specific implementations given below is also possible.

[0067] In a first embodiment, a longer predicted clear lens can be implemented that projects forward based on the direction between some of the latest detected gaze points. The length of the predicted clear lens can be a function of the position and / or of the distance between some of the latest gaze points. The number of the latest gaze points, and the weight parameters attributed to position and / or distance of each of the latest gaze points, can vary according to different rules. Saccades are ballistic, e.g., pre-determined, which makes estimation possible.

[0068] In a second embodiment for creation of a predictive clear lens, user behavior can be detected and based on that the clear lens can be positioned, e.g., when detecting that a user is reading, a predicted clear lens can be preemptively added at the beginning of the next row of text.

[0069] In a third embodiment for creation of a predictive clear lens, a predictive model can be implemented that uses machine learning algorithms to predict probabilities of where the next gaze point will be based on either (i) only data relative to historical gaze points, or (ii) a combination of historical gaze points and the contents of the screen of the display device. One or multiple clear lenses would then be placed at the predicted location(s), if the machine learning model outputs that it predicts that the next gaze point will be at one or multiple specific locations with a probability higher than a threshold confidence level.

[0070] In some embodiments of the present invention, decoy lenses, e.g., lensshaped obfuscations in parts of the screen where the authorized viewer is not looking, can be added. Thus, the unauthorized user would be confused in such a way that it becomes more difficult to try following the gaze of the authorized user. The attention of the onlooker is substantially diverted from the content on the screen that is being visualized by the authorized user.

[0071] These decoy lenses can display content that appears to not be obfuscated but does not contain any sensitive information. This will make it harder to distinguish them with respect to the actual clear lens.

[0072] Decoy lenses can be used as a complement to different types of obfuscation masks, to enhance obfuscation effectiveness.

[0073] In some embodiments of the present invention, possibly in combination with any of compatible procedures above described, the method of the present invention can automatically calculate and adapt the size of the clear lens as a function of the calculated total latency of the system, so that the obfuscation grade can be optimized while maintaining a consistent user experience.

[0074] As already mentioned, the noticeability of foveated privacy to the authorized user depends on several factors. Various types of hardware and software modules cause latencies such as the camera system used for eye-tracking, the processing used for eye-tracking (edge-NPU / CPU / GPU, etc.), the processing used for doing the screen obfuscation (CPU / GPU, etc.), the refresh-rate of the screen, the pixel switching time, etc. Therefore, there exists an inverse relationship between the quality of hardware and the level of sophistication of software and the required size of the clear lens needed to achieve a low level of noticeability to the authorized user. Naturally, the larger the clear lens is, the less powerful the obfuscation becomes in the sense that it might become more easily identifiable to an unauthorized viewer if not compensated for.

[0075] To calculate the total latency of the system, the solution could consider some or all of the following aspects: eye tracking latency; obfuscation software latency; display latency.

[0076] The concept of adaptive lens sizing can additionally be applied to compensate for the fact that eye tracking devices have different levels of performance on different parts of the screen, typically with better performance closer to the center and worse performance closer to the edges. Accordingly, by increasing the lens size as it moves further away from the center of the screen, the present invention achieves to mitigate this phenomenon.

[0077] Also, adaptive lens sizing according to the present invention can be carried out based on relative importance of screen regions on a display device, considering that the center region of the screen incorporates typically more sensitive content than the regions on the edges. In fact, more standard functional and content-neutral interfaces are traditionally positioned at the edges, such as headers, scroll bars, task bar, navigation pane. Therefore, the present method advantageously achieves tocreate a smaller lens at the center region of the screen of a display device, which increases obfuscation grade.

[0078] To determine hardware and software latencies, the method according to the present invention can carry out any of the following steps, possibly in combination:- request latency characteristics of the various components of the system . For instance, in a modern Windows OS, it is possible to inquire the display driver about the refresh rate of the display hardware;- use computer logic to estimate latencies. For instance, the software could know that an eye tracker of a certain configuration and model has a latency of X milliseconds or that the obfuscation software will typically have a latency of Y milliseconds when it runs on a particular CPU;- use a database to lookup latencies relating to, for instance, configuration and model of eye tracking systems, computer models, CPU models, GPU models, etc. Such database could either be implemented offline (stored locally at the user’s computer) or be refreshed against an online database that is continuously updated with new models;- use a special technique to automatically measure latencies in the system, such as during a calibration procedure the system can show stimulus at random locations, and measure how long it takes for the system to receive a corresponding eye movement from the user. If this is repeated, the solution can use the latency of this to infer an approximation of the total latency of the system for foveated privacy.

[0079] The latency of the software for eye tracking system and the latency of the software for obfuscation can be measured and compounded, to obtain an estimate of the overall latency of the complete system.

[0080] In some embodiments, compatible with substantially all of the alternative hereby recited for obfuscation mask creation and clear lens creation, the method according to the present invention may comprise a destination lens function, configured to allow an authorized user to comfortably transition from gazing at first area on the screen to a second, further area on the screen.

[0081] In fact, when an authorized user undergoes one or several longer saccades into a further, second area of the screen, relative to a first area where the authorized user’s gaze has been previously focused, it is advantageous to provide such user with an idea of what is to be expected in this new, “unexplored” second area.

[0082] While it is desirable to provide an intuitive impression of the content of the second area being detected as becoming of interest for the authorized user, normally this is not enabled by the inadequate size of the clear lens as such. The method according to the present invention therefore detects that a number of saccades of the authorized user within a set time period, resulting in an overall gaze displacement having been made to a second area or destination; and then instructs the system to create and display an additional, larger clear lens at such second area or destination. Advantageously, this larger clear lens over the second area on the screen can decay over a set amount of time, after the need for the authorized user to assess the content output on the second area has substantially expired. This larger clear lens is not functionally connected to the gaze directed clear lens and will be added to a further second area on the screen as described, after which it will remain in that position and start decaying, e.g., shrinking, fading or any other suitable decaying function known to a skilled person.

[0083] In some embodiments, it may be desirable that the clear lens conforms to the boundaries of a specific viewed object, or, that, when the clear lens comes into contact with certain types of objects, such as image or video content, an additional clear lens is created, possibly overlapping in part with the original clear lens, taking on the size and shape of the very object to be displayed. This solution is especially advantageous when certain types of viewed content, such as images or video, are larger than a clear lens created according to the present invention. Without the intelligibility expanding to the images or video contacted by the so called “Content contact lens”, i.e. , an additional or complementary lens to the clear lens conforming to the boundaries of the viewed object, which may also interact with or work independently of the clear lens, the authorized user’s experience would be negatively affected as the immersion of his or her visual experience would be limited.

[0084] In some embodiments, a small region around the cursor can be revealed and made intelligible, e.g., by adding an additional clear lens which follows the cursorposition. Thus, flexibility and fluidity in the authorized user’s visual experience are increased.

[0085] Building on the creation of the obfuscated mask and of the clear lens as per the above description, obtained by implementing steps or instructions of a foveated privacy, computer-implemented method and the computer program according to the present invention, an encryption method for securing documents displayed on the screen can be created preventing unauthorized access. Such encryption method prevents the access to the documents, unless the above foveated privacy software is running and, possibly, unless an additional condition that an eye tracker system is operative is verified. This can be applied to, or triggered by, conventional structures of organizational document security coding. In such an application, documents with a set level of confidentiality cannot be opened without foveated privacy software running; or the foveated privacy software according to the present invention can run continuously in the background, but the encryption method is only applied to documents of above the set confidentiality level.

[0086] The above encryption solution, carrying out a document lock in connection with foveated privacy software, is particularly advantageous when the content of some documents is extremely sensitive, and the author(s) / owner(s) have a high interest in that the contents is not compromised.

[0087] In association with the abovementioned document lock based on foveated privacy software, or as an extension to it, the present invention may comprise a foveated privacy document reader that always, as default, displays contents as obfuscated, except for in a region containing the gaze point of the authorized user. Based on this approach, it can be commanded that only a new, special file format can be decoded by the document reader. The document reader may be configured to convert (and by extension encrypt) conventional document types (.doc, .pdf, etc.) to such new document format. Moreover, an ID of a set of specific eye trackers belonging to specific individual(s) may be embedded in the file format. This solution may incorporate several of the innovative obfuscation mask and clear lens related features mentioned above, enhancing simplicity of implementation thereof, such as characterreplacement obfuscation, semantic-based obfuscation, content-dependent blurring, semi-static lens, predictive lens, as well as decoy lenses.

[0088] As explained, the noticeability to the authorized user of a foveated privacybased method as per the present invention, however minimized, remains in part physically bound to the inherent latency of hardware components currently available. Various types of hardware cause latencies such as eye-tracking device, CPU / GPU, screen, etc. Therefore, there exists an inverse relationship between the quality of hardware and the required size of the clear lens needed to achieve unnoticeability.

[0089] If the above obfuscation mask is created and / or applied in a chip that is very close to the screen of the display device itself (e.g., inside the monitor), ideally directly wired to the eye tracking system, it can be possible to dramatically lower the latency, and hence improve the performance of the present foveated privacy method.Brief Description of the Drawings

[0090] The method and system according to the invention are described in more detail herein below by way of exemplary embodiments and with reference to the attached drawings, in which:Fig. 1 is a block diagram of components of an exemplary embodiment of the present invention for providing content privacy for an authorized user with an image sensor, such as a webcam or gaze tracker, in accordance with embodiments of the present invention;Fig. 2 illustrates an exemplary flow-chart for a process of creating an obfuscation mask on a screen of a display device;Fig. 3 shows an exemplary flow-chart for a process of creating a semantic-based obfuscation mask on a screen of a display device according to the present invention;Fig. 4A shows an exemplary flow-chart for a process of creating a first non-semantic- based obfuscation mask according to the present invention by curtain segmentation of a screen of a display device;Fig. 4B shows an exemplary flow-chart for a process of creating a second non- semantic-based obfuscation mask according to the present invention by grid segmentation of a screen of a display device;Fig. 5 shows an exemplary flow-chart for a process of creating a character-replacement obfuscation mask according to the present invention;Fig. 6 illustrates an exemplary flow-chart for a process of creating a luminance invariant blur according to the present invention, showing a a screen which has not been affected by luminosity changes after application of an inventive blurring function, as opposed to the application of a traditional blur function;Fig. 7 illustrates an exemplary flow-chart for a process of creating an intermittent obfuscation mask according to the present invention;Fig. 8 illustrates an exemplary flow-chart for a process of creating a gaze-directed clear lens on a screen of a display device;Fig. 9 illustrates an exemplary visualization of a current implementation for creating a semi-static clear lens according to the present invention;Figs. 10A, 10 B and 10 C illustrate three different process options for creating a clear lens at a predicted position corresponding to the position of an authorized user’s next gaze point, according to the present invention;Fig. 11 illustrates an exemplary flow-chart for a process of creating decoy clear lenses to divert the attention of an onlooker;Fig. 12 illustrates an exemplary flow-chart for a process of automatically calculating and adapting the size of the clear lens as a function of the calculated total latency of a system;Fig. 13 shows an exemplary flow-chart for a process of automatically creating a distinct, larger clear lens when an eye tracking system detects that an authorized user intends to get an idea of content on a yet unexplored area of the screen of a display device, such distinct, larger clear lens being applied on the unexplored area;Fig. 14 shows an exemplary flow-chart for a clear lens conforming to the boundaries of a viewed object, according to the present invention; andFig. 15 illustrates a simplified exemplary view of a screen comprising a clear lens at the gaze point of an authorized user and an additional cursor lens, according to the present invention.Description of Embodiments

[0091] In the following description certain terms are used for reasons of convenience and are not intended to limit the invention. The terms “right”, “left”, “up”, “down”, “under" and “above" refer to directions in the figures. The terminology comprises the explicitly mentioned terms as well as their derivations and terms with a similar meaning. Also, spatially relative terms, such as "beneath", "below", "lower", "above", "upper", "proximal", "distal", and the like, may be used to describe one element’s or feature’s relationship to another element or feature as illustrated in the figures. These spatially relative terms are intended to encompass different positions and orientations of the devices in use or operation in addition to the position and orientation shown in the figures. For example, if a device in the figures is turned over, elements described as "below" or "beneath" other elements or features would then be "above" or "over" the other elements or features. Thus, the exemplary term "below" can encompass both positions and orientations of above and below. The devices may be otherwise oriented (rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein interpreted accordingly. Likewise, descriptions of movement along and around various axes include various special device positions and orientations.

[0092] To avoid repetition in the figures and the descriptions of the various aspects and illustrative embodiments, it should be understood that many features are common to many aspects and embodiments. Omission of an aspect from a description or figure does not imply that the aspect is missing from embodiments that incorporate that aspect. Instead, the aspect may have been omitted for clarity and to avoid prolix description. In this context, the following applies to the rest of this description: if, in order to clarify the drawings, a figure contains reference signs which are not explained in the directly associated part of the description, then it is referred to previous or following description sections. Further, for reason of lucidity, if in a drawing not all features of a part are provided with reference signs it is referred to other drawings showing the same part. Like numbers in two or more figures represent the same or similar elements.

[0093] The present invention provides systems, software, and methods to maintain privacy of data shown on a screen of a display device by using sensor-derived information about an authorized user or other individuals, about the device’s environment, about the user’s interaction with the device, and about the user’s attention to displayed information to detect; and optionally to identify the user,optionally to authorize device use and information display, to detect and notify the user of unauthorized viewers or recording devices, to inhibit unauthorized viewers, and to reduce the amount of displayed information available to unauthorized persons or devices. These capabilities are, in some exemplary embodiments, implemented in such a way as to support centralized security policies and enforcement of these.

[0094] The software and methods provided by the invention can be implemented in computing devices, display devices, sensor devices, virtual machines, cloud-based implementations or in any other device or system possessing the required capabilities for sensor input, sensor data processing, display control or device management, or both, using the description and drawings herein.

[0095] The various embodiments of the current invention use one or more sensors, such as video cameras, infrared cameras, sonar, radar and lidar systems, gaze trackers, or other appropriate sensor systems in conjunction with appropriate processing software to monitor the area from which the display is visible and to determine whether an authorized user or unauthorized individual is looking at the display. In some exemplary embodiments, any face--like object is treated as an authorized user. As used herein, a "face-like object" is any sensor input that possesses characteristics that cause facial recognition software to determine that a face is within the detection range of the sensor.

[0096] With respect to Figure 1 , an exemplary embodiment of a system 1000 in accordance with the present invention comprises a computer-driven display system 1200 for presenting visual information, or content or data, securely to a viewer. The exemplary system 1000 includes a computer 1050 in electronic communication with input devices such as a keyboard 1100 and mouse 1150. Computer 1050 is electronically coupled with a display device 1200 including a screen 1250, or viewing surface or display, that is viewed by an authorized user 1300. The user’s ability to view intelligibly content displayed on the screen at a gaze point "g" can be determined, in toto or in part, using an image sensor 1400, such as a camera (e.g., a "webcam") or a dedicated gaze tracking system. The image sensor can be electronically coupled with a security processor (not shown), which security processor can be further in communication with a user security parameter database (not shown). Each of theseelements can be of standard design and construction, and will be familiar to those having ordinary skill in the art using the present disclosure and drawings.

[0097] Fig. 2 illustrates an exemplary flow-chart for a process of creating 100 an obfuscation mask on at least a portion of a screen 1250 of a display device 1200 according to the present invention. The obfuscating mask is configured to visually alter the data output on the screen 1250, making the data unintelligible. The mask can be advantageously created at the display device 1200 comprising the screen 1250 on which the data is output, when the display device is provided with computing means 1050 such as one or more processors. The portion of the screen to be provided with the obfuscating mask can be set to a screen portion falling outside of a threshold distance from the detected gaze point of the authorized user 1300, that is an area of the screen, away from the centre of gaze or gaze point by a pre-set distance, which renders peripheral screen content for the authorized user’s peripheral vision.

[0098] Thus, the data, or content, visually output on such portion of the screen 1250, currently not at the centre of attention of the authorized use 1300, is made unintelligible, and particularly unintelligible to a non-authorized onlooker or occasional bystander.

[0099] Fig. 3 shows an exemplary flow-chart for a process of creating 101 a semanticbased obfuscation mask on a screen 1250 of a display device according to the present invention. For the specific example, three areas of screen 1250, bearing respective different types of original content 1 -3, have been identified. Different obfuscation methods, or degrees of obfuscation, are applied to such different areas, creating in fact three corresponding obfuscation mask variants 1 -3.

[0100] Fig. 4A shows an exemplary flow-chart for a process of creating 102 a first non-semantic-based obfuscation mask according to the present invention. This first non-semantic-based obfuscation mask is created by carrying out a curtain segmentation of the screen 1250 of the display device, wherein obfuscated curtains 102' are applied to the screen irrespective of the type of content rendered on three different areas of the screen.

[0101] Fig. 4B shows an exemplary flow-chart for a process of creating 103 a second non-semantic-based obfuscation mask according to the present invention. Suchsecond non-semantic-based obfuscation mask is created by carrying out a grid segmentation of the screen 1250 of the display device, wherein obfuscated rectangles 103', constituting a part of the grid, are applied to the screen irrespective of the type of content rendered on three different areas of the screen.

[0102] Fig. 5 shows an exemplary flow-chart for a process of creating a characterreplacement obfuscation mask according to the present invention. At a first step 104, processing means of the computer 1050 in system 1000 detects that the original content to be visually output on the screen 1250 comprises either text or a symbol. If this is verified, at the same step 104 the processing means determines which specific character or integer the symbol is. For each character or symbol, then the system determines, at step 105, what font the character or symbol is displayed in; and, at step 106, the size of the character or symbol is determined. At step 107, based on a database of character and symbol replacement combinations, the system determines a suitable replacement character or symbol or graphical element. If a valid alternative according to the embedded processing logic is found, the processing means instructs at step 108 that such suitable replacement character or symbol or graphical element be inserted in the obfuscation mask, at the place of the original symbol, in the same position.

[0103] Fig. 6 shows an exemplary flow-chart for a process of creating a luminance invariant blur according to the present invention. In fact, in the second illustration, shown in the lower part of Figure 6, it is exemplified how the application 108 of an inventive blurring function according to the present invention to the screen 1250 does not affect luminosity of the screen, as opposed to the result obtained through the application 107 of a traditional blur function, shown in the top part of Figure 6. A traditional blur function evidently has the drawback of generally altering the luminosity of the screen, once the display content has been obfuscated.

[0104] Fig. 7 illustrates an exemplary flow-chart for a process of creating 109 an intermittent obfuscation mask according to the present invention, based on current visualization conditions and user behaviour. The updated creation of the obfuscation mask is, in this specific case, triggered by certain established criteria 110. If any, or all, of the set criteria 110 are met, the obfuscation mask is generated accordingly, to output obfuscated content on the screen 1250. Criteria 110 can be based on detection ofoccurrence of specific events, such as when the underlying content of the display is changed, or when the user initiates certain actions, for instance by scrolling or moving a content window. One option is to increase obfuscation mask refresh rate during scrolling actions.

[0105] Fig. 8 illustrates an exemplary flow-chart for a process of creating 111 a gaze- directed clear lens 1260 on the screen 1250 of a display device 1200. The method according to the present invention creates, or keeps, a clear lens 1260, or clear window, within a threshold distance from the detected gaze point “g” of the authorized user 1300, leaving data output within the clear lens 1260 visually unaltered, i.e., not manipulated by application of the above obfuscation mask, and thus intelligible for the authorized user 1300. Accordingly, the screen content that fits within the clear lens 1260 - directed by, and positioned at, the authorized users’ gaze “g”- will be intelligible to the authorized user 1300, and the screen content that does not coincide with a gaze region will remain unintelligible.

[0106] Fig. 9 illustrates an exemplary visualization of an implementation of the method for creating a semi-static clear lens 1260 according to the present invention. A clear lens 1260 thus conceived has a radius A and its position is defined by point C, substantially corresponding to a user’s gaze point. While shown as an ellipse with two semi-axis having radius of A in Figure 9, it will be understood that a circular region with radius A is contemplated by the current disclosure. A gaze point threshold T is implemented that controls the movement of the clear lens 1260, limiting its movement to when actually needed based on substantial displacement of the gaze of the authorized user 1300. Thus, number and disturbance of artifacts due to movement are effectively reduced or eliminated.

[0107] When a new gaze point of the authorized user is detected, following a displacement thereof on the screen 1250, and taking into account the radius A of the clear lens 1260 in relation to the pre-set gaze point threshold T, the method according to the present invention can determine whether it is required that the clear lens 1260 be moved or not.

[0108] Thus, for any gaze point G1 registered by an eye tracking system 1400 within the pre-set gaze point threshold T on the screen 1250 (or a domain or domain areathereof), the method according to the present invention instructs no change in the position of the clear lens 1260. Whereas, if gaze point G2 is registered by the eye tracking system 1400 outside of the domain area of the pre-set gaze point threshold T, the centre C of the clear lens 1260 is instructed to move to that G2 position. The preset gaze point threshold T preferably is a function of the radius A of the clear lens 1260, for instance according to a relation f(A) = A * X = T, where X is a number in the range [0,1 ], The semi-static clear lens 1260 created according to this approach stays put until a gaze point G1 is registered inside the pre-set gaze point threshold T. As soon as a gaze point G2 is registered outside the edge of the domain defined by the pre-set gaze point threshold T, the clear lens can be centred on the newly registered gaze point position G2.

[0109] Figs. 10A, 10 B and 10 C illustrate three different process options for creating a clear lens at a predicted position corresponding to the position of an authorized user’s next gaze point, according to the present invention.

[0110] With respect to Fig. 10A, a longer predicted clear lens shape P can be implemented that projects forward based on the direction between some of the latest detected gaze points Gi, G2, G3. The length of the predicted clear lens P can be a function of the position and / or of the relative distance between some of the latest gaze points G1, G2, G3. The number of the latest gaze points G1, G2, G3, and the weight parameters attributed to position and / or relative distance of each of the latest gaze points G1, G2, G3, can be varied according to defined rules. If V is a vector whose magnitude and direction is based on the position of the latest gaze points G1, G2, G3, the predicted lens shape P inherits direction of V and is also scaled based on the magnitude of V. The predicted lens shape preferably extends past the position of X, the predicted gaze landing spot, by an extra distance.

[0111] With respect to Fig. 10B, when the system 1000 detects that a user 1300 is reading a text at step 112, based on the determination 113 that the user has reached the end of a row of text, the clear lens is pre-emptively added 114 at the beginning of the next row of text.

[0112] In a third embodiment for creation of a predictive clear lens, shown in Fig. 10C, a predictive model 115 is implemented that uses machine learning algorithms topredict probabilities of where the next gaze point will be, based on data relative to historical gaze points, or a combination of historical gaze points and the contents of the screen of the display device. Using user gaze behaviour data 116, the predictive model 115 outputs a predicted clear lens position on the screen at step 117.

[0113] Fig. 11 shows how decoy clear lenses 1260* are created at step 118, to divert the attention of an onlooker. These decoy clear lenses 1260* display content that appears to not be obfuscated, but does not contain any sensitive information, as opposed to actual clear lens 1260 for the benefit of the authorized user.

[0114] With reference to Fig. 12, the method according to the present invention automatically calculates the total latency of the system 1000 and adapts the size of the clear lens 1260 as a function of the calculated total latency of the system. As portrayed, a system with high combined system latency 1000H, as measured by a dedicated system latency measuring function at step 119, supports preferably a rather large clear lens 12601, to optimize the obfuscation grade while maintaining a consistent user experience. Conversely, a system with low combined system latency WOOL, as measured by a dedicated system latency measuring function at step 119, can support a comparatively smaller clear lens 1260s, with increased obfuscation grade.

[0115] In Fig. 13 exemplifies a case when an authorized user undergoes a sequence of saccades Mi, M2, M3, extending the gaze to an area at a relatively high distance V from the original gaze point contained within region of the original lens A where the authorized user’s gaze has been previously focused and the original clear lens 1260 has been accordingly centred. At step 120, a destination lens D, larger than original lens A 1260, is created at the destination, or end point of the saccades, allowing the user to get an idea of the content at the corresponding unexplored area of the screen. The destination lens D shrinks and decays over an established amount of time, once the exploration of the user has finished. Other examples of changing a size / transparency of the destination lens, for example, fading or similar methods may be utilized. For the destination lens D to be created, the distance V preferably exceeds a pre-set distance threshold.

[0116] In Fig.14, a clear lens CL 1260, conforms to the boundaries of a viewed object which is part of obfuscated content on the screen 1250. This happens when the clearlens CL comes into contact with objects or hovers over the displayed objects, also understood as partially coinciding / overlapping in the coordinate system of the display, such as image or video content, which are previously kept obfuscated. An additional clear lens, designated as Content Contact Lens or CCL, is created, so that it overlaps in part with the original clear leans CL. The Content Contact Lens, CCL, takes on size and shape of the very object being in contact with the clear leans CL, corresponding to the previously obfuscated content, for the time the clear lens CL keeps in contact with the contacted object previously obfuscated. The object corresponding to the previously obfuscated content, contacted by the clear lens CL, is in this case larger than the clear lens CL. Thanks to the fact that the intelligibility expands to the object contacted by the originally created contact lens CL, the authorized user’s visual experience is enhanced under the given circumstances.

[0117] In Fig. 15, on top of a clear lens A analogous to previously described clear lenses 1260, a small region B around the cursor is revealed and made intelligible, by adding an additional clear lens which follows the cursor position. Thus, flexibility and fluidity in the authorized user’s visual experience are increased.

[0118] This description and the accompanying drawings that illustrate aspects and embodiments of the present invention should not be taken as limiting-the claims defining the protected invention. In other words, while the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. Various mechanical, compositional, structural, electrical, and operational changes may be made without departing from the spirit and scope of this description and the claims. In some instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure the invention. Thus, it will be understood that changes and modifications may be made by those of ordinary skill within the scope and spirit of the following claims. In particular, the present invention covers further embodiments with any combination of features from different embodiments described above and below. The disclosure also covers all further features shown in the figures individually although they may not have been described in the afore or following description. Also, single alternatives of the embodiments described in the figures and the description and single alternatives of features thereof can be disclaimed from the subject matter of the invention or from disclosed subject matter. The disclosure comprises subject matterconsisting of the features defined in the claims or the exemplary embodiments as well as subject matter comprising said features.

[0119] Furthermore, in the claims the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single unit or step may fulfil the functions of several features recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. The terms “essentially”, “about”, “approximately” and the like in connection with an attribute or a value particularly also define exactly the attribute or exactly the value, respectively. The term “about” in the context of a given numerate value or range refers to a value or range that is, e.g., within 20%, within 10%, within 5%, or within 2% of the given value or range. Components described as coupled or connected may be electrically or mechanically directly coupled, or they may be indirectly coupled via one or more intermediate components. Any reference signs in the claims should not be construed as limiting the scope.

[0120] A computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. In particular, e.g., a computer program can be a computer program product stored on a computer readable medium which computer program product can have computer executable program code adapted to be executed to implement a specific method such as the method according to the invention. Furthermore, a computer program can also be a data structure product or a signal for embodying a specific method such as the method according to the invention.

Claims

CLAIMS1. A computer-implemented method for securing privacy of data, or content, visually output on a screen (1250) of a display device (1200) for an authorized user (1300), comprising: detecting, by an eye-tracking system (1400), a gaze point (g) of the authorized user (1300) on the screen; creating at least a clear lens (1260) of a predetermined size and shape at a location of the screen (1250) corresponding to the detected gaze point (g) of the authorized user, and leaving data output in the clear lens (1260) visually unaltered and intelligible for the authorized user (1300); creating an obfuscating mask configured to visually alter the data output on the screen, making the data unintelligible within a region covered by the obfuscation mask; and applying the obfuscating mask to at least a portion of the screen of the display device (1200) outside of the clear lens (1260), to make data visually output on the portion of the screen unintelligible to a non-authorized user.

2. The method according to claim 1 , wherein the clear lens substantially corresponds to a region of the screen with a defined shape and size containing and / or being around the detected gaze point of the authorized user, such that the displayed content with this region is unaltered and legible for the authorized user.

3. The method according to any of the preceding claims, wherein the data output in the clear lens being visually unaltered and intelligible is exclusively available for the authorized user only.

4. The method according to any of the preceding claims, wherein the method further comprises providing a portion of the screen with the obfuscating mask based on a screen portion falling outside of a threshold distance from the detected gaze point of the authorized user.

5. The method according to claim 5, wherein said screen portion, away from the center of gaze or gaze point by a pre-set distance, renders peripheral screen content for the authorized user’s peripheral vision.

6. The method according to any of the preceding claims, wherein the visual difference between obfuscated and non-obfuscated display content is reduced by minimizing substantial movement artifacts of the displayed content.

7. The method according to any of the preceding claims, wherein the method further comprises maintaining the clear lens within said threshold distance from the detected gaze point of the authorized user.

8. The method according to any of the preceding claims, further comprising providing an intelligible output to a second user, different from the authorized user, contained within a data output of the clear lens.

9. The method according to any of the preceding claims, wherein the location on the screen containing the clear lens is substantially not visually discernible for the non-authorized user.

10. The method according to any of the preceding claims, further comprising visually rendering of the original content on the screen of the display device by optical character recognition and machine learning or machine learning powered image segmentation, to divide displayed original content into coherent and semantically meaningful segments.11 . The method according to any of the preceding claims, wherein the displayed content comprises any combination of: images; videos; text; spreadsheets; graphs; games and advertisements.

12. The method according to any of the preceding claims, wherein the created obfuscating mask comprises character-based obfuscation and / or a blur-based obfuscation and / or non-semantic obfuscation.

13. The method according to any of the preceding claims, wherein a degree of the created obfuscating mask is dependent on the type of the displayed content.

14. The method according to any of claims 10-13, further comprising creating obfuscation mask by replacing all or some letters of recognized text in the visually output content with other letters or graphical elements that have similar appearance.

15. A computing system, configured to carry out the computer-implemented method according to any of the preceding claims, wherein the computing system comprises: a computing device including one or more processors; one or more input elements; a memory; and one or more computer programs stored in the memory including instructions for implementing the method according to any of the preceding claims.

16. The computing system of claim 15, further comprising an eye tracking system for detecting a gaze point of a user.

17. A non-transitory computer-readable storage medium, storing one or more computer programs configured to be executed by one or more processors of the computing system with one or more input elements, the one or more programs including instructions for carrying out the computer implemented method according to any of claim 1 -14.

Citation Information

Patent Citations

  • Video-based privacy supporting system

    US20220245288A1

  • Gaze-dependent display encryption

    US20200081527A1

  • Document spatial layout feature extraction to simplify template classification

    US20220292862A1

  • Display security using gaze tracking

    US9355612B1

Cited By

  • Multi-screen content access monitoring method and system based on position tracking

    CN120145466A