Prevention of User Interaction Hijacking Attacks by Blocking Interaction with Hidden Components

Abstract

The method includes detecting a user interaction with a particular region of a graphical user interface (GUI) and determining that the user interaction will be consumed by a particular user interface (UI) component, the particular UI component being covered by a plurality of UI components configured to allow the user interaction to pass through to the particular UI component. The method also includes determining a cumulative opacity of the plurality of UI components in the particular region of the GUI based on the determination that the user interaction will be consumed by the particular UI component. The method further includes determining that the cumulative opacity exceeds a threshold opacity and preventing the particular UI component from consuming the user interaction based on the determination that the cumulative opacity exceeds the threshold opacity.

Description

Background Art

[0001] Background Malicious software can be configured to, among a number of malicious functions, deliberately damage computing resources, monitor the use of computing resources, and / or obtain information about the users of a computing device. In some cases, malicious software can operate by exploiting known vulnerabilities in a computing device, its operating system, and / or an application executed thereon. Thus, it is desirable to mitigate and / or eliminate such vulnerabilities without preferably affecting the normal operation and / or functions of the computing device, the operating system, and / or the software application.

Summary of the Invention

Problems to be Solved by the Invention

[0002] Summary A user interaction hijacking attack may include that the user interaction is passed to a specific user interface (UI) component blocked by a plurality of UI components overlaid as part of a graphical user interface (GUI). The user interaction may be consumed by the specific UI component rather than the plurality of overlying UI components. This is because each of the plurality of UI components can be associated with an attribute that configures each UI component to allow the user interaction to pass through to the layer below the GUI by ignoring the user interaction. The user interaction hijacking attack can be mitigated and / or eliminated by determining the cumulative opacity of the plurality of UI components in the area associated with the user interaction. If the corresponding part of the specific UI component is not sufficiently visible through the plurality of UI components due to the cumulative opacity exceeding the threshold opacity, the user interaction can be blocked.

Means for Solving the Problem

[0003] The first exemplary embodiment includes a computer-implemented method, the computer-implemented method including the step of detecting a user interaction with a specific region of a GUI. The computer-implemented method also includes the step of determining that the user interaction is to be consumed by a specific UI component, the specific UI component being covered by a plurality of UI components configured to allow the user interaction to pass through to the specific UI component. The computer-implemented method further includes the step of determining a cumulative opacity of the plurality of UI components in the specific region of the GUI based on the determination that the user interaction is to be consumed by the specific UI component. The computer-implemented method further includes the step of determining that the cumulative opacity exceeds a threshold opacity, and based on the determination that the cumulative opacity exceeds the threshold opacity, the step of preventing the specific UI component from consuming the user interaction.

[0004] The second exemplary embodiment includes a system, the system including a processor and a non-transitory computer-readable medium storing instructions that, when executed by the processor, cause the processor to perform operations. The operations include detecting a user interaction with a specific region of a GUI. The operations also include determining that the user interaction is to be consumed by a specific UI component, the specific UI component being covered by a plurality of UI components configured to allow the user interaction to pass through to the specific UI component. The operations further include determining a cumulative opacity of the plurality of UI components in the specific region of the GUI based on the determination that the user interaction is to be consumed by the specific UI component. The operations further include determining that the cumulative opacity exceeds a threshold opacity and, based on the determination that the cumulative opacity exceeds the threshold opacity, preventing the specific UI component from consuming the user interaction.

[0005] A third exemplary embodiment includes a manufactured article that includes a non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform operations. The operations include detecting a user interaction with a particular region of a GUI. The operations also include determining that the user interaction is to be consumed by a particular UI component, where the particular UI component is covered by a plurality of UI components configured to allow the user interaction to pass through the particular UI component. The operations further include determining a cumulative opacity of the plurality of UI components in the particular region of the GUI based on the determination that the user interaction is to be consumed by the particular UI component. The operations further include determining that the cumulative opacity exceeds a threshold opacity and, based on the determination that the cumulative opacity exceeds the threshold opacity, preventing the particular UI component from consuming the user interaction.

[0006] A fourth exemplary embodiment includes a system that includes means for detecting a user interaction with a particular region of a GUI. The system also includes means for determining that the user interaction is to be consumed by a particular UI component, where the particular UI component is covered by a plurality of UI components configured to allow the user interaction to pass through the particular UI component. The system further includes means for determining a cumulative opacity of the plurality of UI components in the particular region of the GUI based on the determination that the user interaction is to be consumed by the particular UI component. The system further includes means for determining that the cumulative opacity exceeds a threshold opacity and, based on the determination that the cumulative opacity exceeds the threshold opacity, means for preventing the particular UI component from consuming the user interaction.

[0007] These and other embodiments, aspects, advantages, and alternatives will become apparent to those skilled in the art by reading the following detailed description with appropriate reference to the accompanying drawings. Further, this summary and other descriptions and drawings provided herein are intended to illustrate embodiments by way of example only, and thus, numerous variations are possible. For example, structural elements and process steps may be rearranged, combined, distributed, removed, or otherwise changed while remaining within the scope of the claimed embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

[0009] Detailed Description Exemplary methods, devices, and systems are described herein. It should be understood that the terms "example" and "exemplary" are used herein to mean "an example, instance, or illustration." Any embodiments or features described herein as "example," "exemplary," and / or "illustrative" are not necessarily to be construed as preferred or advantageous over other embodiments or features, unless otherwise specified. Thus, other embodiments may be utilized and other changes may be made without departing from the scope of the subject matter presented herein.

[0010] Accordingly, the exemplary embodiments described herein are not intended to be limiting. It will be readily understood that aspects of the present disclosure, generally described and illustrated in the figures herein, may be arranged, substituted, combined, separated, and designed in a wide variety of different configurations.

[0011] Furthermore, unless the context suggests otherwise, the features shown in each of the figures may be used in combination with one another. Thus, the figures are generally to be regarded as component aspects of one or more overall embodiments, understanding that not all illustrated features are necessary for each embodiment.

[0012] Furthermore, any listing of elements, blocks, or steps in this specification or the claims is for the purpose of clarity only. Thus, such listing should not be construed as requiring or implying that these elements, blocks, or steps adhere to a particular arrangement or are to be performed in a particular order. Unless otherwise noted, the figures are not drawn to scale.

[0013] I. Overview User interaction hijacking attacks, such as tapjacking, may involve displaying a UI component that appears to be the target of the user interaction and that blocks another UI component that will actually consume the user interaction. The blocking UI component may display seemingly innocuous content that is designed to direct the user to interact with a particular part of the GUI, but these interactions may be used to control and / or change the state of the blocked UI component without the user's awareness. Thus, for example, a malicious application may, among other possibilities, trick the user into granting permissions that the user would not otherwise grant to the malicious application, such as permission to use certain hardware of the computing device, permission to monitor the operation of other software applications, and / or permission to execute certain system functions. Additionally or alternatively, a malicious application may, among other possibilities, trick the user into pressing a payment button, taking a photo, and / or making a phone call. In this way, a malicious application can behave in ways that the user did not intend.

[0014] The possibility of such user interaction hijacking attacks can be attributed to the fact that some computing devices and / or operating systems can provide overlapping and / or covering UI components. Some of these UI components can be configured to pass user interactions to underlying UI components while ignoring the user interactions. User interaction hijacking attacks can be particularly problematic when the UI components are not manually movable and / or repositionable by the user. This is because it can prevent the user from noticing that the user interaction is being misdirected. Since this feature of covering some UI components that can ignore user interactions is desirable for implementing certain GUI effects and / or functions, generally, eliminating this feature to prevent user interaction hijacking attacks is not a desirable solution.

[0015] Accordingly, to eliminate and / or mitigate the possibility of a user interaction hijacking attack, a computing device may be configured to block certain user interactions that pass through a UI component that would otherwise consume user interaction and make the UI component effectively invisible to the user. If a particular UI component is covered by a single UI component, the opacity of this single UI component may be compared to a threshold opacity. If the opacity is below the threshold opacity, user interaction may be permitted to pass through the particular UI component and be consumed by the particular UI component. Otherwise, user interaction may be blocked. However, still considering the blocking of UI components individually rather than as a whole may potentially enable a user interaction hijacking attack. For example, each of two different UI components may have respective opacities lower than the threshold opacity. However, when these two different UI components are overlaid on each other (and on top of a particular UI component), the net effect of the layering of the two UI components may still result in the blocking of the particular UI component.

[0016] Thus, rather than comparing the opacity of each UI component independently with a threshold opacity, a computing device may be configured to determine a cumulative opacity associated with a stack of multiple UI components configured to allow user interactions to pass through. Specifically, the opacity of each UI component may be associated with a corresponding transparency (e.g., the opacity and transparency may be considered complementary as their sum can be a predefined value). The cumulative opacity may be based on the product (scaled to the unit interval) of the transparencies associated with the multiple UI components covering a particular UI component that will consume the user interaction. In some cases, since the opacity of a UI component may vary across its entire area, the cumulative opacity may be determined for the area to which the user interaction is directed (e.g., the area of the GUI that the user touches or clicks) and / or based on that area. Thus, the cumulative opacity can quantify the extent to which the area of a particular UI component will be visible to and / or visible in some circumstances to the user.

[0017] In some implementations, the cumulative opacity may be application-independent. Specifically, the cumulative opacity can be determined for a plurality of UI components regardless of whether each UI component of the plurality of UI components was generated by the same software application or by different software applications. If the application-independent cumulative opacity indicates that the user can clearly see the relevant part of a particular UI component (as determined by the threshold opacity), the particular UI component may be permitted to consume user interaction. If the user cannot clearly see the relevant part of a particular UI component, the particular UI component may be blocked from consuming user interaction. In some cases, an overlying UI component generated by the same software application as the particular UI component that will consume user interaction may be included as part of the application-independent cumulative opacity, or in other cases, it may be excluded therefrom.

[0018] In other implementation examples, the cumulative opacity can be based on the cumulative opacity specific to one or more applications. For example, the opacity specific to the first application can be determined based on a first plurality of UI components associated with the first software application, and the opacity specific to the second application can be determined based on a second plurality of UI components associated with the second software application. The first software application and the second software application can be different from a third software application associated with specific UI components expected to consume user interactions. The cumulative opacity can be the higher of the opacity specific to the first application or the opacity specific to the second application. Thus, the application-specific cumulative opacity can quantify whether any single software application can prevent the user from viewing a given area of a specific UI component with sufficient clarity, but whether multiple different software applications can independently stack their respective UI components such that they can occlude the given area of the specific UI component.

[0019] Often, the computing device and / or its operating system can implement the independent operation of these multiple software applications. Therefore, a given software application is likely unaware of the operations being performed by other software applications, and thus it is not possible to deterministically correlate the display of the UI components of a given software application with the UI components of other software applications. Thus, since there is unlikely to be malicious correlation between different software applications, the application-specific cumulative opacity can reduce and / or eliminate the possibility of user interaction hijacking while making it difficult to interfere with the benign operation of the computing device.

[0020] In further implementation examples, some UI components may be excluded from the cumulative opacity calculation. For example, UI components generated by an operating system and / or a clearly trustworthy software application do not contribute to the cumulative opacity. This is because these UI components can be considered trustworthy based on their sources. Additionally, UI components with an opacity equal to zero may be excluded. This is because these UI components are completely transparent and thus do not affect the cumulative opacity. Other exceptions based on other signs of trustworthiness are also possible.

[0021] In some implementation examples, among the operations discussed herein, monitoring of UI components, determination of cumulative opacity, and / or blocking of user interactions can be performed by the operating system of a computing device. For example, these operations can be performed automatically without including an opt-in process and / or without providing an opt-out process to a software application. By automatically providing these operations as part of the operating system, each software application executed by the operating system can be automatically protected from user interaction hijacking attacks. That is, avoidance of user interaction hijacking attacks does not depend on these protections and / or other protections that can be provided by the operating system for each software application that calls them, and thus an intentionally security-protected system can be obtained.

[0022] II. Exemplary Computing Devices and Systems FIG. 1 is a diagram showing an exemplary computing device 100. The computing device 100 is shown in the form factor of a mobile phone. However, the computing device 100 may alternatively be implemented as, among other possibilities, a desktop computer, a laptop computer, a tablet computer, or a wearable computing device (e.g., a watch). The computing device 100 may include a body 102, a display 106, and various elements such as buttons 108 and 110. The computing device 100 may further include one or more cameras such as a front camera 104 and a rear camera 112.

[0023] The front camera 104 may generally be positioned on the side of the body 102 that faces the user during operation (e.g., the same side as the display 106). The rear camera 112 may be positioned on the side of the body 102 opposite the front camera 104. It is optional to refer to the cameras as front camera and rear camera, and the computing device 100 may include a plurality of cameras positioned on various sides of the body 102.

[0024] The display 106 may represent a cathode ray tube (CRT) display, a light emitting diode (LED) display, a liquid crystal (LCD) display, a plasma display, an organic light emitting diode (OLED) display, or other types of displays known in the art. In some examples, the display 106 can serve as a viewfinder for the front camera 104 and / or the rear camera 112. Also, the display 106 can support a touch screen function that enables interaction with aspects of the computing device 100.

[0025] In addition, computing device 100 may include an ambient light sensor that can continuously or occasionally determine the ambient luminance of the environment in which computing device 100 is present. In some implementations, the ambient light sensor can be used to adjust the display luminance of display 106. Further, the ambient light sensor can be used to determine or assist in determining the exposure length of one or more of cameras 104 or 112.

[0026] Figure 2 is a simplified block diagram showing some of the components of an exemplary computing system 200. As an example, computing system 200 may be a cellular mobile phone (e.g., a smartphone), a computer (such as a desktop, laptop, tablet, or handheld computer), a home automation component, a digital video recorder (DVR), a digital television, a remote control, a wearable computing device, a gaming console, a robotic device, a vehicle, or any other type of device, but is not limited thereto. Computing system 200 may represent, for example, aspects of computing device 100. As shown in FIG. 2, computing system 200 may include a communication interface 202, a user interface 204, a processor 206, and a data storage 208, all of which may be communicatively coupled by a system bus, network, or other connection mechanism 210.

[0027] The communication interface 202 may enable the computing system 200 to communicate with other devices, access networks, and / or transfer networks using analog or digital modulation. Therefore, the communication interface 202 can facilitate circuit-switched communication and / or packet-switched communication such as basic telephone service (POTS) communication and / or Internet Protocol (IP) or other packetized communication. For example, the communication interface 202 may include a chipset and an antenna arranged for wireless communication with a wireless access network or an access point. Also, the communication interface 202 may take the form of a wireline interface such as an Ethernet (registered trademark), Universal Serial Bus (USB), or High-Definition Multimedia Interface (HDMI) (registered trademark) port, or may include such a wireline interface. Also, the communication interface 202 may take the form of a wireless interface such as Wi-Fi, BLUETOOTH (registered trademark), Global Positioning System (GPS), or wide area wireless interface (e.g., WiMAX, 3GPP (registered trademark) Long-Term Evolution (LTE), and / or 3GPP 5G), or may include such a wireless interface. However, other forms of physical layer interfaces and other types of standard or proprietary communication protocols may be used on the communication interface 202. Further, the communication interface 202 may include multiple physical communication interfaces (e.g., a Wi-Fi interface, a BLUETOOTH (registered trademark) interface, and a wide area wireless interface).

[0028] The user interface 204 may function to enable the computing system 200 to interact with a human or non - human user, such as receiving input from the user and providing output to the user. Thus, the user interface 204 may include input components such as a keypad, keyboard, touch - sensitive panel, computer mouse, trackball, joystick, microphone, etc. Also, the user interface 204 may include one or more output components such as a display screen, which can be combined with, for example, a touch - sensitive panel. The display screen may be based on CRT, LCD and / or LED technology, or other technologies that are currently known or may be developed later. Also, the user interface 204 may be configured to generate audible output via speakers, speaker jacks, audio output ports, audio output devices, earphones and / or other similar devices. Also, the user interface 204 may be configured to receive and / or capture audible speech, noise and / or signals by a microphone and / or other similar devices.

[0029] In some examples, the user interface 204 may include a display that serves as a viewfinder for still and / or video camera functions supported by the computing system 200. Further, the user interface 204 may include one or more buttons, switches, knobs and / or dials that facilitate the configuration and focusing of the camera function and the capture of images. Some or all of these buttons, switches, knobs and / or dials may be realized by a touch - sensitive panel.

[0030] Processor 206 may include one or more general-purpose processors such as, for example, a microprocessor, and / or one or more special-purpose processors such as, for example, a digital signal processor (DSP), a graphics processing unit (GPU), a floating point unit (FPU), a network processor, or an application-specific integrated circuit (ASIC). In some cases, the special-purpose processor may be capable of, among other possibilities, image processing and / or executing a machine learning model. Data storage 208 may include one or more volatile and / or non-volatile storage components such as magnetic storage, optical storage, flash storage, or organic storage, and may be integrated, in whole or in part, with processor 206. Data storage 208 may include removable components and / or non-removable components.

[0031] Processor 206 may be capable of executing program instructions 218 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 208 to perform the various functions described herein. Thus, data storage 208 may include a non-transitory computer-readable medium storing program instructions that, when executed by computing system 200, cause computing system 200 to perform any of the methods, processes, or operations disclosed herein and / or in the accompanying drawings. By executing program instructions 218 by processor 206, processor 206 can use data 212.

[0032] As an example, the program instructions 218 may include an operating system 222 (e.g., an operating system kernel, a device driver, and / or other modules), and one or more application programs 220 installed in the computing system 200 (e.g., a camera function, an address book, an email, a web browsing, a social networking, an audio-text function, a text translation function, and / or a game application). Similarly, the data 212 may include operating system data 216 and application data 214. The operating system data 216 may be mainly accessible by the operating system 222, and the application data 214 may be mainly accessible by one or more of the application programs 220. The application data 214 may be placed in a file system visible to the user of the computing system 200 or hidden from the user of the computing system 200.

[0033] The application program 220 may communicate with the operating system 222 via one or more application programming interfaces (APIs). These APIs can facilitate, for example, the application program 220 reading and / or writing the application data 214, the application program 220 transmitting or receiving information via the communication interface 202, and the application program 220 receiving information and / or displaying the information on the user interface 204.

[0034] In some cases, the application program 220 may be referred to simply as an "app". Further, the application program 220 may be downloadable to the computing system 200 through one or more online application stores or application markets. However, the application program can also be installed on the computing system 200 in other ways, such as via a web browser or via a physical interface (e.g., USB port) on the computing system 200.

[0035] III. Exemplary Tapjacking Attacks FIG. 3 is a diagram showing an exemplary tapjacking (short for tap hijacking) attack, which includes malicious software application takeover and tricking the user into doing something unintended by leading the user's touch input in the wrong direction. Throughout this specification, tapjacking is used as an example of the general concept of user interaction takeover. Therefore, it should be understood that the exemplary techniques discussed herein are generally applicable in the context of a GUI regardless of the specific modality of interaction with the GUI (e.g., tap, click, press, gaze, etc.).

[0036] Specifically, as shown in FIG. 3, a tapjacking attack may include a malicious application generating a UI component 302 that overlays a UI component 300 generated by another software application and / or an operating system. The UI component 302 may be configured such that user interactions pass through to a layer below the GUI. That is, the UI component 302 may be configured to allow these interactions to pass through to the UI component 300 that consumes these interactions by ignoring the user interactions. The UI component 300 may be configured to consume user interactions. Consumption of a user interaction by a given UI component may include the given UI component receiving the user interaction and responding to the user interaction (e.g., triggering the execution of an operation based on and / or in response to the user interaction). When a given UI component is configured to consume a user interaction, the user interaction does not pass through the given UI component but rather interacts with the given UI component and thus is not available for consumption by other UI components.

[0037] Further, the UI component 302 may be opaque such that the UI component 300 is not visible to the user. Thus, the UI component 302 can function as a facade that makes the UI component 300, with which the user is interacting unbeknownst to the user, invisible to the user. Further, in some cases, the respective positions of the UI components 300 and 302 may be fixed relative to the GUI, such that the user cannot move the UI component 302 to reveal the UI component 300.

[0038] In the example of FIG. 3, the UI component 300 displays and provides a change in permissions associated with the application that generated the UI component 302. Specifically, the UI component 300 includes and provides changes to permissions 308 and permissions 310 - 312 (i.e., permissions 308 - 312), and granting those permissions may enable the application that generated the UI component 302 to, among other possibilities, use the hardware of the computing device on which the application is run (e.g., camera, microphone, etc.), take screenshots using the computing device, monitor keyboard input received by the computing device, and / or call operating system functions. In other cases, the UI component 300 may instead provide, among other possibilities, selection of a "pay" button, taking a photo, and / or making a phone call.

[0039] The UI component 302 may provide content that appears benign, thereby directing the user to interact with the button 304. The button 304 of the UI component 302 covers a toggle switch corresponding to the permission 310, as indicated by the enclosure 314. As such, since the UI component 302 is configured to ignore user interaction, interaction with the button 304 may actually result in granting the permission 310 to the application. The content of the UI component 302 appears benign and the UI component 302 is opaque, so the user may be tricked into interacting with the button 304, and as a result, the permission 310 may be granted to the application without the user's awareness.

[0040] The application may be configured to generate additional content that appears to be benign, is opaque to the user, and secretly obtains permissions that may allow the application to operate in ways the user did not intend, including buttons that cover other permissions among permissions 308 - 312. In other cases, rather than covering UI component 300, UI component 302 may instead cover other UI components that display and provide changes to other system settings and / or other UI components provided by other software applications. Thus, the application that generated UI component 302 may deceive the user into configuring other system settings and / or operating other software applications in ways the user did not intend.

[0041] IV. Exemplary Systems and Operations for Preventing Tapjacking Attacks FIG. 4 shows an exemplary scenario related to preventing tapjacking attacks, where multiple UI components configured to ignore or (instead of consuming) pass user interactions are overlaid on a UI component configured to consume (e.g., receive and react to) user interactions. Specifically, FIG. 4 shows UI components 400, 402, and 404, and user interaction 410. UI components 400, 402, and 404, and other UI components discussed herein may include system alert windows, overlays, toasts, dialog boxes, cards, forms, notifications, sidebars, and / or other UI structures or elements that can be displayed as part of a multi - layer GUI and / or may take such forms.

[0042] UI component 404 covers UI component 402, and UI component 402 covers UI component 400. Each of UI components 404 and 402 can be associated with an attribute that configures each respective UI component to allow user interaction 410 to pass through to UI component 400 by ignoring the user interaction. Thus, user interaction 410 may appear (e.g., to the user) to be directed at UI component 404 as indicated by region 406, but the user interaction is actually consumed by UI component 400 as indicated by region 408.

[0043] If UI component 400 is covered by only one UI component (e.g., only UI component 404), a computing device providing the GUI can be made to allow user interaction to pass through to UI component 400 when the opacity of UI component 404 is below a threshold opacity, and to prevent UI component 400 from consuming the user interaction when the opacity of UI component 404 meets or exceeds the threshold opacity, thereby avoiding a tapjacking attack. That is, when the user can clearly see UI component 400 through UI component 404 well enough, UI component 400 is covered by UI component 404 but the user can interact with UI component 400. The opacity of a UI component (e.g., UI component 404) can represent, for example, the degree and / or extent of blocking by the UI component of visual signals emanating from another underlying UI component (e.g., UI component 400).

[0044] However, an approach that considers UI components individually can allow for stacking multiple UI components that may each have an opacity below a threshold opacity, effectively making the UI components that consume user interaction invisible to the user. Thus, a computing device can be configured to determine whether to block or permit consumption of user interaction by a particular UI component based on the cumulative opacity of the UI components covering the particular UI component. Therefore, in the scenario shown in FIG. 4, the computing device can be configured to determine the cumulative opacity of UI components 404 and 402. If the cumulative opacity of UI components 404 and 402 meets or exceeds the threshold opacity, user interaction 410 can be blocked from being consumed by UI component 400. If the cumulative opacity of UI components 404 and 402 is below the threshold opacity, consumption of user interaction 410 by UI component 400 can be permitted.

[0045] In one example, the cumulative opacity can be calculated according to the formula: θ Z =1-Π i∈I (1 - o i ), where θ Z represents the cumulative opacity for UI component Z, I represents the set of UI components covering UI component Z, and o i (in the range from 0 to 1) represents the opacity of the i-th UI component in set I (excluding UI component Z). The cumulative opacity θ Zis based on UI component i∈I that covers UI component Z, rather than on the opacity of UI component Z itself. This is because the opacity of UI component Z does not affect how extensively UI component Z is blocked by the UI components above it. The cumulative opacity of multiple UI components (e.g., UI components 402 and 404) can represent, for example, the total degree and / or extent of the blocking of visual signals originating from another underlying UI component (e.g., UI component 400) by the multiple UI components.

[0046] Therefore, in the example shown in FIG. 4, the cumulative opacity is θ 400 =1 - ((1 - o 402 )(1 - o 404 )) and can be expressed as such. For example, if each of UI components 404 and 402 has an opacity of 0.7, the cumulative opacity can be equal to (1 - (1 - 0.7)(1 - 0.7)) = 0.91. Thus, if the threshold opacity is 0.8, each of UI components 404 and 402 individually falls below the threshold, but the cumulative opacity of UI components 404 and 402 exceeds the threshold. Therefore, relying on the cumulative opacity can prevent malicious applications from allowing user interaction 410 to pass through to UI component 400 when UI components 404 and 402 are stacked on top of each other and effectively make UI component 400 invisible to the user.

[0047] The opacity of a given UI component and the transparency of a given UI component can serve complementary roles. That is, the opacity of a given UI component and the transparency of a given UI component can be interchangeable because their sum can be a predetermined value (e.g., 1 or 255). Specifically, λ = o i + t i where λ represents a predetermined value (e.g., λ = 1 in the above example), and t irepresents the transparency of the i-th UI component. Therefore, the opacity of the i-th UI component is, for example, determined by how the GUI attributes are represented by the computing device, operating system, and / or software application, o i may be explicitly represented using, or t i may be implicitly represented using.

[0048] Therefore, the maximum opacity (corresponding to the minimum transparency) can be expressed as o i = λ, and the minimum opacity (corresponding to the maximum transparency) can be expressed as o i = 0, the maximum transparency can be expressed as t i = λ, and the minimum transparency can be expressed as t i = 0. Therefore, the transparency of the UI component can represent, for example, the degree and / or extent of transmission of visual signals generated from another underlying UI component by the UI component. The cumulative transparency of multiple UI components can represent, for example, the total degree and / or extent of transmission of visual signals generated from another underlying UI component by the multiple UI components.

[0049] Alternatively, the above formula θ Z = 1 - Π i∈I (1 - o i )(λ = 1) is (i) θ Z = λ - λΠ i∈I ((λ - o i ) / λ), (ii) θ Z = λ - λΠ i∈I (t i / λ), (iii) T Z = λΠ i∈I ((λ - o i ) / λ) or (iv) T Z = λΠ i∈I (t i / λ) and can be expressed as, where T Z represents the cumulative transparency, λ = T Z + θ Z and λ > 0. That is, the cumulative opacity is λΠi∈I ((λ - o i ) / λ) or λΠ i∈I (t i / λ), it can be obtained based on the product of the transparency of each UI component covering the UI component Z. The cumulative opacity may be explicitly expressed based on the difference between a predetermined value and the product as seen in (i) or (ii), or may be implicitly expressed using the cumulative transparency as seen in (iii) or (iv).

[0050] Similarly, the threshold opacity θ THRESHOLD can be expressed as λ = θ THRESHOLD + T THRESHOLD . Therefore, when the cumulative opacity is less than or equal to the threshold opacity (i.e., θ Z ≦ θ THRESHOLD ) or when the cumulative transparency is greater than or equal to the threshold transparency (i.e., T Z ≧ T THRESHOLD ), the consumption of user interaction by the UI component Z can be permitted. When the cumulative opacity is greater than the threshold opacity (i.e., θ Z > θ THRESHOLD ) or when the cumulative transparency is less than the threshold transparency (i.e., T Z < T THRESHOLD ), the consumption of user interaction by the UI component Z can be blocked.

[0051] In some cases, the determination of the cumulative opacity can be specific to the region of the GUI towards which the user interaction 410 is directed. Thus, in the example of FIG. 4, the cumulative opacity can be determined based on (i) the region 406 of the UI component 404 and (ii) the region of the UI component 402 corresponding to the region 406. If a given region is associated with two or more opacity values (e.g., of its different parts), the highest opacity (or the lowest transparency) can be used for the determination of the cumulative opacity. Thus, if the user interaction 410 is provided low along the y-axis of FIG. 4 such that it is outside the area of the UI component 404, the cumulative opacity is based on the UI component 402 but not on the UI component 404. By considering the opacity of the specific region towards which the interaction is directed rather than, for example, the maximum opacity of the entire given UI component, the frequency with which user interactions are blocked in some non-malicious situations can be reduced.

[0052] The size of the specific region used for the determination of the cumulative opacity can be selected based on (i) the frequency of user interaction blocking in non-malicious cases and / or (ii) the success rate of blocking malicious attacks. The size of the specific region can be based on the position of the user interaction (e.g., center, centroid, etc.), the size and / or area of the user interaction (e.g., the area covered by the user's touch input), and / or a buffer zone provided around the position and / or area of the user interaction. In one example, the size of the specific region can be equal to the sum of the area of the user interaction and the buffer zone, defined by adding a predetermined number of pixels at all points along the outer perimeter of the area of the user interaction. If the determination of the cumulative opacity is not specific to the region of the GUI towards which the user interaction 410 is directed, but instead is based on the opacity of each entire UI component, the highest opacity (or the lowest transparency) of each UI component can be used for the determination of the cumulative opacity.

[0053] In some implementations, the cumulative opacity may be application-independent. That is, a single cumulative opacity may be determined for UI component 400 regardless of whether UI components 402 and 404 are generated by the same software application or different software applications. By using such application-independent cumulative opacity, when the user is permitted for user interaction 410 to pass through UI component 400 and be consumed by UI component 400, the user can view the content of UI component 400 with sufficient clarity (at least in region 408). Thus, if the content of UI component 400 (at least in region 408) is not sufficiently visible to the user (as determined by comparing the application-independent cumulative opacity with a threshold opacity), user interaction 410 may be blocked from passing through UI component 400 and being consumed by UI component 400.

[0054] In particular, in some cases, a given UI component (e.g., 402) may be excluded from the calculation of cumulative opacity if it is generated by the same application as the UI component (e.g., 400) configured to consume the user interaction. This would be the case because it is unlikely that a software application would generate a UI component that intends to obscure its own content or a UI component that contributes to the obscuring of its own content. Thus, the application-independent cumulative opacity may be independent of the relationship between the covering UI components (e.g., the applications that generate these UI components), yet still be able to take into account how related the underlying UI component consuming the user interaction is to the covering UI components.

[0055] FIG. 5 is a diagram illustrating another exemplary scenario related to preventing tapjacking attacks, where a plurality of UI components configured to ignore or pass through user interactions are overlaid on top of a UI component configured to consume user interactions. Specifically, FIG. 5 shows a first plurality of UI components including UI components 502 and 502 (shown in a hatched pattern) generated by a first software application, and a second plurality of UI components including UI components 504, 508, and 510 (shown in a dotted pattern) generated by a second software application different from the first software application. A user interaction 516 that appears to target region 512 of UI component 510 can actually pass through to UI component 500 as indicated by region 514. The first and second software applications can be configured to operate independently of each other, and since each application may not be aware of the operation of the other application, it is unlikely that these two applications will cooperate to perform malicious operations.

[0056] Thus, in some cases, the cumulative opacity can be determined in an application-specific manner. That is, the cumulative opacity specific to the first application can be determined for the first software application based on UI components 502 and 506, and the cumulative opacity specific to the second application can be determined for the second software application based on UI components 504, 508, and 510. The overall cumulative opacity can be based on the cumulative opacity specific to the first application and / or the cumulative opacity specific to the second application. For example, the overall cumulative opacity can be expressed as follows.

[0057]

Equation

[0058] In the formula, u represents the identifier of a specific software application in the set {uids} of identifiers of each of a plurality of software applications in a state where the UI component covers the UI component Z. That is, the overall cumulative opacity can be based on (i) the maximum application-specific opacity among the plurality of application-specific cumulative opacities associated with the plurality of software applications, and / or (ii) the minimum application-specific transparency among the plurality of application-specific cumulative transparencies associated with the plurality of software applications.

[0059] In the example of FIG. 5, the overall cumulative opacity is (i) θ 500 = max{λ - λ((λ - o 502 ) / λ)((λ - o 506 ) / λ), λ - λ((λ - o 504 ) / λ)((λ - o 508 ) / λ)((λ - o 510 ) / λ)}, (ii) θ 500 = max{λ - λ(t 502 / λ)(t 506 / λ), λ - λ(t 504 / λ)(t 508 / λ)(t 510 / λ)}, (iii) T 500 = min{λ((λ - o 502 ) / λ)((λ - o 506 ) / λ), λ((λ - o 504 ) / λ)((λ - o 508 ) / λ)((λ - o 510 ) / λ)}, and / or (iv) T 500 = min{λ(t 502 / λ)(t 506 / λ), λ(t 504 / λ)(t 508 / λ)(t 510It can be evaluated according to {...} / λ). Therefore, if (i) the application-specific cumulative opacity of UI components 502 and 506 and / or (ii) the maximum value of the application-specific cumulative opacity of UI components 504, 508, and 510 exceeds the threshold opacity, user interaction 516 can be blocked by UI component 500.

[0060] In some cases, as a result of this configuration, when the consumption of user interaction 516 by UI component 500 is permitted, the user may not be able to see (e.g., with sufficient clarity and / or at all) the area 514 of UI component 500 corresponding to user interaction 516. Nevertheless, this occlusion of area 514 is likely to be the result of the accidental, non-malicious, and / or unrelated operation of at least two different software applications, none of which is the sole independent cause of the occlusion.

[0061] Specifically, the exploitation of the operation of a non-malicious application by a malicious application may include the malicious software application determining what the non-malicious application is doing. For example, the malicious software application may attempt to determine whether the non-malicious application is displaying a UI component that, when combined with further other malicious UI components, would completely occlude the UI component targeted for attack. However, without special permission (which is generally not given by default), many computing devices and / or operating systems do not permit software applications to monitor each other's execution and / or operation. Therefore, it is likely that a malicious application will not be able to monitor the operation of other applications and thus will not be able to exploit the independent operation of non-malicious software applications.

[0062] Further collusion between two malicious applications may include the user installing the two malicious applications and / or granting permission to a first malicious application among the malicious applications that would enable the installation of yet other malicious applications. It is unlikely that the user will install two malicious applications configured to cooperate with each other. Further, the operating system provides safeguards (e.g., permission dialogs) for applications that are granted permission to install other applications. Therefore, it is also unlikely that the first malicious application will install yet other malicious applications, and the techniques discussed herein further reduce the likelihood.

[0063] Accordingly, the computing device assumes that accidental and / or uncorrelated obstructions are safe, and thus, user interaction 516 may be permitted to pass through UI component 500 and be consumed by UI component 500 as long as there are no applications generating UI components that exceed the threshold opacity. By using application-specific cumulative opacity instead of application-independent cumulative opacity, the number of user interactions blocked in response to benign obstructions can be reduced. Thus, application-specific cumulative opacity can provide additional flexibility while incurring a non-significant sacrifice in terms of the security of the computing device, if any.

[0064] For example, a non-malicious software application can be configured to change the color scheme of a GUI (e.g., to control the wavelength of light emitted by the GUI) by displaying a transparent UI component colored over the entire area of the GUI. Under application-independent calculations of cumulative opacity, this non-malicious application will effectively reduce the extent to which other software applications can occlude a given UI component before receiving user input is blocked. On the other hand, under application-specific calculations of cumulative opacity, this non-malicious application will not affect the extent to which other software applications can occlude a given UI component before receiving user input is blocked.

[0065] In some cases, both application-specific and application-independent cumulative opacity may be calculated and used simultaneously. For example, a response to application-specific cumulative opacity exceeding a threshold opacity may include preventing consumption of user interaction, and a response to application-independent cumulative opacity exceeding the threshold opacity may include displaying a prompt. This prompt may (i) indicate that user interaction is about to be consumed by a UI component that is not sufficiently visible to the user, and (ii) explicitly request permission to allow consumption of the user interaction. Such an approach provides both enhanced security and flexibility, and can allow user interaction to pass through UI components that are completely obscured in some situations.

[0066] FIG. 6 is a diagram showing an exemplary system configured to filter user interactions based on the opacity of UI components. Specifically, system 660 includes an overlap calculator 632, a touch filter 636, a type filter 640, an accumulated opacity calculator 644, and a user interaction filter 650. System 660 may form part of an operating system of a computing device (e.g., operating system 222) and / or may be a stand-alone component provided as part of a computing device. The components of system 660 may be implemented as hardware, software, and / or combinations thereof that form part of computing device 100 and / or computing system 200. In some implementations, the order of operation of overlap calculator 632, touch filter 636, and / or type filter 640 may be different from that shown.

[0067] System 660 may be configured to receive, as input, attributes that define UI components 600 - 620 (i.e., UI components 600 - 620) and / or attributes associated with UI components 600 - 620 and user interactions 612 that define and / or are associated with user interactions 612. Based on these attributes, system 660 may be configured to determine whether user interaction 612 is passed or blocked to UI components configured to consume user interactions, as shown by enclosure 652.

[0068] The UI component 600 can be associated with, among a number of possible attributes, an application identifier 602, a type 604, an opacity 606, a position 608, and a touchability 610. The application identifier 602 can indicate the software application that generated and / or requested the generation of the UI component 600. The type 604 can be one of a predetermined number of possible types or classes of the UI component 600. The opacity 606 can include one or more opacities (which may be expressed in terms of transparency) corresponding to different regions of the UI component 600. The opacity of a given region can represent the highest opacity present in that region (i.e., some parts of the region may have a lower opacity), or the corresponding lowest transparency present in that region (i.e., some parts of the region may have a higher transparency).

[0069] The position 608 can include the vertical position of the UI component 600 (e.g., along the z - axis as shown in FIGS. 4 and 5) and / or the horizontal area of the UI component 600 (e.g., along the x - y plane as shown in FIGS. 4 and 5). The touchability 610 can indicate whether the UI component 600 is configured to consume user interaction or is configured to allow user interaction to pass through to underlying layers (e.g., layers with lower z - axis values) by ignoring the user interaction. Similarly, the UI component 620 can be associated with, among a number of possibilities, an application identifier 622, a type 624, an opacity 626, a position 628, and a touchability 630.

[0070] User interaction 612 may be associated with a GUI area 614 that the user interaction 612 covers. For example, the GUI area 614 may be a subset of the x-y plane shown in FIGS. 4 and 5 and may be determined based on user touches, clicks, and / or other forms of interaction with the GUI. For example, the GUI area 614 may correspond to area 406 and / or area 408 in FIG. 4, and / or area 512 and / or area 514 in FIG. 5.

[0071] The overlap calculation unit 632 may be configured to identify overlapping UI components 634 based on the UI components 600-620 and the user interaction 612. Specifically, the overlapping UI components 634 may include a subset of the UI components 600-620, and each UI component in this subset is associated with a horizontal area that overlaps with the GUI area 614. For example, the overlap calculation unit 632 may be configured to determine whether the UI component 600 overlaps with the GUI area 614 based on the horizontal component of the position 608. The overlapping UI components 634 may include UI components configured to ignore the user interaction 612 and UI components configured to consume the user interaction 612. The overlapping UI components 634 may not include UI components that do not overlap and / or are non-overlapping with the user interaction 612, and thus may not be considered part of the calculation of the cumulative opacity.

[0072] The touch filter 636 can be configured to determine untouchable UI components 638 based on overlapping UI components 634 and their attributes. Specifically, the untouchable UI components 638 can include a subset of the overlapping UI components 634, and each UI component in this subset is configured to ignore, pass through, and / or not consume in other ways user interactions 612. For example, the touch filter 636 can be configured to determine whether a UI component 600 is touchable based on touchability 610.

[0073] The type filter 640 can be configured to determine untrustworthy UI components 642 based on untouchable UI components 638 and their attributes. Specifically, the untrustworthy UI components 642 can include a subset of the untouchable UI components 638, and each UI component in this subset is a component of a type classified as untrustworthy. For example, the type filter 640 can be configured to determine whether a UI component 600 is untrustworthy based on type 604.

[0074] Type 604 can take on a plurality of different pre-defined values. Some of these values are classified as reliable and / or considered to be reliable, while others are classified as unreliable and / or considered to be unreliable. For example, a UI component generated by an operating system can be associated with a first type that is classified as reliable and thus may not be considered part of the calculation of the cumulative opacity. In another example, a UI component generated by a software application that has been explicitly given high permission (e.g., by a user or the operating system) can be associated with a second type that is classified as reliable and thus may not be considered part of the calculation of the cumulative opacity. On the other hand, a UI component generated by a software application that has not been explicitly given high permission can be associated with a third type that is classified as unreliable and thus may be considered part of the calculation of the cumulative opacity.

[0075] The cumulative opacity calculation unit 644 may be configured to determine the cumulative opacity 646 based on the untrusted UI components 642, the overlapping UI components 634, and their attributes. Specifically, the cumulative opacity calculation unit 644 may be configured to select at least one UI component configured to consume the user interaction 612 based on the overlapping UI components 634. Since the overlapping UI components 634 depend on the GUI area 614 associated with the user interaction 612, at least one UI component configured to consume the user interaction 612 is also based on the GUI area 614. For example, the cumulative opacity calculation unit 644 may be configured to select the topmost UI component from the overlapping UI components 634, and the topmost UI component is configured to consume the user interaction 612 (which will be ignored by all the covering UI components) because it is touchable, and (ii) is associated with an application identifier different from at least one of the untrusted UI components 642 (since it is unlikely that a given application will attempt to attack itself).

[0076] In the case of the UI component 600, the cumulative opacity calculation unit 644 may be configured to determine whether the UI component 600 is the topmost touchable component based on the position 608 and the touchability 610. The cumulative opacity calculation unit 644 may be configured to determine whether the application identifier of the UI component 600 is different from at least one other application identifier of the untrusted UI components 642 based on the application identifier 602 and the corresponding application identifier of the untrusted UI components 642. In some cases, the selection of the topmost touchable component may instead be performed by the touchability filter 636.

[0077] The cumulative opacity calculation unit 644 may be configured to determine a cumulative opacity 646 for the topmost touchable UI component using any of the techniques discussed herein. Specifically, the cumulative opacity calculation unit 644 may be configured to determine a cumulative opacity 646 for the topmost touchable UI component based on a subset of the untrusted UI components 642, where each UI component in the subset has an application identifier different from the application identifier of the topmost touchable UI component.

[0078] In one example, the untrusted UI components 642 may include the UI components 404 and 402 of FIG. 4, and the cumulative opacity 646 may be an application-independent opacity. In another example, the untrusted UI components 642 may include the UI components 502, 504, 506, 508, and 510 of FIG. 5, and the cumulative opacity 646 may be an application-specific opacity.

[0079] The user interaction filter 650 may be configured to determine whether to pass or block the user interaction 612 based on the cumulative opacity 646 and the threshold opacity 648, and each of the cumulative opacity 646 and the threshold opacity 648 can be expressed in terms of opacity and / or transparency. Specifically, the user interaction filter 650 may be configured to block the user interaction 612 when the cumulative opacity 646 exceeds the threshold opacity 648 (or when the corresponding cumulative transparency is less than the corresponding threshold transparency). In some implementations, based on and / or in response to the user interaction filter 650 blocking the user interaction 612, the system 660 may be configured to additionally or alternatively take other protective measures. For example, the system 660 may generate a prompt indicating that the user interaction 612 has been blocked, and / or a prompt asking whether to discard the user interaction 612 or allow the user interaction 612 to be consumed.

[0080] In some implementations, the system 660 may be configured to track how often a given software application generates UI components that cause and / or contribute to the blocking of user interactions. These tracked frequencies may be aggregated across multiple devices (e.g., by an application store operator), and these tracked frequencies can be used to identify one or more applications that generate UI components that cause and / or contribute to the blocking of user interactions more frequently than other applications. Thus, when used across multiple computing devices, the data generated by the system 660 can further be used to identify software applications that may be malicious.

[0081] V. Further Exemplary Operations FIG. 7 is a flowchart of operations related to preventing tapjacking attacks. These operations may be performed, among other possibilities, by computing device 100, computing system 200, and / or system 660. The embodiment of FIG. 7 may be simplified by removing any one or more of the features shown herein. Further, these embodiments may be combined with any of the previous drawings, or with the features, aspects, and / or implementations described herein.

[0082] Block 700 includes detecting user interaction with a particular region of the GUI.

[0083] Block 702 includes determining that the user interaction is to be consumed by a particular UI component, where the particular UI component is covered by a plurality of UI components configured to allow the user interaction to pass through to the particular UI component.

[0084] Block 704 includes determining the cumulative opacity of a plurality of UI components in a particular region of the GUI based on the determination that the user interaction is to be consumed by a particular UI component.

[0085] Block 706 includes determining that the cumulative opacity exceeds a threshold opacity. Block 708 includes preventing the particular UI component from consuming the user interaction based on the determination that the cumulative opacity exceeds the threshold opacity.

[0086] In some embodiments, each of the plurality of UI components may be associated with a corresponding opacity of the specific region of the GUI. The step of determining the cumulative opacity may include determining the cumulative opacity of the plurality of UI components in the specific region of the GUI based on the corresponding opacity of each of the UI components.

[0087] In some embodiments, the cumulative opacity of the plurality of UI components in the specific region of the GUI may be different from the corresponding opacity of each of the UI components.

[0088] In some embodiments, the corresponding opacity of each of the UI components may indicate the maximum opacity of the content of each of the UI components in the specific region of the GUI.

[0089] In some embodiments, the plurality of UI components may include (i) a first plurality of UI components generated by a first software application and (ii) a second plurality of UI components generated by a second software application. The step of determining the cumulative opacity of the plurality of UI components may include (i) determining a first application-specific cumulative opacity of the first plurality of UI components in the specific region of the GUI and (ii) determining a second application-specific cumulative opacity of the second plurality of UI components in the specific region of the GUI. The step of determining that the cumulative opacity exceeds the threshold opacity may include determining that at least one of the first application-specific cumulative opacity or the second application-specific cumulative opacity exceeds the threshold opacity.

[0090] In some embodiments, the plurality of UI components may include (i) a first plurality of UI components generated by a first software application and (ii) a second plurality of UI components generated by a second software application. The step of determining the cumulative opacity of the plurality of UI components may include the step of determining the application-independent cumulative opacity of the first plurality of UI components and the second plurality of UI components in the specific region of the GUI. The step of determining that the cumulative opacity exceeds the threshold opacity may include the step of determining that the application-independent cumulative opacity exceeds the threshold opacity.

[0091] In some embodiments, the specific UI component may be generated by a third software application different from the first software application and the second software application.

[0092] In some embodiments, the corresponding opacity may be from a value of zero to a predetermined value. The value of zero may represent the minimum opacity, and the predetermined value may represent the maximum opacity. The step of determining the cumulative opacity may include, for each respective UI component, the step of determining the corresponding transparency based on the difference between (i) the predetermined value and (ii) the corresponding opacity of the respective UI component. Also, the step of determining the cumulative opacity may include the step of determining the product of the corresponding transparencies of the plurality of UI components and the step of determining the cumulative opacity based on the product.

[0093] In some embodiments, the corresponding opacity may be expressed as the corresponding transparency from a value of zero to a predetermined value. The value of zero may represent the minimum transparency, and the predetermined value may represent the maximum transparency. The step of determining the cumulative opacity may include the step of determining the product of the corresponding transparencies of the plurality of UI components and the step of determining the cumulative opacity based on the product.

[0094] In some embodiments, the step of determining the cumulative opacity based on the product may include the step of determining the difference between (i) the predetermined value and (ii) the product and the step of determining the cumulative opacity based on the difference.

[0095] In some embodiments, the cumulative opacity may be expressed as cumulative transparency. The threshold opacity may be expressed as threshold transparency. The step of determining that the cumulative opacity exceeds the threshold opacity may include the step of determining that the cumulative transparency is less than the threshold transparency.

[0096] In some embodiments, the step of determining that the user interaction is to be consumed by the specific UI component may include the step of determining that each of the plurality of UI components is associated with an attribute, where the attribute indicates that each of the respective UI components is configured to allow the user interaction to pass through to a layer of the GUI below the respective UI component, ignoring the user interaction.

[0097] In some embodiments, the user interaction may include touching a specific area of the GUI via a touch interface of a mobile computing device.

[0098] In some embodiments, each of (i) the step of detecting the user interaction, (ii) the step of determining that the user interaction is to be consumed by the specific UI component, (iii) the step of determining the cumulative opacity, (iv) the step of determining that the cumulative opacity exceeds the threshold opacity, and (iv) the step of preventing the specific UI component from consuming the user interaction can be executed by the operating system of the computing device independently of a software application executed on the computing device.

[0099] In some embodiments, the GUI can be provided by the operating system of a mobile computing device. Each of the specific UI component and the plurality of UI components can be generated by the operating system of the mobile computing device in response to respective requests from corresponding software applications executed by the mobile computing device.

[0100] In some embodiments, for each respective UI component of the plurality of UI components, it can be determined that the type of the respective UI component is one of a predetermined number of types of UI components. Further, based on the determination that the type of the respective UI component is one of the predetermined number of types of UI components, for each respective UI component, it can be determined that the respective UI component is untrustworthy. Based further on the determination that each respective UI component is untrustworthy, the cumulative opacity of the plurality of UI components can be determined.

[0101] In some embodiments, the GUI may include a horizontal area and a plurality of vertically stacked layers. The specific area of the GUI may include a subset of the horizontal area. Each respective UI component of the plurality of UI components may be disposed in a corresponding one of the plurality of vertically stacked layers.

[0102] In some embodiments, Each of the plurality of UI components is within a fixed area of the GUI such that each respective UI component is not repositionable by user interaction a thereof.

[0103] VI. Conclusion The present disclosure should not be limited to the specific embodiments described herein, and these specific embodiments are for the purpose of explaining various aspects. As will be apparent to those skilled in the art, numerous changes and modifications are possible without departing from the scope of the present disclosure. In addition to what is described herein, methods and apparatuses that are functionally equivalent within the scope of the present disclosure will be apparent to those skilled in the art from the above description. Such changes and modifications are also intended to be within the scope of the appended claims.

[0104] In the foregoing detailed description, various features and operations of the disclosed systems, devices, and methods have been described with reference to the accompanying drawings. In these drawings, unless the context dictates otherwise, like reference numerals typically identify like components. The exemplary embodiments described herein and in the drawings are not intended to be limiting. It is also possible to utilize other embodiments and make other changes without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure generally described herein and shown in the drawings can be arranged, substituted, combined, separated, and designed in a variety of different configurations.

[0105] For any or all of the message flow diagrams, scenarios, and flowcharts shown in the drawings and described herein, each step, block, and / or communication may represent the processing of information and / or the transmission of information according to an exemplary embodiment. Alternative embodiments are also included within the scope of these exemplary embodiments. In these alternative embodiments, for example, operations described as steps, blocks, transmissions, communications, requests, responses, and / or messages may be performed in an order different from that illustrated or described, depending on the functions involved (including being performed substantially simultaneously or in the reverse order). Further, the blocks and / or operations used in any of the message flow diagrams, scenarios, and flowcharts described herein may be more or fewer. Also, these message flow diagrams, scenarios, and flowcharts may be combined with each other, in part or in whole.

[0106] A step or block representing the processing of information may correspond to a circuit configured to perform a specific logic function of the method or technique described herein. Alternatively or additionally, a block representing the processing of information may correspond to a part of a module, segment, or program code (including related data). The program code may include one or more instructions executable by a processor to implement a specific logical operation or action in the above method or technique. The program code and / or related data may be stored in any type of computer-readable medium, such as a random access memory (RAM), a disk drive, a solid-state drive, or other storage devices including storage media.

[0107] A computer-readable medium may include a non-transitory computer-readable medium, such as a register memory, a processor cache, and a RAM, which are computer-readable media for storing short-term data. The computer-readable medium may also include a non-transitory computer-readable medium for storing program code and / or data for a longer period. Thus, the computer-readable medium may include, for example, secondary or persistent long-term storage devices such as read-only memory (ROM), optical or magnetic disks, solid-state drives, and compact-disc read-only memory (CD-ROM). The computer-readable medium may be other volatile or non-volatile memory systems. The computer-readable medium may be regarded as, for example, a computer-readable storage medium or a tangible storage device.

[0108] Furthermore, one or more steps or blocks representing information transmission may correspond to information transmission between software modules and / or between hardware modules of the same physical device. However, other information transmissions may be performed between software modules and / or between hardware modules of different physical devices.

[0109] The specific arrangements shown in the drawings should not be regarded as limiting. It should be understood that other embodiments may include more or fewer of each element shown in a given drawing. Furthermore, some of the elements shown may be combined or omitted. Additionally, exemplary embodiments may include elements not shown in the drawings.

[0110] Although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are presented for purposes of illustration and not of limitation. Its true scope is indicated by the following claims.

Claims

1. A method implemented by a computer, comprising: detecting a user interaction with a specific area of a graphical user interface (GUI); determining that the user interaction is to be consumed by a specific user interface (UI) component, the specific UI component being covered by a plurality of UI components configured to allow the user interaction to pass through the specific UI component; and the method implemented by the computer further comprises: determining a cumulative opacity of the plurality of UI components in the specific area of the GUI based on the determination that the user interaction is to be consumed by the specific UI component; determining that the cumulative opacity exceeds a threshold opacity; and based on the determination that the cumulative opacity exceeds the threshold opacity, blocking the specific UI component from consuming the user interaction.

2. Each of the plurality of UI components is associated with a corresponding opacity of the specific area of the GUI, The method implemented by the computer according to claim 1, wherein the step of determining the cumulative opacity includes determining the cumulative opacity of the plurality of UI components in the specific area of the GUI based on the corresponding opacity of each of the UI components.

3. The method implemented by the computer according to claim 2, wherein the cumulative opacity of the plurality of UI components in the specific area of the GUI is different from the corresponding opacity of each of the UI components.

4. The method implemented by the computer according to any one of claims 2 to 3, wherein the corresponding opacity of each of the UI components indicates a maximum opacity of the content of each of the UI components in the specific area of the GUI.

5. The plurality of UI components include (i) a first plurality of UI components generated by a first software application and (ii) a second plurality of UI components generated by a second software application. The step of determining the cumulative opacity of the plurality of UI components includes (i) a step of determining a first application-specific cumulative opacity of the first plurality of UI components in the specific region of the GUI and (ii) a step of determining a second application-specific cumulative opacity of the second plurality of UI components in the specific region of the GUI. The step of determining that the cumulative opacity exceeds the threshold opacity includes the step of determining that at least one of the first application-specific cumulative opacity or the second application-specific cumulative opacity exceeds the threshold opacity. The method realized by a computer according to any one of claims 2 to 4.

6. The plurality of UI components include (i) a first plurality of UI components generated by a first software application and (ii) a second plurality of UI components generated by a second software application. The step of determining the cumulative opacity of the plurality of UI components includes the step of determining an application-independent cumulative opacity of the first plurality of UI components and the second plurality of UI components in the specific region of the GUI. The step of determining that the cumulative opacity exceeds the threshold opacity includes the step of determining that the application-independent cumulative opacity exceeds the threshold opacity. The method realized by a computer according to any one of claims 2 to 4.

7. The specific UI component is generated by a third software application different from the first software application and the second software application. The method realized by a computer according to any one of claims 5 to 6.

8. The corresponding opacity ranges from a value of zero to a predetermined value, where the value of zero represents the minimum opacity and the predetermined value represents the maximum opacity, and the step of determining the cumulative opacity comprises for each respective UI component, determining a corresponding transparency based on the difference between (i) the predetermined value and (ii) the corresponding opacity of the respective UI component determining the product of the corresponding transparencies of the plurality of UI components determining the cumulative opacity based on the product, the method realized by a computer according to any one of claims 2 to 7.

9. The corresponding opacity is represented as a corresponding transparency from a value of zero to a predetermined value, where the value of zero represents the minimum transparency and the predetermined value represents the maximum transparency, and the step of determining the cumulative opacity comprises determining the product of the corresponding transparencies of the plurality of UI components determining the cumulative opacity based on the product, the method realized by a computer according to any one of claims 2 to 7.

10. The step of determining the cumulative opacity based on the product comprises determining the difference between (i) the predetermined value and (ii) the product determining the cumulative opacity based on the difference, the method realized by a computer according to any one of claims 8 to 9.

11. The cumulative opacity is represented as a cumulative transparency, the threshold opacity is represented as a threshold transparency, and the step of determining that the cumulative opacity exceeds the threshold opacity comprises determining that the cumulative transparency is less than the threshold transparency, the method realized by a computer according to any one of claims 1 to 10.

12. The step of determining that the user interaction is to be consumed by the specific UI component comprises including a step of determining that each of the plurality of UI components is associated with an attribute, the attribute indicating that each of the UI components is configured to allow the user interaction to pass through to a layer of the GUI below the respective UI component while ignoring the user interaction, the method realized by a computer according to any one of claims 1 to 11.

13. The method realized by a computer according to any one of claims 1 to 12, wherein the user interaction includes touching the specific area of the GUI via a touch interface of a mobile computing device.

14. Each of: (i) a step of detecting the user interaction; (ii) a step of determining that the user interaction is to be consumed by the specific UI component; (iii) a step of determining the cumulative opacity; (iv) a step of determining that the cumulative opacity exceeds the threshold opacity; and (v) a step of preventing the specific UI component from consuming the user interaction is executed by an operating system of the computing device independently of a software application executed on the computing device, the method realized by a computer according to any one of claims 1 to 13.

15. The GUI is provided by an operating system of a mobile computing device, and each of the specific UI component and the plurality of UI components is generated by the operating system of the mobile computing device in response to respective requests from corresponding software applications executed by the mobile computing device, the method realized by a computer according to any one of claims 1 to 14.

16. For each of the plurality of UI components, a step of determining that the type of each of the UI components is one of a predetermined number of types of UI components; Based on the determination that the type of each of the UI components is one of the types of the predetermined number of UI components, for each respective UI component, a step of determining that each respective UI component is unreliable; The method realized by a computer according to any one of claims 1 to 15, further comprising a step of determining the cumulative opacity of the plurality of UI components based on the further determination that each respective UI component is unreliable.

17. The GUI includes a horizontal area and a plurality of vertically stacked layers, the specific area of the GUI includes a subset of the horizontal area, and each respective UI component of the plurality of UI components is arranged in a corresponding layer among the plurality of vertically stacked layers. The method realized by a computer according to any one of claims 1 to 16.

18. The method realized by a computer according to any one of claims 1 to 17, wherein each respective UI component of the plurality of UI components is within a fixed area of the GUI such that each respective UI component is not repositionable by user interaction.

19. A system, a processor; a non-transitory computer-readable medium storing instructions that, when executed by the processor, cause the processor to execute the method according to any one of claims 1 to 18.

20. A computer program that, when executed by a computing device, causes the computing device to execute the method according to any one of claims 1 to 18.

Images