Systems and methods for user interface adjustment, customization, and placement

Abstract

According to a particular embodiment, an interactive display system includes: (1) one or more interactive display devices, which may include, for example, any suitable interactive display device; and (2) one or more connected computing devices operatively coupled to the one or more interactive display devices (e.g., via any suitable wireless and / or wired connection). In some embodiments, the one or more interactive display devices are configured to display data (e.g., video data, image data, etc.) from the one or more connected computing devices on at least a portion of a display screen of the one or more interactive display devices. In a particular embodiment, the one or more interactive display devices are configured to modify a user interface location based on one or more user characteristics.

Description

Background Art

[0001] Using an interactive display or other display devices, especially when using multiple connected devices through a single display, can be troublesome. Additionally, using a particularly large display screen for demonstration and / or teaching purposes may cause physical fatigue due to the size of the display screen. Therefore, there is a need for systems and methods that address these difficulties in addition to other deficiencies found in existing technology systems. Summary of the Invention

[0002] According to various embodiments, an interactive display system includes: (1) a display device having at least one display processor and a display screen; (2) one or more connected computing devices operatively connected to the display device, each of the one or more connected computing devices having a corresponding connected computing device processor; and (3) a user interface running on the display device, configured to display a list of software applications available on each of the one or more connected computing devices. In some embodiments, the at least one display processor is configured to: (1) receive a list of software applications available on each of the one or more connected computing devices; (2) display the list of software applications on the display device; (3) receive, via an input provided to the display device, a selection of a particular software application from the list of software applications; (4) in response to receiving the selection of the particular software application: (A) identify a particular connected computing device associated with the particular software application; (B) cause the particular connected computing device to run the particular software application; and (C) while the particular software application is running on the connected computing device, display the particular software application on at least a portion of the display screen.

[0003] In some embodiments, the at least one display processor is further configured to: (A) receive a touch input at a first location on an interactive touch screen display input; (B) determine, at least in part based on the first location, a final destination of the touch input; (C) in response to determining that the final destination is the interactive touch screen display device, process the touch input; and (D) in response to determining that the final destination is a particular connected computing device, route the touch input from the interactive touch screen display device to the particular connected computing device via the interactive touch screen display input for processing by the corresponding connected computing device processor.

[0004] In various embodiments, a computer-implemented data processing method for providing a unified list of software applications available on a connected computing device operatively connected to a first computing device includes: (1) a connected computing device including a connected computing device processor; and (2) a first computing device including a first computing device processor and a display screen. In various embodiments, the method includes: (1) receiving, by the first computing device processor, a list of one or more software applications available on the connected computing device; (2) displaying, by the first computing device processor, the list of the one or more software applications on the display screen; (3) receiving, by the first computing device processor, a selection of a first software application among the one or more software applications; (4) causing, by the first computing device processor, the connected computing device processor to open the first software application on the connected computing device; and (5) while the first software application is opened on the connected computing device, causing, by the first computing device processor, at least a portion of the display screen to display the first software application.

[0005] In some embodiments, the method further includes: (1) providing a first software compilation application for installation on the first computing device; (2) providing a second software application compilation application for installation on the connected computing device; and (3) transferring a list of one or more software applications available on the connected computing device from the second software compilation application to the first software compilation application. In still other embodiments, the method further includes: (1) receiving, from a remote computing device, a request to project the display of the remote computing device onto at least a portion of the display screen; and (2) in response to receiving the request to project the display of the remote computing device onto at least a portion of the display screen, causing, by the first computing device processor, at least a portion of the display screen to project the display of the remote computing device.

[0006] In some embodiments, a computer-implemented data processing method for providing a unified list of software applications available on a plurality of connected computing devices operatively connected to an interactive touchscreen display device includes: (1) receiving, by one or more processors, a list of one or more first software applications available on a first connected computing device; (2) receiving, by the one or more processors, a list of one or more second software applications available on a second connected computing device; (3) aggregating, by the one or more processors, the list of one or more first software applications and the list of one or more second software applications into an aggregated list of available software applications; (4) displaying, by the one or more processors, the aggregated list of available software applications on a touch input display screen; (5) receiving, by the one or more processors, a selection of a particular software application from the aggregated list of available software applications; (6) determining, by the one or more processors, on which of the plurality of connected computing devices the particular software application is available; (7) in response to determining that the particular software application is available on the first connected computing device, causing, by the one or more processors, the first connected computing device to open the particular software application on the first connected computing device; and (8) in response to determining that the particular software application is available on the first connected computing device, causing, by the first computing device processor, at least a portion of the touch input display screen to display the particular software application while the particular software application is being opened on the first connected computing device.

[0007] In various embodiments, the interactive touchscreen display device is a first interactive touchscreen display device disposed in a first room of a building, the first interactive touchscreen display device including a first microphone; and the method further includes providing a second interactive touchscreen display device including one or more processors, the second interactive touchscreen display device being disposed in a second room of the building and including a second microphone. In still other embodiments, the method further includes: (1) monitoring, by the one or more processors, a first audio cue using the first microphone and the second microphone; (2) receiving, at the first interactive touchscreen display device via the first microphone, the first audio cue; (3) receiving, at the second interactive touchscreen display device via the second microphone, the first audio cue; (4) analyzing, by the one or more processors, the first audio cue at each of the first interactive touchscreen display device and the second interactive touchscreen display device; (5) based on the analysis, determining the location of the source of the first audio cue relative to the first interactive touchscreen display device and the second interactive touchscreen display device; and storing the location of the source in a computer memory.

[0008] An interactive display system according to various embodiments includes a display device having at least one processor and a display screen. In some embodiments, the at least one processor is configured to: (1) receive one or more user interface positioning determination data of a user of the display device, the one or more user interface positioning determination data including at least one reference height; (2) determine whether to modify a default position of at least one user interface on the display device based at least in part on the one or more user interface positioning determination data; and (3) in response to determining to modify the default position of the at least one user interface on the display device, modify a default output position of the at least one user interface, wherein modifying the default output position includes modifying the default output position to adjust a height of the default output position.

[0009] In some embodiments, the system is further configured to: (1) display a plurality of user-selectable markers to a user on the display screen, the plurality of user-selectable markers including: (A) a first user-selectable marker at a first height; and (B) a second user-selectable marker at a second height higher than the first height; (2) receive a selection of a marker among the plurality of user-selectable markers via an interactive touch screen display input; (3) in response to receiving a selection of the second user-selectable marker, modify the default output position to increase the height of the default output position; and (4) in response to receiving a selection of the first user-selectable marker, modify the default output position to decrease the height of the default output position. In some embodiments, the at least one processor is further configured to: (1) receive a plurality of user inputs via an interactive touch screen display input; (2) analyze the plurality of user inputs to determine whether more than a threshold number of the user inputs have an input position higher than the at least one reference height; and (3) in response to determining that more than the threshold number of the user inputs have an input position higher than the at least one reference height, modify the default output position to decrease the height of the default output position.

[0010] In a particular embodiment, a computer-implemented data processing method for modifying the position of at least one user interface on an interactive touchscreen display device includes: (1) receiving, by one or more processors, one or more pieces of user interface positioning determination data regarding a user of the interactive touchscreen display device; (2) determining, by one or more processors, a default position of the at least one user interface at least in part based on the one or more pieces of user interface positioning determination data; and (3) displaying, by one or more processors, the at least one user interface on the interactive touchscreen display device based on the determined default position. In a particular embodiment, the method further includes: (1) receiving, by one or more processors, an input on at least one user interface at a first position on the interactive touchscreen display device, the input triggering the production of a second user interface; (2) defining, in response to receiving the input, by one or more processors, a viewing cone at least in part based on the first position; and (3) causing, by one or more processors, a visual indication of the second user interface to be produced at least temporarily within the viewing cone.

[0011] In some embodiments, the method further includes: (1) receiving, via the interactive touchscreen display device, a plurality of user inputs; (2) analyzing the plurality of user inputs to determine whether more than a threshold number of the user inputs have input positions that are higher than a threshold height; and (3) in response to determining that more than the threshold number of the user inputs have input positions that are higher than the threshold height, modifying the default position to lower the height of the default position. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Various embodiments of an interactive display system are described below. In the course of this description, reference will be made to the accompanying drawings, which are not necessarily drawn to scale and in which:

[0013] Figure 1 is a block diagram of an interactive display system according to an embodiment of the present system;

[0014] Figure 2 is a schematic diagram of a computer suitable for use in various embodiments, such as Figure 1 one or more interactive display devices;

[0015] Figure 3 depicts a perspective view of an interactive display according to a particular embodiment;

[0016] Figure 4 depicts an input event routing process for a first computing device (e.g., an interactive display) having one or more connected devices according to various embodiments;

[0017] Figure 5Depicts a flowchart illustrating various steps performed by a specific embodiment of an input routing module 500, which can be performed, for example, by Figure 1 one or more interactive displays 110 of

[0018] Figures 6 - 7 Depicts exemplary screen displays and graphical user interfaces (GUIs) according to various embodiments of the system, which can display information associated with the system or enable one or more users to access or interact with the system (e.g., when using an interactive display to access both the functionality native to the active display and software applications running on a connected computing device).

[0019] Figure 8 Depicts a flowchart illustrating various steps performed by a specific embodiment of a connected computing device software integration module 800, which can be performed, for example, by Figure 1 one or more interactive displays 110 of

[0020] Figure 9 Depicts an exemplary process by which a user can access and use a selected application running on a connected computing device via a first computing device (e.g., an interactive display) operatively connected to the connected computing device;

[0021] Figures 10 - 12 Depicts exemplary screen displays and graphical user interfaces (GUIs) according to various embodiments of the system, which can display information associated with the system or enable one or more users to access one or more features of the system or interact with one or more features of the system (e.g., when using an interactive display device to access one or more software applications installed on and / or running on one or more computing devices operatively connected to the interactive display device).

[0022] Figure 13 Depicts a flowchart illustrating various steps performed by a specific embodiment of a user interface adjustment, customization, and placement module 1300, which can be performed, for example, by Figure 1 one or more interactive displays 110 of

[0023] Figures 14 - 30 Depicts exemplary screen displays and graphical user interfaces (GUIs) according to various embodiments of the system, which can display information associated with the system or enable one or more users to access one or more features of the system or interact with one or more features of the system;

[0024] Figure 31Depicts a flowchart illustrating various steps performed by a particular embodiment of a software availability monitoring module 3100, which can be implemented, for example, by Figure 1 one or more remote servers 130;

[0025] Figure 32 Depicts a flowchart illustrating various steps performed by a particular embodiment of a context environment analysis module 3100, which can be implemented, for example, by Figure 1 one or more interactive displays 110 and / or one or more remote servers 130; and

[0026] Figure 33 Depicts a top - down floor plan of a building (e.g., a school building) having multiple interactive display devices during an exemplary incident, and one or more of the multiple interactive display devices can identify the exemplary incident using one or more suitable techniques described herein. Detailed Description

[0027] Various embodiments will now be described more fully hereinafter with reference to the accompanying drawings. It should be understood that the present invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numerals refer to like elements throughout.

[0028] Overview

[0029] According to a particular embodiment, an interactive display system 100 includes: (1) one or more interactive display devices 110, which can include, for example, any suitable interactive display device (e.g., any suitable Promethean Active Panel display); and (2) one or more connected computing devices 112 operatively coupled to the one or more interactive display devices 110 (e.g., via any suitable wireless and / or wired connection). In some embodiments, the one or more interactive display devices 110 are configured to display data (e.g., video data, image data, etc.) from the one or more connected computing devices 112 on at least a portion of a display screen of the one or more interactive display devices 110. In a particular embodiment, the one or more interactive display devices 110 are configured to pass one or more inputs received at the one or more interactive display devices 110 to the one or more connected computing devices 112.

[0030] In certain embodiments, an interactive display system is configured to: (1) process one or more inputs received via one or more input devices (e.g., a touchscreen display) of an interactive display device 110 having one or more connected computing devices 112; (2) determine a final destination of the processed one or more inputs (e.g., by processing the inputs using a processor of the interactive display device such as a scaler board); and (3) route each respective input to the determined final destination for consumption and / or processing (e.g., by routing each respective input to a particular connected computing device of the interactive display device or the one or more connected computing devices. As a specific example, the system may be configured to: (1) receive a touch input on an interactive display having a connected computing device (e.g., a Chromebox); (2) process the touch input using a processor of the interactive display to determine a final destination of the touch input (e.g., determine that the final destination of the touch input is the interactive display or the connected computing device); (3) in response to determining that the final destination of the touch input is the interactive display, process the touch input as an input on the interactive display; and (4) route the touch input to the connected computing device (e.g., via an input device) for processing at the connected computing device.

[0031] In various embodiments, when using a computing device or multiple connected computing devices (e.g., such as a Chromebox, an Apple TV, a Microsoft Xbox One; a Sony PlayStation, a laptop computer, a tablet computer, or any other suitable computing device) connected to any suitable display device (e.g., an interactive display device, a smart TV, a television display, or any other suitable display device), a user may need to: (1) know which connected computing device has a particular application that the user desires to use: (2) switch the input of the display device to the appropriate connected computing device; and (3) launch the application on the connected computing device.

[0032] In some embodiments, an interactive display system 100 is configured to aggregate a list of one or more available applications installed, running, or otherwise available on each of the one or more connected devices to a particular computing device (e.g., a display device). The system may further be configured to: (1) provide the aggregated list of available applications to a user of the particular computing device; (2) receive a selection of a particular application from the aggregated list of available applications; and (3) automatically cause the particular computing device to display data from the particular application on the particular computing device while the particular application is running on a connected computing device. In various embodiments, from the perspective of the user, the selected particular application may appear to be running on the particular computing device (e.g., even though the application is strictly speaking running on a connected computing device).

[0033] In various embodiments, the interactive display system 100 may further be configured to reduce user fatigue (e.g., due to having to reach different parts of the interactive display, backing up to see different parts of the interactive display, stretching the user's neck to see different parts of the display, etc.). The system may be configured, for example, to modify and / or adjust one or more user interfaces on the interactive display device at least in part based on one or more user characteristics. The system may be configured, for example, to modify and / or adjust the position, size, etc. of one or more user interfaces based at least in part on the user of the interactive display device. In some embodiments, the system may further be configured to cause a user interface to pop up at least in part based on the location of an input provided by the user (e.g., such that the system causes the user interface to appear in a location on the display screen such that the user can see the user interface and be able to interact with the user interface with substantially no movement (e.g., substantially not having to stretch their neck, reposition themselves adjacent to the interactive display device, etc.)).

[0034] In some embodiments, the system may be configured with default user interface features that the user can use to interact towards the lower portion of the interactive display, e.g., to make the user interface features more accessible to the user. As discussed herein, particular embodiments of the interactive display device may include a particularly large screen that may, for example, have portions of its display that are difficult to access due to the height of those portions relative to the user's support surface (e.g., the floor).

[0035] In certain embodiments, the system is configured to modify and / or adjust user interface positioning, size, etc. based on, for example: (1) user preference data (e.g., the system may receive and / or determine the user preference data from a user profile when the user logs in or otherwise provides such preference data to the interactive display device using any of the techniques described herein); (2) user selection of screen markers indicating a preferred position / size of the user interface; (3) user body characteristics (e.g., the system may use one or more cameras to determine the user body characteristics to identify the user's height, reach, etc.).

[0036] In still other embodiments, the system is configured to: (1) track user touches (e.g., inputs) in one or more specific regions of the interactive display device (e.g., one or more regions above a specific height such as shoulder height); (2) automatically modify the default position of one or more user interfaces, at least in part based on detecting more than a threshold number of inputs above the specific height. This can, for example, enable the system to reduce the number of times the user has to reach above the specific height (e.g., this can reduce injuries, fatigue, etc.) by modifying the user interface positioning (e.g., by lowering the position), thereby reducing user fatigue.

[0037] In any of the embodiments described herein, the interactive display system is configured to enable a user to log in to any specific interactive display in order to apply user-specific settings, preferences, etc. to that specific interactive display such that the user can walk up to any specific interactive display, log in, and have the same user experience when using any interactive display. The interactive display can, for example, be configured to: (1) receive user identification data; (2) optionally authenticate the user (e.g., by requiring the user to enter a password or other unique identifier); (3) retrieve user profile data in response to receiving the user identification data and optional authentication data (e.g., via a network from one or more remote servers); (3) apply the user profile data to one or more settings of the interactive display; (4) optionally automatically log in to one or more connected computing devices using the user profile data; and / or (5) optionally modify one or more features and / or functionality of the interactive display, at least in part based on the role of the logged-in user (e.g., whether the user is a teacher, student, administrator, etc.).

[0038] In certain embodiments, the interactive display system 100 is configured to provide a plurality of available software applications (e.g., via an app store or a similar catalog of available software applications), which include first-party software (e.g., provided by the manufacturer of the interactive display device) and approved and / or vetted third-party applications for use on the interactive display device. In certain embodiments, the system is configured to provide an administrator or other authorized user with the ability to select a particular application (e.g., and / or content) from the list of available software applications to become available on one or more interactive display devices where they are an administrator (e.g., one or more devices within a particular school, school district, etc.). In certain embodiments, the system is configured to generate a curated app store based on the selections of the administrator (e.g., or other authorized user), which includes only those applications selected by the administrator. Individual users of the interactive displays managed by the administrator can then select from the curated list of approved applications for use on a particular interactive display device. In this way, the system can be configured to provide a whitelist of applications from a larger catalog of applications for selection by users (e.g., teachers) in a particular group (e.g., a particular school, school district, county, city, etc.).

[0039] In various embodiments, one or more interactive display devices 110 (e.g., or other computing devices) can be used for environmental analysis, with a particular application in the real-time localization of gunfire in an active shooter scenario. In such embodiments, a plurality of interactive display panels in one or more fixed physical locations can be configured to perform passive analysis of the audio environment to infer information about the environment. In such embodiments, each of the plurality of interactive display devices can include at least one microphone. By substantially continuously monitoring the audio environment, the plurality of interactive display devices can be configured to: (1) identify at least one audio cue; (2) determine the location of the identified audio cue based on sound data received at one or more of the plurality of interactive display devices; and (3) generate one or more warnings associated with the identified audio cue.

[0040] Exemplary Technology Platform

[0041] As those skilled in the relevant art will appreciate, the present invention may be embodied, for example, as a computer system, a method (e.g., a computer-implemented method, a computer-implemented data processing method, etc.), or a computer program product. Thus, various embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. In addition, a particular embodiment may take the form of a computer program product stored on a computer-readable storage medium (e.g., a non-transitory computer-readable medium), where the computer-readable storage medium embodies computer-readable instructions (e.g., software). Various embodiments may take the form of web-implemented computer software. Any suitable computer-readable storage medium may be utilized, including, for example, a hard disk, a compact disk, a DVD, an optical storage device, and / or a magnetic storage device.

[0042] Various embodiments are described below with reference to block diagrams and flowcharts of methods (e.g., computer-implemented methods), apparatuses (e.g., systems), and computer program products. It should be understood that each block in the block diagrams and flowcharts, and combinations of blocks in the block diagrams and flowcharts, can be implemented respectively by a computer executing computer program instructions. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executed on the computer or other programmable data processing apparatus create means for implementing the functions specified in one or more of the blocks of the flowchart.

[0043] These computer program instructions can also be stored in a computer-readable memory, which can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture configured to implement the functions specified in one or more of the blocks of the flowchart. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus, thereby producing a computer-implemented process, such that the instructions executed on the computer or other programmable apparatus provide the steps for implementing the functions specified in one or more of the blocks of the flowchart.

[0044] Accordingly, the blocks of the block diagrams and flowcharts support combinations of means for performing the specified functions, combinations of steps for performing the specified functions, and program instructions for performing the specified functions. It should also be understood that each block in the block diagrams and flowcharts, and combinations of blocks in the block diagrams and flowcharts, can be implemented by a special-purpose hardware-based computer system for performing the specified functions or steps, or by a combination of special-purpose hardware and other hardware executing appropriate computer instructions.

[0045] Example System Architecture

[0046] Figure 1 It is a block diagram of an interactive display system 100 according to a specific embodiment. As can be understood from this figure, the interactive display system 100 according to the first embodiment includes: (1) one or more interactive display devices 110, which may include, for example, any suitable interactive display device (e.g., such as any suitable Promethean active panel display); and (2) one or more connected computing devices 112 operatively coupled to the one or more interactive display devices 110 (e.g., via any suitable wireless and / or wired connection). In some embodiments, the one or more interactive display devices 110 are configured to display data (e.g., video data, image data, etc.) from the one or more connected computing devices 112 on at least a portion of the display screen of the one or more interactive display devices 110. In a particular embodiment, the one or more interactive display devices 110 are configured to pass one or more inputs received at the one or more interactive display devices 110 to the one or more connected computing devices 112.

[0047] In various embodiments, one or more interactive display devices 110 may be operatively coupled (e.g., connected) to one or more connected computing devices 112 in any suitable manner using any suitable connection type (e.g., wired and / or wireless connection types). For example, in a particular embodiment, one or more interactive display devices 110 may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable video cable (e.g., (one or more) composite video cables, (one or more) s-video cables, (one or more) component video cables, (one or more) VGA cables, (one or more) DVI cables, (one or more) HDMI cables, (one or more) DisplayPort cables, (one or more) Mini DisplayPort cables, etc.) or a combination thereof. In still other embodiments, one or more interactive display devices 110 may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable data cable (e.g., (one or more) Ethernet cables, (one or more) token ring cables, (one or more) coaxial cables, (one or more) fiber optic cables, USB cables, etc.) or a suitable combination thereof. In still other embodiments, one or more interactive display devices 110 may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable wireless connection (e.g., Bluetooth, WIFI, NFC, Bluetooth Low Energy, ZigBee, etc.) or a combination of wireless connections. In still other embodiments, one or more interactive display devices 110 may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable network discussed herein or any other suitable network. In other embodiments, one or more interactive display devices 110 may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable combination of the connection types described herein or in any other suitable manner.

[0048] In various embodiments of the interactive display system 100, one or more interactive display devices 110 include one or more touchscreen display devices, which may include, for example, any suitable touchscreen device configured to receive input via contact on the display device (e.g., via a user's finger or other body part, via a stylus or other pen-shaped device, etc.). For example, in various embodiments, one or more interactive display devices 110 include one or more resistive touchscreen displays (e.g., one or more 5-wire resistive touchscreen displays), one or more surface capacitive touchscreen displays, one or more projected capacitive touchscreen displays, one or more surface acoustic wave touchscreen displays, one or more infrared touchscreen displays (e.g., which may include, for example, one or more touch overlays at least partially embedded in the bezel of one or more interactive display devices 110), or any other suitable touchscreen display. In a particular embodiment, one or more interactive display devices 110 include one or more processors and memory. In such an embodiment, one or more interactive display devices 110 may include a stand-alone computing device, such as a handheld computing device (e.g., a tablet computer or a smart phone), a wall-mounted display device (e.g., such as a touch-enabled computerized LED and / or LCD display), etc.

[0049] In a particular embodiment, one or more interactive display devices 110 are configured to detect multiple simultaneous inputs (e.g., from multiple users). In various embodiments, one or more interactive display devices 110 are configured to enable a user to interact with one or more displayed images in the same manner as a user would interact with any other computing display device (e.g., by dragging on one or more projected images), etc. In still other embodiments, one or more interactive display devices 110 include one or more computing components (e.g., one or more processors and memory), such that one or more interactive display devices 110 embody a stand-alone computing device.

[0050] In certain embodiments, one or more interactive display devices 110 are configured to run software locally on the one or more interactive display devices 110. For example, in one embodiment of the interactive display system 100, one or more interactive display devices 110 may store software applications on local memory associated with the one or more interactive display devices 110, use one or more local processors associated with the one or more interactive display devices 110 to execute software instructions from the interactive overlay software applications, locally detect input using any suitable input detection techniques for the one or more interactive display devices 110, and so on. In certain embodiments, the software applications native to the one or more interactive display devices may include, for example, any suitable software, such as: (1) one or more word processing applications; (2) one or more annotation software segments; (3) one or more video applications; (4) one or more whiteboards simulating software applications; and / or (5) any other suitable software applications.

[0051] In certain embodiments, one or more connected computing devices 112 may include any suitable connected computing devices, such as, for example, one or more laptop computers, one or more tablet computers, one or more Chromeboxes, one or more Amazon Fire TV devices, one or more Apple TV devices, one or more gaming systems (e.g., one or more Microsoft, Sony, Nintendo, etc. gaming systems), or any other suitable computing and / or video playback devices that may have one or more software segments installed thereon.

[0052] In various other embodiments, the interactive display system 100 may utilize one or more suitable cloud computing technologies to execute overlay software, underlying software, store and access one or more documents, etc. In such embodiments, one or more computer networks 115 may facilitate communication between one or more interactive display devices 110, one or more remote servers 130, one or more databases 140, one or more remote computing devices 150, one or more third-party servers 160, etc. For example, one or more networks 115 may facilitate communication between one or more interactive display devices 110 and one or more remote servers 130 to access profile data of users of the one or more interactive display devices 110 to customize the user experience while using the one or more interactive display devices 110.

[0053] One or more networks 115 can further facilitate access between one or more interactive display devices 110 and suitable documents stored in one or more remote databases 140 (e.g., stored in one or more remote databases managed by a third-party cloud-based storage service such as Google Drive, Dropbox, or any other suitable service). One or more computer networks 115 can include any of a variety of types of wired or wireless computer networks, such as the Internet, a private intranet, a mesh network, a public switched telephone network (PSTN), or any other type of network (e.g., a network that uses Bluetooth or near field communication to facilitate communication between computers). The communication links between one or more interactive display devices 110, one or more databases 140, and one or more remote servers 130 can be implemented, for example, via a local area network (LAN) or via the Internet.

[0054] Figure 2 FIG. illustrates a graphical representation of a computer architecture 200 that can be used within an interactive display system 100, the computer architecture 200 being, for example, a client computer (e.g., Figure 1 one or more of the interactive display devices 110 shown in ), or a server computer (e.g., Figure 1 one or more of the remote servers 130 shown in ). In a particular embodiment, the computer 200 can be suitable for use as a computer within the context of an interactive display system 100, the computer being configured to receive input from an interactive display device (e.g., such as one or more of the interactive display devices 110), determine whether the input is intended for one or more of the interactive display devices 110 or for one or more connected computing devices 112, and pass the input to the appropriate computing device for processing.

[0055] In a particular embodiment, the computer 200 can be connected (e.g., networked) to other computers in a LAN, intranet, extranet, and / or the Internet. As noted above, the computer 200 can operate in the capacity of a server or client computer in a client-server network environment, or as a peer computer in a peer-to-peer (or distributed) network environment. The computer 200 can be a desktop personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a web appliance, a server, a network router, a switch, or a bridge, or any other computer capable of executing a set of instructions (sequentially or otherwise) that specify actions to be taken by that computer. Further, although only a single computer is illustrated, the term "computer" should also be considered to include any collection of computers that individually or jointly execute a set of instructions (or multiple sets of instructions) to perform any one or more of the method techniques discussed herein.

[0056] The exemplary computer 200 includes a processing device 202 (e.g., one or more computer processors), a main memory 204 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous dynamic DRAM (SDRAM) or Rambus DRAM (RDRAM)), a static memory 206 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage device 218, which communicate with each other via a bus 232.

[0057] The processing device 202 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, etc. More particularly, the processing device 202 can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a scalar board, or a processor implementing other instruction sets, or a processor implementing a combination of instruction sets. The processing device 202 can also be one or more dedicated processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. The processing device 202 can be configured to execute processing logic 226 for performing the various operations and steps discussed herein.

[0058] The computer 200 can further include a network interface device 208. The computer 200 can also include a video display unit 210 (e.g., a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a plasma display, a projector, a cathode ray tube (CRT), any suitable display described herein, or any other suitable display), an alphanumeric or other input device 212 (e.g., a keyboard), a cursor control or other input device 214 (e.g., a mouse, a stylus, a pen, a touch-sensitive input device, etc.), and a signal generation device 216 (e.g., a speaker).

[0059] The data storage device 218 can include a non-transitory computer-accessible storage medium 230 (also referred to as a non-transitory computer-readable storage medium or a non-transitory computer-readable medium) having stored thereon one or more instruction sets (e.g., software 222) embodying any one or more of the method techniques or functions described herein. The software 222 can also reside, completely or at least partially, within the main memory 204 and / or within the processing device 202 during execution by the computer 200 - the main memory 204 and the processing device 202 also constituting computer-accessible storage media. The software 222 can further be transmitted or received via the network interface device 208 over a network 115.

[0060] Although computer-accessible storage medium 230 is shown as a single medium in the exemplary embodiment, the term "computer-accessible storage medium" should be understood to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of instructions. The term "computer-accessible storage medium" should also be understood to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by a computer and that causes the computer to perform any one or more of the method techniques of the present invention. Thus, the term "computer-accessible storage medium" should be understood to include, but not be limited to, solid-state memories, optical and magnetic media, and the like.

[0061] Exemplary Interactive Display Device

[0062] Figure 3 An interactive display device 110 is depicted in accordance with a particular embodiment. As can be understood from Figure 3 the figure, the interactive display device 110 in the illustrated embodiment includes an interactive whiteboard that includes: (1) a display 310; (2) a bezel 312; and (3) a control panel 320.

[0063] In various embodiments, the interactive display device 110 includes one or more touchscreen displays 310, which may include, for example, any suitable touchscreen device configured to receive input via contact on the display device (e.g., via a user's finger or other body part, via a stylus or other pen-like device, etc.). For example, in various embodiments, the interactive display device 110 includes one or more resistive touchscreen displays (e.g., one or more 5-wire resistive touchscreen displays), one or more surface capacitive touchscreen displays, one or more projected capacitive touchscreen displays, one or more surface acoustic wave touchscreen displays, one or more infrared touchscreen displays (e.g., which may include, for example, one or more touch overlays at least partially embedded in the bezel 312 of the interactive display device 110), or any other suitable touchscreen display 310. In a particular embodiment, the interactive display device 110 includes one or more processors and a memory. In such an embodiment, the interactive display device 110 may include a stand-alone computing device, such as a wall-mounted display device (e.g., such as a touch-enabled computerized LED and / or LCD display), and the like.

[0064] In certain embodiments, the interactive display device 110 is configured to detect multiple simultaneous inputs (e.g., from multiple users) via the display 310. In various embodiments, the interactive display device 110 is configured to enable a user to interact with one or more displayed images as if the user were interacting with any other computing display device (e.g., by dragging on one or more projected images), etc. In still other embodiments, the interactive display device 110 includes one or more computing components (e.g., one or more processors and memory) such that one or more interactive display devices 110 embody a stand-alone computing device.

[0065] In various embodiments, the interactive display device 110 includes a control panel 320 that may be disposed, for example, adjacent to a lower portion of the interactive display device 110 below the display 310. As can be seen from Figure 3 it can be appreciated that the control panel 322 may define a face portion 322 that includes one or more interactive buttons 324 that may control one or more functions and / or operations of the interactive display device 110. As can be seen from Figure 3 it can be appreciated that the face 322 of the control panel 320 may be angled outwardly at least partially from the front of the interactive display device 110 such that the face 322 forms an angle between approximately zero degrees and approximately forty-five degrees with the display screen 310. In still other embodiments, the face 322 of the control panel 320 may be angled outwardly at least partially from the front of the interactive display device 110 such that the face 322 forms any other suitable angle with the display screen 310 (e.g., approximately five degrees, approximately ten degrees, approximately thirty degrees, approximately forty-five degrees, up to approximately sixty degrees, or any suitable angle between approximately zero degrees and approximately ninety degrees).

[0066] In embodiments where the face 322 is at least partially angled outwardly from the interactive display device 110, the control panel 320 can be configured such that a user of the interactive display device 110 can see each of one or more interactive buttons 324 while the user stands adjacent to the interactive display device 110 (e.g., within reach of the interactive display device 110 with an arm). As can be appreciated in view of the present disclosure, the interactive display device 110 can be particularly large (e.g., up to approximately 100 inches or more in diagonal measurement). As such, when standing adjacent to the interactive display device 110 (e.g., within reach of the interactive display device 110 with an arm), it may be difficult for a user to reach and / or see the display screen 310 or every part of the interactive display device 110. By angling the face 322 of the control panel 320 at least partially outwardly, the interactive display device 110 can be configured such that a user can view each of one or more interactive buttons 324 without having to step back from the interactive display device 110 (e.g., by looking down at the face 322 of the control panel 320). In other embodiments where one or more interactive buttons 324 are flush with and / or substantially parallel (e.g., parallel) to the display screen 310, a user may not be able to see the buttons when standing close enough to the display screen 310 to interact with it.

[0067] Exemplary System Platform

[0068] Various embodiments of the interactive display system 100 can be implemented in the context of any suitable interactive display device, interactive display device system, or any other suitable system. For example, particular embodiments can be implemented on an interactive display device, a touchscreen display device, one or more remote servers, or any other suitable computing device or combination of computing devices. Aspects of the system functionality can be performed by certain system modules, including an input routing module 500; a connected computing device software integration module 800; a user interface adjustment, customization, and placement module 1300; a software usability regulation module 3100; and a context environment analysis module 3100. These modules are discussed in more detail below.

[0069] Although these modules are presented as a series of steps, it should be understood in view of the present disclosure that the various embodiments of the input routing module 500; the connected computing device software integration module 800; the user interface adjustment, customization, and placement module 1300; the software usability monitoring module 3100; and the context environment analysis module 3100 described herein may perform the steps described below in an order other than the order in which they are presented. In still other embodiments, the input routing module 500; the connected computing device software integration module 800; the user interface adjustment, customization, and placement module 1300; the software usability monitoring module 3100; and the context environment analysis module 3100 may omit certain steps described below. In various other embodiments, the input routing module 500; the connected computing device software integration module 800; the user interface adjustment, customization, and placement module 1300; the software usability monitoring module 3100; and the context environment analysis module 3100 may perform steps other than those described (e.g., one or more steps described with respect to one or more other modules, etc.).

[0070] Input Routing Module

[0071] In a particular embodiment, the interactive display system 100 is configured to: (1) process one or more inputs received via one or more input devices of a first composite device having one or more connected computing devices; (2) determine a final destination of the processed one or more inputs; and (3) route each respective input to the determined final destination for consumption and / or processing (e.g., by routing each respective input to a first computing device or a particular connected computing device among one or more connected computing devices). As a particular example, the system may be configured to: (1) receive a touch input on an interactive display having a connected computing device (e.g., a Chromebox); (2) process the touch input using a processor of the interactive display to determine a final destination of the touch input (e.g., determine that the final destination of the touch input is the interactive display or a connected computing device); (3) in response to determining that the final destination of the touch input is the interactive display, process the touch input as an input on the interactive display; and (4) route the touch input to the connected computing device (e.g., via an input device) for processing at the connected computing device.

[0072] Turning to Figure 4, as can be understood from the figure, a first computing device (e.g., the interactive display device 110) can be operatively connected to one or more connected devices 112. The interactive display device 110 can include one or more input devices (e.g., the touchscreen display 310, the bezel 312, or other suitable input devices), which are configured to act as one or more input devices 310 for one or more connected devices while the one or more connected devices 112 are operatively connected to the first computing device 110.

[0073] Reference Figure 4 , in the embodiment shown in the figure, the interactive display system 100 is configured to receive all input data via one or more input devices 310. The input data received by the one or more input devices 310 first flows to one or more first computing device processors 202 of the first computing device 110. The one or more first computing device processors 202 can include any suitable processing device, such as, for example, a scalar board or other suitable computer processor. The one or more first computing device processors 202 can be configured to process the input data received from the one or more input devices 310 to determine where the input should "land" (e.g., determine the final destination of the input). The system can be configured to determine the final destination of each particular input based on, for example, the location of the input (e.g., the first computing device 110 itself, or any particular connected computing device 114 among the one or more connected devices 112). As can be understood in view of the present disclosure, the first computing device 110 can display data from the one or more connected devices 112 on at least a portion of the display screen (e.g., on the full screen, in a window environment on a portion of the screen, etc.). The first computing device 110 can further display one or more overlays or other interactive elements or software applications native to the first computing device 110 on one or more other portions of the display. The system can be configured to determine the final destination of each particular input based at least in part on whether the location of the input coincides with: (1) an overlay, menu, software application, or other feature running natively on the interactive display device 110; or (2) data displayed from the one or more connected devices 112.

[0074] In response to determining that the ultimate destination of the input is an overlay or native application on the first computing device 110, the system is configured to process the input at the first computing device 110 (e.g., using one or more first computing device processors 202). In response to determining that the ultimate destination is a particular connected computing device 114 among one or more connected devices 112, the system is configured to transfer data associated with the input (e.g., input location) from the first computing device 110 (e.g., one or more first computing device processors 202) to the particular connected computing device 114 for consumption by the particular connected computing device 114. In a particular embodiment of the interactive display system 100 shown in this figure, in response to determining that the ultimate destination is the particular connected computing device 114, the system is configured to transfer data associated with the input (e.g., input location) from the first computing device 110 (e.g., one or more first computing device processors 202) to the particular connected computing device 114 via one or more input devices 310 (e.g., bezel 312).

[0075] Turning Figure 5 , in a particular embodiment, when the input routing module 500 is executed, the system is configured to receive first input data at step 510 via one or more input devices of a first computing device having one or more connected computing devices. In a particular embodiment, the first input data includes touch input data received via a touchscreen display or other interactive display (such as any suitable interactive display described herein). In other embodiments, the first input data includes the location at which the input is made on the interactive display device 110.

[0076] In various embodiments, the first computing device includes any suitable computing device described herein (e.g., the interactive display device 110). In some embodiments, the first computing device includes any suitable display device that includes at least one processor capable of processing the first input data to determine the destination of the input associated with the input data.

[0077] In a particular embodiment, one or more connected computing devices include any suitable connected computing device operatively connected to a first computing device. In various embodiments, the first computing device may be operatively coupled (e.g., connected) to one or more connected computing devices 112 in any suitable manner using any suitable connection type (e.g., wired and / or wireless connection types). For example, in a particular embodiment, the first computing device may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable video cable (e.g., (one or more) composite video cables, (one or more) s-video cables, (one or more) component video cables, (one or more) VGA cables, (one or more) DVI cables, (one or more) HDMI cables, (one or more) DisplayPort cables, (one or more) Mini DisplayPort cables, etc.) or a combination thereof. In still other embodiments, the first computing device may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable data cable (e.g., (one or more) Ethernet cables, (one or more) token ring cables, (one or more) coaxial cables, (one or more) fiber optic cables, USB cables, etc.) or a suitable combination thereof. In still other embodiments, the first computing device may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable wireless connection (e.g., Bluetooth, WIFI, NFC, Bluetooth Low Energy, ZigBee, etc.) or a combination of wireless connections. In still other embodiments, the first computing device may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable network discussed herein or any other suitable network. In other embodiments, the first computing device may be operatively coupled (e.g., connected) to one or more connected computing devices 112 via any suitable combination of the connection types described herein or in any other suitable manner.

[0078] In a particular embodiment, one or more connected computing devices 112 may include any suitable connected computing device, such as for example one or more laptop computers, one or more tablet computers, one or more Chromeboxes, one or more Amazon Fire TV devices, one or more Apple TV devices, one or more gaming systems (e.g., one or more Microsoft, Sony, Nintendo, etc. gaming systems) or any other suitable computing and / or video playback device that may have one or more segments of software installed thereon.

[0079] Proceeding to step 520, the system is configured to transfer first input data from one or more input devices to one or more first computing device processors. As described above, with respect to Figure 4, the first input data received by one or more input devices 310 first flows to one or more first computing device processors 202 of the first computing device 110. The one or more first computing device processors 202 can include any suitable processing device, such as, for example, a scalar board or other suitable computer processor. The one or more first computing device processors 202 can be configured to process the input data received from the one or more input devices 310 to determine where the input should "land" (e.g., determine the final destination of the input).

[0080] Next, at step 530, the system is configured to process the first input data to determine the final destination of the first input data. The system can be configured to determine the final destination of each particular input based on, for example, the location of the input (e.g., the first computing device 110 itself, or any particular connected computing device 114 among the one or more connected devices 112). As can be understood in view of the present disclosure, the first computing device 110 can display data from the one or more connected devices 112 on at least a portion of the display screen (e.g., full screen, in a window environment on a portion of the screen, etc.). The first computing device 110 can further display one or more overlays, menus, or other interactive elements or software applications native to the first computing device 110 on one or more other portions of the display. The system can be configured to determine the final destination of each particular input based at least in part on whether the location of the input coincides with: (1) an overlay, menu, software application, or other feature running natively on the interactive display device 110; or (2) data displayed from the one or more connected devices 112.

[0081] Returning to step 540, in response to determining (e.g., at step 530) that the first input data has a final destination of the first computing device, the system is configured to process the first input data at the first computing device (e.g., using one or more processors of the first computing device).

[0082] At step 550, in response to determining (e.g., at step 530) that the first input data has a final destination of a particular one of the one or more connected computing devices, the system is configured to: (1) transmit the first input data from the first computing device to the particular connected computing device (e.g., via one or more input devices); and (2) cause the particular connected computing device to process the first input data. In still other embodiments, the system is configured to transmit the first input data to the particular connected computing device for processing.

[0083] In response to determining that the final destination is a particular connected computing device 114 among one or more connected devices 112, the system can be configured, for example, to transfer data associated with an input (e.g., an input location) from a first computing device 110 (e.g., one or more first computing device processors 202) to the particular connected computing device 114 for consumption by the particular connected computing device 114. In a particular embodiment of the interactive display system 100, in response to determining that the final destination is the particular connected computing device 114, the system is configured to transfer data associated with an input (e.g., an input location) from the first computing device 110 (e.g., one or more first computing device processors 202) to the particular connected computing device 114 via one or more input devices 310 (e.g., Figure 3 the bezel 312 shown in

[0084] Input Routing User Experience

[0085] Figure 6 An exemplary display screen 600 that a user may encounter when using an interactive display device 110 having a connected computing device (e.g., a Google Chromebox) is depicted. As can be understood from an embodiment of the user interface 600 shown in this figure, the user interface includes a menu 610 native to the interactive display device (e.g., running locally on the interactive display device), a window interface 630 that displays a software application (e.g., a web browser) running on the connected computing device, and a wrapper 620 around the window interface 630 native to the interactive display device.

[0086] In a particular embodiment, the system is configured to display the menu 610 on top of any source displayed by the interactive display device (e.g., even when using one or more connected computing devices in full-screen mode). In various embodiments, the system is configured to enable a user to expand and collapse the menu 610 using a menu expander 613. In a particular embodiment, the system is further configured to enable a user to access the menu 610 on the left and right sides of the display screen using corresponding menu expanders 611, 612. In a particular embodiment, when the menu 610 is displayed on the left or right hand side of the display, the system is configured to present the menu below the center of the display. For example, in the case of a larger display, this can facilitate access to the menu when it is on the left or right hand side.

[0087] In various embodiments, such as Figure 6In the embodiment shown, when the system is displaying a view of a connected computing device in window interface 630, the system is configured to enable a user to move the position of window interface 630, for example, by selecting and dragging wrapper 620 around display screen 600. In this way, the system is configured to enable a user to modify a portion of display screen 600 of interactive display device 110 that is displaying one or more images from a connected computing device (e.g., in window interface 630).

[0088] As can be understood from Figure 6 Interactive display device 110 may include a touch screen display configured to receive user input via touch inputs (e.g., using a finger, pen, or other touch device). Interface 600 shown in the figure may be configured to: (1) receive an input at a specific location; (2) determine a final destination of the input based at least in part on the location; and (3) route the input to an appropriate device for processing. For example, in the interface shown in the figure, the system may be configured to determine that any input at the location on display screen 600 within window interface 630 has a final destination of a connected computing device. The system may then route the input to the connected computing device for processing at the connected computing device. The system may then update screen display 600 to show a live version of the software application running on the connected computing device as shown in window interface 630 to reflect a substantially up-to-date (e.g., live) view of the software application running on the connected computing device (e.g., which reflects any changes to the software application running on the connected computing device resulting from the input routed to the connected device).

[0089] The system may be configured to determine that any input outside of window interface 630 has a final destination of the interactive display device. The system may then be configured to process the input at the input location at the interactive display device. For example, the system may be configured to process an input at menu 610 or wrapper 620 on the interactive display device (e.g., by initiating any specific function or performing any specific action initiated by the input at the detected location).

[0090] Figure 7Depicts another exemplary display screen 700 that a user may encounter when using an interactive display device 110 with a connected computing device (e.g., a Google Chromebox). As can be understood from the illustrated embodiment of the user interface 700 in this figure, the user interface includes a menu 610 native to the interactive display device (e.g., running locally on the interactive display device), an interface 730 for full-screen displaying a software application (e.g., a web browser) running on the connected computing device (e.g., being executed by one or more connected computing device processors), and an annotation overlay 720 native to the interactive display device 100 (e.g., running on the interactive display device and / or one or more interactive display device processors). In the embodiment shown in this figure, the interface 730 may include a user interface from a software application running on the connected computing device to the display device (e.g., an interactive display), which is transferring one or more images from the connected computing device to the interactive display device for display (e.g., using any suitable wired and / or wireless connection described herein).

[0091] As can be understood from Figure 7 and from this disclosure, the interactive display device 110 may include a touch screen display (e.g., or other display with any suitable input device), which is configured to receive user input via touch or other input (e.g., using a finger, a pen, or other touch device). The interface 700 shown in this figure may be configured to: (1) receive input at a specific location; (2) determine the ultimate destination of the input at least in part based on that location; and (3) route the input to the appropriate device for processing. For example, in the display interface 700 shown in this figure, the system may be configured to determine that any input at the location of the display screen 700 on the connected device interface 730 (e.g., which is running on the connected device and displayed on the interactive display device in full-screen settings) has an ultimate destination of the connected computing device. The system may then route the input to the connected computing device for processing at the connected computing device (e.g., such that the connected computing device processes the input as if the input was received by the connected computing device at the location corresponding to the location where the input was received on the interface 730).

[0092] The system may then update the screen display 730 (e.g., substantially in real time) to show a live version of the software application running on the connected computing device as shown in the full-screen interface 730, to reflect a substantially up-to-date (e.g., live) view of the software application running on the connected computing device (e.g., which reflects any changes to the software application running on the connected computing device caused by the input routed to the connected device).

[0093] The system can be further configured to process any input at an input location on the display screen 700. For example, the system can be configured to process input at a menu 610 or an annotation overlay 720 on the interactive display device itself (e.g., by initiating any specific function or performing any specific action from the input at the detected location). For example, in response to detecting input at an annotation overlay 720 on a "pen" icon, the system can be configured to: (1) determine (e.g., using a processor associated with the interactive display device) that the ultimate destination of the input is the interactive display device; and (2) process the input at the input location on the interactive display device (e.g., by selecting "pen" from the annotation overlay 720). In various embodiments, the system can then be configured to: (1) determine that subsequent input when "pen" is selected has the ultimate destination of the interactive display device (e.g., even if the user provides input on a portion of the display screen 700 that includes a connected device interface 730); and (2) process the input at the interactive display device by dragging at one or more input locations on the underlying connected device interface 730. In a particular embodiment, the system can be configured to continue to interpret additional input as input having the ultimate destination of the interactive display device until the user deselects "pen" from the annotation menu 720 or otherwise selects any other system option to revert to the above-described input event routing functionality. In this way, the system can be configured to interpret a particular input (e.g., determine the destination of a particular input) differently depending on what particular mode the interactive display system is in when the input is received (e.g., whether the interactive display is in a "normal" mode, an annotation mode, etc.).

[0094] Connected Computing Device Software Integration Module

[0095] In various embodiments, when using a computing device or multiple connected computing devices (e.g., such as a Chromebox, an Apple TV, a Microsoft Xbox One; a Sony PlayStation, a laptop computer, a tablet computer, or any other suitable computing device or combination of computing devices) connected to any suitable display device (e.g., an interactive display device, a smart TV, a television display, or any other suitable display device), a user may need to: (1) know which connected computing device has a particular application that the user desires to use; (2) switch the input of the display device to the appropriate connected computing device; and (3) launch the application on the connected computing device (e.g., using an input device associated with the connected computing device).

[0096] For example, if a user has a Chromebox with Netflix installed, and the Chromebox is operatively connected to a Samsung Smart TV (e.g., via an HDMI cable), and the user desires to use Netflix to view a piece of multimedia content, the user may have to: (1) turn on the power of the Samsung Smart TV (e.g., using an input device, such as the remote control of the Samsung Smart TV, or pressing the power button located on the physical Samsung Smart TV); (2) turn on the power of the Chromebox (e.g., using an input device, such as the remote control of the Chromebox, pressing the power button located on the physical Chromebox, etc.); (3) set the input of the Samsung Smart TV to the input via which the Chromebox is operatively connected to the Samsung Smart TV (e.g., HDMI 1); and (4) select the Netflix application from the Chromebox using a suitable Chromebox input device different from the input device used for the Samsung Smart TV (e.g., remote control). Although this may be a common experience for the user, the experience of using a software application on a computing device connected to a display device may be more complex, for example: (1) when the display device has multiple connected computing devices, each with a corresponding available application; (2) when each of the connected computing devices is connected to the display device via a different corresponding input; (3) when the display device itself has at least some applications available for the user to use; and / or (4) and so on.

[0097] In some embodiments, the interactive display system 100 is configured to aggregate a list of one or more available applications installed, running, or otherwise available on each of the one or more connected devices to a particular computing device (e.g., the display device). For example, when using a display device with one or more connected devices, this can enable a user of the interactive display system 100 to enjoy a more unified user experience. From the user's perspective, when using the interactive display system 100 described herein, any selected software application (e.g., which may be running on any computing device connected to the display device) may appear to be running on the display device itself, even though it is strictly speaking running on one of the connected devices.

[0098] In certain embodiments, an interactive display system may be configured to: (1) provide a user of a particular display device with a list of aggregated available applications, where the aggregated available applications include each application available on any connected computing device in addition to any applications available on the display device itself; (2) receive a selection of a particular application from the aggregated list of available applications; and (3) automatically cause a particular connected computing device (e.g., the particular connected computing device on which the selected particular application is available) to display data from the particular selected application on a particular computing device while the particular application is running on the connected computing device. In various embodiments, from the user's perspective, the selected particular application may appear to be running on the particular display device (e.g., even though the application is strictly running on the connected computing device).

[0099] Turning Figure 8 , in certain embodiments, when the connected computing device software integration module 800 is executed, the system is configured to initiate a discovery and registration process for one or more connected computing devices that are operatively connected to a first computing device (e.g., a display device) at step 810.

[0100] In certain embodiments, the system is configured to register the intention to make available on a first computing device (e.g., a display device) the (one or more) applications of a particular connected computing device that are operatively connected via each of the one or more connected computing devices. For example, the system may be configured to register each of the one or more connected computing devices with the first computing device in order to facilitate the software application discovery process described below at step 820. In various embodiments, the system is configured to receive any suitable data strip associated with each respective connected computing device in order to register each device. The system may be configured, for example, to receive: (1) a unique device ID for each of the one or more connected computing devices; (2) an IP address for each of the one or more connected computing devices; and / or (3) any other suitable identifier for each particular connected computing device.

[0101] In certain embodiments, a display device (e.g., a first computing device) is configured to register each of one or more connected computing devices via any suitable connection (e.g., via a serial connection through a serial port in a backplane of the first computing device or the display device). In still other embodiments, the system is configured to establish at least a temporary registration connection between the first computing device (e.g., the display device) and each of one or more connected computing devices via any suitable wired and / or wireless connection (e.g., WIFI, Bluetooth, etc.). In certain embodiments, a connected computing device may be configured to transmit unique device information to the display device (e.g., via one or more encoded pixels (e.g., transmitting encoded data via a suitable wired connection such as via HDMI) to the first computing device). The display device (e.g., the display device processor) may then be configured to receive the unique identifier via one or more encoded pixels.

[0102] In various embodiments, the system is configured to discover one or more connected devices using any suitable discovery protocol. The discovery protocol may include, for example, transmitting one or more registration messages between each of one or more connected computing devices and the first computing device. The one or more registration messages may include, for example, a device ID, an IP address, or any other suitable information.

[0103] Continuing to step 820, when the connected computing device software integration module 800 is executed, the system may be configured to initiate a software application discovery process to identify one or more software applications on each of one or more connected computing devices. In certain embodiments, the system is configured to provide software applications for installation on a particular connected computing device. In various embodiments, the system is configured to provide software applications during the registration process described above. In various embodiments, the software application is configured to identify and maintain a substantially up-to-date (e.g., the most recent) list of information related to one or more software applications available on the connected computing device (e.g., installed on the connected computing device, available for installation and execution on the connected computing device, or otherwise available for execution by the connected computing device).

[0104] In certain embodiments, the system is configured to complete a software application discovery process based at least in part on analyzing (e.g., accessing) data associated with a user account associated with a connected computing device. For example, in various embodiments, a user may have a user account for a particular connected computing device (e.g., a user account with the manufacturer of the connected device, a user account with the provider of the operating system used to connect the computing device, etc.). In certain embodiments, the user account may include associated data identifying one or more software applications that the user has downloaded on the connected computing device (e.g., or one or more associated computing devices also associated with the user account). For example, in certain embodiments, a user may have a Google account that stores data related to one or more applications that the user has downloaded and / or installed on a particular connected computing device (e.g., a Google Chromebox), as well as additional application data that the user has installed on one or more other Google computing devices (e.g., or computing devices utilizing one or more Google operating systems, such as Android), even if those applications are not currently installed on the particular connected computing device (e.g., the computing device connected to the display device or interactive touch display device described herein).

[0105] Next, at step 830, the system may be configured to generate a unified list of each of one or more software applications on each of one or more connected computing devices. In certain embodiments, an interactive display system may be configured, for example, to aggregate a list of one or more available applications on each of a plurality of connected computing devices (e.g., to a particular display device). In some embodiments, a software application on a display device (e.g., an interactive display device) may be configured to: (1) receive application data for each of a plurality of connected devices (e.g., from the respective software applications on each particular connected device); and (2) aggregate the available applications, which includes: (a) the respective devices on which each particular application is installed; (2) input information for the particular respective devices associated with each particular application (e.g., to facilitate an automatic transition to the input associated with the respective connected device after the user selects an application available on the respective connected device, etc.).

[0106] In various embodiments, the display device is configured to communicate with each respective application running on each respective connected device in order to orchestrate a unified experience for the user (e.g., such that from the user's perspective, the system presents to the display device that the user is using an aggregated list of all applications available on any connected device). In certain embodiments, a locker application on the display device (e.g., an interactive display device) presents a list of all available applications on any connected device to the user of the display device.

[0107] Return to step 840, where the system is configured to display a unified list of one or more software applications to a user of the first computing device. For example, Figure 10 FIG. Figure 10 depicts an exemplary screen display showing a unified app repository 1010 (e.g., which may be accessed via menu 610), the unified app repository 1010 including an app list 1020 of all apps available on (1) any connected device to an interactive display device; and (2) on the interactive display device itself. The system may be configured to display the unified list on the first computing device (e.g., the display device) in any suitable manner. For example, the system may be configured to display the unified list via a suitable menu on the first computing device (e.g., a unified overlay menu), via a software repository app on the first computing device, or in any other suitable manner.

[0108] At step 850, the system is configured to receive a selection of a particular application from among one or more software applications from the user. For example, the system may be configured to receive the selection in response to the user's selection of a particular application from a user interface listing each available application (e.g., as shown in FIG. Figure 10 ). The system may be configured to receive the selection from one or more input devices of the first computing device (e.g., one or more touch input devices or other suitable input devices), for example. Figure 10

[0109] Proceed to step 860, where the system is configured to cause a particular connected computing device among one or more connected computing devices on which the particular application is installed to launch the particular application (e.g., in response to receiving the selection at step 580). The system may be configured to, for example: (1) determine which of the one or more connected computing devices has the particular application available; and (2) cause the determined connected computing device to launch the particular application. In a particular embodiment, the system is configured to cause the connected computing device to launch the particular application using one or more remote intent techniques.

[0110] ​In some embodiments, multiple connected computing devices may have the same available software applications (e.g., both a connected Chromebox and a connected Apple TV may have a Netflix application available for use). In such embodiments, the system may be configured to cause any suitable connected device having the selected software application to be available to launch the selected software application. In other embodiments, the system is configured to determine which one of one or more connected devices having a particular software application is most suitable for use in launching the selected particular software application. For example, the system may be configured to determine which one of two or more connected devices having the same particular software application for use on a display device: (1) provides a higher resolution output (e.g., 4k versus 1080p); (2) is operatively coupled to the display device (e.g., a first computing device) using a more reliable connection (e.g., HDMI versus a wireless connection - which may be more prone to packet loss or other issues); (3) has lower power consumption; (4) has the available application already installed for use (e.g., immediately available); (5) has a more recently updated version of the software application available for launch; and / or (6) makes any other suitable determination related to two or more connected computing devices to determine on which one of the connected computing devices (e.g., or the first computing device) to launch the selected software application (e.g., based on processing capabilities, based on connectivity, at least in part based on which connected device the user typically manually selects to use the selected particular application, etc.).

[0111] At step 870, the system is further configured to cause the first computing device to display data from the particular application while the particular application is running on the particular connected computing device.

[0112] Connected Computing Device Software Integration Experience

[0113] Figure 9Depicts an exemplary process flow 900 illustrating a process by which: (1) a connected computing device registers with a first computing device 110 (e.g., first computing device processor 202) operatively coupled to the connected computing device 114; (2) the first computing device 110 (e.g., first computing device processor 202) receives a list of applications installed on the connected computing device 114; (3) a user uses the first computing device 110 to select a particular software application available on the connected computing device 114; (4) the first computing device 110 (e.g., first computing device processor 202) causes the connected computing device to launch, open, or switch to the selected particular application; and (5) the first computing device 110 (e.g., first computing device processor 202) or a display sets the first computing device input to the connected computing device 114 (e.g., such that the first computing device 110 or the display displays a particular software application running on the connected computing device on at least a portion of the display screen of the first computing device).

[0114] Figure 10 Depicts an exemplary screen display 1000 that a user may encounter when using an interactive display device 110 having at least one connected computing device (e.g., Google Chromebox). As can be seen from Figure 10 It can be understood that the screen display 1000 includes a menu 610 through which a user can access various features, software, files, etc. on the interactive display device. The screen display 1000 further includes a unified application repository 1010 (e.g., which may be accessible via the menu 610) that includes an application list 1020 of all applications available on: (1) any connected device to the interactive display device; and (2) on the interactive display device itself. As can be understood in view of the present disclosure, in response to a user selecting a particular application from the application list 1020, the system can be configured to: (1) identify the connected device on which the selected application is available; (2) launch the selected application on the connected device; and (3) set the input of the interactive display 110 to the connected computing device (e.g., such that the interactive display device displays the selected application running on the connected computing device on at least a portion of the display screen of the interactive display device).

[0115] Figure 11 Depicts an exemplary screen display 1100 that a user may encounter when using an interactive display device 110 having at least two connected computing devices (e.g., Google Chromebox and a laptop computer). As can be seen from Figure 11It will be appreciated that the screen display 1100 includes a menu 610 via which a user can access various features, software, files, etc. on the interactive display device. The screen display 1100 further includes an input menu 1110 (e.g., which may be accessible via the menu 610) that shows the current display of each respective connected device 1120, 1130. As can be understood from this figure, the input menu 1110 may display a substantially current (e.g., current) view of what each respective connected device 1120, 1130 is currently displaying. As can be understood from Figure 11 the user interface shown in, a user can select a particular input from the input menu 1110 to switch between the connected devices displayed on the interactive display in a substantially seamless (e.g., seamless) manner. In some embodiments, the system is configured to enable a user to select multiple connected devices to be displayed simultaneously on the interactive display device (e.g., on different respective portions of the interactive display screen, such as in windows or other suitable contexts).

[0116] Figure 12 Exemplary screen display 1200 that a user may encounter when using an interactive display device 110 having at least one connected computing device (e.g., a Google Chromebox) is depicted. As can be understood from Figure 12 It will be appreciated that the screen display 1200 includes a menu 610 via which a user can access various features, software, files, etc. on the interactive display device. The screen display 1200 further includes an open applications menu 1210 (e.g., which may be accessible via the menu 610) that shows the current display of each respective application 1220, 1222, 1224, 1226, 1228, 1230 running on the interactive display or any connected computing device. As can be understood from this figure, the open applications menu 1210 may display a substantially current (e.g., current) view of what each respective open application 1220, 1222, 1224, 1226, 1228, 1230 is currently displaying. As can be understood from Figure 12As can be understood by the user interface shown in [FIGURE REFERENCE], the user can select a specific application (e.g., Application 1224) from the open application menu 1210 to switch to the selected application (e.g., and to the connected device on which the selected application is running) substantially seamlessly (e.g., seamlessly). In some embodiments, the system is configured to enable the user to select multiple applications to be displayed simultaneously on the interactive display device (e.g., on different respective portions of the interactive display screen, such as in windows or other suitable environments). Accordingly, the system can be configured to display multiple applications running on multiple different connected devices. The open application menu 1210 can further include user-selectable markers for closing each respective open application (e.g., causing the system to close the application on the corresponding connected device on which the application is running), and a user-selectable marker for closing all running applications.

[0117] User Interface Adjustment, Customization, and Placement Module

[0118] In various embodiments, the interactive display system 100 can be configured to reduce user fatigue (e.g., due to having to reach different parts of the interactive display, lean back to see different parts of the interactive display, stretch the user's neck to see different parts of the display, etc.). The system can be configured, for example, to modify and / or adjust one or more user interfaces on the interactive display device at least in part based on one or more user characteristics. The system can be configured, for example, to modify and / or adjust the position, size, etc. of one or more user interfaces at least in part based on the user of the interactive display device. In some embodiments, the system can further be configured to cause a user interface to pop up at least in part based on the location of the input provided by the user (e.g., such that the system causes the user interface to appear in a location on the display screen such that the user can see the user interface and be able to interact with the user interface with substantially no movement (e.g., substantially without having to stretch their neck, reposition themselves adjacent to the interactive display device, etc.)).

[0119] In some embodiments, the system can be configured to default the position of one or more user interface features that the user can interact with towards the lower portion of the interactive display, e.g., to make the user interface features more accessible to the user. As discussed above, a particular embodiment of the interactive display device can include a particularly large screen, which can have portions of its display, for example, that are difficult to access due to the height of those portions relative to the user's support surface (e.g., the floor).

[0120] In certain embodiments, the system is configured to modify and / or adjust user interface positioning, sizing, etc. based on, for example: (1) user preference data (e.g., the system can receive and / or determine the user preference data from a user profile when the user logs in or otherwise provides such preference data to an interactive display device using any of the techniques described herein); (2) user selection of screen markers indicating a preferred location / size of the user interface; (3) user body characteristics (e.g., the system can use one or more cameras to determine the user body characteristics to identify the user's height, reach, etc.).

[0121] In still other embodiments, the system is configured to: (1) track user touches (e.g., inputs) in one or more specific regions of the interactive display device (e.g., one or more regions above a specific height such as shoulder height); (2) automatically modify the default position of one or more user interfaces at least in part based on detecting more than a threshold number of inputs above a specific height. This can, for example, enable the system to reduce the number of times the user has to reach above a specific height (e.g., this can reduce injuries, fatigue, etc.) by modifying the user interface positioning (e.g., by lowering the position), thereby reducing user fatigue.

[0122] Turning Figure 13 , in certain embodiments, when executing the user interface adjustment, customization, and placement module 800, the system is configured to receive one or more pieces of user data at step 1310, including one or more of the following: (1) user interface preference data; (2) user body characteristic data; (3) user marker selection data; (4) and so on.

[0123] The system can receive user interface preference data, for example, in response to the user logging in to a specific interactive display (e.g., using any suitable login technique described herein). The system can be configured to store user interface preference data associated with a user account, for example. The system can be configured to prompt each user to provide user interface location preference data at any suitable point during an account registration process (e.g., during registration, when the user first uses the interactive display, etc.). The system can then be configured to store the user preference data so that, in response to the user logging in to the interactive display device (e.g., at a later time), one or more user settings are applied substantially automatically (e.g., automatically).

[0124] In still other embodiments, the system is configured to use one or more imaging devices (e.g., one or more cameras) operatively connected to a particular interactive display device to determine a user's height (e.g., when the user approaches the interactive display board). The system can use any suitable image analysis technique to determine the user's height. In a particular embodiment, the system is configured to determine the height of any suitable part of the user's body (e.g., the user's total height, the user's shoulder height, the user's elbow height, etc.). In a particular embodiment, the system can be configured to use one or more image analysis techniques to determine the length of one or more of the user's arms. In various embodiments, the system is configured to store height and / or reach data associated with the user associated with the user profile and / or account.

[0125] In still other embodiments, the system is configured to provide the user with a selection of one or more selectable markers on the interactive display screen. The system can be configured to receive user marker selection data at least in part based on the user's selection of one or more particular markers. The system can provide one or more user-selectable markers at one or more varying locations (e.g., heights) on the interactive display screen, for example. The system can then be configured to receive a user selection of a particular marker at a particular height.

[0126] Proceeding to step 1320, the system is configured to modify the placement of the user interface on the display screen at least in part based on one or more pieces of user data.

[0127] The system can be configured to store user interface preference data associated with the user account, for example. The system can be configured to prompt each user to provide user interface location preference data at any suitable point during the account registration process (e.g., during registration, when the user first uses the interactive display, etc.), for example. The system can then be configured to store the user preference data so that, in response to the user logging in to the interactive display device (e.g., at a later time), one or more user settings are applied substantially automatically (e.g., automatically).

[0128] In certain embodiments, an interactive display device may enable a user to interact with a display using one or more user interfaces. In certain embodiments, the interactive display device may be configured to display various user interfaces at one or more specific default heights. In some embodiments, the system may be configured to modify the position of various user interfaces at least in part based on one or more user data received at step 1310. For example, the system may be configured to modify the default height of one or more specific user interfaces at least in part based on: (1) stored user preference data; (2) determined user height and / or reach; and / or (3) the position of a marker selected by the user during the above-mentioned marker selection process. For example, the system may be configured to modify the position of one or more user interface features in response to determining that the user is at least a specific height (e.g., using one or more imaging techniques, based on the height of a marker selected by the user, etc.), such as by modifying the user interface position at least slightly upward on the interactive display device. As another example, the system may be configured to modify the position of one or more user interface features in response to determining that the user is less than a specific height (e.g., using one or more imaging techniques, based on the height of a marker selected by the user, etc.), such as by modifying the user interface position at least slightly downward on the interactive display device.

[0129] In still other embodiments, the system is configured to increase and / or decrease the size of one or more user interface features at least in part based on user preference data, user marker selections, etc. In still other embodiments, the system is configured to modify the position or configuration of one or more user interfaces in part based on the user's side preference (e.g., at least in part based on whether the user is right-handed or left-handed).

[0130] At step 1330, the system is configured to optionally track one or more user inputs on one or more specific portions of a display screen (e.g., the interactive display device 110). The specific portions of the display screen can include, for example, a defined "red zone" that includes one or more areas that may be difficult for a user to reach. The system can define the "red zone" at least in part based on one or more user characteristics (e.g., arm reach, height, etc.) described herein. Then, at step 1340, the system can optionally modify the user interface placement at least in part based on the tracking of one or more user inputs. In such an embodiment, the system can be configured to: (1) track user touches (e.g., inputs) in one or more specific areas of the interactive display device (e.g., one or more areas above a specific height such as shoulder height); (2) automatically modify the default position of one or more user interfaces at least in part based on detecting more than a threshold number of inputs above a specific height. This can, for example, enable the system to reduce the number of times a user has to reach above a specific height (e.g., this can reduce injuries, fatigue, etc.) by modifying the user interface positioning (e.g., by lowering the position), thereby reducing user fatigue.

[0131] In some embodiments, the system is configured to track user inputs and modify the default position of one or more interfaces. In other embodiments, the system is configured to track user inputs to modify user-supplied interface positioning preferences.

[0132] Returning to step 1350, the system can be configured to determine the user input location during the selection of a specific user interface feature. Then, at step 1360, the system can display the specific user interface feature at least in part based on the user input location (e.g., and further in part based on one or more user characteristics, frustum characteristics, etc.). The system can, for example, be configured to modify and / or adjust the position, size, etc. of one or more user interfaces at least in part based on the user of the interactive display device. In some embodiments, the system can further be configured to cause a user interface to pop up (e.g., spawn) at least in part based on the location of an input provided by the user (e.g., such that the system causes the user interface to appear in a location on the display screen such that the user can see the user interface and be able to interact with the user interface with substantially no movement (e.g., substantially without having to stretch their necks, reposition themselves adjacent to the interactive display device, etc.)).

[0133] In a particular example, the system can be configured to define a viewing cone for a user based at least in part on: (1) one or more viewing cone definitions; (2) the relative height of an input; (3) the relative height of the user; (4) the relative height of the interactive display device; and / or (5) any other suitable factor. For example, the system can be configured to define a viewing cone for the user that is substantially ovoid (e.g., egg-shaped). In various embodiments, the system is configured to define the viewing cone of the user based at least in part on the line of sight from the user to the location of a touch input provided by the user. For example, a typical user may have a viewing cone that extends at least approximately thirty degrees to the left and right, approximately five degrees upward, and approximately forty-five degrees downward to form a substantially ovoid (e.g., egg-shaped) viewing cone 2200 as shown in the figure. In various embodiments, the system is configured to pop up any user interface within the viewing cone of the user, where the line of sight of the user is defined as being substantially centered (e.g., centered) on the input location provided by the user (e.g., touch point). In additional embodiments, the system can be configured to define the viewing cone based at least in part on the reach of the user (e.g., the approximate distance the user may be standing from the interactive display when providing one or more touch inputs to the interactive display device).

[0134] In various embodiments, the system is configured to pop up (e.g., produce) user interface features within a viewing cone defined by the location of a touch input that causes the production of the user interface features. The system can further be configured to define the viewing cone based on one or more of: (1) one or more viewing cone definitions; (2) the relative height of an input; (3) the relative height of the user; (4) the relative height of the interactive display device; and / or (5) any other suitable factor.

[0135] In still other embodiments, the system is configured to produce user interface features outside of a defined cone of vision. In such embodiments, the system may be configured to at least initially produce user interface features (e.g., or other visual indicators) within the defined cone of vision, and provide a visual indication of the final location of the newly produced user interface features. For example, in response to a user selecting a text entry box, the system may be configured to produce a keyboard interface so that the user can provide text input via a touch display. In various embodiments, the system is configured to cause an interactive display to produce an on-screen keyboard outside of the cone of vision (e.g., because launching the keyboard within the cone of vision may at least partially block the interface where the user is attempting to provide text entry). In this example, the system may be configured to display a visual indication of the final location of the keyboard interface by: (1) displaying at least some visual indicators within the defined cone of vision; and (2) tracking a patch from at least some of the visual indicators to the final location of the keyboard. The system may be configured to use any suitable visual animation (e.g., expanding box, arrow, etc.) to display the transition.

[0136] In still other embodiments, the system may be configured to at least partially distort (e.g., blur) the view of any portion of the display screen that does not include the most recently accessed user interface. In this way, the system may be configured to draw the user's eye to the location of the most recently opened interface feature.

[0137] User Interface Adjustment, Customization, and Placement User Experience

[0138] Figures 14 - 21 An exemplary screen display depicting a particular user interface is shown, through which a user can provide input to the system, and when one or more user interfaces are presented to the user, the system can use the input to determine the location and / or sizing of one or more particular user interfaces.

[0139] As Figure 14As shown, the display screen 1400 includes a user interface 1410 and a plurality of user-selectable markers 1420, 1430, 1440, 1450 at varying heights. As can be understood in view of the present disclosure, the system can be configured to instruct the user to select a specific marker among the user-selectable markers 1420, 1430, 1440, 1450 that is at a "comfortable height" (e.g., elbow height, below shoulder height, etc.). The system can then be configured to modify the position of the user interface 1410 at least in part based on the selected user-selectable markers 1420, 1430, 1440, 1450. For example, in response to the user selecting a relatively low marker (e.g., 1420), the system can be configured to display the user interface (1410) at a relatively low height. In response to the user selecting a relatively high marker (e.g., 1450), the system can be configured to display the user interface (1410) at a relatively high height, as Figure 15 shown. As can be understood from the Figure 14 and 15 screen displays shown, the system can be configured to modify the height of the user interface based on the marker selection provided by the user. In various embodiments, the system is configured to provide the user-selectable markers 1420, 1430, 1440, 1450 at varying heights to the user during the user's first use of the interactive display device. In some embodiments, the system can then retrieve user height preference data and apply the user preferences during the user's subsequent use of the interactive display device. In some embodiments, when applying user preference settings regarding user interface positioning (e.g., height, size, orientation, etc.) to a particular interactive display device, the system can be configured to further modify one or more interface positions at least in part based on the installation height of the particular interactive display device. For example, when applying height settings to one or more user interfaces on a display device, the system can be configured to take into account the height of the display device via which the user initially provided his or her preferences (e.g., using the Figure 14 and 15 shown markers) compared to the height of a second display device that the user will use later.

[0140] Figure 16 and 17 depict screen displays 1600, 1700 that a user may encounter when user interface size preferences are provided to the system. The system can, for example, display the user interface 1410 along with a plurality of size selection markers. As can be understood from the Figure 16 and 17 , the user's selection of different size selection markers 1620 can cause the system to modify the display size of one or more system interface features (e.g., icons). Figure 16Depicts a user interface 1410 that the system can provide in response to a user's selection of a selection marker 1620 of a relatively small (e.g., small) size. Figure 17 Depicts an exemplary user interface 1720 that the system can be configured to display in response to a user's selection of a selection marker 1620 of a relatively large (e.g., large) size.

[0141] Figure 18 and 19 Depicts exemplary screen displays 1800, 1900 that a user may encounter when one or more user interface positioning preferences are provided to the system. As can be seen from Figure 18 and 19 As can be understood, the screen display can include a user interface 1800 having one or more user-selectable buttons 1830. As can be understood from these figures, the system can be configured to enable a user to modify (e.g., and / or automatically modify) the default positions of such user-selectable buttons 1830 within the user interface 1810. The system can, for example, be configured to enable a user to select different positions 1820 for placing the user-selectable buttons 1830 within the user interface. In this way, the system can be configured to provide more accessible user interface features to the user based at least in part on: (1) the user's preferences; (2) the user's dominant hand; and / or (3) any other suitable factors.

[0142] Figure 20Depicts an exemplary display screen 2000 that a user may encounter when using an interactive display device 110 with a connected computing device (e.g., a Google Chromebox). As can be understood from an embodiment of the user interface 2000 shown in this figure, the user interface includes a menu 610 native to the interactive display device (e.g., running locally on the interactive display device), a window interface 630 that displays a software application (e.g., a web browser) running on the connected computing device, and a wrapper 620 around the window interface 630 native to the interactive display device. As can be understood from this figure, the wrapper 620 includes a plurality of user-selectable buttons 2015, which can be configured, for example, to perform any suitable function (e.g., closing the window interface, minimizing the user interface, etc.). In some embodiments, the wrapper 620 may include a text entry box 2010, which can be configured, for example, to mirror one or more text entry boxes 635 within the window interface 630 that displays a software application (e.g., a web browser) running on the connected computing device. In this way, the system can be configured to receive text entry box 635 data from the connected computing device and display the text entry data in the text entry box 2010 in the wrapper for easy viewing by the user. For example, the user may be able to view what the user is typing in the software application displayed in the window interface 630 within the wrapper. In a particular embodiment, the system is configured to transmit and receive information to / from both the text entry box 2010 in the wrapper 620 and the text entry box 635 on the software application 630 running on the connected device. In this way, the user may be able to easily view, based on, for example, the following: (1) where the user is standing relative to the interactive display device; (2) how close the user is standing to the interactive display device; (3) and so on, via which text entry box it is easier to view the information via the screen display 2000.

[0143] In a particular embodiment, the system is configured to interpret one or more inputs received at the wrapper 620 (e.g., which is running on the interactive display device itself) as one or more inputs to the software application 630 running on the connected computing device. One or more buttons 2015 may include, for example, a back button, a favorites button, etc., which can be configured, for example, to cause the connected device to perform certain functionality on the software application 630 running on the connected device (e.g., even if the system receives the input at a location consistent with an application running locally on the interactive display device (e.g., the wrapper 620)). In this way, the system can be configured to provide certain functionality via the wrapper 620 via one or more buttons 2015 physically positioned lower on the display (e.g., at the bottom of the wrapper 620).

[0144] In various embodiments, as described herein, the interactive display device 110 can be particularly large (e.g., up to approximately 100 inches or greater in diagonal measurement). As such, when standing adjacent to the interactive display device 110 (e.g., within arm's reach of the interactive display device 110), a user may have difficulty reaching and / or seeing each part of the display screen 310 or the interactive display device 110. As such, by providing one or more inputs (e.g., text entry box 2010 and / or one or more buttons 2015) configured to transmit functionality or other input to a software application 630 running on a connected computing device, the system is configured to enable a user to more easily reach a particular function of the software application 630 (e.g., which may typically be located at the upper portion of the interface for the software application 630).

[0145] Figure 21 depicts Figure 20 a screen display 2100 having an on-screen keyboard 2110 for entering text via a text entry box 2120, and the system can be configured to transmit the input to a text entry box 635 of a software application 630 running on a connected device and displayed on the interactive display screen. For example, in response to a user selecting the text entry box 635, the system can be configured to produce a keyboard interface to enable the user to provide text input via the touch display. In various embodiments, the system is configured to cause the interactive display 2110 to produce an on-screen keyboard within the user's cone of vision (e.g., as discussed above). In other embodiments, the system can be configured to produce an on-screen keyboard outside of the cone of vision (e.g., because initiating the keyboard within the cone of vision may at least partially block the interface where the user is attempting to provide text entry). In this example, the system can be configured to display a visual indication of the final position of the keyboard interface by: (1) displaying at least some visual indicators within the defined cone of vision; and (2) tracking a patch from at least some of the visual indicators to the final position of the keyboard. The system can be configured to use any suitable visual animation (e.g., expanding box, arrow, etc.) to display the transition.

[0146] In some embodiments, the on-screen keyboard 620 may include a text entry box 2110, which may be configured, for example, to mirror one or more text entry boxes 635 within a window interface 630 that displays a software application (e.g., a web browser) running on a connected computing device. In this way, the system may be configured to receive text entry box 635 data from the connected computing device and display the text entry data in a text entry box 2120 in the on-screen keyboard 2110 for easy viewing by the user. For example, the user may be able to view what the user is typing in the software application 630 displayed in the window interface 630 within the on-screen keyboard 2110. In certain embodiments, the system is configured to transmit and receive information to / from both the text entry box 2120 in the on-screen keyboard 2110 and the text entry box 635 on the software application 630 running on the connected device. In this way, the user may be able to easily view, via the screen display 2100, based on, for example, the following: (1) where the user is standing relative to the interactive display device; (2) how close the user is standing to the interactive display device; (3) and so on, which text entry box it is easier to view the information through.

[0147] Figure 22 An exemplary representation of a user's visual cone is depicted. As can be understood from the figure, the system may be configured to define a user's visual cone at least in part based on the line of sight from the user to the location of a touch input provided by the user. For example, a typical user may have a visual cone that extends at least approximately thirty degrees to the left and right, approximately five degrees upward, and approximately forty-five degrees downward to form a generally oval-shaped (e.g., ovoid) visual cone 2200 as shown in the figure. In various embodiments, the system is configured to pop up any user interface within the user's visual cone, where the user's line of sight is defined as being substantially centered (e.g., centered) on the input location (e.g., touch point) provided by the user.

[0148] Single Sign - on User Experience

[0149] In certain embodiments, the interactive display system is configured to enable a user to log in to any particular interactive display in order to apply user-specific settings, preferences, etc. to that particular interactive display, such that the user can walk up to any particular interactive display, log in, and have the same user experience when using any interactive display.

[0150] In various embodiments, each particular interactive display device is configured to operate in at least two modes: (1) a logged-in mode, where the user has logged in to the device at least temporarily; and (2) a non-logged-in mode, where the interactive display panel is configured to operate using one or more default settings.

[0151] In some embodiments, the system is configured to enable a user to log in using a username and password (e.g., associated with a user account previously created by the user). In some embodiments, the system is configured to enable a user to register a user account on an interactive display device. In still other embodiments, the system is configured to enable a user to register a user account in any other suitable location (e.g., at a website associated with the developer of the interactive display device, using a software application provided by the developer of the interactive display device, using a different computing device, etc.).

[0152] Figure 23 Depicts an exemplary screen display 2300 that a user may encounter when logging in to an interactive display device. As can be understood from the figure, the system can be configured, for example, to provide a user interface 2310 through which a user can log in to the system (e.g., by directly logging in to the interactive display device).

[0153] In some embodiments, the system is configured to enable a user to log in using any other suitable technique. For example, in some embodiments, the system is configured to receive identification data (e.g., profile data) of the user via one or more remote computing devices (e.g., mobile computing devices), and the user logs in to the user account via the remote computing device. For example, the system can be configured to provide an accompanying application for installation on a mobile computing device (e.g., a smart phone) associated with the user. In response to determining that the user is attempting to log in to the interactive display device using the logged-in mobile computing device (e.g., or other suitable computing device), the system can also be configured to automatically log the user in to the interactive display device. For example, the system can be configured to wirelessly receive data indicating that the user has logged in to the accompanying application from the mobile computing device (e.g., via Bluetooth, NFC, etc.).

[0154] In still other embodiments, the system is configured to automatically log the user in to the interactive display device in response to the user scanning a machine-readable marker on the interactive display device using an accompanying application that the user has logged in to on a remote computing device (e.g., a smart phone). For example, as Figure 24 shown, the system can display a user login interface 2410 that includes a machine-readable marker (e.g., a QR code, a linear barcode, or any other suitable machine-readable marker). Then, the user can use their remote computing device having at least one imaging device (e.g., a camera) to scan the machine-readable marker. In response to receiving the scan of the machine-readable marker, the interactive display system can be configured to identify the interactive display device based on the machine-readable marker and automatically log the user in to the interactive display device. In some embodiments, the machine-readable marker is unique to the interactive display device.

[0155] In still other embodiments, the system is configured to log a user into the interactive display device using any other suitable technique, such as, for example: (1) one or more facial recognition techniques (e.g., using one or more imaging devices operatively connected to the interactive display device); (2) one or more suitable voice recognition techniques (e.g., using one or more microphones operatively connected to the interactive display device); and / or (3) any other technique for identifying the user to retrieve account / profile / preference data for the user.

[0156] In a particular embodiment, in response to the system logging a particular user into a particular interactive display device, the system is configured to: (1) retrieve one or more settings and / or preferences associated with the user; (2) apply the one or more settings and / or preferences to the particular interactive display device. The settings and / or preferences can include, for example: (1) one or more user interface settings discussed herein; (2) brightness and other hardware settings; (3) volume settings, etc. In some embodiments, the system is configured to retrieve account data for one or more accounts connected to the user's interactive display user account. For example, the system can be configured to enable the user to associate one or more third-party accounts with the user's interactive display device account. The system can be configured to store one or more third-party account credentials associated with the user account and automatically log the user into one or more third-party services or applications available on the interactive display device. For example, the system can be configured to automatically log the user into a cloud storage account on the interactive display device and configured to make one or more files from the user's associated cloud storage account available to the user via the interactive display device when the user logs into the interactive display device.

[0157] In certain embodiments, when a user logs in to an interactive display device, the system is configured to provide one or more functions via an accompanying application installed on an auxiliary computing device (e.g., a mobile computing device). For example, the system can be configured to use the accompanying application such that the user can easily log in to the interactive display device, and the system can be configured to provide one or more inputs to the interactive display device via the connected accompanying application. The system can be configured, for example, to transmit user interface selections and inputs made via the accompanying application to the interactive display device in any suitable manner (e.g., via a suitable wireless and / or wired connection). In some embodiments, the accompanying application can provide, for example, an on-screen keyboard for entering text on the interactive display device or any other suitable functionality. In some embodiments, the accompanying application is configured to provide functionality at least in part based on the software application that the user is currently using on the interactive display device (e.g., whether the application is running natively on the interactive device as described herein or on one or more connected computing devices).

[0158] In some embodiments, the system is configured to substantially automatically log the user in to one or more connected computing devices in response to the user also logging in to the interactive display device. For example, the system can be configured to automatically log the user in to a third-party computing device (e.g., using third-party user profiles and / or account data stored by the system associated with the user's interactive display device account). For example, a user can link their Google account to their interactive display device account such that the system can automatically log the user in to a Google Chromebox operatively connected to the interactive display device in response to the user logging in to the interactive display device.

[0159] Figures 23 - 25 Depicts an exemplary user interface that a user may encounter when logging in to a particular interactive display. The interactive display can be configured, for example, to: (1) receive user identification data; (2) optionally authenticate the user (e.g., by requiring the user to enter a password or other unique identifier); (3) retrieve user profile data in response to receiving the user identification data and optional authentication data (e.g., via a network from one or more remote servers); (3) apply the user profile data to one or more settings of the interactive display; (4) optionally use the user profile data to automatically log in to one or more connected computing devices; and / or (5) optionally modify one or more features and / or functionality of the interactive display at least in part based on the role of the logged-in user (e.g., whether the user is a teacher, student, administrator, etc.).

[0160] In certain embodiments (e.g., in an educational setting), the user is enabled to easily approach the interactive display device and easily log in, have access to any applications the user typically uses, have access to any files the user may need, etc. In various embodiments, the system is configured such that the user can approach any interactive display device and enjoy the same experience in terms of the applications, content, preferences, etc. available via it.

[0161] In some embodiments, the system is configured to log out the user substantially automatically (e.g., automatically) according to a specific schedule (e.g., at the same time each day, such as in the evening). In other embodiments, the system is configured to enable the user to log out once the user has finished using the interactive display device. In some embodiments, logging out may also cause the system to automatically log the user out of any third-party services and / or connected devices.

[0162] In various embodiments, the system is configured to modify one or more functions of the interactive display device at least in part based on the role of the logged-in user. For example, the system may modify functionality based on whether the logged-in user is a teacher, student, IT staff, administrator, etc.

[0163] Projection User Experience

[0164] Figure 26 and 27 Exemplary screen displays 2600, 2700 that a user may encounter when using the interactive display device 110 to project a remote computing device 150 to the interactive display device 110 are depicted. During the projection, the system may be configured to cause a wireless touchback from the interactive display device 110 to the remote computing device 150 that is projecting to the interactive display device 110 while the remote computing device 150 is projecting to the interactive computing device. For example, a student may project a worksheet completed by the student (e.g., via any suitable network described herein) from their tablet computer to the interactive display device.

[0165] In some embodiments, the system is configured to project a remote computing device (e.g., wirelessly) onto at least a portion of the display screen of the interactive display device. In various embodiments, the system is configured to transfer data from the remote computing device to the interactive display device via a software application on the remote computing device. The system can, for example: (1) provide a unique interactive display device identifier; (2) receive the unique interactive display device identifier at the remote computing device (e.g., via a software application on the remote computing device); and (3) in response to receiving the unique interactive display device identifier at the remote computing device, initiate projecting at least a portion of the display screen of the remote computing device (e.g., or one or more software applications running on the remote computing device) onto the interactive display device. As Figure 26 shown, the system can, for example, display a unique panel code on the interactive display device. The user can then enter the panel code on the remote computing device to initiate the projection.

[0166] In some embodiments, the system is configured to display a live preview of one or more devices that are attempting to project to a particular interactive display device. As Figure 27 shown, the system can display a thumbnail of each of the multiple devices that are attempting to project to the interactive display device. The user of the interactive display device can then select a particular remote device for projection.

[0167] In various embodiments, the system is configured to facilitate a connection between the interactive display device and each particular remote computing device using any suitable technique (e.g., wired and / or wireless connections, via a direct connection, over a network, using an ad-hoc network, etc.).

[0168] Additional User Experience Descriptions

[0169] Figures 28 - 30 Exemplary screen displays that a user may encounter when accessing various other features of the system are depicted, and these features are discussed more fully below.

[0170] Figure 28 Exemplary screen display 2800 that a user may encounter when using an interactive display device 110 with at least one connected computing device (e.g., a Google Chromebox) is depicted. As can be seen from Figure 28It is understandable that the screen display 2800 includes a menu 610 through which a user can access various features, software, files, etc. on the interactive display device. The screen display 2800 further includes an open application menu 2800 (e.g., which can be accessed via the menu 610), which shows the current display of each corresponding application running on the interactive display or any connected computing device. As can be understood from this figure, the open application menu 2800 can display a substantially current (e.g., current) view of what each corresponding open application is currently displaying. From Figure 28 the user interface shown in, a user can select a specific application from the open application menu 2800 to switch to the selected application (e.g., and to the connected device on which the selected application is running) substantially seamlessly (e.g., seamlessly). In some embodiments, the system is configured to enable a user to select multiple applications to be displayed simultaneously on the interactive display device (e.g., on different corresponding portions of the interactive display screen, such as in windows or other suitable environments). Accordingly, the system can be configured to display multiple applications running on multiple different connected devices. The open application menu 2800 can further include user-selectable markers for closing each corresponding open application (e.g., causing the system to close the application on the corresponding connected device on which the application is running), and a user-selectable marker for closing all running applications 2805.

[0171] Figure 29 depicts an exemplary screen display 2900 that a user may encounter when using the interactive display device 110. From Figure 29 it is understandable that the screen display 2900 includes a menu 610 through which a user can access various features, software, files, etc. on the interactive display device. The screen display 2900 further includes a file repository menu 2910 (e.g., which can be accessed via the menu 610). In the embodiment shown in this figure, the system can be configured to aggregate multiple files, the files being: (1) locally stored on the interactive display device; (2) remotely stored in one or more remote storage locations; (3) stored in cloud storage or a similar third-party file storage service; (4) and so on. In various embodiments, the system is configured to enable a user to access any such file via Figure 29 the file repository menu 2910 shown in.

[0172] Figure 30 depicts an exemplary screen display 3000 that a user may encounter when using the interactive display device 110. As from Figure 30It can be understood that the screen display 3000 includes a menu 610 through which a user can access various features, software, files, etc. on the interactive display device. The screen display 3000 further includes a settings menu 3010 that a user can use to, for example: (1) modify one or more settings of the interactive display device; (2) modify one or more settings of one or more connected computing devices; (3) and so on.

[0173] In some embodiments, the system is configured to automatically modify system settings (e.g., as well as one or more connected device settings) in response to a user logging in to the interactive display device as discussed herein. In response to a user modifying one or more settings via the settings menu 3010, the system can be configured to modify profile data (e.g., preference data) associated with the user account and / or user profile to reflect the changes (e.g., such that the system can automatically implement similar changes to the settings when the user logs in in the future).

[0174] Software Usability Supervision Module

[0175] In certain embodiments, the interactive display system 100 is configured to provide a plurality of available software applications (e.g., via an app store or a similar catalog of available software applications), which includes first-party software (e.g., provided by the manufacturer of the interactive display device) and approved and / or vetted third-party applications for use on the interactive display device. In certain embodiments, the system is configured to provide an administrator or other authorized user with the ability to select specific applications (e.g., and / or content) from the list of available software applications such that they are available on one or more interactive display devices for which they are an administrator (e.g., one or more devices within a particular school, school district, etc.). In certain embodiments, the system is configured to generate a curated app store based on the selections of the administrator (e.g., or other authorized user), which includes only those applications selected by the administrator. Individual users of the interactive displays managed by the administrator can then select from the curated list of approved applications for use on a particular interactive display device. In this way, the system can be configured to provide a whitelist of applications from a larger app catalog for selection by users (e.g., teachers) within a particular group (e.g., a particular school, school district, county, city, etc.).

[0176] Turning Figure 31 , in certain embodiments, when the software availability regulatory module 3100 is executed, at step 3110, the system is configured to provide a plurality of available software applications and / or content for use on a plurality of computing devices.

[0177] Next, at step 3120, the system is configured to receive a selection of approved software applications and / or content from among a plurality of available software applications and / or content for availability to a particular subset of a plurality of computing devices.

[0178] Returning to step 3130, the system is configured to provide the approved software applications and / or content to one or more users for use on a particular subset of the plurality of computing devices.

[0179] In some embodiments, the system is configured to generate a curated app store that can include apps that have been subscribed to and purchased as well as approved apps / content that teachers / users can elect to purchase. In some embodiments, an administrator can also select what information (e.g., username, email address, etc.) to transmit from an interactive display device to one or more app developers. In certain embodiments, the system is configured to make internal content (e.g., within a particular school district / school) available. In still other embodiments, the system is configured to make teacher / student-created apps available, e.g., via the curated app store.

[0180] In some embodiments, the system is configured to enable a plurality of users to form a group of app and / or content sources (e.g., a group of social studies teachers in the Atlanta school district). In some embodiments, relying on a school administrator to select particular apps can enable an app store provider to fulfill various legal obligations related to privacy. By providing consent to use a particular app and enabling sub-users to select from a list of approved apps, a school administrator can provide the necessary consent on behalf of end users of one or more interactive display devices (e.g., of an app store having at least some available apps).

[0181] Situational Environment Analysis Module

[0182] In various embodiments, one or more interactive display devices 110 (e.g., or other computing devices) can be used for environmental analysis, with a particular application in the real-time localization of gunfire in an active shooter scenario. In such embodiments, multiple interactive display panels in one or more fixed physical locations can be configured to perform passive analysis of the audio environment to infer information about the environment. In such embodiments, each of the multiple interactive display devices can include at least one microphone. By substantially continuously monitoring the audio environment, the multiple interactive display devices can be configured to: (1) identify at least one audio cue; (2) determine the location of the identified audio cue based on sound data received at one or more of the multiple interactive display devices; and (3) generate one or more warnings associated with the identified audio cue. Although the following examples will generally be described with respect to identifying and analyzing the sound of gunfire, it should be understood that in other embodiments, the system can be configured to identify any other suitable audio cue. In such embodiments, one or more additional audio cues can include, for example, glass breaking, smoke alarms, detection of certain words (e.g., "help", "fire", etc.).

[0183] In still other embodiments, the system can be configured to use any suitable technique to analyze the identified audio cue. For example, the system can be configured to utilize one or more audio fast Fourier transform techniques. When utilizing one or more audio fast Fourier transform techniques, the system can be configured to filter the audio spectrum to analyze only those regions of sound that match the frequency and loudness of what the system is set to monitor (e.g., gunshots, etc.). The fast Fourier transform can, for example, enable low-latency passive monitoring while maintaining the privacy of the interactive display device users. In various embodiments, utilizing the fast Fourier transform, the interactive display device can be configured to listen to the room at all times, but ignore every human voice and focus only on the selected audio cues.

[0184] In yet other embodiments, the system can be configured to use one or more sound triangulation techniques to identify the location of the registered audio cue. For example, the system can be configured to use multiple audio microphones (e.g., in multiple interactive display devices spread throughout a building, e.g., in various different classrooms) to compare the differences between the same sound source and its known geographical coordinates to determine its geographical location.

[0185] In response to identifying a particular audio cue, such as gunfire, the audio cue can pass through the fast Fourier transform system and trigger each interactive display device that detected the audio cue to compare the sound with other interactive display devices in the area. Each interactive display device that registered the audio cue can use the loudness registered at each device to infer the location of the audio cue in space.

[0186] Illustrative Example of the Situational Environment Analysis Process

[0187] Figure 33 Depicts a top - down floor plan of school 3300. As shown in this figure, school 3300 has a plurality of interactive display devices 110 installed throughout the various rooms of the school (such as classrooms, media rooms, etc.). As in Figure 33 shown, audio cues can be located at various locations 3301, 3302, 3303 throughout the school. The audio cue can initially be identified at a first location 3301 and then later at a second location 3302. This can, for example, indicate that the source of the audio cue (such as an active shooter) may be moving throughout the school. Then, the system can transmit substantially up - to - date location information to law enforcement officials or others.

[0188] Conclusion

[0189] Benefiting from the teachings presented in the foregoing description and the associated drawings, those skilled in the art to which this invention pertains will envision many modifications and other embodiments of this invention. Although this specification contains many specific implementation details, these should not be construed as limitations on any invention or the scope that may be claimed, but rather as descriptions of features specific to particular embodiments that may be specific to a particular invention. Certain features described in the context of separate embodiments in this specification can also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment can also be implemented separately in multiple embodiments or in any suitable sub - combination. Additionally, although features may be described as acting in certain combinations above and even initially claimed as such, in some cases, one or more features from a claimed combination can be deleted from that combination, and the claimed combination can be directed to a sub - combination or a variation of a sub - combination.

[0190] Similarly, although operations are depicted in the drawings in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in a sequential order, or that all illustrated operations be performed to achieve a desired result. Additionally, although some steps may be described as occurring in response to (e.g., or at least in part in response to) certain other steps, it should be understood that in other embodiments, such steps can occur independently of each other (e.g., or concurrently). In certain cases, multitasking and parallel processing may be advantageous. Moreover, the separation of the various system components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0191] Benefiting from the teachings presented in the foregoing description and the associated drawings, many modifications and other embodiments of the present invention will occur to those skilled in the art to which this invention pertains. Accordingly, it is to be understood that the invention is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. An interactive display system, comprising: A display device having at least one processor and a display screen, wherein the at least one processor is configured to: Receive one or more user interface location determination data of a user of the display device, the one or more user interface location determination data including at least one reference height; Determine whether to modify a default position of at least one user interface on the display device at least in part based on the one or more user interface location determination data; In response to determining to modify the default position of the at least one user interface on the display device, modify a default output position of the at least one user interface, wherein modifying the default output position includes modifying the default output position to adjust a height of the default output position, wherein The display device includes an interactive touch screen display device having an interactive touch screen display input; and The at least one processor is further configured to: Receive a plurality of user inputs via the interactive touch screen display input; Analyze the plurality of user inputs to determine whether more than a threshold number of the user inputs have input positions higher than the at least one reference height; In response to determining that more than a threshold number of the user inputs have input positions higher than the at least one reference height, modify the default output position to reduce the height of the default output position.

2. The interactive display system according to claim 1, wherein, The at least one reference height is selected from the group consisting of: The height of the user; The height of the user's shoulder; The height of the user's elbow.

3. The interactive display system according to claim 2, wherein: The display device includes at least one imaging device; The at least one processor is further configured to: Use the at least one imaging device to capture at least one image of the user; and Analyze the at least one image to determine the at least one reference height.

4. The interactive display system according to claim 2, wherein: Determining whether to modify the default position of the at least one user interface on the display device at least in part based on the one or more user interface location determination data includes: determining whether the at least one reference height is higher than or lower than a default reference height; The at least one processor is further configured to, in response to determining that the at least one reference height is lower than the default reference height, modify the default output position to reduce the height of the default output position.

5. The interactive display system according to claim 2, wherein: The display device includes an interactive touch screen display device having an interactive touch screen display input; The at least one processor is further configured to: Display a plurality of user-selectable markers to the user on the display screen, the plurality of user-selectable markers including: A first user-selectable marker at a first height; and A second user-selectable marker at a second height higher than the first height; Receive a selection of a marker among the plurality of user-selectable markers via the interactive touch screen display input; In response to receiving a selection of the second user-selectable marker, modify the default output position to increase the height of the default output position; and In response to receiving a selection of a first user-selectable marker, modify a default output position to reduce the height of the default output position.

6. The interactive display system according to claim 5, wherein, the at least one processor is further configured to: store an indication of a selection of a marker among the plurality of user-selectable markers in a computer memory; associate the indication of the selection of the marker with a user in the computer memory.

7. The interactive display system according to claim 1, wherein, the at least one processor is further configured to determine whether to modify a default position of the at least one user interface on a display device based at least in part on: the size of the display screen; and the mounting height of the display screen.

8. A computer-implemented data processing method for modifying a position of at least one user interface on an interactive touch screen display device, the method comprises: receiving, by one or more processors, one or more pieces of user interface positioning determination data of a user of the interactive touch screen display device; determining, by one or more processors, a default position of the at least one user interface based at least in part on the one or more pieces of user interface positioning determination data; displaying, by one or more processors, the at least one user interface on the interactive touch screen display device based on the determined default position; receiving, via the interactive touch screen display device, a plurality of user inputs; analyzing the plurality of user inputs to determine whether more than a threshold number of the user inputs have an input position higher than a threshold height; and in response to determining that more than a threshold number of the user inputs have an input position higher than a threshold height, modifying the default position to reduce the height of the default position.

9. The computer-implemented data processing method according to claim 8, the method further comprises: receiving, by one or more processors, an input on at least one user interface at a first position on the interactive touch screen display device, the input triggering the output of a second user interface; defining, in response to receiving the input, a viewing cone by one or more processors based at least in part on the first position; and causing, by one or more processors, a visual indication of the second user interface to be output at least temporarily within the viewing cone.

10. The computer-implemented data processing method according to claim 9, wherein: the one or more pieces of user interface positioning determination data include a user's reach; defining the viewing cone is further based at least in part on the user's reach.

11. The computer-implemented data processing method according to claim 8, the method further comprises: displaying, on the interactive touch screen display device, a plurality of user-selectable markers to a user, the plurality of user-selectable markers including: a first user-selectable marker at a first height; and a second user-selectable marker at a second height higher than the first height; receiving, via the interactive touch screen display device, a selection of a marker among the plurality of user-selectable markers; in response to receiving a selection of the second user-selectable marker, modifying the default position of the at least one user interface to increase the height of the default position; and In response to receiving a selection of a first user-selectable marker, modify a default position of the at least one user interface to reduce a height of the default position.

12. The computer-implemented data processing method according to claim 8, wherein: the one or more user interface location determination data includes a user's dominant hand; the method further includes modifying a position of at least one user-selectable marker on the at least one user interface at least in part based on the user's dominant hand.

13. The computer-implemented data processing method according to claim 8, wherein the method further includes: modifying a size of at least one user-selectable marker on the at least one user interface at least in part based on the one or more user interface location determination data.

14. An interactive display panel, comprising: at least one processor; a touch input operatively coupled to the at least one processor; and a display operatively coupled to the at least one processor, wherein the at least one processor is configured to: receive a first marker from a first user; determine a first user identity of the first user at least in part based on the received first marker; configure a first user interface at least in part based on one or more characteristics associated with the first user identity; display the first user interface on the display, wherein the one or more characteristics associated with the first user identity are selected from the group consisting of: a height of the first user; a reach of the first user; a mounting height of the interactive display panel above the floor; and whether the first user is right-handed or left-handed; and receive a plurality of user inputs via the touch input; analyze the plurality of user inputs to determine whether more than a threshold number of the user inputs have input positions above a threshold height; and in response to determining that more than the threshold number of the user inputs have input positions above the threshold height, modify a default position of the first user interface on the display to reduce the height of the default position.

15. The interactive display panel according to claim 14, wherein, the at least one processor is further configured to: display a plurality of selectable markers on the display; receive a first input from a second user from the plurality of selectable markers displayed on the display; reconfigure the first interface to a second interface at least in part based on the first input from the second user; and display the reconfigured first interface on the display.

16. The interactive display panel according to claim 14, wherein: the first input provides height information for the second user; and reconfiguring the first interface to the second interface further includes adjusting the first interface based on the height information for the second user.

Images