Cross-Application Sharing of User Interface Customization

Abstract

Methods, systems, and computer program products are provided for sharing user interface (UI) customization across applications. Application developers can use control clusters that can be customized as clusters to create a UI (e.g., a game control layout). A control cluster can include multiple controls that are selectively assigned to a socket cluster (e.g., a socket cluster) in a layout template. The controls in a control cluster can be customized as a cluster, for example, by changing the position, rotation, and / or scale of the control cluster. A user can customize the controls for comfort, switch the controls for playing games with the left and right hands, and share the customized preferences across applications. UI customization can be shared across applications with different UIs. For example, the customization of one or more control clusters in a first UI can be applied to one or more control clusters in other UIs of other applications. A user can avoid making tedious and time-consuming individual control changes for many UIs.

Description

[0001] Background

[0002] Application developers can develop user interfaces for users to interact with applications on a device. An example of a user interface (UI) is a touch screen interface that provides a graphical user interface (GUI), which enables a user to interact with the device using touch (e.g., by applying one or more fingers). The GUI may include one or more displayed controls for the user to interact with on the touch screen. The layout of the UI tends to be unique for each application.

[0003] Overview

[0004] This summary is provided to introduce a selection of concepts that are further described below in the detailed description in a simplified form. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0005] Methods, systems, and computer program products are provided for sharing user interface (UI) customization across applications. Application developers can use control clusters that can be customized as a group to create a UI, such as a game control layout. A control cluster may include multiple controls that are selectively assigned to a socket cluster in a layout template, and the socket cluster may be referred to as a socket group. The controls in a control cluster can be customized as a group by modifying the control cluster as a whole. Users can also customize controls individually. In addition, UI customization can be shared across applications with different UIs.

[0006] The following further describes the features and advantages of the embodiments and the structure and operation of each embodiment in detail with reference to the accompanying drawings. Note that the methods and systems are not limited to the specific embodiments described herein. These embodiments are presented herein for illustrative purposes only. Based on the teachings contained herein, additional embodiments will be apparent to those skilled in the relevant art. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings incorporated herein and constituting a part of this specification illustrate embodiments of the present application and, together with the specification, further serve to explain the principles of the embodiments and to enable those skilled in the relevant art to implement and use these embodiments.

[0008] Figure 1 A block diagram of an example system for user interface customization across applications according to an embodiment is shown.

[0009] Figure 2 An example of a control socket template on a touch screen according to an embodiment is shown.

[0010] Figures 3A - 3GShows an example of the selective configuration of sockets and socket grouping in a touch control socket template according to various embodiments.

[0011] Figure 4A Shows an example of the control layout and control layout library on a touch screen according to an embodiment.

[0012] Figure 4B Shows an example of the active first control layout on a touch screen according to an embodiment.

[0013] Figure 5 Shows an example of an interface control manager and a first layout grouping on a touch screen according to an embodiment.

[0014] Figure 6 Shows an example of selecting a second control layout in a first layout grouping on a touch screen according to an embodiment.

[0015] Figure 7 Shows an example of the active second control layout on a touch screen according to an embodiment.

[0016] Figure 8 Shows an example of the first control layout in a layout editor on a touch screen according to an embodiment.

[0017] Figure 9 Shows an example of scaling and rotating the left control grouping in the first control layout on a touch screen according to an embodiment.

[0018] Figure 10 Shows an example of simultaneously repositioning the right and left control groupings in the first control layout on a touch screen according to an embodiment.

[0019] Figure 11 Shows an example of an activated customized version of the first control layout on a touch screen according to an embodiment.

[0020] Figure 12 Shows an example of sharing the customization of the first control layout with a second control layout in the same layout grouping by repositioning, scaling, and rotating the right and left control groupings on a touch screen according to an embodiment.

[0021] Figure 13 Shows an example of browsing a control layout grouping library on a touch screen according to an embodiment.

[0022] Figure 14 Shows an example of sharing the customization of the first control layout in the first layout grouping with the first control layout in a second layout grouping on a touch screen according to an embodiment.

[0023] Figure 15An example of customization that shares the first control layout in the first layout group with the first control layout in the third layout group on a touch screen according to an embodiment.

[0024] Figure 16 A flowchart illustrating an example method for providing a control socket template according to an embodiment.

[0025] Figure 17 A flowchart illustrating an example method for providing a control layout for an application according to an embodiment.

[0026] Figure 18 A flowchart illustrating an example method for providing a control layout manager according to an embodiment.

[0027] Figure 19 A flowchart illustrating an example method for using a control layout manager to select a control layout according to an embodiment.

[0028] Figure 20 A flowchart illustrating an example method for editing a control layout according to an embodiment.

[0029] Figure 21 A flowchart illustrating an example method for browsing a control layout library according to an embodiment.

[0030] Figure 22 A block diagram of an example mobile device that can be used to implement various embodiments.

[0031] Figure 23 A block diagram of a computing device that can be used to implement various embodiments.

[0032] The features and advantages of the embodiments described herein will become more apparent from the detailed description set forth below when considered in conjunction with the accompanying drawings in which like reference numerals identify corresponding elements. In the drawings, like reference numerals generally denote equivalent, functionally similar, and / or structurally similar elements. The drawing in which an element first appears is indicated by the leftmost digit(s) in the corresponding reference numeral.

[0033] Detailed Description

[0034] I. Introduction

[0035] This specification and the accompanying drawings disclose one or more embodiments that include the features of the disclosed embodiments. The scope of the embodiments is not limited to the aspects disclosed herein. The disclosed embodiments merely illustrate the desired scope, and modified versions of the disclosed embodiments are also covered. The embodiments are defined by the appended claims.

[0036] References in the specification to "one embodiment", "an embodiment", "an example embodiment", etc., indicate that the described embodiment may include a particular feature, structure, or characteristic, but that each embodiment may not necessarily include that particular feature, structure, or characteristic. Moreover, these phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is considered within the knowledge of one of ordinary skill in the art to implement that feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0037] In addition, it should be understood that the spatial descriptions used herein (e.g., "above", "below", "upper", "left", "right", "lower", "top", "bottom", "vertical", "horizontal", etc.) are for illustrative purposes only, and the actual implementation of the structures described herein may be spatially arranged in any orientation or manner.

[0038] In discussions, unless otherwise specified, adjectives such as "substantially" and "about" that modify a condition or relationship characteristic of one or more features of an embodiment of the present disclosure should be understood to mean that the condition or characteristic is defined within the tolerance of the operation of the embodiment that is acceptable for the intended application of the embodiment.

[0039] The following describes multiple exemplary embodiments. It should be noted that any chapter / subchapter headings provided herein are not intended to be limiting. The embodiments are described herein, and any type of embodiment may be included under any chapter / subchapter. Moreover, the embodiments disclosed in any chapter / subchapter may be combined with any other embodiments described in the same chapter / subchapter and / or different chapter / subchapters in any manner.

[0040] II. Exemplary Embodiments

[0041] An application developer may develop a user interface for a user to interact with an application on a device. An example of a user interface (UI) is a touchscreen interface that provides a graphical user interface (GUI) that enables a user to interact with the device using touch (e.g., by applying one or more fingers). The GUI may include one or more displayed controls for the user to interact with on the touchscreen. In the context of a GUI, a control is a graphical element configured for a user to interact with by touch manipulation to provide input to an application associated with the GUI. Examples of controls include graphical elements such as buttons, sliders, spinners, drop-down lists / menus, toolbars, icons, etc. The number of controls, control types, control combinations, and control positioning in the UI tend to be unique for each application.

[0042] Accordingly, methods, systems, and computer program products are provided for UI customization and cross-application sharing of user interface (UI) customization. Application developers can create a UI (e.g., a game control layout) using a control group that can be customized as a cluster. The control group can include multiple controls that are selectively assigned to a socket cluster (e.g., a socket group) in a layout template. The controls in the control group can be customized as a cluster, for example, by changing the position, rotation, and / or scaling of the control group. A user can customize the controls for comfort, switch the controls for playing games with the left or right hand, and share the customized preferences across applications. UI customization can be shared across applications with different UIs. For example, the customization of one or more control groups in a first UI can be (e.g., automatically) applied to one or more control groups in other UIs of other applications. A user can avoid making tedious and time-consuming individual control changes for many UIs.

[0043] Figure 1 FIG. 4 shows a block diagram of an example system 100 for cross-application sharing of user interface customization according to an example embodiment. Example system 100 presents one of several possible computing environments. In Figure 1 several components are shown with dashed lines to indicate several optional and alternative configurations, such as to provide a compact representation of multiple example environments. Many other environments not shown can also be used to implement the subject matter described herein. The various components presented may or may not be present in Figure 1 any given environment shown in the example in FIG. 4 or in an example not shown.

[0044] As Figure 1 shown, system 100 includes a computing device 106, a computing device 136, a controlled device 146, and a display screen 148. Computing device 106 includes a UI layout manager 108, a UI layout editor 112, a UI layout library 114, a display screen 118, a touch screen 124, a streaming client 128, and one or more UI control applications 140. Computing device 136 includes a streaming service 138, which can include one or more UI control applications 140. Streaming client 128 includes a content engine 126. These features of system 100 are described in further detail below.

[0045] Referring to Figure 1In the system 100, an example environment may include user 134 interacting with a UI control application (app) 140 executed by a content engine 134 in a computing device 106. User 134 may interact with UI control application 140 via a touch screen 124, for example, based on a UI (shown as the displayed UI layout 120 displayed by a display screen 118). User 134 may use the controls on UI layout 120 to control real and / or virtual scenes and / or objects, for example, based on in-person viewing or based on the displayed content 122 displayed by display screen 118.

[0046] Another example environment may include user 134 interacting with UI layout 120 while computing device 106 acts as a streaming client 128 coupled to a streaming service 138 in computing device 136, which may be local (e.g., a game console) or remote (e.g., a server-based game console on a network such as the Internet). The displayed content 122 that may be generated by executing UI control application 140 may be provided, for example, to a display screen 148 (e.g., a TV, a desktop or laptop computer monitor) and / or to display screen 118. In one example, a user may use a smart phone or a tablet (e.g., computing device 106) as a touch screen controller while watching content displayed on a TV.

[0047] User 134 may or may not observe the displayed content on display screen 118 or 148. For example, user 134 may interact with UI layout 120 to control a device in person (e.g., a controlled device 146, such as a wired or wirelessly controlled flying drone, a water or ground vehicle, or a ground robot) via wired or wireless communication 152. In one example, computing device 106 may include wired or wireless local or remote controls for real or virtual scenes or objects that user 134 may observe and control in one way or another. In another example, computing device 106 may display content and an integrated or overlapping UI. In another example, user 134 may interact with the displayed UI layout 120 via a pointing device such as a mouse or other user input device (e.g., a keyboard key).

[0048] Computing devices 106 and 136 may include any type of mobile or fixed computing device, such as a desktop computer, a server, a video game console, a mobile computing device (e.g., a device, a personal digital assistant (PDA), a laptop computer, a notebook computer, a tablet computer (such as an Apple iPad TM ), a netbook, etc.), a mobile phone (e.g., a cellular phone, a smart phone (such as a Microsoft phone, an Apple iPhone, executing Android TMoperating systems such as telephones), fixed computing devices (such as desktop computers or PCs (personal computers)), gaming consoles / systems (e.g., Microsoft Sony Nintendo or etc.). Examples of computing devices 106 and 136 are further presented below in Figure 22 and 23 which illustrate various user input devices including a touch screen 124 and supplementary to the touch screen 124.

[0049] The content engine 126 can generate application (“app”) data (e.g., by executing the UI control application 140) or receive application data streamed by the streaming service 138 from the computing device 136 executing the UI control application 140. The content engine 126 can provide display content 130 to the display screen 118 generated by the content engine 126 or received by the content engine 126 from the streaming service 138. The content engine 126 can be configured to display the UI layout 120 alone or together with the displayed content 122 (e.g., as a component of the displayed content 122 or as a graphical overlay on the displayed content 122). The content engine 126 can receive user input events 132 provided by a user 134 of the computing device 106 interacting with input controls (e.g., the displayed UI layout 120) (e.g., via the touch screen 124). User input can be provided, for example, via the touch screen 124 and / or other user input devices (see, for example, the examples shown in Figure 22 and 23 ). User input data 132 can indicate a user action (e.g., touching the touch screen 124 near a particular displayed control in the displayed UI layout 120 on the display screen 118) in the case of observing real and / or virtual scenes and / or objects either in person or on a display. As Figure 1 shown, the content engine 126 can process or transmit the input data 132 (e.g., to be transmitted as user input data 142 to the streaming service 138). The user input data 132 can be processed by the UI content application 140 to determine control signals and / or subsequent UI control application executions, which may affect the application data presented to the user 134.

[0050] Application (e.g., game) streaming transfers video game data from a first device (e.g., a "source" or "originating" device) running the application to a second device (e.g., a "client" device) to enable a user to view or interact with the application using the second device. In one example, video game data can be streamed from a desktop computer or game console to, for example, a smart phone, a tablet, or other device with a touch screen (e.g., a "touch device") to enable the user to interact with the UI control application using the touch screen. Streaming can enable a user to play a video game on a device for which the video game was not originally designed. For example, game streaming can enable a game designed to be played on a video game console with a physical game controller to be interacted with using a touch device. A graphical overlay can be presented on the touch screen device, e.g., to permit the user to interact with the UI control application on a touch screen device for which the UI control application was not designed. The graphical overlay can provide touch screen controls with which the user can interact. The user's interaction with the touch controls can be based on or mapped to physical controls in a physical game controller.

[0051] Computing device 106 can be used to view and interact with applications (e.g., video games) that may not be designed for the display device and / or input device available to computing device 106. For example, application content can be streamed from a computing device (e.g., computing device 136). UI control application 140 can be executed on computing device 136 (e.g., a desktop computer, a game console, a remote server with an Internet connection, etc.). Streaming service 138 can be configured to stream application data 144 from UI control application 140 to another device, such as computing device 106. Streaming service 138 can be configured to transmit application data 144 via (e.g., wired and / or wireless) networks (e.g., a local area network (LAN) (such as a wireless LAN (WLAN) or "Wi-Fi") and / or a wide area network (WAN) (such as the Internet)).

[0052] A streaming service 138 (e.g., at computing device 136) and a streaming client 128 (e.g., at computing device 106) can cooperate to present a streamed video generated by a UI control application 140 (e.g., a video game application) executing at computing device 136 as displayed content 122 on a display screen 118 coupled to (e.g., integrated with) computing device 106. The streaming service 138 can stream application data 144 (e.g., video and audio data) generated through the execution of the UI control application 140 to the streaming client 128, which can present the streamed video to user 134 at computing device 106. The streaming client 128 can stream user input data 142 (e.g., received from user 134 interacting with the displayed content 122 via a touch screen 124 at computing device 106) as a user input event to the UI control application 140 to the streaming service 138. User 134 can view and interact with the UI control application 140, which may or may not be designed to be viewed and interacted with (e.g., via a touch screen 124) on computing device 106. Examples of server-client streaming services into which embodiments may be incorporated can include, for example, the server-client streaming service provided by Microsoft the server-client streaming service provided by the platform and the one provided by Valve Corporation platform.

[0053] The UI control application 140 can include any application that accepts user input to control real and / or virtual objects, which can be viewed in person or on a display. Application developer 104 can develop a UI control application 140 (e.g., games, drones, robots, and other UI control applications) with which user 134 can interact via one or more UIs. Application developer 104 can provide a UI integrated in the UI control application (e.g., UI control application 140) and / or a separate UI to users and / or service providers (such as a streaming game service provider).

[0054] UI control applications (e.g., created by an application developer 104 using one or more UIs) can include any application that controls anything (real or virtual) based on feedback provided by a user interacting with the UI. In one example, a user can use visual UI controls to control real or virtual objects or physical / real devices (e.g., cameras, gimbals, drones, robots) in the displayed content. A UI control application may or may not provide the displayed content. UI control applications can include, for example, video games, drone control applications, robot control applications, etc. A user can control real and / or virtual objects, for example, by interacting with a displayed computer-generated user interface (UI) (e.g., a graphical UI (GUI) or a head-up display (HUD)) to provide user input to the UI control application. A user can interact with the UI, for example, by using a touch device, a pointing device (e.g., a mouse, a stylus), a keyboard, or other user input devices.

[0055] UI control applications can be adapted to or initially developed for touch devices. In one example, a UI control can be displayed alone or as an overlay on a UI application (e.g., a game) displayed on a touch device display.

[0056] UI control applications may or may not provide display content. In one example, a UI control application 140 can generate real and / or virtual scenes and / or real and / or virtual objects (e.g., a person or character, a vehicle, a weapon, a tool) for display on a display screen 118 and / or a display screen 148. In one example, a user can view a controllable real / physical object (e.g., a ground robot or a flying drone) in person or as a video in a first- or third-person perspective. The display content provided by the UI control application can be displayed, for example, on the same device on which the UI is displayed and / or on a separate display. Accordingly, a user can view real or virtual objects on the display of the computing device providing the UI control and / or on a different display. In one example, a UI control can be displayed together with the display content (e.g., embedded in a game), the UI can be displayed as a graphical overlay on the display content, the UI can be displayed on a first display while the display content can be displayed on a second display (e.g., a computer monitor, a television, or a digital whiteboard), etc.

[0057] UI control application 140 can communicate user control, for example, by leveraging computing device resources (e.g., wireless communication (e.g., cellular, WiFi, Bluetooth) or wired communication (e.g., USB)).

[0058] Application developer 104 can utilize, for example, UI templates for developing one or more UIs to control real or virtual objects via one or more UI control applications 140. The templates can be platform-independent and / or application-independent. The application developer can develop UI control templates. Platform developer 102 can develop UI control templates and provide them to the application developer (e.g., application developer 104) to develop UIs for various UI control applications. In one example, the UI control templates can be provided to application (e.g., game) developers by, for example, platform developers developing game platforms (e.g., Microsoft Sony PlayStation Nintendo ). The templates can have control socket configurations or layouts. An infinite number of socket templates can be implemented. The socket template design (e.g., configuration or layout) can be based on, for example, the type of UI application, the type of control, the type of display device, the type of interaction (e.g., handheld finger touch, pointing device), etc. The templates can include proposed UI control groupings (e.g., central or main control(s) surrounded by satellite controls), proposed locations (e.g., right, left, top, bottom), and proposed uses (e.g., primary, secondary, menu, social media).

[0059] In one example, a single UI control application may be compatible with multiple UIs from the same or different developers. In one example, the UI can be used to control real or virtual objects in one or more applications. The UI can be provided with the application and / or provided separately (e.g., including being provided by a third party). Users of many applications (e.g., gamers) may use many UIs and may potentially use many different types of UIs. There can be many UI categories and subcategories or UI groupings and sub-groupings, such as action (e.g., shooting, fighter, platform, stealth), sports (e.g., racing, soccer), adventure, strategy (e.g., tactical wargame, arena), simulation (e.g., flight, combat flight, driving, racing, construction), role-playing.

[0060] The UI layout 120 can be displayed alone or in conjunction with the displayed content 122 (e.g., as a component of the displayed content 122 or as a graphical overlay on the displayed content 122). The displayed UI layout 120 can include an arrangement (e.g., configuration or layout) of one or more graphical controls referred to herein as a user interface (UI) control configuration or layout with which a user 134 can interact on a display screen 118 while the UI control application 140 is being executed. The UI layout 120 can include any number and type of controls for real and / or fictional objects with which the user 134 can interact to control (e.g., via the UI control application 140) in person and / or view on the display. In one example, the UI layout 120 can include graphical controls for, e.g., steering, flying, landing, accelerating, braking, running, jumping, shooting, boxing, kicking, dodging, etc.

[0061] The UI layout manager 108 can manage the UI for one or more UI control applications, e.g., by providing a menu system with which the user 134 can interact to, e.g., retrieve, store, browse, select, organize, edit (e.g., customize), create preferences. The UI layout manager 108 can be implemented as, e.g., a stand-alone application, part of an operating system, a service to an application or operating system, and / or a component of a platform application (e.g., Microsoft Xbox). The platform developer 102 or application developer 104 can develop the UI layout manager 108, e.g., to manage a UI created using one or more UI templates (e.g., provided by the platform developer 102 to the application developer 104). The UI layout manager 108 can include, e.g., a UI layout menu 108, a UI layout editor 112, a UI layout library 114, and UI layout preferences 116. Other implementations can include the same, more, or fewer components, which can include the same and / or different components.

[0062] The UI layout menu 110 can provide a navigation menu for the user 134 to access and use the features provided by the UI layout manager 108, such as, by way of example, obtaining, storing, browsing, selecting, deleting, organizing, editing (e.g., customizing), creating preferences for UI control layouts, and so on. In one example, the layout menu 110 can be selectable. The UI layout menu 110 can, for example, show the user 134 the currently selected UI layout group and / or one or more specific UI layouts. The UI layout menu 110 can provide the user 134 with navigation options such as, by way of example: (i) enter the active / usage mode (e.g., with the first UI control layout in the currently selected UI layout group); (ii) select a different UI control layout in the currently selected UI control layout group; (iii) select the edit mode to edit one of the multiple UI control layouts in the currently selected UI layout group; (iv) select the library mode to browse multiple UI control layout groups, and so on. Figure 5 Shows one of many possible example implementations of the UI layout menu 110.

[0063] The UI layout library 114 can provide a library of UI control layouts that can be organized, for example, by group (e.g., groups A, B, C) and / or subgroup, by application type, by application title, by UI control, and so on. The UI layout library 114 can be searchable, for example, by group and / or subgroup, by application type, by application title, by UI control, and so on. The UI layout library 114 can be provided with UI control layouts and / or can search for UI control layouts. The UI layout library 114 can organize the UI control layouts based on default settings and / or user settings. The UI layout library 114 can include one or more groups. Each group in the UI layout library 114 can include one or more UI control layouts. Figure 13 Shows one of many possible example implementations of browsing the UI layout library 114.

[0064] The UI layout editor 112 can allow the user 134 to edit (e.g., customize) one or more UI control layouts. The user can select to customize, for example, for comfort, for handedness, or for any other purpose. For example, the size of the hand and fingers can vary significantly between children and adults. Similarly, the computer device and the way it is held can vary depending on the device and the user. A left-handed user can, for example, swap the first control group side and the second control group side. The editing can be made, for example, specific to the UI, specific to the UI category, general, and so on.

[0065] In one example, user 134 may customize one or more control clusters in a UI control layout, for example, by customizing the position, rotation (e.g., angle), or scaling of one or more control clusters. User 134 may, for example, create user preferences regarding that a control cluster (such as a right control cluster and a left control cluster (e.g., regardless of the UI control application)) should (e.g., always) be in a specific position, have a specific size (e.g., the size of the active / touch area and / or the icon), have a specific rotation, and / or a specific scaling.

[0066] In one example, user 134 may customize one or more controls, for example, by moving a control from a first set of interfaces to a second set of interfaces. User 134 may, for example, select (e.g., touch and hold) a control (e.g., in a first or second control cluster) and drag it to a new position (e.g., in a first or second control cluster). User 134 may, for example, create user preferences regarding that a specific control (e.g., regardless of the UI control application) should (e.g., always) be in a specific position with a specific size (e.g., the size of the active / touch area and / or the icon).

[0067] Figures 8 - 10 Some of many possible example implementations of the UI layout editor 112 are shown for illustrative purposes and are described in further detail below.

[0068] UI layout preferences 116 may be specified by user 134. The preferences may include, for example, a list of settings that apply to one or more UI control layouts (e.g., icon size). User 134 may specify whether one or more preferences should (e.g., automatically) be applied to one or more UI control layouts (e.g., a subset or all of the UI control layouts). Preferences may (e.g., also) be created by editing one or more UI control layouts. For example, user 134 may edit a first UI control layout having two control clusters, for example, by customizing the position, rotation, and / or scaling of the control clusters. The customization may be applied to (e.g., based on default and / or user preferences) other UI control layouts, for example, having two control clusters. Figures 8 - 11 Figures 12, 14, and 15 show some of many possible example implementations of creating user-customized preferences by editing a UI control layout and applying them to a UI control layout, and are described in further detail below.

[0069] Figure 2 An example of a control set interface template according to an example embodiment is shown. Example template 200 is displayed on a touchscreen display 202 (e.g., a cellular phone, a tablet, a digital whiteboard), but the embodiments may be implemented with any type of computing device and any type of display. Example template 200 presents a visual representation of computer-executable code (e.g., a program) (the display output generated thereby).

[0070] A control socket template can be provided to application (e.g., game) developers by, for example, platform developers of a development platform (e.g., Microsoft Sony PlayStation Nintendo ). The template can be platform-independent and / or application-independent. An application developer can have its own template. The socket configuration or layout of the template can provide, for example, a suggested UI control grouping (e.g., a central or main control surrounded by satellite controls), a suggested location (e.g., right, left, top, bottom), and a suggested use (e.g., primary, secondary, menu, social media). An infinite number of socket templates can be implemented. The socket design (e.g., configuration or layout) of the template can be based on, for example, the type of UI application, the type of control, the type of display device, the type of interaction (e.g., handheld finger touch, pointing device), etc.

[0071] An application developer can utilize, for example, a template for developing one or more UIs to control real or virtual objects via one or more applications. In one example, a single UI control application may be compatible with multiple UIs from the same or different developers. In one example, a single UI can be used to control real or virtual objects in one or more applications. The UI can be provided with the application and / or separately provided (e.g., including being provided by a third party). Users of many applications (e.g., gamers) may use many UIs and may potentially use many different types of UIs. There can be many UI categories and subcategories or UI groupings and sub-groupings, such as action (e.g., shooting, fighter, platform, stealth), sports (e.g., racing, soccer), adventure, strategy (e.g., tactical wargame, arena), simulation (e.g., flight, combat flight, driving, racing, construction), and role-playing.

[0072] The platform developer or other developers can (e.g., further) provide a UI manager to application users to manage UIs created using one or more templates. The UI manager can be, for example, a stand-alone application, a part of an operating system, a service to the application or operating system, and / or integrated into a platform application. The UIs can be managed, for example, in a library that can be (e.g., selectively) organized, for example, by grouping and / or sub-grouping, by application type, by application title, by UI control, etc. One or more sockets can be used to access UI management.

[0073] The template can include any number of individual and / or warp-knit group interfaces. An interface can represent a programmable location into which one or more controls can be inserted (e.g., assigned to, associated with). An interface can be used in whole or in part (e.g., associated with a control) or unused (e.g., not associated with a control). In one example, an interface can be split and / or combined (e.g., combined with other parts of the interface(s) or a complete interface). Interfaces can be grouped, for example, by one or more characteristics such as relative position or proximity. The warp-knit group interfaces can be manipulated as a group. For example, interface grouping can collectively change the position, rotation, and / or scale as a group.

[0074] In Figure 2 the example of, template 200 includes a first interface group 206, a second interface group 208, a third interface group 210, and a fourth interface group 212. Interfaces can have different sizes. In Figure 2 the example of, the interfaces of template 200 can be different from the central interfaces 228, 236 in the first and second control groups 206, 208, which can have a second size (e.g., size 2), and have a first size (e.g., size 1). The combined first interface group 206 and the combined second interface group 208 can have a third size (e.g., size 3). Controls can have different sizes suitable for one or more interface sizes, for example, based on the user's and / or device's ability to reliably distinguish the controls. In one example, buttons, joysticks, touch pads, and throttle controls can be suitable for sizes 1, 2, and 3. A larger direction (D) pad control can be suitable for size 2 and size 3 interfaces (e.g., but may not be suitable for size 1). A set of flight buttons can be suitable for size 3 (e.g., but may not be suitable for sizes 1 and 2). Scaling the interface group (e.g., to increase and / or decrease the distance between interfaces) can affect the set size.

[0075] The first set of interface clusters 206 can be referred to as, for example, the left hand (LH) interface cluster or the left wheel. The left wheel 206 is shown as having concentric circles (e.g., a center interface surrounded by satellite interfaces). The left wheel 206 can have any shape, such as circular, square, rectangular, triangular, regular polygon, irregular polygon, star-shaped, hub and spoke, etc. The left wheel 206 includes a center interface 228, which can represent the active (e.g., hit or touch) area of the control associated with the interface 228. An icon 230 can be associated with the center interface 228, for example, to graphically depict the purpose of the control associated with the center interface 228 and / or to identify the center and / or active area of the center interface 228. Surrounding the center interface 228 are, for example, eight outer or satellite interfaces represented by satellite interfaces 224. The touch area of the satellite interface 224 is shown as having an irregular shape with straight and curved sides (e.g., a pie slice with the center removed). The satellite interface can have any shape. An icon 226 can be associated with the satellite interface 224, for example, to graphically depict the purpose of the control associated with the satellite interface 224 and / or to identify the center and / or active area of the satellite interface 224.

[0076] The second set of interface clusters 208 can be referred to as, for example, the right hand (RH) interface cluster or the right wheel. The right wheel 208 is shown as having concentric circles (e.g., a center interface surrounded by satellite interfaces). The right wheel 208 can have any shape, such as circular, square, rectangular, triangular, regular polygon, irregular polygon, star-shaped, hub and spoke, etc. The right wheel 208 includes a center interface 236, which can represent the active (e.g., hit or touch) area of the control associated with the center interface 236. An icon 238 can be associated with the center interface 236, for example, to graphically depict the purpose of the control associated with the center interface 236 and / or to identify the center and / or active area of the center interface 236. Surrounding the center interface 236 are, for example, eight outer or satellite interfaces represented by satellite interfaces 232. The touch area of the satellite interface 232 is shown as having an irregular shape with straight and curved sides (e.g., a pie slice with the center removed). The satellite interface can have any shape. An icon 234 can be associated with the satellite interface 232, for example, to graphically depict the purpose of the control associated with the satellite interface 232 and / or to identify the center and / or active area of the satellite interface 232.

[0077] The third set of interface clusters 210 can be referred to as, for example, the upper tray. The upper tray 210 can include, for example, a left interface cluster 214, a center interface cluster 216, and a right interface cluster 218. Each interface cluster is shown with three interfaces having a circular shape. Various implementations of the template may or may not have an upper tray and may have any number of interface clusters (e.g., including zero), any number of interfaces, and any shape suitable for display and interaction. The interface clusters in the upper tray 210 can be organized for, for example, menu access controls (e.g., Xbox menu, UI manager menu), social media access controls, input controls, reserved, etc. As indicated, the interfaces can be associated with icons to graphically depict the purpose of the controls associated with the interfaces and / or identify the center and / or active area of the interfaces.

[0078] The fourth set of interface clusters 212 can be referred to as, for example, the lower tray. The lower tray 212 can include, for example, a center interface cluster. The fourth set of interface clusters 212 is shown with three interfaces having a circular shape. Various implementations of the template may or may not have a lower tray and may have any number of interface clusters (e.g., including zero), any number of interfaces, and any shape suitable for display and interaction. The interfaces in the lower tray 212 can be organized for, for example, secondary control actions. As indicated, the interfaces can be associated with icons to graphically depict the purpose of the controls associated with the interfaces and / or identify the center and / or active area of the interfaces.

[0079] Figure 2 The interfaces shown can be reshaped, repositioned, separated, split, combined, and / or manipulated in any other way. Figure 2 The interfaces shown can be assigned to controls or may not be used to create control clusters and UI control layouts. Figure 2 The interfaces shown are merely examples. An infinite number of examples can be implemented.

[0080] Figures 3A - 3G An example of a selective configuration of interfaces and interface clusters in a touch control interface template according to an example embodiment is shown. Figures 3A - 3G (By way of example) shows how an application developer 104 or a user 134 can selectively utilize the interfaces in an example interface cluster 300 to create an example control cluster 312. It can be observed that the example control cluster 312 provides controls available on the right hand side of a Microsoft Xbox controller. Of course, Figures 3A - 3G represents one of an infinite number of examples of the selective assignment of interfaces and controls to the interfaces.

[0081] Figure 3AIllustrates an example socket grouping 300, which can be implemented in a UI control socket template. The socket grouping 300 includes a first (e.g., central) region, a second region formed by two concentric circles, and a third region having four lobes. The first region includes four sockets 0, 1, 2, 3. The second region includes eight sockets numbered relative to each of the four lobes (e.g., second region socket 0.0 and 0.3 are adjacent to first lobe sockets 0.1 and 0.2, second region sockets 1.0 and 1.3 are adjacent to second lobe sockets 1.1 and 1.2, second region sockets 2.0 and 2.3 are adjacent to third lobe sockets 2.1 and 2.2, and second region sockets 3.0 and 3.3 are adjacent to fourth lobe sockets 3.1 and 3.2). The sockets can identify the active (e.g., touch, hit, point) regions (e.g., for the controls assigned to the sockets).

[0082] Figure 3B Illustrates central socket 302 and various examples of combining, dividing, and reconfiguring sockets 1, 2, 3, 4. As shown in the example, sockets 1, 2, 3, 4 can be combined (e.g., with a larger combined size) and assigned to a single control B1, combined and assigned to two controls B1, B2, combined and / or split (e.g., partially) and assigned to three sockets B1, B2, B3, or assigned to four controls B1, B2, B3, B4 as four sockets.

[0083] Figures 3C - 3F Illustrates assigning controls to sockets in the second region adjacent to the third lobe in the Figure 3A third region. Figures 3C - 3F Illustrates sockets (e.g., faded) assigned to controls, such as central sockets 0, 1, 2, 3, sockets 0, 1, 2 (empty) and 3 in the second region. Figure 3C Illustrates unassigned sockets 4.0, 4.1, 4.2, and 4.3, which are renumbered from the Figure 3A sockets 2.0, 2.1, 2.2, 2.3 in the Figure 3C As shown, sockets 4.0, 4.1, 4.2, and 4.3 can be used as is (e.g., as provided in the template) and assigned to four controls.

[0084] Figure 3D Assigns Figure 3C A replacement configuration of the four sockets 4.0, 4.1, 4.2, and 4.3 shown in the Figure 3D is shown as three sockets 4.0, 4.3, and 4.1. The socket 4.1 in the Figure 3C consumes a portion of the active region defined by the sockets 4.0, 4.1, 4.2, and 4.3 shown in the Figure 3CThe sockets 4.1 and 4.2 in [[]] have different shapes (e.g., circular). Figure 3D The shapes of the sockets 4.0 and 4.3 in [[]] (e.g., active touch areas) are different from Figure 3C the shapes of the sockets 4.0 and 4.1 in [[]] due to the applied circular shape of the socket 4.1.

[0085] Figure 3E The Figure 3C replacement configurations of the four sockets 4.0, 4.1, 4.2, and 4.3 shown in [[]] are shown as two sockets 4.0 and 4.1, where Figure 3E the socket 4.0 in [[]] combines Figure 3C the sockets 4.0 and 4.3 in [[]]. Figure 3E The socket 4.1 in [[]] consumes a part of the active area defined by the Figure 3C sockets 4.1 and 4.2 shown in [[]], and has a shape (e.g., circular) different from Figure 3C the sockets 4.1 and 4.2 in [[]].

[0086] Figure 3F The Figure 3C replacement configurations of the four sockets 4.0, 4.1, 4.2, and 4.3 shown in [[]] are shown as one socket 4.0. Figure 3E The socket 4.0 in [[]] combines Figure 3C the sockets 4.0, 4.1, 4.2, and 4.3 in [[]]. Figure 3E The socket 4.0 in [[]] has a different shape (e.g., circular) and consumes a part (e.g., less than all) of the active area defined by the Figure 3C sockets 4.1, 4.2, 4.3, and 4.4 shown in [[]].

[0087] Figure 3G Shows Figure 3F the socket grouping configuration (e.g., layout) assigned to controls in [[]] to create a control grouping configuration (e.g., layout) 312. For example, Figure 3F the socket 4.0 in [[]] is assigned to the right joystick (R) control. The second (e.g., satellite) area socket 3 is assigned to the right joystick press control. The second (e.g., satellite) area socket 2 is assigned to empty (unused or TBA). The second (e.g., satellite) area socket 1 is assigned to the right trigger (RT) control. The second (e.g., satellite) area socket 0 is assigned to the right front button (RT) control. The center sockets 0, 1, 2, 3 are assigned to the B4, B2, B3, B1 button controls respectively. The icons associated with the sockets are displayed in the control grouping 312.

[0088] The control grouping 312 can be created, for example, by a developer (e.g., based on a socket template). The control grouping 312 can be customized (e.g., by an end user) to group (e.g., by collectively changing the position, rotation, and / or scaling as a group). Each individual socket and control can be customized (e.g., by an end user), such as by moving a control from a first socket to a second socket, swapping control positions (e.g., by swapping sockets), etc. In one example, a user can swap the positions of an RT control and an RB control or move the RB control from socket 0.0 (see, e.g., Figure 3A ) to socket 0.1.

[0089] In various embodiments, the control layout including the control grouping can be configured in various arrangements, and such control layouts in various arrangements can be stored in a library for selection and editing during gameplay. For example, Figure 4A FIG. shows a block diagram of a control layout and a control library displayed on a display screen 118 of an example display device 400 according to one embodiment. Specifically, the display device 400 displays a split-screen example presented to a user (e.g., user 134) on the display screen 118, which includes a UI layout 120 and a UI layout library 114. The displayed UI layout 120 can include, for example, the current selection of a UI layout from the UI layout library 114. The displayed UI layout 120 includes, for example, at least a first control grouping 156 and a second control grouping 158. The first control grouping 156 can include any number of controls, such as, for example, at least FG-C1, FG-C2, and FG-C3. The second control grouping 158 can include any number of controls, such as, for example, at least SG-C1, SG-C2, and SG-C3. Other examples may display more or fewer control groupings and controls within the control groupings. The UI layout library 114 can be minimized (e.g., reduced to small icons or indicators to increase the size of the displayed UI layout 120). The UI layout library 114 includes, for example, at least a first UI layout 150, a second UI layout 152, and a third UI layout 154. The user can, for example, scroll up and down to view (e.g., select) other UI layout groupings in the UI layout library 114. One (e.g., each) grouping can include zero or more UI layouts.

[0090] Any number of UI layouts can be included in the UI layout library 114. In one example, the UI layout groupings can be organized (e.g., as a group) based on an application title. In other implementations, the UI layout groupings can be organized (e.g., as a group) based on any organizational characteristic (e.g., by socket template, by control type, etc.). A search can be performed to, for example, find, select, and / or organize UI control layouts based on features or characteristics. Such as according to the following regarding Figure 4B and Figures 5 - 21In the described examples, such UI control layouts can be configured to include any number of controls in any number of control groupings and can be selected from any number and configuration of control layouts in a library.

[0091] Figure 4B FIG. 4 shows an example of an active first control layout according to an example embodiment. An example (e.g., first) UI control layout 401 can be displayed on (e.g., positioned and scaled for use on) any display device. In one example, (e.g., as Figure 4B shown), the UI control layout 401 can be displayed on a touchscreen device display 402 (e.g., smartphone, tablet). The first control layout 401 shows the controls assigned to the control sockets (e.g., in a control socket template).

[0092] The user can interact with the controls in the first control layout 401 to provide user input that can be processed to control real and / or virtual objects. The first UI control layout 401 can include, for example, a UI layout manager (UILM) 406, a first control grouping 408, a second control grouping 412, and a third control grouping 410.

[0093] When positioned as Figure 4B shown, the first control grouping 408 can be referred to as, for example, a left or left hand (LH) control grouping or wheel. The first control grouping 408 can include any number and combination of controls. For example, in various embodiments, the first control grouping 408 can include, for example, one or more of a left joystick (C5), a left stick press (B5), a d-pad (C6), a left trigger (T1), a left finger button (FB1), a button, a touch pad, etc.

[0094] When positioned as Figure 4B shown, the second control grouping 412 can be referred to as, for example, a right or right hand (RH) control grouping or wheel. The second control grouping 412 can include one or more of any number and combination of controls. For example, in various embodiments, the second control grouping 412 can include, for example, a right joystick (C7), a right stick press (B6), one or more face buttons (B1, B2, B3, B4), a right trigger (T2), a right finger button (FB2), a direction stick (C7), a touch pad, etc.

[0095] When positioned as Figure 4BWhen positioned as shown, the third control group 410 may be referred to as, for example, an upper tray. In other embodiments, the third control group 410 may be positioned in other locations within the UI control layout 401, including the left or right side of the UI control layout 401, as well as middle or lower positions on the UI control layout 401. The third control group 410 may include any number and combination of controls, such as the first through fourth controls C1 - C4. For example, in one embodiment, the third control group 410 may include back, guide, and start controls, one or more menu access controls, etc. In various embodiments, the controls within the UI control layout 401 may have any graphical form factor, including being displayed and interactive as buttons, touch pads, switches, D - pads, sticks, etc. In one example, the controls within the third control group 410 may fade out when the right - hand and / or left - hand control groups 408, 412 are in use and return (e.g., after a delay) when, for example, inactive.

[0096] The UI layout manager 406 may be selected to, for example, leave the active UI control mode to access the UI layout manager menu. The UI layout manager may manage many (e.g., dozens, hundreds) of control layouts created for many different applications based on one or more socket templates. The user may select the UI layout manager 406 to, for example, edit the first control layout 401, select a different control layout to use or edit within the same group, browse a library of control layouts within each group, etc.

[0097] Figure 5 An example of an interface control manager and a first layout group is shown in accordance with an example embodiment. Figure 5 An example of a UI layout manager menu 500 is shown, which may be accessed, for example, by selecting the UI layout manager 406 from the Figure 4B example screen shown. The example layout manager 500 is one of many possible examples for managing multiple control layouts for one or more control applications. Other examples may implement different options and screens.

[0098] The example UI manager menu 500 may show a representation 400A of the first control layout 401 along with menu options. The representation 400A may include, for example, a highlighted, reduced - size, and / or inactive (e.g., unresponsive) version of the first control layout 401. The first control layout 401 may be associated with a label (e.g., a title or name). In one example, (e.g., as Figure 4BAs shown, the first control layout is marked as "standard" in the first control layout grouping. In one example, the second control layout in the first grouping is marked as "Custom 1". The representation 700B of Custom 1 can be shown in the UI manager menu 500. An indication can be provided (e.g., a highlighted triangle below standard) to explain which control layout in the grouping is selected (e.g., and can become active when exiting the UI manager menu 500).

[0099] Menu navigation options can include, for example, Exit 514, Edit 516, Library 518, and an option to select a different UI control layout in the current layout grouping (e.g., a control layout named "Custom 1").

[0100] Selecting Exit 514 (e.g., without making any other selections) can, for example, exit the UI manager menu 500 and return to the active control mode with the standard / first control layout 401 as the active / used control layout, e.g., as shown in the example of Figure 4B shown.

[0101] Selecting Custom 1 can, for example, replace the standard representation 400A with the representation 700B of Custom 1 as the selected control layout in the grouping. The user can select the Custom 1 control layout, for example, by selecting (e.g., touching the displayed control on the touchscreen 402) "Custom 1" or the representation 700B of Custom 1. Selecting Exit 514 (e.g., after selecting Custom 1) can, for example, exit the UI manager menu 500 to the active control mode with the Custom 1 control layout as the active / used control layout. Figure 6 and 7 show examples of these selections.

[0102] Selecting Edit 516 can, for example, open the layout editor to edit the selected control layout, which (as shown) is "standard" or the first control layout 401. The user can, for example, alternatively select Custom 1, then select Edit 516 to edit Custom 1 or select Library 518, select any control layout in the library, and select Edit to select another control layout. Figure 5 shown. Figure 8 shows an example of selecting Edit 516 to edit the first control layout (marked as "standard") 401, while Figure 9 and 10 show editing of the first control layout (marked as "standard") 401.

[0103] Selecting Library 518 can open the control layout library for browsing and selection. Figure 13 shows an example of selecting Library 518, while Figure 14 and 15 show control layouts from different groupings selected from the control layout library.

[0104] Figure 6 Shows an example of selecting a second control layout in a first layout grouping according to an example embodiment. Figure 6 Shows an example of selecting Custom 1 in the example layout manager menu 500.

[0105] The example UI manager menu display 600 may show a representation 700A of the second control layout (Custom 1) 700 along with menu options. The representation 700A may include, for example, a highlighted, reduced-size, and / or inactive (e.g., unresponsive) version of the second control layout 700. A representation 400B of the unselected first control layout (Standard) may be shown in the UI manager menu display 600. An indication may be provided (e.g., a highlighted triangle below Custom 1) to illustrate which control layout in the grouping is selected (e.g., and may become active upon exiting the UI manager menu display 600).

[0106] The menu navigation options may include, for example, Exit 514, Edit 516, Library 518, and an option to select a different UI control layout (e.g., the first control layout “Standard”) in the current (e.g., first) layout grouping.

[0107] Selecting Exit 514 (e.g., without making any other selection) may, for example, exit the UI manager menu to an active control mode with the second control layout Custom 1 700 as the active / active-in-use control layout, e.g., as shown in the example of Figure 7 as shown.

[0108] Selecting Standard may, for example, return to the manager menu 500. The user may select the standard control layout, for example, by selecting (e.g., touching the displayed control on the touch screen 402) “Standard” or the representation 400B of Standard. Selecting Exit 514 (e.g., after selecting Standard) may, for example, exit the UI manager menu 600 to an active control mode with the standard control layout as the active / active-in-use control layout, e.g., as shown in the example of Figure 4B as shown.

[0109] Selecting Edit 516 may, for example, open a layout editor to edit the selected control layout, which (as shown in Figure 6 is the second control layout Custom 1. The user may alternatively select Standard, then select Edit 516 to edit Standard or select Library 518, select any control layout in the library, and select Edit to select another control layout. Figure 8 Shows an example of selecting Standard then Edit 516 to edit the first control layout (Standard) 401, while Figure 9 and 10Editing of a first control layout (labeled "standard") 401 is shown.

[0110] The Selection Library 518 can be selected to open the control layout library for browsing and selection. Figure 13 An example of the Selection Library 518 is shown, while Figure 14 and 15 show control layouts from different groupings selected from the control layout library.

[0111] Figure 7 An example of an active second control layout according to an example embodiment is shown. An example (e.g., second) UI control layout 700 can be displayed on (e.g., positioned and scaled for use on) any display device. In one example, (e.g., as Figure 7 shown), a second control layout (Custom 1) 700 in a first layout grouping can be displayed on a touchscreen device display 402 (e.g., smartphone, tablet). The second control layout 700 shows controls assigned to control sockets (e.g., in a control socket template).

[0112] The user can interact with the controls in the second control layout 700 to provide user input that can be processed to control real and / or virtual objects. The second UI control layout 700 can include, for example, a UI layout manager 406, a first control grouping 708, a second control grouping 712, and a third control grouping 410.

[0113] The first control grouping 708 can be referred to as, for example, a left or left - hand (LH) control grouping or wheel. The second control grouping 712 can be referred to as, for example, a right or right - hand (RH) control grouping or wheel. The third control grouping 710 can be referred to as, for example, an upper tray 710.

[0114] Figure 8 An example of a first control layout in a layout editor according to an example embodiment is shown. The example layout editor 800 shows an editable representation 400C of the first control layout (standard) 401. The editable representation 400C includes an editable first control grouping 808 and an editable second control grouping 812. In various embodiments, the UI layout manager 406 and the third control grouping 810 may or may not be editable.

[0115] The editable first control grouping 808 can be referred to as, for example, an editable left or left - hand (LH) control grouping or wheel. The editable first control grouping 808 can include, for example, a left joystick (C5), a left stick press (B5), a direction pad (C6), a left trigger (T1), and a left finger button (FB1). The editable first control grouping 808 can (e.g., further) include custom controls, such as a first grouping rotation and scale control 814 and a first grouping positioning control 816.

[0116] The first group rotation and scaling control 814 may include, for example, a selectable ring near, on, or around the editable first control group 808. The first group positioning control 816 may include, for example, a selectable area inside the first group rotation and scaling control 814 (e.g., the central area of the editable first control group 808). The user may select the first group rotation and scaling control 814 or the first group positioning control 816, for example, by touching and holding any point on the selectable area.

[0117] The editable second control group 812 may be referred to as, for example, the editable right or right hand (RH) control group or wheel. The editable second control group 812 may include, for example, a right joystick (C7), a right stick press (B6), face buttons (B1, B2, B3, B4), a right trigger (T2), and a right finger button (FB2). The editable second control group 812 may (e.g., further) include custom controls, such as a second group rotation and scaling control 818 and a second group positioning control 820.

[0118] The second group rotation and scaling control 818 may include, for example, a selectable ring near, on, or around the editable second control group 812. The second group positioning control 820 may include, for example, a selectable area inside the second group rotation and scaling control 818 (e.g., the central area of the editable second control group 812). The user may select the second group rotation and scaling control 818 or the first group positioning control 820, for example, by touching and holding any point on the selectable area.

[0119] The user may customize the first control layout 401, for example, by interacting with the custom controls (e.g., the first group rotation and scaling control 814, the first group positioning control 816, the second group rotation and scaling control 818, and the second group positioning control 820).

[0120] In one example, the user may selectively edit the position, rotation, and scaling of the editable first control group 808 and the editable second control group 812. The user may, for example, customize the editable first control group 808 and the editable second control group 812 simultaneously.

[0121] For example, the user can select the first group rotation and scaling control 814 (e.g., by touching and holding any point on the ring defined by the first group rotation and scaling control 814). Dragging the held position clockwise or counterclockwise (e.g., along the ring defined by the first group rotation and scaling control 814) can rotate the editable first control group 808 to the right or left, respectively. Dragging the held position away from or towards the center of the editable first control group 808 can expand or contract (increase or decrease) the scaling of the editable first control group 808, respectively. An increase in scaling can increase the distance between the controls, while a decrease in scaling can decrease the distance between the controls.

[0122] For example, the user can select the first group position control 816 (e.g., by touching and holding any point in the selectable area defined by the first group position control 816). Dragging the held position (e.g., in any direction) can move the editable first control group 808.

[0123] The size of the socket / control active area and / or the control icon can be customized, for example, by specifying user preferences (e.g., small, medium, large, extra large).

[0124] The customization of the UI control layout can be (e.g., selectively and / or automatically) applied to other UI control layouts (e.g., having the same socket template, in the same group, for the same type or class of UI control application, in all UI control layouts, etc.). The customization can be applied automatically with or without user selection. In one example, the user can specify the applicability of the customization generally (e.g., in user preferences) and / or specifically (e.g., per customization session).

[0125] As presented previously (e.g., in the discussion of FIG. 3 regarding selectively customizing the position of controls in a socket), the user can customize one or more controls (e.g., in the first editable control group 808, the second editable control group 812, etc.) by moving the controls from a first socket to a second socket, e.g., as Figure 2 shown in the examples of FIGS. 3 and 4. The user can, for example, select (e.g., touch and hold) a control (e.g., in the first or second control group) and drag it to a new position (e.g., in the first or second control group).

[0126] The user can select customization, for example, for comfort, for handedness, or for any other purpose. For example, the sizes of hands and fingers can vary significantly between children and adults. Similarly, the computer device and the way it is held can vary depending on the device and the user. A left-handed user can, for example, swap the sides of the editable first control group 808 and the editable second control group 812. The user can, for example, select to accept or reject the customization by selecting the finish control 920 or the reset control 922.

[0127] Figure 9 An example of scaling and rotating the left control group in the first control layout according to an example embodiment is shown. The example layout editor display 900 shows that the user selects the first group rotation and scale control 814 and drags the held position away from the center of the editable first control group 808 and drags it counterclockwise (e.g., down and to the right) to the position 914, which has the effect of increasing the scale of the editable first control group 808 (e.g., increasing the active touch area of the socket / control and / or the spacing between controls) and rotating the positions of the left stick press (B5), the direction pad (C6), the left trigger (T1), and the left finger button (FB1) counterclockwise. These edits create the customized first control group 908. The user can continue to drag the position 914 to continue changing the configuration, including dragging it to the starting configuration of the editable first control group 808. The user can release the position 914 and reselect it for further editing. The user can select the reset control 822 to return the editable first and second control groups to their configurations at the start of the editing session or select the finish control 820 to save the customization.

[0128] Figure 10 An example of simultaneously repositioning the right and left control groups in the first control layout according to an example embodiment is shown. The customized first control layout 1000 shows that the first group position control 816 is selected and dragged down to the position 1016, while the second group position control 820 is selected and dragged up to the position 1020, which has the effect of moving the editable first control group 808 down and moving the editable second control group 812 up. These edits create the customized first control group 1008 and the customized second control group 1012. The user can continue to drag the positions 1016 and 1020 to continue changing the configuration, including dragging them to the starting configuration of the editable first control group 808. The user can release the positions 1016 and / or 1020 and reselect them for further editing. The user can select the reset control 822 to return the editable first and second control groups to their configurations at the start of the editing session or select the finish control 820 to save the customization.

[0129] Figures 11 - 15 Presents based on the user selecting the finish control 822 to accept Figure 10An example of a customized first control layout 1000 as shown.

[0130] Figure 11 An example of an activated and customized version of the first control layout according to an example embodiment is shown. The example customized first UI control layout 1100 can be compared with the first UI control layout 401. It can be observed (e.g., by comparing Figure 4B with Figure 11 ) that the differences between the first (e.g., left - hand side) control cluster 408 and the customized first control cluster 1108 include re - positioning downwards, rotating counter - clockwise, and an expanded scale. It can be observed (e.g., by comparing Figure 4B with Figure 11 ) that the differences between the second (e.g., right - hand side) control cluster 412 and the customized second control cluster 1112 include re - positioning upwards, rotating clockwise, and an expanded scale. It can be observed (e.g., by comparing Figure 4B and Figure 11 ) that the central control socket with face buttons (e.g., B1, B2, B3, B4 controls) does not rotate with the right joystick (C7), right - stick press (B6), right trigger (T2), and right finger button (FB2). The rotation (e.g., of the central socket / controls) can be selectable, for example, generally (e.g., in user preferences) and / or specifically (e.g., per customization session).

[0131] The UI layout manager 406 can be selected to, for example, leave the active UI control mode showing the customized first UI control layout 1100 to access other UI control layouts via the UI layout manager menu.

[0132] Figures 12 - 15 It can be explained that, for example, for other applications, customizations to the first UI control layout in the first layout cluster (e.g., customizing the first control layout 401 to the customized first control layout 1100) can be applied to other UI control layouts in the same cluster and other clusters, thereby providing cross - application customization. The customization can be (e.g., selectively) general or global (e.g., for all UI control layouts), for example, based on the same or similar socket templates and / or other layout similarities.

[0133] Figure 12 An example of customizing a first control layout to share with a second control layout in the same layout cluster by re - positioning, scaling, and rotating the left - hand side and right - hand side control clusters according to an example embodiment is shown. Figure 12Shows an example of a customized second UI control layout (customization 1 1200). The customized customization 1 1200 can be compared with customization 1 700. It can be observed that customization 1 700 has been customized without editing customization 1 700. Similar to the comparison of the customized standard 1100 with the standard 401, it can be observed that (e.g., by comparing Figure 7 with Figure 12 ), the differences between the first (e.g., left - hand side) control grouping 708 and the customized first control grouping 1208 include downward repositioning, counter - clockwise rotation, and expanded scaling. It can be observed that (e.g., by comparing Figure 7 with Figure 12 ), the differences between the second (e.g., right - hand side) control grouping 712 and the customized second control grouping 1212 include upward repositioning, clockwise rotation, and expanded scaling.

[0134] Shared customization can implement (e.g., broadcast) user customization on multiple user interface controls, thus avoiding time - consuming customization of each control in each user interface one by one for each control and each application.

[0135] Figure 13 Shows an example of a browsing control layout grouping library according to an example embodiment. Figure 13 Shows an example library browsing menu 1300 that is displayed, for example, when the library 518 in the selection UI layout manager menu 500 is selected. An example list of UI layout groupings 1307 visible on the touch screen 402 includes, for example, the following groupings: Sports, FPS / TPS (First - Person Shooter / Third - Person Shooter), Action, Racing, Puzzle, and Adventure. The user can scroll up and down, for example, to view other UI layout groupings. In one example, the UI layout groupings can be organized (e.g., as groupings) based on the application title or the type or genre of the application (e.g., sports video games, FPS / TPS video games, action video games, racing video games, puzzle video games, and adventure video games). One (e.g., each) grouping can have zero or more sub - groupings (e.g., having UI default and customized controls for a specific application title). In other implementations, the UI layout groupings can be organized (e.g., as groupings) based on any organizational characteristic (e.g., by socket template, by control type, etc.). A search can be performed to, for example, find, select, and / or organize UI control layouts based on features or characteristics. The user can navigate and select, for example, a UI control layout from the Action grouping (e.g., as shown in the example of Figure 14 ) or a UI control layout from the War grouping (e.g., as shown in the example of Figure 15 ).

[0136] Figure 14Shows an example of customization that shares the first control layout in the first layout group with the first control layout in the second layout group according to an example embodiment. Figure 14 Shows a customized UI control layout 1400 from an action group selected from the Figure 13 library shown. In one example, the customized UI control layout 1400 (e.g., customized based on the first and second control groups 408, 412 in the first control layout 401 to the customized first and second control layouts 1108, 1112 in the customized first control layout 1200) is customized by downward repositioning, counterclockwise rotation, and scaled expansion of the first (e.g., left) control group 1408 and upward repositioning, clockwise rotation, and scaled expansion of the second control group 1412.

[0137] Figure 15 Is an example of customization that shares the first control layout in the first layout group with the first control layout in the third layout group according to an example embodiment. Figure 15 Shows a customized UI control layout from a war group selected from the Figure 13 library shown. The war layout group is shown in the layout manager menu 1500. The war group includes three UI control layouts (e.g., third person, third person (gyro), and standard). The third person UI control layout is selected. A reduced-size, highlighted, customized version of a portion of the third person UI control layout is shown along with the first and second control groups 1508, 1512. A reduced-size, customized version of the third person (gyro) UI control layout 1524 is also shown. In one example, the customized third person UI control layout (e.g., customized based on the first and second control groups 408, 412 in the first control layout 401 to the customized first and second control layouts 1108, 1112 in the customized first control layout 1200) is customized by downward repositioning, counterclockwise rotation, and scaled expansion of the first (e.g., left) control group 1508 and upward repositioning, clockwise rotation, and scaled expansion of the second control group 1512.

[0138] Various embodiments may also be implemented in processes or methods. For example, Figure 16FIG. 0 is a flow chart showing an example method for providing a control set interface template according to an example embodiment. Embodiments disclosed herein and other embodiments may operate according to example method 1600. Method 600 includes step 1602. However, other embodiments may operate according to other methods. Based on the foregoing discussion of the embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant arts. The order of the steps is not required unless explicitly stated or inherently required. Method embodiments are not required to implement all of the steps illustrated in FIG. 1600. Figure 16 is just one of many possible embodiments. Embodiments may implement fewer, more, or different steps.

[0139] Method 600 includes step 1602. In step 1602, a UI set interface template having a set of interfaces (e.g., a cluster) (e.g., an interface set) may be provided. Controls may be selectively assigned to parts and / or the whole of the interface. The interface (e.g., and the assigned controls) may be selectively customized (e.g., as a cluster). For example, as Figure 1 shown, platform developer 102 may provide a UI set interface template having at least one interface set (e.g., as shown in the example of Figure 2 ), and application developer 104 may utilize the UI set interface template to create a UI control layout for an application (e.g., a game) by selectively assigning controls to the whole and / or separate parts of the interface (e.g., as shown in the example of FIG. 3). App developer 104 may provide a content application (content application) 140 having a UI, for example, to one or more computing devices 106, 136. User 134 may interact with content application 140 using a UI that may be managed by UI layout manager 108. User 134 may customize the UI, for example, by changing the position, rotation, and / or scale of one or more control clusters created using one or more interface sets.

[0140] Figure 17 FIG. 13 is a flow chart showing an example method for providing a control layout for an application according to an example embodiment. Embodiments disclosed herein and other embodiments may operate according to example method 1700. Method 1700 includes step 1702. However, other embodiments may operate according to other methods. Based on the foregoing discussion of the embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant arts. The order of the steps is not required unless explicitly stated or inherently required. Method embodiments are not required to implement all of the steps illustrated in FIG. 1700. Figure 17 is just one of many possible embodiments. Embodiments may implement fewer, more, or different steps.

[0141] Method 1700 includes step 1702. In step 1702, at least one UI control layout can be created (e.g., using at least one UI control socket layout template) for each of the N applications. Each UI control layout can include a UI control grouping based on a UI socket grouping that can be customized for clustering. For example, as Figure 1 shown, application developer 104 can utilize a UI socket template having at least one socket grouping (e.g., as shown in the example of Figure 2 ) to create one or more UI control layouts for an application (e.g., a game) by selectively assigning controls to the whole and / or separate parts of the socket (e.g., as shown in the example of Figures 3A - 3G ). App developer 104 can provide content application (content app) 140 to one or more computing devices 106, 136. App development can provide the UI separately or can provide the UI together with the content application. User 134 can interact with content application 140 using a UI that can be managed by UI layout manager 108. User 134 can customize the UI, for example, by changing the position, rotation, and / or scaling of one or more control groupings created using one or more socket groupings.

[0142] Figure 18 FIG. shows a flowchart of an example method for providing a control layout manager according to an example embodiment. The embodiments disclosed herein and other embodiments can operate according to example method 1800. Method 1800 includes step 1802. However, other embodiments can operate according to other methods. Based on the foregoing discussion of the embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant art. The order of steps is not required unless explicitly stated or inherently required. It is not required that the method embodiments implement all the steps illustrated in FIG. 1800. Figure 18 is just one of many possible embodiments. The embodiments can implement fewer, more, or different steps.

[0143] Method 1800 includes step 1802. In step 1802, a UI manager can be provided to manage the UI control layouts for applications 1 - K. The UI manager can have a menu for selecting UI control layouts for an application, a library for organizing, browsing, and selecting UI control layouts, and a layout editor for customizing layouts and user preferences to apply the customization for one UI control layout to multiple UI control layouts. For example, as Figure 1 shown, platform developer 102 can provide UI layout manager 108 to a user device to manage the UI developed by application developer 104. UI layout manager 108 can include UI layout menu 110 (e.g., as shown in the example of Figure 5as shown in the example in) to select a UI control layout (e.g., standard and custom 1). The UI layout manager 108 may include a UI layout library 114 (e.g., as Figure 13 shown in the example in) to organize, browse, and select UI control layouts (e.g., UI control layouts in layout group 1307). The UI layout manager 108 may include a layout editor 112 (e.g., as Figures 8 - 10 shown in the example in) to customize a layout (e.g., from standard layout 401 to customized standard layout 1100). The UI layout manager 108 may include UI layout preferences 116 to apply the customization for one UI control layout to multiple UI control layouts (e.g., as Figure 12 , 14 and as shown in 15).

[0144] Figure 19 FIG. shows a flowchart of an example method 1900 for using a control layout manager to select a control layout according to an example embodiment. The embodiments disclosed herein and other embodiments may operate according to example method 1900. Method 1900 includes steps 1902-1908. However, other embodiments may operate according to other methods. Based on the foregoing discussion of the embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant art. The order of the steps is not required unless explicitly stated or inherently required. It is not required that a method embodiment implement all the steps illustrated in FIG. 1900. Figure 19 is just one of many possible embodiments. Embodiments may implement fewer, more, or different steps.

[0145] Method 1900 includes step 1902. In step 1902, the UI control application and the layout control manager may be started. For example, as Figure 1As shown, user 134 can interact with touch screen 124 to direct computing device 106 or computing device 136 to open a content application (e.g., by selecting a content application from content applications 140). UI layout manager 108 can, for example, already be running, be called by a content application, or be selected by user 134 to run. In one example, the user can open an Xbox game application and access many different game application titles, which can run locally, stream locally, or stream over the Internet. UI layout manager 108 can, for example, be integrated into the Xbox game application. User 134 can select content applications 140 through another application (e.g., the Xbox game application). Each game application can be associated with a (e.g., unique) layout of UI controls that can be displayed on display screen 118 and / or display screen 148. In another example, the user can open a flying drone control application, where the drone can provide video and user 134 interacts with the displayed UI control layout to control one or more virtual or physical objects associated with the drone (e.g., drone direction, speed, altitude, camera, gimbal, camera zoom or focus, etc.). In another application, user 134 can open a robot control application. The UI control layout can be displayed to user 134 who can view and control the robot in person (e.g., using a touch screen UI).

[0146] In step 1904, the UI control layout for the UI control application can be activated and displayed, where user customization preferences are applied to the UI control layout. For example, as Figure 1 shown, the UI control layout can be activated, for example, when UI control application 140 and / or UI layout manager 108 are activated. The UI control layout can be embedded in content application 140 and activated by content application 140 or be separately selected and activated, e.g., user 134 interacts with UI layout manager 108. As Figure 12 shown in the example of, the UI control layout (e.g., customization 1 layout 700) can be activated as customized customization 1 layout 1200 according to user customization preferences based on user customization preferences created by customizing different UI control layouts (e.g., standard layout 401 is modified to customized standard layout 1100).

[0147] In step 1906, the UI control layout manager menu can be selected. For example, as Figure 1 shown, user 134 can interact with touch screen 134 to select UI layout menu 110 shown in the displayed content 122 on display screen 118 of computing device 106 (e.g., a smart phone or tablet). As Figure 4B shown, user 134 can select UI layout manager 406.

[0148] In step 1908, selections can be made (e.g., in the UI layout manager) to return to the active / usage mode, select a different UI control layout for the UI application, edit the UI control layout, or browse the UI control layout library. For example, as Figure 5 shown, user 134 can select to exit ("X") to return to the active / usage mode (e.g., as shown in the example of Figure 4B ), select the Custom 1 layout and then exit to activate the Custom 1 layout as the UI for the application (e.g., as shown in the example of Figure 7 ), select the Edit mode 516 to edit the standard layout (e.g., as shown in the example of Figures 8 - 10 ), or select the Library mode 518 to browse the UI control layout library (e.g., as shown in the example of Figure 13 ).

[0149] Figure 20 FIG. shows a flowchart of an example method for editing a control layout according to an example embodiment. The embodiments disclosed herein and other embodiments can operate according to example method 2000. Method 2000 includes steps 2002-2008. However, other embodiments can operate according to other methods. Based on the foregoing discussion of the embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant art. The order of the steps is not required unless explicitly stated or inherently required. It is not required that the method embodiments implement all the steps illustrated in FIG. 2000. Figure 20 is just one of many possible embodiments. The embodiments can implement fewer, more, or different steps.

[0150] Method 2000 includes step 2002. In step 2002, an editor can be opened to edit a first UI control layout having at least one UI control group. For example, as Figure 1 and 5 shown, user 134 can select the editor 516 to edit the standard layout. An example of the UI layout editor 112 is shown in Figure 8 . The example layout editor 800 shows an editable representation 400C of a first control layout (standard layout) 401. The editable representation 400C includes an editable first control group 808 and an editable second control group 812.

[0151] In step 2004, the first UI control layout can be customized by changing at least one of the position, scale, and rotation of the first UI control group and / or the second UI control group. For example, as shown in the example of Figure 9 , the rotation and scale of the first control group 808 are changed to a customized first control group 908. As shown in Figure 10As shown in the example in, the positions of the first and second control groupings 808 and 812 are changed to customized first and second control groupings 1008 and 1012.

[0152] In step 2006, the first UI control layout can be edited by changing the socket of the first control in the first UI control grouping and / or the second UI control grouping. For example, as shown and discussed with respect to Figure 2 , Figures 3 and 8, user 134 can customize one or more controls (e.g., in the first editable control grouping 808 and / or the editable second control grouping 812) by moving the control from the first socket to the second socket (e.g., as shown in the example in Figure 2 and Figure 3). The user can, for example, select (e.g., touch and hold) a control (e.g., in the first or second control grouping) and drag it to a new position (e.g., in the first or second control grouping). In one example, user 134 can swap the positions of the RT control and the RB control or move the RB control from socket 0.0 (see, for example, Figure 3A ) to socket 0.1.

[0153] In step 2008, the customization of the first UI control layout can be applied to other UI control layouts of other UI control applications. For example, as shown in Figure 1 , the UI layout manager 108 can include UI layout preferences 116 to apply the customization for one UI control layout to multiple UI control layouts (e.g., as shown in the examples in Figure 12 , 14 and 15). As shown in the example in Figure 12 , a UI control layout (e.g., the customized 1 layout 700) can be activated as a customized customized 1 layout 1200 according to user customization preferences based on user customization preferences created by customizing different UI control layouts (e.g., the standard layout 401 is modified to the customized standard layout 1100).

[0154] Figure 21 FIG. shows a flowchart of an example method 2100 for browsing a control layout library according to an example embodiment. The embodiments disclosed herein and other embodiments can operate according to example method 2100. Method 2100 includes steps 2102 - 2108. However, other embodiments can operate according to other methods. Based on the foregoing discussion of the embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant art. The order of the steps is not required unless explicitly stated or inherently required. It is not required that the method embodiments implement all of the steps illustrated in FIG. 2100. Figure 21 is just one of many possible embodiments. The embodiments can implement fewer, more, or different steps.

[0155] Method 2100 includes step 2102. In step 2102, a UI control layout library can be selected and displayed. For example, as Figure 1 , 5 and 13 shown, user 134 can select UI layout library 114 by selecting library 516. An example of the user interface for library 114 is as Figure 13 shown.

[0156] In step 2104, multiple UI control layout groups developed for multiple applications can be browsed. For example, as Figure 13 shown, user 134 can browse UI control layout group 1307, for example, by scrolling up and down, performing a search, selecting a group to view the UI control layouts in the group, and so on.

[0157] In step 2106, a UI control layout group can be selected. For example, as Figure 13 and 15 shown, user 134 can select the UI control layout group "War".

[0158] In step 2108, a second UI control layout can be selected from the selected group. For example, user 134 can select the "Third Person" UI control layout in the group War, which can be displayed according to the customization based on the customization of the first UI control layout (such as the standard layout 401 being modified to the customized standard layout 1100).

[0159] III. Example Mobile Device and Computing Device Embodiments

[0160] The embodiments described herein can be implemented in hardware or in hardware in combination with software and / or firmware. For example, the embodiments described herein can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, the embodiments described herein can be implemented as hardware logic / circuitry.

[0161] As mentioned herein, the described embodiments, including Figures 1 - 21Together with any of its modules, components, and / or sub-components, and the flowcharts / flow diagrams (including their parts) described herein and / or other examples described herein, can be implemented in hardware or in hardware with any combination of software and / or firmware, including being implemented as computer program code configured to execute in one or more processors and stored in a computer-readable storage medium, or being implemented as hardware logic / circuitry, such as being implemented together in a system-on-chip (SoC), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). The SoC may include an integrated circuit chip that includes one or more of the following: a processor (such as a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or further circuitry and / or embedded firmware for performing its functions.

[0162] The embodiments described herein can be implemented in one or more computing devices that are similar to the mobile systems and / or computing devices in the stationary or mobile computer embodiments, including one or more features and alternative features of the mobile systems and / or computing devices described herein. The descriptions of the mobile systems and computing devices provided herein are for illustrative purposes and are not intended to be restrictive. The embodiments can also be implemented in other types of computer systems known to those skilled in the relevant art.

[0163] Figure 22 is a block diagram of an exemplary mobile system 2202 that includes a mobile device 2200 that can implement the embodiments described herein. For example, in the preceding section, the mobile device 2202 can be used to implement any system, client, or device or its components / sub-components. As Figure 22 shown, the mobile device 2202 includes various optional hardware and software components. Any component in the mobile device 2202 can communicate with any other component, but not all connections are shown for ease of illustration. The mobile device 2202 can be any of a variety of computing devices (e.g., a cellular phone, a smart phone, a handheld computer, a personal digital assistant (PDA), etc.), and can permit wireless two-way communication with one or more mobile communication networks 2204 such as a cellular or satellite network or with a local area network or a wide area network.

[0164] The mobile device 2202 may include a controller or processor 2210 (e.g., a signal processor, a microprocessor, an ASIC, or other control and processing logic circuitry) for performing tasks such as signal encoding, data processing, input / output processing, power control, and / or other functions. The operating system 2212 may control the allocation and use of the mobile device 2202 and provide support for one or more application programs 2214 (also referred to as “applications,” “apps,” etc.). The application programs 2214 may include common mobile computing applications (e.g., an email application, a calendar, a contact manager, a web browser, a messaging application), or any other computing application (e.g., a word processing application, a drawing application, a media player application).

[0165] The mobile device 2202 may include a memory 2220. The memory 2220 may include non-removable memory 2222 and / or removable memory 2224. The non-removable memory 2222 may include RAM, ROM, flash memory, a hard disk, or other well-known memory devices or technologies. The removable memory 2224 may include flash memory or a subscriber identity module (SIM) card well-known in GSM communication systems, or other well-known memory devices or technologies such as “smart cards.” The memory 2220 may be used to store data and / or code for running the operating system 2212 and the application programs 2214. Example data may include web pages, text, images, sound files, video data, or other data sets sent to and / or received from one or more network servers or other devices via one or more wired or wireless networks. The memory 2220 may be used to store subscriber identifiers such as the International Mobile Subscriber Identity (IMSI), and device identifiers such as the International Mobile Equipment Identity (IMEI). Such identifiers may be transmitted to network servers to identify the user and the device.

[0166] Multiple programs may be stored in the memory 2220. These programs include the operating system 2212, one or more application programs 2214, and other program modules and program data. Examples of such application programs or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing one or more of those in the system 100 along with any of its components and / or sub-components, and the flowcharts / process diagrams described herein, including their parts, and / or other examples described herein. Figure 1

[0167] The mobile device 2202 may support one or more input devices 2230 (such as a touch screen 2232, a microphone 2234, a camera 2236, a physical keyboard 2238, and / or a trackball 2240) and one or more output devices 2250 (such as a speaker 2252 and a display 2254). Other possible output devices (not shown) may include piezoelectric or other haptic output devices. Some devices may provide more than one input / output function. For example, the touch screen 2232 and the display 2254 may be combined in a single input / output device. The input devices 2230 may include a natural user interface (NUI).

[0168] One or more wireless modems 2260 may be coupled to (an) antenna(s) (not shown) and may support two-way communication between the processor 2210 and external devices, as is well understood in the art. The modem 2260 is generally shown and may include a cellular modem 2266 for communicating with a mobile communication network 2204 and / or other radio-based modems (e.g., Bluetooth 2264 and / or Wi-Fi 2262). At least one wireless modem 2260 is typically configured to communicate with one or more cellular networks (such as a GSM network) for data and voice communication within a single cellular network, between multiple cellular networks, or between the mobile device and the public switched telephone network (PSTN).

[0169] The mobile device 2202 may further include at least one input / output port 2280, a power supply 2282, a satellite navigation system receiver 2284 such as a global positioning system (GPS) receiver, an accelerometer 2286, and / or a physical connector 2290, which may be a USB port, an IEEE 2294 (FireWire) port, and / or an RS-232 port. The components of the mobile device 2202 shown are not necessarily required or exhaustive, as any component may be absent and additionally other components may be present, as understood by those skilled in the art.

[0170] In one embodiment, the mobile device 2202 is configured to implement any of the above features of the flowcharts herein. The computer program logic for performing any of the operations, steps, and / or functions described herein may be stored in the memory 2220 and executed by the processor 2210.

[0171] Figure 23Depicts an exemplary implementation of a computing device 2300 in which various embodiments may be implemented. For example, the various embodiments described herein may be implemented in one or more computing devices similar to the computing device 2300 in the stationary or mobile computer embodiments, including one or more features and / or alternative features of the computing device 2300. The description of the computing device 2300 provided herein is for illustrative purposes only and is not intended to be limiting. Embodiments may also be implemented in other types of computer systems and / or game consoles known to those skilled in the relevant art.

[0172] As Figure 23 shown, the computing device 2300 includes one or more processors (referred to as processor circuitry 2302), system memory 2304, and a bus 2306 that couples the various system components including the system memory 2304 to the processor circuitry 2302. The processor circuitry 2302 is implemented as a central processing unit (CPU), microcontroller, microprocessor, and / or other physical hardware processor circuitry using one or more physical hardware electronic circuit device elements and / or integrated circuit devices (semiconductor material chips or dies). The processor circuitry 2302 may execute program code stored in a computer-readable medium, such as the program code of an operating system 2330, an application program 2332, other programs 2334, etc. The bus 2306 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a local bus using any of a variety of bus architectures. The system memory 2304 includes read-only memory (ROM) 2308 and random access memory (RAM) 2310. The basic input / output system 2312 (BIOS) is stored in the ROM 2308.

[0173] The computing device 2300 also has one or more of the following drives: a hard disk drive 2314 for reading from and writing to a hard disk, a disk drive 2316 for reading from and writing to a removable disk 2318, and an optical disk drive 2320 for reading from and writing to a removable optical disk 2322 (such as a CD-ROM, a DVD, or other optical medium). The hard disk drive 2314, the disk drive 2316, and the optical disk drive 2320 are connected to the bus 2306 through a hard disk drive interface 2324, a disk drive interface 2326, and an optical disk drive interface 2328, respectively. These drives and their associated computer-readable media provide the computer with non-volatile storage of computer-readable instructions, data structures, program modules, and other data. Although hard disks, removable disks, and removable optical disks are described, other types of hardware-based computer-readable storage media, such as flash memory cards, digital video disks, RAM, ROM, and other hardware storage media, may also be used to store data.

[0174] Several program modules may be stored on the hard disk, disk, optical disk, ROM, or RAM. These programs include an operating system 2330, one or more application programs 2332, other programs 2334, and program data 2336. The application programs 2332 or other programs 2334 may include, for example, computer program logic (such as computer program code or instructions) for implementing the various embodiments described herein, along with any of its modules, components, and / or sub-components, and the flowcharts / flow diagrams described herein, including their parts, and / or other examples described herein.

[0175] The user may input commands and information into the computing device 2300 through input devices such as a keyboard 2338 and a pointing device 2340. Other input devices (not shown) may include a microphone, a joystick, a gamepad, a dish satellite antenna, a scanner, a touch screen and / or a touch pad, a voice recognition system for receiving voice input, a gesture recognition system for receiving gesture input, and so on. These and other input devices are typically connected to the processor circuit 2302 through a serial port interface 2342 coupled to the bus 2306, but may also be connected through other interfaces (such as a parallel port, a game port, or a universal serial bus (USB)).

[0176] The display screen 2344 is also connected to the bus 2306 through an interface such as a video adapter 2346. The display screen 2344 may be external to or incorporated within the computing device 2300. The display screen 2344 may display information and serve as a user interface for receiving user commands and / or other information (such as through touch, finger gestures, a virtual keyboard, etc.). In addition to the display screen 2344, the computing device 2300 may also include other peripheral output devices (not shown), such as speakers and printers.

[0177] The computing device 2300 is connected to a network 2348 (e.g., the Internet) via an adapter or network interface 2350, a modem 2352, or other means for establishing communication over a network. The modem 2352 (which may be internal or external) can be connected to the bus 2306 via a serial port interface 2342, as Figure 23 shown, or can use another interface type including a parallel interface to connect to the bus 2306.

[0178] As used herein, terms such as "computer program medium", "computer-readable medium", and "computer-readable storage medium" are used to refer to physical hardware media. Examples of such physical hardware media include hard disks associated with hard disk drives 2314, removable disks 2318, removable optical disks 2322, other physical hardware media such as RAM, ROM, flash memory cards, digital video discs, zip disks, MEMs, nanotechnology-based storage devices, and other types of physical / tangible hardware storage media (including Figure 23 the memory 2320). Such computer-readable media and / or storage media are distinguished from and do not overlap with (excluding) communication media and propagated signals. Communication media carry computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media include wireless media (such as acoustic, RF, infrared, and other wireless media) and wired media. Embodiments also relate to such communication media that are separate from and do not overlap with embodiments directed to computer-readable storage media.

[0179] As noted above, computer programs and modules (including the application program 2332 and other programs 2334) can be stored on hard disks, disks, optical disks, ROMs, RAMs, or other hardware storage media. Such computer programs can also be received via the network interface 2350, the serial port interface 2342, or any other interface type. These computer programs, when executed or loaded by an application, enable the computer 2300 to implement the features of the embodiments discussed herein. Accordingly, such computer programs represent the controller of the computer system 2300.

[0180] Embodiments also relate to computer program products that include computer code or instructions stored on any computer-readable medium or computer-readable storage medium. Such computer program products include hard disk drives, optical disk drives, memory device packages, portable memory sticks, memory cards, and other types of physical storage hardware.

[0181] IV. Additional Example Embodiments

[0182] Methods, systems, and computer program products for sharing user interface (UI) customization across applications are provided. Application developers can use a control cluster that can be customized as a swarm to create a UI (e.g., a game control layout). The control cluster can include multiple controls that are selectively assigned to a socket cluster (e.g., a socket swarm) in a layout template. The controls in the control cluster can be customized as a swarm, for example, by changing the position, rotation, and / or scaling of the control cluster. A user can customize the controls for comfort, switch the controls for playing games with the left and right hands, and share the customized preferences across applications. The UI customization can be shared across applications with different UIs. For example, the customization of one or more control clusters in a first UI can be (e.g., automatically) applied to one or more control clusters in other UIs of other applications. A user can avoid making tedious and time-consuming individual control changes for many UIs.

[0183] In one example, a method for sharing UI customization across applications can include, for example, providing a first UI configured with a first plurality of controls in a first layout; providing a second UI configured with a second plurality of controls in a second layout; and displaying the second UI with the second plurality of controls in a customized layout based on the customized layout of the first plurality of controls on a display device. The second plurality of controls can be different from the first plurality of controls.

[0184] In one example, the first UI can include a first control cluster, which can include a first plurality of controls that can be customized as a swarm. The second UI can include a second control cluster, which can include a second plurality of controls that can be customized as a swarm.

[0185] In one example, the method can (e.g., further) include customizing the first UI by changing at least one of the position, rotation, and scaling of the first control cluster.

[0186] In one example, the first control cluster can be based on a first socket swarm that includes a plurality of control sockets. The first plurality of controls can be selectively assigned to the plurality of control sockets. The second control cluster can be based on a second socket swarm that includes a plurality of control sockets. The second plurality of controls can be selectively assigned to the plurality of control sockets.

[0187] In one example, the first socket swarm can be part of a first template that has a plurality of socket swarms. The second socket swarm can be part of a second template that has a plurality of socket swarms.

[0188] In one example, the first UI can include a plurality of control clusters, for example, based on the plurality of socket swarms in the first template. The second UI can include a plurality of control clusters, for example, based on the plurality of socket swarms in the second template.

[0189] In one example, the method can (e.g., further) include customizing a first UI by changing at least one of the position, rotation, and scaling of each of a plurality of control groupings. In one example, the customization of the first and second control groupings can be simultaneous.

[0190] In one example, a first control grouping can be configured to be displayed on a first side of a display and at least one other control grouping can be configured to be displayed on a second side of the display. In one example, the customization of the first UI can include moving the first control grouping to the second side of the display; and moving at least one other control grouping to the first side of the display.

[0191] In one example, a first UI can be configured for a first application; and a second UI can be configured for a second application.

[0192] In one example, the method can (e.g., further) include customizing a first UI by moving a first control in a first control grouping from a first set of interfaces in a first set of interface groupings to a second set of interfaces. In one example, the first control can be assigned to a plurality of sets of interfaces in a first set of interface groupings.

[0193] In one example, the method can (e.g., further) include receiving user input for a first application from a second UI displayed on a computing device; switching from the second UI to the first UI by browsing a library and selecting the first UI from a plurality of UIs in the library; and receiving user input for the first application from the second UI displayed on the computing device.

[0194] In one example, the method can (e.g., further) include displaying the second UI as an overlay on the displayed content generated based on the first application on a display device.

[0195] In one example, a computer-readable medium can have program code recorded thereon that, when executed by at least one processor, causes the at least one processor to perform a method. In one example, the method includes, e.g., providing a first user interface (UI) configured with a first control grouping for display on a display device, the first control grouping including a first plurality of controls that can be customized as a grouping by changing at least one of the position, rotation, and scaling of the first control grouping.

[0196] In one example, the method may (e.g., further) include providing a second UI configured with a second control grouping for display on a display device, the second control grouping including a second plurality of controls that are customizable as a grouping; and customizing the second UI by changing at least one of a position, rotation, and scale of the second control grouping based on a customization of the first UI. The second plurality of controls may be different from the first plurality of controls.

[0197] In one example, the first UI may be configured for a first application; and the second UI may be configured for a second application.

[0198] In one example, the first control grouping may be based on a first interface grouping including a plurality of control sockets. The first plurality of controls may be selectively assigned to the plurality of control sockets. The second control grouping may be based on a second interface grouping including a plurality of control sockets. The second plurality of controls may be selectively assigned to the plurality of control sockets.

[0199] In one example, a system may include at least one computing device configured to provide a first user interface (UI) for display on a display device, the first user interface (UI) being configured for a first application and configured with a first control grouping including a first plurality of controls that are customizable as a grouping. The at least one computing device may be configured to apply a customization of the first control grouping in the first UI to a second control grouping in a second UI, the second control grouping including a second plurality of controls that are customizable as a grouping, wherein the second plurality of controls is different from the first plurality of controls.

[0200] In one example, the at least one computing device may (e.g., further) be configured to provide a UI editor for display on a display device, the UI editor being configured to provide customization of a UI configured with a control grouping by changing at least one of a position, rotation, and scale of the control grouping.

[0201] V. Conclusion

[0202] Although the embodiments of the present application have been described above, it should be understood that they are presented by way of example only and not as limitations. Those skilled in the relevant art will understand that various modifications can be made in form and detail without departing from the spirit and scope of the present application as defined by the appended claims. Accordingly, the scope of the present application should not be limited by any of the above-described exemplary embodiments, but should be defined only by the following claims and their equivalents.

Claims

1. A method for sharing user interface customization across applications, comprising: Providing a first user interface in a first layout configured with a first plurality of controls; In response to customization input, performing a first instance of a specific layout modification on a first set of controls in the first layout to obtain a customized first layout of the first user interface; Providing a second user interface in a second layout configured with a second plurality of controls, wherein the second plurality of controls is different from the first plurality of controls and the second user interface includes a second set of controls corresponding to the spatial location of the first set of controls of the first user interface; and Automatically performing a second instance of the same specific layout modification on the second set of controls in the second layout without user selection as was performed on the first set of controls in the first layout to obtain a customized second layout of the second user interface, wherein the first instance of the specific layout modification involves rotating the first set of controls in response to the customization input, and the second instance of the specific layout modification involves automatically rotating the second set of controls.

2. The method according to claim 1, wherein the first set of controls and the second set of controls respectively share a position relative to other sets of controls in the first layout and the second layout.

3. The method according to claim 2, further comprising: Customizing the first user interface by modifying the scaling of the first set of controls in response to the customization input; And Automatically modifying the scaling of the second set of controls in the same manner as modifying the scaling of the first set of controls without user selection.

4. The method according to claim 2, wherein: The first set of controls is grouped based on a first set of interfaces including a plurality of first control sockets, and the first plurality of controls are selectively assigned to the plurality of first control sockets; and The second set of controls is grouped based on a second set of interfaces including a plurality of second control sockets, and the second plurality of controls are selectively assigned to the plurality of second control sockets.

5. The method according to claim 4, wherein: The first set of interface groupings is part of a first template having a plurality of first set of interface groupings; and The second set of interface groupings is part of a second template having a plurality of second set of interface groupings.

6. The method according to claim 5, wherein: The first user interface includes a first plurality of sets of controls based on the plurality of first set of interface groupings in the first template; and The second user interface includes a second plurality of sets of controls based on the plurality of second set of interface groupings in the second template.

7. The method according to claim 1, further comprising, in response to the customization input, simultaneously modifying at least one other set of controls in the first set of controls and the first layout.

8. The method according to claim 7, wherein, The first set of controls is configured to be displayed on a first side of a display and at least one other set of controls is configured to be displayed on a second side of the display, wherein the method further comprises: In response to the customization input, moving the first set of controls to the second side of the display; and In response to the customized input, move the at least one other set of controls to the first side of the display.

9. The method according to claim 1, wherein the first user interface and the second user interface are configured for different applications.

10. The method according to claim 4, further comprising: Customize the first user interface by moving a first control in the first set of controls from a first set of interfaces in the first set of interfaces to a second set of interfaces.

11. The method according to claim 1, further comprising: Provide a library of available control layouts; And Select the first layout from the library in response to user input from the user browsing the available control layouts in the library.

12. The method according to claim 1, further comprising: Display the second user interface on the display device as an overlay on the content generated by the displayed application.

13. A computer-readable medium having program code recorded thereon, the program code causing the at least one processor to perform a method for cross-application sharing of user interface customization when executed by the at least one processor, the method comprising: Provide a first user interface configured with a first set of a first plurality of controls for display on a display device, the first plurality of controls being customizable as a group; In response to a customization input, perform a first instance of a specific layout modification on the first set of the first plurality of controls to obtain a customized first layout of the first user interface; And Automatically perform a second instance of the same specific layout modification on a second set of a second plurality of controls of a second user interface without user selection as the specific layout modification performed on the first set of the first plurality of controls to obtain a customized second layout of the second user interface, Wherein the first user interface and the second user interface are configured with different controls for different applications, the first instance of the specific layout modification involves rotating the first set of the first plurality of controls in response to the customization input, and the second instance of the specific layout modification involves automatically rotating the second set of the second plurality of controls.

14. The computer-readable medium according to claim 13, wherein: The first user interface is configured for a first application; The second user interface is configured for a second application different from the first application; And The second set has at least one control not present in the first set.

15. The computer-readable medium according to claim 13, wherein: The first set of the first plurality of controls is grouped based on a first set of interfaces including a plurality of control interfaces; and The second set of the second plurality of controls is grouped based on a second set of interfaces including a plurality of control interfaces.

16. A cross-application sharing user interface customization system, comprising: At least one computing device configured to: Determine that a first instance of specific customization has been performed on a first set of controls of a first user interface; And Automatically apply a second instance of the same specific customization as the specific customization performed on the first set of controls of the first user interface to a second set of controls without user selection, wherein the second set of controls is spatially corresponding to the first set of controls, and wherein automatically applying the second instance of the customization includes rotating the second set of controls based at least on a first instance of a customization in response to a user customization input that has rotated a first set of controls.

17. The system of claim 16, wherein the at least one computing device is further configured to: provide an editor for display on a display device, the editor being configured to receive the customization input and perform the first instance of the customization of the first user interface in response to the customization input.

Images