Sharing user interface customization across applications

Abstract

Methods, systems, and computer program products are provided for sharing user interface (UI) customization across applications. Application developers may create UIs (e.g., game control layouts) with control groups that may be customized as groups. The control grouping may include a plurality of controls selectively assigned to a socket cluster (e.g., socket grouping) in a layout template. The controls in the control groups may be customized to groups, for example, by changing the position, rotation, and / or scaling of the control groups. A user may customize controls to achieve comfort, switch controls for left and right hand games, and share customized preferences across applications. The UI customization may be shared across applications with different UIs. For example, customization of one or more control groups in a first UI may be applied to one or more control groups in other UIs of other applications. The user can avoid tedious and time-consuming individual control changes for many UIs.

Description

[0001] This application is a divisional application of the application with international application number PCT / US2020 / 030006, international application date April 27, 2020, national phase application number 202080042119.9, and titled "Cross-application Shared User Interface Customization". background

[0002] Application developers can develop user interfaces for users to interact with applications on a device. An example of a user interface (UI) is a touchscreen interface that provides a graphical user interface (GUI) that enables a user to interact with the device using touch (e.g., by using one or more fingers). The GUI can include one or more displayed controls for the user to interact with on the touchscreen. The layout of the UI is often unique to each application.

[0003] Overview

[0004] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0005] Methods, systems, and computer program products are provided for sharing user interface (UI) customizations across applications. Application developers can create UIs, such as game control layouts, using control groups that can be customized into groups. A control group can include multiple controls that are selectively assigned to a socket cluster in a layout template. A socket cluster can be referred to as a socket group. Controls within a control group can be customized into a group by modifying the control group as a whole. Users can also customize controls individually. Furthermore, UI customizations can be shared across applications with different UIs.

[0006] Further features and advantages of various embodiments, as well as the structure and operation of various embodiments, are described in detail below with reference to the accompanying drawings. Note that the methods and systems are not limited to the specific embodiments described herein. These embodiments are presented herein for illustrative purposes only. Additional embodiments will be apparent to those skilled in the relevant art based on the teachings contained herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings, which are incorporated herein and constitute a part of the specification, illustrate various embodiments of the present application and, together with the description, further serve to explain the principles of the various embodiments and enable those skilled in the relevant art to make and use the embodiments.

[0008] Figure 1 A block diagram of an example system for cross-application user interface customization is shown, according to an embodiment.

[0009] Figure 2 An example of a control socket template on a touch screen according to one embodiment is shown.

[0010] Figures 3A-3G Shown are examples of selective configuration of sockets and socket groups in a touch control socket template according to various embodiments.

[0011] Figure 4A An example of a control layout and a control layout library on a touch screen according to an embodiment is shown.

[0012] Figure 4B An example of an active first control layout on a touch screen according to an embodiment is shown.

[0013] Figure 5 An example of an interface control manager and a first layout grouping on a touch screen according to one embodiment is shown.

[0014] Figure 6 An example of selecting a second widget layout in a first layout group on a touch screen according to one embodiment is shown.

[0015] Figure 7 An example of an active second control layout on a touch screen is shown according to one embodiment.

[0016] Figure 8 An example of a first control layout in a layout editor on a touch screen according to an embodiment is shown.

[0017] Figure 9 An example of scaling and rotating a left-side control group in a first control layout on a touch screen is shown according to one embodiment.

[0018] Figure 10 An example of simultaneously repositioning right and left control groupings in a first control layout on a touch screen is shown according to one embodiment.

[0019] Figure 11 An example of an activated customized version of a first widget layout on a touch screen is shown in accordance with an embodiment.

[0020] Figure 12 An example of sharing customization of a first control layout with a second control layout in the same layout group by repositioning, scaling, and rotating right and left control groupings on a touch screen is shown according to an embodiment.

[0021] Figure 13 An example of browsing a widget layout grouping library on a touch screen according to one embodiment is shown.

[0022] Figure 14An example of sharing customization of a first widget layout in a first layout group with a first widget layout in a second layout group on a touch screen is shown according to an embodiment.

[0023] Figure 15 is an example of sharing customization of a first widget layout in a first layout group with a first widget layout in a third layout group on a touch screen according to an embodiment.

[0024] Figure 16 A flow chart of an example method for providing a control socket template according to one embodiment is shown.

[0025] Figure 17 A flowchart of an example method for providing a control layout for an application according to an embodiment is shown.

[0026] Figure 18 A flowchart illustrating an example method for providing a control layout manager according to an embodiment is shown.

[0027] Figure 19 A flow chart illustrating an example method for selecting a control layout using a control layout manager according to an embodiment is shown.

[0028] Figure 20 A flowchart of an example method for editing a control layout according to an embodiment is shown.

[0029] Figure 21 A flowchart of an example method for browsing a control layout library according to one embodiment is shown.

[0030] Figure 22 A block diagram is shown of an example mobile device that can be used to implement various embodiments.

[0031] Figure 23 A block diagram of a computing device that can be used to implement various embodiments is shown.

[0032] The features and advantages of the embodiments described herein will become more apparent from the detailed description set forth below when taken in conjunction with the accompanying drawings in which like reference numerals identify corresponding elements. In the drawings, like reference numerals generally indicate identical, functionally similar, and / or structurally similar elements. The drawing in which an element first appears is indicated by the leftmost digit(s) in the corresponding reference numeral. Detailed description

[0033] I. Introduction

[0034] This specification and the accompanying drawings disclose one or more embodiments that include the features of the disclosed embodiments. The scope of the embodiments is not limited to the aspects disclosed herein. The disclosed embodiments are merely illustrative of the intended scope, and modifications of the disclosed embodiments are also contemplated. The embodiments are defined by the appended claims.

[0035] References in the specification to "one embodiment," "an embodiment," "an example embodiment," etc., indicate that the described embodiment may include a particular feature, structure, or characteristic. However, every embodiment may not necessarily include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is considered within the knowledge of those skilled in the art to implement the feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.

[0036] Furthermore, it should be understood that the spatial descriptions used herein (e.g., "above," "below," "up," "left," "right," "down," "top," "bottom," "vertical," "horizontal," etc.) are for illustrative purposes only and that actual implementations of the structures described herein may be spatially arranged in any orientation or manner.

[0037] In the discussion, unless otherwise indicated, adjectives such as "substantially" and "approximately" that modify a condition or relational characteristic of one or more features of an embodiment of the present disclosure should be understood to mean that the condition or characteristic is defined within a tolerance of the operation of the embodiment that is acceptable for the application for which the embodiment is intended.

[0038] A number of exemplary embodiments are described below. It should be noted that any section / subsection titles provided herein are not intended to be limiting. Various embodiments are described in this document, and any type of embodiment may be included under any section / subsection. In addition, the embodiments disclosed in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or in different sections / subsections.

[0039] II. Example Embodiments

[0040] Application developers can develop user interfaces for users to interact with applications on a device. An example of a user interface (UI) is a touch screen interface that provides a graphical user interface (GUI) that enables a user to interact with the device using touch (e.g., by applying one or more fingers). The GUI may include one or more displayed controls for the user to interact with on the touch screen. In the context of a GUI, a control is a graphical element that is configured for a user to interact with through touch manipulation to provide input to an application associated with the GUI. Examples of controls include graphical elements such as buttons, sliders, spinners, drop-down lists / menus, toolbars, icons, and the like. The number, type, combination, and positioning of controls in a UI are often unique to each application.

[0041] Accordingly, methods, systems, and computer program products for UI customization and cross-application sharing of user interface (UI) customization are provided. Application developers can create UIs (e.g., game control layouts) using control groups that can be customized as groups. A control group can include multiple controls that are selectively assigned to a socket cluster (e.g., a socket group) in a layout template. The controls in a control group can be customized as a group, for example, by changing the position, rotation, and / or scaling of the control group. Users can customize controls for comfort, switch controls for left-handed and right-handed gaming, and share customized preferences across applications. UI customizations can be shared across applications with different UIs. For example, customizations to one or more control groups in a first UI can be (e.g., automatically) applied to one or more control groups in other UIs of other applications. Users can avoid tedious, time-consuming individual control changes for many UIs.

[0042] Figure 1 A block diagram of an example system 100 for sharing user interface customizations across applications according to an example embodiment is shown. The example system 100 presents several of many possible computing environments. Figure 1 Several components are shown with dashes to indicate several optional and alternative configurations, such as to provide a compact representation of multiple example environments. Many other environments, not shown, can also be used to implement the subject matter described herein. The various components shown may or may not be present in Figure 1 in any given environment as shown in the examples or in examples not shown.

[0043] like Figure 1As shown, system 100 includes computing device 106, computing device 136, controlled device 146, and display screen 148. Computing device 106 includes UI layout manager 108, UI layout editor 112, UI layout library 114, display screen 118, touch screen 124, streaming client 128, and one or more UI control applications 140. Computing device 136 includes streaming service 138, which may include one or more UI control applications 140. Streaming client 128 includes content engine 126. These features of system 100 are described in further detail below.

[0044] refer to Figure 1 10, an example environment may include a user 134 interacting with a UI control application (app) 140 executed by a content engine 134 in a computing device 106. The user 134 may interact with the UI control application 140, for example, based on a UI (shown as a displayed UI layout 120 displayed by a display screen 118) via a touch screen 124. The user 134 may use controls on the UI layout 120 to control real and / or virtual scenes and / or objects, for example, based on viewing in person or based on displayed content 122 displayed by the display screen 118.

[0045] Another example environment may include a user 134 interacting with a UI layout 120 while the computing device 106 acts as a streaming client 128 coupled to a streaming service 138 in the computing device 136, which may be local (e.g., a game console) or remote (e.g., a server-based game console on a network such as the Internet). Displayed content 122, which may be generated by executing the UI control application 140, may be provided, for example, to a display screen 148 (e.g., a television, a desktop or laptop computer display) and / or to a display screen 118. In one example, a user may use a smartphone or tablet (e.g., the computing device 106) as a touchscreen controller while viewing content displayed on a television.

[0046] User 134 may or may not observe the displayed content on display screen 118 or 148. For example, the user may interact with UI layout 120 to personally control a device (e.g., a controlled device 146, such as a wired or wireless controlled flying drone, a water or ground vehicle, or a ground robot) via wired or wireless communication 152. In one example, computing device 106 may include wired or wireless local or remote controls for real or virtual scenes or objects that user 134 can observe and control in one way or another. In another example, computing device 106 may display content and an integrated or overlaid UI. In another example, user 134 may interact with displayed UI layout 120 via a pointing device such as a mouse or other user input device (e.g., keyboard keys).

[0047] Computing devices 106 and 136 may include any type of mobile or stationary computing device, such as a desktop computer, a server, a video game console, a mobile computing device (e.g., devices, personal digital assistants (PDAs), laptops, notebooks, tablets (such as the Apple iPad TM ), netbooks, etc.), mobile phones (e.g., cellular phones, smartphones (such as Microsoft Phone, Apple iPhone, Execution Android TM operating system phones, etc.), stationary computing devices (such as desktop computers or PCs (personal computers)), game consoles / systems (e.g., Microsoft Sony Nintendo or etc. Examples of computing devices 106 and 136 are shown below. Figure 22 and 23 124 , which shows various user input devices including and in addition to the touch screen 124 .

[0048] The content engine 126 can generate application ("app") data (e.g., by executing the UI control application 140) or receive application data streamed by the streaming service 138 from the computing device 136 executing the UI control application 140. The content engine 126 can provide display content 130 to the display screen 118 generated by the content engine 126 or received by the content engine 126 from the streaming service 138. The content engine 126 can be configured to display the UI layout 120 alone or in conjunction with the displayed content 122 (e.g., as an integral part of the displayed content 122 or as a graphical overlay on the displayed content 122). The content engine 126 can receive user input events 132 provided by a user 134 of the computing device 106 interacting with an input control (e.g., the displayed UI layout 120) (e.g., via the touch screen 124). The user input can be provided, for example, via the touch screen 124 and / or other user input devices (see, e.g., Figure 22 and 23 The user input data 132 may indicate a user action (e.g., touching the touch screen 124 near a specific displayed control in the displayed UI layout 120 on the display screen 118) while observing real and / or virtual scenes and / or objects in person or on a display. Figure 1As shown, the content engine 126 can process or transmit input data 132 (e.g., to the streaming service 138 as user input data 142). The user input data 132 can be processed by the UI content application 140 to determine control signals and / or subsequent UI control application execution, which may affect the application data presented to the user 134.

[0049] Application (e.g., game) streaming transmits video game data from a first device running the application (e.g., an "originating" or "source" device) to a second device (e.g., a "client" device) to enable a user to view or interact with the application using the second device. In one example, data for a video game can be streamed from a desktop computer or game console to, for example, a smartphone, tablet, or other device with a touch screen (e.g., a "touch device") to enable the user to interact with the UI control application using the touch screen. Streaming can enable users to play video games on devices for which the video game was not originally designed. For example, game streaming can enable games designed to be played on video game consoles with physical game controllers to be interacted with using a touch device. A graphical overlay can be presented on a touch screen device, for example, to allow a user to interact with the UI control application on a touch screen device for which the UI control application was not designed. The graphical overlay can provide touch screen controls for the user to interact with. The user's interaction with the touch controls can be based on or mapped to physical controls in a physical game controller.

[0050] Computing device 106 can be used to view and interact with applications (e.g., video games) that may not be designed for the display devices and / or input devices available to computing device 106. For example, application content can be streamed from a computing device (e.g., computing device 136). UI control application 140 can be executed on computing device 136 (e.g., a desktop computer, a game console, a remote server with an Internet connection, etc.). Streaming service 138 can be configured to stream application data 144 from UI control application 140 to another device, such as computing device 106. Streaming service 138 can be configured to transmit application data 144 over a (e.g., wired and / or wireless) network (e.g., a local area network (LAN) (such as a wireless LAN (WLAN) or "Wi-Fi") and / or a wide area network (WAN) (such as the Internet)).

[0051] A streaming service 138 (e.g., at a computing device 136) and a streaming client 128 (e.g., at a computing device 106) can collaborate to present streaming video generated by a UI control application 140 (e.g., a video game application) executing at the computing device 136 as displayed content 122 on a display screen 118 coupled to (e.g., integrated with) the computing device 106. The streaming service 138 can stream application data 144 (e.g., video and audio data) generated by execution of the UI control application 140 to the streaming client 128, which can present the streamed video to the user 134 at the computing device 106. The streaming client 128 can stream user input data 142 (e.g., received from a user 134 interacting with the displayed content 122 via the touch screen 124 at the computing device 106) to the streaming service 138 as user input events to the UI control application 140. User 134 can view and interact with UI control application 140, which may or may not be designed for viewing and interaction on computing device 106 (e.g., via touch screen 124). Examples of server-client streaming services in which embodiments may be incorporated may include, for example, those provided by Microsoft The server-client streaming service provided by the platform and the one provided by Valve platform.

[0052] The UI control application 140 may include any application that accepts user input to control real and / or virtual objects that can be viewed in person or on a display. Application developers 104 may develop UI control applications 140 (e.g., games, drones, robots, and other UI control applications) that users 134 may interact with via one or more UIs. Application developers 104 may provide UIs integrated into UI control applications (e.g., UI control applications 140) and integrated or separate UIs to users and / or service providers (such as streaming game service providers).

[0053] UI control applications (e.g., created by application developer 104 using one or more UIs) may include any application that controls anything (whether real or virtual) based on the feedback provided by the user through interaction with the UI. In one example, a user can use visual UI controls to control real or virtual objects or physical / real devices (e.g., cameras, gimbals, drones, robots) in the displayed content. The UI control application may or may not provide the displayed content. The UI control application may include, for example, video games, drone control applications, robot control applications, etc. The user can control real and / or virtual objects, for example, by interacting with a displayed computer-generated user interface (UI) (e.g., a graphical UI (GUI) or a head-up display (HUD)) to provide user input to the UI control application. The user can interact with the UI, for example, by using a touch device, a pointing device (e.g., a mouse, a stylus), a keyboard, or other user input devices.

[0054] The UI control application can be adapted to or initially developed for a touch device and can be displayed separately or as an overlay on a UI application (e.g., a game) displayed on a touch device display.

[0055] The UI control application may or may not provide display content. In one example, the UI control application 140 can generate real and / or virtual scenes and / or real and / or virtual objects (e.g., people or characters, vehicles, weapons, tools) for display on the display screen 118 and / or the display screen 148. In one example, a user can view a controllable real / physical object in person (e.g., a ground robot or a flying drone) or view the controllable real / physical object as a video in a first or third person perspective. The display content provided by the UI control application can, for example, be displayed on the same device on which the UI is displayed and / or on a separate display. Accordingly, the user can view real or virtual objects on the display of the computing device that provides the UI controls and / or on different displays. In one example, the UI controls can be displayed together with the display content (e.g., embedded in a game), the UI can be displayed as a graphical overlay on the display content, the UI can be displayed on a first display and the display content can be displayed on a second display (e.g., a computer monitor, a television, or a digital whiteboard), etc.

[0056] The UI control application 140 may communicate user controls, for example, by utilizing computing device resources such as wireless communications (eg, cellular, WiFi, Bluetooth) or wired communications (eg, USB).

[0057] Application developers 104 can utilize, for example, UI templates for developing one or more UIs to control real or virtual objects via one or more UI control applications 140. Templates can be platform-independent and / or application-independent. Application developers can develop UI control templates. Platform developers 102 can develop UI control templates and provide them to application developers (e.g., application developers 104) to develop UIs for various UI control applications. In one example, UI control templates can be developed by, for example, a gaming platform (e.g., Microsoft Sony PlayStation Nintendo ) platform developers to application (e.g., game) developers. Templates can have control socket configurations or layouts. An unlimited variety of socket templates can be implemented. Template socket designs (e.g., configurations or layouts) can be based on, for example, the type of UI application, the type of control, the type of display device, the type of interaction (e.g., handheld finger touch, pointing device), etc. Templates can include suggested UI control groupings (e.g., central or master control(s) surrounded by satellite controls), suggested locations (e.g., right, left, top, bottom), and suggested uses (e.g., primary, secondary, menu, social media).

[0058] In one example, a single UI control application may be compatible with multiple UIs from the same or different developers. In one example, a UI may be used to control real or virtual objects in one or more applications. The UI may be provided together with the application and / or provided separately (e.g., including provided by a third party). Users of many applications (e.g., people playing games) may use many UIs and may potentially use many different types of UIs. There may be many UI categories and subcategories or UI groups and subgroups, such as action (e.g., shooting, fighter, platform, stealth), sports (e.g., racing, football), adventure, strategy (e.g., tactical war games, arena), simulation (e.g., flight, combat flight, driving, racing, architecture), role-playing.

[0059] The UI layout 120 can be displayed alone or in conjunction with the displayed content 122 (e.g., as an integral part of the displayed content 122 or as a graphical overlay on the displayed content 122). The displayed UI layout 120 can include an arrangement (e.g., a configuration or layout) of one or more graphical controls, referred to herein as a user interface (UI) control configuration or layout, that the user 134 can interact with on the display screen 118 while the UI control application 140 is being executed. The UI layout 120 can include any number and type of controls that the user 134 can interact with to control real and / or imaginary objects that they are viewing (e.g., via the UI control application 140) in person and / or on the display. In one example, the UI layout 120 can include graphical controls for steering, flying, landing, accelerating, braking, running, jumping, shooting, punching, kicking, dodging, and the like.

[0060] The UI layout manager 108 can manage the UI for one or more UI control applications, for example, by providing a menu system with which the user 134 can interact to, for example, obtain, store, browse, select, organize, edit (e.g., customize), and create preferences. The UI layout manager 108 can be implemented as, for example, a stand-alone application, a part of an operating system, a service to an application or operating system, and / or a component of a platform application (e.g., Microsoft Xbox). The platform developer 102 or the application developer 104 can, for example, develop the UI layout manager 108 to manage UIs that can be created using one or more UI templates (e.g., provided by the platform developer 102 to the application developer 104). The UI layout manager 108 can include, for example, a UI layout menu 108, a UI layout editor 112, a UI layout library 114, and UI layout preferences 116. Other implementations can include the same, more, or fewer components, which can include the same and / or different components.

[0061] The UI layout menu 110 can provide a navigation menu for the user 134 to access and use features provided by the UI layout manager 108, such as, for example, obtaining, storing, browsing, selecting, deleting, organizing, editing (e.g., customizing), creating preferences for UI control layouts, etc. In one example, the layout menu 110 can be selectable. The UI layout menu 110 can, for example, show the user 134 the currently selected UI layout group and / or one or more specific UI layouts. The UI layout menu 110 can provide the user 134 with navigation options, such as, for example: (i) entering active / use mode (e.g., with the first UI control layout in the currently selected UI layout group); (ii) selecting a different UI control layout in the currently selected UI control layout group; (iii) selecting an edit mode to edit one of the multiple UI control layouts in the currently selected UI layout group; (iv) selecting a library mode to browse multiple UI control layout groups, etc. Figure 5 One of many possible example implementations of the UI layout menu 110 is shown.

[0062] The UI layout library 114 may provide a library of UI control layouts that may be organized, for example, by editor groups (e.g., editor groups A, B, C) and / or sub-editing groups, by application type, by application title, by UI control, and the like. The UI layout library 114 may be searchable, for example, by editor groups and / or sub-editing groups, by application type, by application title, by UI control, and the like. The UI layout library 114 may be provided with UI control layouts and / or may be searchable for UI control layouts. The UI layout library 114 may organize UI control layouts based on default settings and / or user settings. The UI layout library 114 may include one or more editor groups. Each editor group in the UI layout library 114 may include one or more UI control layouts. Figure 13 One of many possible example implementations of browsing the UI layout library 114 is shown.

[0063] The UI layout editor 112 can allow a user 134 to edit (e.g., customize) one or more UI control layouts. A user may choose to customize, for example, for comfort, for handedness, or for any other purpose. For example, the size of hands and fingers may differ significantly between children and adults. Similarly, computer devices and the way they are held may vary depending on the device and the user. A left-handed user can, for example, swap a first control grouping side with a second control grouping side. Edits can be made, for example, to be UI-specific, UI category-specific, general, and the like.

[0064] In one example, user 134 can customize one or more control groups in a UI control layout, for example, by customizing the position, rotation (e.g., angle), or scale of the one or more control groups. User 134 can, for example, create user preferences that control groups (such as the right control group and the left control group (e.g., regardless of the UI control application)) should (e.g., always) be in a specific position, have a specific size (e.g., the size of the active / contact area and / or icon), have a specific rotation, and / or a specific scale.

[0065] In one example, user 134 can customize one or more controls, for example, by moving the controls from a first interface to a second interface. User 134 can, for example, select (e.g., touch and hold) a control (e.g., in a first or second control group) and drag it to a new location (e.g., in a first or second control group). User 134 can, for example, create a user preference that a particular control (e.g., regardless of the UI control application) should (e.g., always) be in a particular location with a particular size (e.g., the size of the active / contact area and / or icon).

[0066] Figure 8-10 Some of many possible example implementations of the UI layout editor 112 are shown for illustration purposes and are described in further detail below.

[0067] UI layout preferences 116 can be specified by user 134. Preferences may include, for example, a list of settings that specify that apply to one or more UI control layouts (e.g., icon size). User 134 can specify whether one or more preferences should (e.g., automatically) be applied to one or more UI control layouts (e.g., a subset or all of a UI control layout). Preferences can (e.g., also) be created by editing one or more UI control layouts. For example, user 134 can edit a first UI control layout with two control groups, for example, by customizing the position, rotation, and / or scaling of the control group. Customizations can be applied to other UI control layouts, such as those with two control groups, (e.g., based on default and / or user preferences). Figure 8-11 , 12, 14 and 15 illustrate some of many possible example implementations of creating user customization preferences by editing a UI control layout and applying them to a UI control layout, and are described in further detail below.

[0068] Figure 2 An example of a control socket template according to an example embodiment is shown. The example template 200 is displayed on a touch screen display 202 (e.g., a cell phone, a tablet, a digital whiteboard), but embodiments can be implemented using any type of computing device and any type of display. The example template 200 presents a visual representation of computer executable code (e.g., a program) (and the display output generated by it).

[0069] The control socket template can be provided by a development platform (e.g., Microsoft Sony PlayStation Nintendo ) to application (e.g., game) developers. Templates can be platform-independent and / or application-independent. Application developers can have their own templates. Template socket configurations or layouts can, for example, provide suggested groupings of UI controls (e.g., central or master control(s) surrounded by satellite controls), suggested locations (e.g., right, left, top, bottom), suggested uses (e.g., primary, secondary, menu, social media). An unlimited variety of socket templates can be implemented. Template socket designs (e.g., configurations or layouts) can be based on, for example, the type of UI application, the type of control, the type of display device, the type of interaction (e.g., handheld finger touch, pointing device), etc.

[0070] Application developers can utilize, for example, templates for developing one or more UIs to control real or virtual objects via one or more applications. In one example, a single UI control application may be compatible with multiple UIs from the same or different developers. In one example, a single UI can be used to control real or virtual objects in one or more applications. The UI can be provided together with the application and / or provided separately (for example, including provided by a third party). Users of many applications (for example, people playing games) may use many UIs and may potentially use many different types of UIs. There may be many UI categories and subcategories or UI groups and subgroups, such as action (for example, shooting, fighter jets, platforms, stealth), sports (for example, racing, football), adventure, strategy (for example, tactical war games, arena), simulation (for example, flight, combat flight, driving, racing, architecture), and role-playing.

[0071] The platform developer or other developer may (e.g., further) provide a UI manager to the application user to manage UIs created using one or more templates. The UI manager may be, for example, a standalone application, part of an operating system, a service for an application or operating system, and / or integrated into a platform application. The UIs may be managed, for example, in a library that may be (e.g., selectively) organized, such as by group and / or subgroup, by application type, by application title, by UI control, etc. One or more sockets may be used to access the UI manager.

[0072] A template may include any number of individual and / or warp group sockets. A socket may represent a programmable location into which one or more controls may be inserted (e.g., assigned to, associated with). A socket may be used in whole or in part (e.g., associated with a control) or unused (e.g., not associated with a control). In one example, a socket may be split and / or combined (e.g., with other parts of (the) sockets or with (the) complete socket combination). A socket may be grouped, for example, by one or more characteristics (such as relative position or proximity). A warp group socket may be manipulated as a group. For example, a socket group may be collectively changed in position, rotation, and / or scaling as a group.

[0073] exist Figure 2 In the example of , template 200 includes a first socket grouping 206, a second socket grouping 208, a third socket grouping 210, and a fourth socket grouping 212. Sockets can be of different sizes. Figure 2 In an example, the sockets of template 200 may have a first size (e.g., size 1) instead of the center sockets 228, 236 in the first and second control groups 206, 208 (which may have a second size (e.g., size 2)). The combined first socket group 206 and the combined second socket group 208 may have a third size (e.g., size 3). Controls may have different sizes that may be suitable for one or more socket sizes, for example, based on the ability of the user and / or device to reliably distinguish between the controls. In one example, buttons, joysticks, touch pads, and throttle controls may be suitable for size 1, size 2, and size 3. A larger directional (D) pad control may be suitable for size 2 and size 3 sockets (e.g., but may not be suitable for size 1). A flying button set may be suitable for size 3 (e.g., but may not be suitable for size 1 and size 2). Scaling the socket groups (e.g., to increase and / or decrease the distance between sockets) may affect the set size.

[0074] The first socket group 206 may be referred to as, for example, the left-hand (LH) socket group or the left wheel. The left wheel 206 is shown as having concentric circles (e.g., a center socket surrounded by satellite sockets). The left wheel 206 may have any shape, such as a circle, a square, a rectangle, a triangle, a regular polygon, an irregular polygon, a star, a hub and spokes, etc. The left wheel 206 includes a center socket 228, which may represent the active (e.g., hit or touch) area of ​​a control associated with the socket 228. An icon 230 may be associated with the center socket 228, for example, to graphically depict the purpose of a control associated with the center socket 228 and / or identify the center and / or active area of ​​the center socket 228. Surrounding the center socket 228 are eight outer or satellite sockets, represented by satellite sockets 224, for example. The touch area of ​​the satellite socket 224 is shown as having an irregular shape with straight sides and curved sides (e.g., a pie slice with the center removed). The satellite sockets may have any shape. Icon 226 may be associated with satellite socket 224 , for example, to graphically depict the purpose of controls associated with satellite socket 224 and / or identify a center and / or active area of ​​satellite socket 224 .

[0075] The second socket group 208 may be referred to as, for example, the right-hand (RH) socket group or the right wheel. The right wheel 208 is shown as having concentric circles (e.g., a center socket surrounded by satellite sockets). The right wheel 208 may have any shape, such as a circle, square, rectangle, triangle, regular polygon, irregular polygon, star, hub and spokes, etc. The right wheel 208 includes a center socket 236, which may represent the active (e.g., hit or touch) area of ​​a control associated with the center socket 236. An icon 238 may be associated with the center socket 236, for example, to graphically depict the purpose of a control associated with the center socket 236 and / or identify the center and / or active area of ​​the center socket 236. Surrounding the center socket 236 are eight outer or satellite sockets, represented by, for example, satellite sockets 232. The touch area of ​​the satellite socket 232 is shown as having an irregular shape, with straight sides and curved sides (e.g., a pie slice with the center removed). The satellite sockets may have any shape. Icon 234 may be associated with satellite socket 232 , for example, to graphically depict the purpose of controls associated with satellite socket 232 and / or identify a center and / or active area of ​​satellite socket 232 .

[0076] The third socket grouping 210 can be referred to as, for example, an upper tray. The upper tray 210 can include, for example, a left socket grouping 214, a center socket grouping 216, and a right socket grouping 218. Each socket grouping is shown as having three sockets with a circle. Various implementations of the template may or may not have an upper tray, and may have any number of socket groups (e.g., including zero), any number of sockets, and any shape suitable for display and interaction. The socket grouping in the upper tray 210 can be organized for, for example, menu access controls (e.g., Xbox menu, UI manager menu), social media access controls, input controls, being retained, etc. As indicated, a socket can be associated with an icon to graphically depict the purpose of the controls associated with a socket and / or identify the center and / or active area of ​​a socket.

[0077] The fourth socket grouping 212 can be referred to as, for example, a lower tray. The lower tray 212 can include, for example, a center socket grouping. The fourth socket grouping 212 is shown with three sockets having a circular shape. Various implementations of the template may or may not have a lower tray and may have any number of socket groups (e.g., including zero), any number of sockets, and any shape suitable for display and interaction. The sockets in the lower tray 212 can be organized for, for example, secondary control actions. As indicated, a socket can be associated with an icon to graphically depict the purpose of the control associated with the socket and / or identify the center and / or active area of ​​the socket.

[0078] Figure 2 The sockets shown in FIG can be reshaped, repositioned, separated, disassembled, combined, and / or manipulated in any other manner. Figure 2 The sockets shown in may be assigned to controls, or may not be used to create control groups and UI control layouts. Figure 2 The sockets shown in are just examples. An unlimited number of examples can be implemented.

[0079] Figures 3A-3G An example of selective configuration of sockets and socket groupings in a touch control socket template according to an example embodiment is shown. Figures 3A-3G It is shown (e.g., by way of example) how an application developer 104 or user 134 may selectively utilize a socket in the example socket group 300 to create an example control group 312. It can be observed that the example control group 312 provides controls available on the right-hand side of a Microsoft Xbox controller. Of course, Figures 3A-3G One of an infinite number of examples of a socket and the selective assignment of controls to the socket is represented.

[0080] Figure 3A2. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid. Sockets are grouped together to form a grid.

[0081] Figure 3B A central socket 302 is shown, along with various examples of combining, splitting, and reconfiguring sockets 1, 2, 3, and 4. As shown in the examples, sockets 1, 2, 3, and 4 can be combined (e.g., have a larger combined size) and assigned to a single widget B1, combined and assigned to two widgets B1, B2, combined and / or split (e.g., partially) and assigned to three sockets B1, B2, and B3, or assigned as four sockets to four widgets B1, B2, B3, and B4.

[0082] Figures 3C-3F Shows the direction and Figure 3A Several examples of socket assignment controls in the second region adjacent to the third lobe in the third region in FIG. Figures 3C-3F Sockets assigned to the controls (eg, faded) are shown, such as center sockets 0, 1, 2, 3, sockets 0, 1, 2 (empty), and 3 in the second region. Figure 3C Unassigned sockets 4.0, 4.1, 4.2 and 4.3 are shown, which are from Figure 3A The sockets 2.0, 2.1, 2.2, and 2.3 in the . Figure 3C As shown, sockets 4.0, 4.1, 4.2, and 4.3 may be used as is (eg, as provided in the template) and assigned to four controls.

[0083] Figure 3D Will Figure 3C An alternative configuration to the four sockets 4.0, 4.1, 4.2 and 4.3 shown in FIG is shown as three sockets 4.0, 4.3 and 4.1. Figure 3D The socket 4.1 in consumes the Figure 3C The sockets 4.0, 4.1, 4.2 and 4.3 shown in FIG are part of the active area defined by the Figure 3CThe sockets 4.1 and 4.2 have different shapes (eg, circular). Figure 3D The shapes of the sockets 4.0 and 4.3 in FIG. 4 are different (eg, the active touch areas) due to the imposed circular shape of the socket 4.1. Figure 3C The socket shape in 4.0 and 4.1.

[0084] Figure 3E Will Figure 3C An alternative configuration of the four sockets 4.0, 4.1, 4.2 and 4.3 shown in FIG is shown as two sockets 4.0 and 4.1, where Figure 3E Socket 4.0 combines Figure 3C Sockets 4.0 and 4.3. Figure 3E The socket 4.1 in consumes the Figure 3C The sockets 4.1 and 4.2 shown in FIG. 4 define a portion of the active area and have Figure 3C The sockets 4.1 and 4.2 have different shapes (eg, circular).

[0085] Figure 3F Will Figure 3C An alternative configuration of the four sockets 4.0, 4.1, 4.2 and 4.3 shown in FIG is shown as one socket 4.0. Figure 3E Socket 4.0 combines Figure 3C Sockets 4.0, 4.1, 4.2, and 4.3. Figure 3E The socket 4.0 in has a different shape (e.g., round) and consumes Figure 3C Sockets 4.1, 4.2, 4.3, and 4.4 shown in FIG. 4 may be a portion (eg, less than all) of the active region defined by the sockets 4.1, 4.2, 4.3, and 4.4.

[0086] Figure 3G Shown Figure 3F 312. For example, Figure 3F Socket 4.0 in the middle is assigned to the right joystick (R) control. Socket 3 in the second (e.g., satellite) region is assigned to the right joystick press control. Socket 2 in the second (e.g., satellite) region is assigned to an empty (unused or TBA) area. Socket 1 in the second (e.g., satellite) region is assigned to the right trigger (RT) control. Socket 0 in the second (e.g., satellite) region is assigned to the right front button (RT) control. Center sockets 0, 1, 2, and 3 are assigned to the B4, B2, B3, and B1 button controls, respectively. Icons associated with the sockets are displayed in control group 312.

[0087] Widget groups 312 can be created, for example, by a developer (e.g., based on a socket template). Widget groups 312 can be customized (e.g., by an end user) as a group (e.g., by collectively changing position, rotation, and / or scale as a group). Individual sockets and widgets can be customized (e.g., by an end user), such as by moving a widget from a first socket to a second socket, swapping widget positions (e.g., by swapping sockets), etc. In one example, a user can swap the positions of an RT widget and an RB widget or move an RB widget from socket 0.0 (see, e.g., Figure 3A ) moves to socket 0.1.

[0088] In various embodiments, the control layout including the control groupings may be configured in various arrangements, and such various arrangements of control layouts may be stored in a library for selection during game play for editing. Figure 4A A block diagram illustrates a control layout and control library displayed on the display screen 118 of an example display device 400, according to one embodiment. Specifically, the display device 400 displays a split-screen example presented to a user (e.g., user 134) on the display screen 118, including a UI layout 120 and a UI layout library 114. The displayed UI layout 120 may include, for example, the current selection of a UI layout from the UI layout library 114. The displayed UI layout 120 includes, for example, at least a first control group 156 and a second control group 158. The first control group 156 may include any number of controls, such as, for example, at least FG-C1, FG-C2, and FG-C3. The second control group 158 may include any number of controls, such as, for example, at least SG-C1, SG-C2, and SG-C3. Other examples may display more or fewer control groups and controls within the control groups. The UI layout library 114 may be minimized (e.g., reduced to a small icon or indicator to increase the size of the displayed UI layout 120). The UI layout library 114 includes, for example, at least a first UI layout 150, a second UI layout 152, and a third UI layout 154. A user can, for example, scroll up and down to view (e.g., select) other UI layout groups in the UI layout library 114. One (e.g., each) group can include zero or more UI layouts.

[0089] Any number of UI layouts may be included in the UI layout library 114. In one example, UI layout groups may be organized (e.g., as groups) based on application titles. In other implementations, UI layout groups may be organized (e.g., as groups) based on any organizational characteristics (e.g., by socket template, by control type, etc.). Searches may be performed to, for example, find, select, and / or organize UI control layouts based on features or characteristics. For example, according to the following description of Figure 4B and Figure 5-21As described in the examples, such UI control layouts can be configured to include any number of controls in any number of control groups, and can be selected from any number and configuration of control layouts in a library.

[0090] Figure 4B An example of an active first control layout according to an example embodiment is shown. The example (e.g., first) UI control layout 401 can be displayed on (e.g., positioned and scaled for) any display device. In one example, (e.g., as Figure 4B As shown in FIG. 4 , a UI control layout 401 may be displayed on a touch screen device display 402 (eg, a smart phone, a tablet). The first control layout 401 shows controls assigned to a control socket (eg, in a control socket template).

[0091] A user may interact with controls in first control layout 401 to provide user input that may be processed to control real and / or virtual objects. First UI control layout 401 may include, for example, a UI layout manager (UILM) 406, a first control group 408, a second control group 412, and a third control group 410.

[0092] When Figure 4B When positioned as shown, first control group 408 may be referred to as, for example, a left or left-hand (LH) control group or wheel. First control group 408 may include any number and combination of controls. For example, in various embodiments, first control group 408 may include, for example, one or more of a left joystick (C5), a left stick press (B5), a directional pad (C6), a left trigger (T1), a left finger button (FB1), a button, a touchpad, and the like.

[0093] When Figure 4B When positioned as shown, the second control group 412 can be referred to as, for example, a right or right-hand (RH) control group or wheel. The second control group 412 can include one or more of any number and combination of controls. For example, in various embodiments, the second control group 412 can include, for example, a right joystick (C7), a right stick press (B6), one or more face buttons (B1, B2, B3, B4), a right trigger (T2), a right finger button (FB2), a direction stick (C7), a touchpad, etc.

[0094] When Figure 4BWhen positioned as shown, the third control group 410 can be referred to as, for example, an upper tray. In other embodiments, the third control group 410 can be positioned elsewhere in the UI control layout 401, including on the left or right side of the UI control layout 401, and in the middle or lower position on the UI control layout 401. The third control group 410 can include any number and combination of controls, such as the first to fourth controls C1-C4. For example, in one embodiment, the third control group 410 can include return, navigation and start controls, one or more menu access controls, and the like. In various embodiments, the controls in the UI control layout 401 can have any graphical form factor, including being displayed and interactable as buttons, touch pads, switches, D pads, sticks, and the like. In one example, the controls in the third control group 410 can fade out when the right and / or left control groups 408, 412 are in use and return, for example, when inactive (e.g., after a delay).

[0095] The UI layout manager 406 can be selected, for example, to exit the active UI control mode to access the UI layout manager menu. The UI layout manager can manage many (e.g., dozens or hundreds) of control layouts created for many different applications based on one or more socket templates. A user can select the UI layout manager 406 to, for example, edit the first control layout 401, select a different control layout to use or edit in the same group, browse a library of control layouts in various groups, etc.

[0096] Figure 5 An example of an interface control manager and a first layout grouping according to an example embodiment is shown. Figure 5 An example of a UI layout manager menu 500 is shown, which can be accessed, for example, by selecting Figure 4B In the example screen shown, UI layout manager 406 is selected to access. Example layout manager 500 is one of many possible examples of managing multiple control layouts for one or more control applications. Other examples may implement different options and screens.

[0097] The example UI manager menu 500 can show a representation 400A of the first control layout 401 along with menu options. The representation 400A can include, for example, a highlighted, reduced size, and / or inactive (e.g., non-responsive) version of the first control layout 401. The first control layout 401 can be associated with a label (e.g., a title or name). In one example, (e.g., as Figure 4B), a first control layout is labeled "Standard" in the first control layout group. In one example, a second control layout in the first group is labeled "Custom 1." Representation 700B of Custom 1 may be shown in UI manager menu 500. An indication may be provided (e.g., a highlighted triangle below Standard) to illustrate which control layout in the group is selected (e.g., and may become active upon exiting UI manager menu 500).

[0098] The menu navigation options may include, for example, Exit 514, Edit 516, Library 518, and an option to select a different UI control layout in the current layout group (eg, a control layout named "Custom 1").

[0099] Selecting Exit 514 (e.g., without making any other selections) may, for example, exit UI manager menu 500 and return to the active controls mode with standard / first control layout 401 as the active / in-use control layout, e.g., as shown in FIG. Figure 4B As shown in the example.

[0100] Selecting Custom 1 can, for example, replace standard representation 400A with a representation of Custom 1 as the selected control layout in the group. A user can select the Custom 1 control layout, for example, by selecting (e.g., touching a displayed control on touch screen 402) "Custom 1" or representation 700B of Custom 1. Selecting Exit 514 (e.g., after selecting Custom 1) can, for example, exit UI manager menu 500 to an active control mode with the Custom 1 control layout as the active / in-use control layout. Figure 6 and 7 Examples of these selections are shown.

[0101] Selecting edit 516 may, for example, open a layout editor to edit the selected control layout. Figure 5 ) is the "standard" or first control layout 401. The user may alternatively select Custom 1, then select Edit 516 to edit Custom 1, or select Library 518, select any control layout in the library, and select Edit to select another control layout, for example. Figure 8 An example of selecting Edit 516 to edit the first control layout (labeled "Standard") 401 is shown, while Figure 9 and 10 Editing of a first control layout (labeled "Standard") 401 is shown.

[0102] Select Library 518 to open a library of widget layouts for browsing and selection. Figure 13 An example of selecting library 518 is shown, while Figure 14 and 15 Widget layouts from different groupings selected from a library of widget layouts are shown.

[0103] Figure 6 An example of selecting a second widget layout in a first layout group is shown according to an example embodiment. Figure 6 An example of Customize 1 being selected in the example layout manager menu 500 is shown.

[0104] The example UI manager menu display 600 can show a representation 700A of a second control layout (Custom 1) 700 along with menu options. Representation 700A can include, for example, a highlighted, reduced-size, and / or inactive (e.g., unresponsive) version of the second control layout 700. A representation 400B of an unselected first control layout (Standard) can be shown in the UI manager menu display 600. An indication can be provided (e.g., a highlighted triangle below Custom 1) to illustrate which control layout in the group is selected (e.g., and can become active upon exiting the UI manager menu display 600).

[0105] The menu navigation options may include, for example, Exit 514, Edit 516, Library 518, and an option to select a different UI control layout (eg, a first control layout "Standard") in the current (eg, first) layout group.

[0106] Selecting Exit 514 (e.g., without making any other selections) may, for example, exit the UI manager menu to the active controls mode with the second control layout customization 1700 as the active / in-use control layout, e.g., as shown in FIG. Figure 7 As shown in the example.

[0107] Selecting Standard may, for example, return to manager menu 500. A user may select the standard control layout, for example, by selecting (e.g., touching a displayed control on touch screen 402) "Standard" or a representation 400B of Standard. Selecting Exit 514 (e.g., after selecting Standard) may, for example, exit UI manager menu 600 to an active control mode with the Standard control layout as the active / in-use control layout, for example, as Figure 4B As shown in the example.

[0108] Selecting edit 516 may, for example, open a layout editor to edit the selected control layout. Figure 6 ) is a second control layout customization 1. The user may alternatively select Standard and then select Edit 516 to edit the standard or select Library 518, select any control layout in the library and select Edit to select another control layout. Figure 8 An example of selecting Standard and then Edit 516 to edit the first control layout (Standard) 401 is shown, while Figure 9 and 10Editing of a first control layout (labeled "Standard") 401 is shown.

[0109] Select Library 518 to open a library of widget layouts for browsing and selection. Figure 13 An example of selecting library 518 is shown, while Figure 14 and 15 Widget layouts from different groupings selected from a library of widget layouts are shown.

[0110] Figure 7 An example of an active second control layout according to an example embodiment is shown. The example (e.g., second) UI control layout 700 can be displayed on (e.g., positioned and scaled for) any display device. In one example, (e.g., as Figure 7 ), a second control layout (custom 1) 700 in the first layout group can be displayed on a touch screen device display 402 (e.g., a smart phone, a tablet). The second control layout 700 shows controls assigned to a control socket (e.g., in a control socket template).

[0111] The user can interact with the controls in the second control layout 700 to provide user input that can be processed to control real and / or virtual objects. The second UI control layout 700 may include, for example, the UI layout manager 406, the first control group 708, the second control group 712, and the third control group 410.

[0112] The first control group 708 may be referred to as, for example, a left or left-hand (LH) control group or wheel. The second control group 712 may be referred to as, for example, a right or right-hand (RH) control group or wheel. The third control group 710 may be referred to as, for example, an upper tray 710.

[0113] Figure 8 An example of a first control layout in a layout editor according to an example embodiment is shown. Example layout editor 800 shows an editable representation 400C of a first control layout (standard) 401. Editable representation 400C includes an editable first control group 808 and an editable second control group 812. In various embodiments, UI layout manager 406 and third control group 810 may or may not be editable.

[0114] The editable first control group 808 may be referred to as, for example, an editable left or left-hand (LH) control group or wheel. The editable first control group 808 may include, for example, a left joystick (C5), a left stick press (B5), a directional pad (C6), a left trigger (T1), and a left finger button (FB1). The editable first control group 808 may (for example, further) include custom controls, such as a first group rotation and zoom control 814 and a first group positioning control 816.

[0115] First grouped rotation and scale controls 814 may include, for example, a selectable ring near, on, or around editable first control cluster 808. First grouped positioning controls 816 may include, for example, a selectable area inside first grouped rotation and scale controls 814 (e.g., a central area of ​​editable first control cluster 808). A user may select first grouped rotation and scale controls 814 or first grouped positioning controls 816, for example, by touching and holding any point on the selectable area.

[0116] The editable second control group 812 may be referred to as, for example, an editable right or right-hand (RH) control group or wheel. The editable second control group 812 may include, for example, a right joystick (C7), a right stick press (B6), face buttons (B1, B2, B3, B4), a right trigger (T2), and a right finger button (FB2). The editable second control group 812 may (e.g., further) include custom controls, such as a second group rotation and zoom control 818 and a second group positioning control 820.

[0117] Second grouped rotation and scale controls 818 may include, for example, a selectable ring near, on, or around editable second control cluster 812. Second grouped positioning controls 820 may include, for example, a selectable area inside second grouped rotation and scale controls 818 (e.g., a central area of ​​editable second control cluster 812). A user may select second grouped rotation and scale controls 818 or first grouped positioning controls 820, for example, by touching and holding any point on the selectable area.

[0118] A user may customize first control layout 401 , for example, by interacting with customization controls (eg, first group rotation and zoom controls 814 , first group positioning controls 816 , second group rotation and zoom controls 818 , and second group positioning controls 820 ).

[0119] In one example, the user can selectively edit the position, rotation, and scale of the editable first control group 808 and the editable second control group 812. The user can, for example, customize the editable first control group 808 and the editable second control group 812 simultaneously.

[0120] For example, a user may select first group rotation and scaling controls 814 (e.g., by touching and holding any point on the ring defined by first group rotation and scaling controls 814). Dragging the held position clockwise or counterclockwise (e.g., along the ring defined by first group rotation and scaling controls 814) may rotate editable first control group 808 to the right or left, respectively. Dragging the held position away from or toward the center of editable first control group 808 may expand or contract (increase or decrease) the scale of editable first control group 808, respectively. An increase in scale may increase the distance between controls, while a decrease in scale may decrease the distance between controls.

[0121] For example, the user can select the first grouped position control 816 (e.g., by touching and holding any point in the selectable area defined by the first grouped position control 816). Dragging the held position (e.g., in any direction) can move the editable first control group 808.

[0122] The size of the socket / control active area and / or control icon can be customized, for example, by specifying a user preference (e.g., small, medium, large, extra large).

[0123] Customizations of a UI control layout can be applied (e.g., selectively and / or automatically) to other UI control layouts (e.g., with the same socket template, in the same group, for UI control applications of the same type or category, in all UI control layouts, etc.). Customizations can be applied automatically with or without user selection. In one example, a user can specify the applicability of a customization generally (e.g., in user preferences) and / or specifically (e.g., per customization session).

[0124] As previously presented (e.g., in the discussion of FIG. 3 regarding selectively customizing control positions in a socket), a user may customize one or more controls (e.g., in a first editable control group 808, an editable second control group 812, and so on) by moving the controls from a first socket to a second socket, e.g., as Figure 2 and 3. The user can, for example, select (e.g., touch and hold) a control (e.g., in the first or second control group) and drag it to a new location (e.g., in the first or second control group).

[0125] The user may, for example, select customization for comfort, for hand dominance, or for any other purpose. For example, the size of hands and fingers may be significantly different between children and adults. Similarly, computer devices and the way they are held may vary between devices and users. A left-handed user may, for example, exchange editable first control group 808 side and editable second control group 812 side. The user may, for example, select to accept or reject customization by selecting finish controls 920 or reset controls 922.

[0126] Figure 9 An example of scaling and rotating the left control group in a first control layout according to an example embodiment is shown. Example layout editor display 900 shows a user selecting the first control group rotation and scaling control 814 and dragging the held position away from the center of the editable first control group 808 and counterclockwise (e.g., downward and rightward) to position 914. This has the effect of increasing the scale of the editable first control group 808 (e.g., increasing the active touch area of ​​the socket / control and / or the spacing between controls) and counterclockwise rotating the positions of the left stick press (B5), directional pad (C6), left trigger (T1), and left finger button (FB1). These edits create a customized first control group 908. The user can continue dragging position 914 to continue changing the configuration, including dragging to the starting configuration of the editable first control group 808. The user can release position 914 and reselect to make further edits. The user can select a reset control 822 to return the editable first and second control groups to their configuration at the beginning of the editing session or select a finish control 820 to save the customizations.

[0127] Figure 10 An example of simultaneously repositioning the right and left control groups in a first control layout according to an exemplary embodiment is shown. Customized first control layout 1000 shows that first group position control 816 is selected and dragged downward to position 1016, while second group position control 820 is selected and dragged upward to position 1020, which has the effect of moving editable first control group 808 downward and editable second control group 812 upward. These edits create customized first control group 1008 and customized second control group 1012. The user can continue to drag positions 1016 and 1020 to continue changing the configuration, including dragging to the starting configuration of editable first control group 808. The user can release positions 1016 and / or 1020 and reselect for further editing. The user can select reset control 822 to return the editable first and second control groups to their configuration at the beginning of the editing session or select finish control 820 to save the customization.

[0128] Figure 11-15 A completion control 822 is presented to accept the Figure 10An example of a customized first control layout 1000 is shown in .

[0129] Figure 11 An example of an activated customized version of a first control layout according to an example embodiment is shown. The example customized first UI control layout 1100 can be compared with the first UI control layout 401. It can be observed (e.g., by comparing Figure 4B and Figure 11 ), the differences between the first (e.g., left) control group 408 and the customized first control group 1108 include downward repositioning, counterclockwise rotation, and expanded scaling. It can be observed (e.g., by comparing Figure 4B and Figure 11 ), the differences between the second (e.g., right) control group 412 and the customized second control group 1112 include upward repositioning, clockwise rotation, and expanded scaling. It can be observed (e.g., by comparing Figure 4B and Figure 11 ), the center control socket with face buttons (e.g., B1, B2, B3, B4 controls) does not rotate with the right joystick (C7), right stick press (B6), right trigger (T2), and right finger button (FB2). Rotation (e.g., of the center socket / controls) may be, for example, generally (e.g., in user preferences) and / or specifically (e.g., per customization session) selectable.

[0130] UI layout manager 406 may be selected, for example, to leave the active UI control mode displaying customized first UI control layout 1100 to access other UI control layouts via a UI layout manager menu.

[0131] Figure 12-15 For example, the customization of the first UI control layout in the first layout group (e.g., customizing the first control layout 401 to the customized first control layout 1100) can be applied to other UI control layouts in the same group and other groups for other applications, thereby providing cross-application customization. The customization can be (e.g., selectively) universal or global (e.g., for all UI control layouts), for example, based on the same or similar socket templates and / or other layout similarities.

[0132] Figure 12 An example of sharing customization of a first control layout with a second control layout in the same layout group by repositioning, scaling, and rotating left and right control groupings is shown according to an example embodiment. Figure 12An example customized second UI control layout (customized Custom 1) 1200 is shown. Customized Custom 1 1200 can be compared to Custom 1 700. It can be observed that Custom 1 700 has been customized without editing Custom 1 700. Similar to the comparison of customized Standard 1 100 to Standard 401, it can be observed (e.g., by comparing Figure 7 and Figure 12 ), the differences between the first (e.g., left) control group 708 and the customized first control group 1208 include downward repositioning, counterclockwise rotation, and expanded scaling. It can be observed (e.g., by comparing Figure 7 and Figure 12 ), the differences between the second (eg, right) control group 712 and the customized second control group 1212 include upward repositioning, clockwise rotation, and extended scaling.

[0133] Shared customizations can, for example, enable (eg, broadcast) user customizations across multiple user interface controls, thereby avoiding the time-consuming process of customizing each control in each user interface on a control-by-control, application-by-application basis.

[0134] Figure 13 An example of browsing a control layout grouping library is shown according to an example embodiment. Figure 13 An example library browsing menu 1300 is shown, for example, when the library 518 in the UI layout manager menu 500 is selected. An example list of UI layout groups 1307 visible on the touch screen 402 includes, for example, the following groups: Sports, FPS / TPS (First Person Shooter / Third Person Shooter), Action, Racing, Puzzle, and Adventure. The user can, for example, scroll up and down to view other UI layout groups. In one example, UI layout groups can be organized (e.g., as groups) based on application titles or application types or genres (e.g., sports video games, FPS / TPS video games, action video games, racing video games, puzzle video games, and adventure video games). One (e.g., each) group can have zero or more subgroups (e.g., with UI default and custom controls for a specific application title). In other implementations, UI layout groups can be organized (e.g., as groups) based on any organizational characteristics (e.g., by socket template, by control type, etc.). Searching can be performed to, for example, find, select, and / or organize UI control layouts based on features or characteristics. A user can navigate and select, for example, a UI control layout from an action group (e.g., Figure 14 ) or from a war group's UI control layout (e.g., as shown in the example in Figure 15 ).

[0135] Figure 14An example of sharing customization of a first widget layout in a first layout group with a first widget layout in a second layout group according to an example embodiment is shown. Figure 14 Shown from Figure 13 14. A customized UI control layout 1400 of an action group selected in the illustrated library. In one example, the customized UI control layout 1400 (e.g., based on the customization of the first and second control groups 408, 412 in the first control layout 401 to the customized first and second control layouts 1108, 1112 in the customized first control layout 1200) is customized by repositioning the first (e.g., left) control group 1408 downward, rotating it counterclockwise, and scaling it to scale, and repositioning the second control group 1412 upward, rotating it clockwise, and scaling it to scale.

[0136] Figure 15 is an example of sharing customization of a first widget layout in a first layout group with a first widget layout in a third layout group according to an example embodiment. Figure 15 Shown from Figure 13 A customized UI control layout for a war group is shown selected in the library. The war layout group is shown in the layout manager menu 1500. The war group includes three UI control layouts (e.g., Third Person, Third Person (Gyro), and Standard). The Third Person UI control layout is selected. A portion of a reduced-size, highlighted, customized version of the Third Person UI control layout is shown along with the first and second control groups 1508 and 1512. A reduced-size, customized version of the Third Person (Gyro) UI control layout 1524 is also shown. In one example, a customized third-person UI control layout (e.g., customization based on the first and second control groups 408, 412 in the first control layout 401 to the customized first and second control layouts 1108, 1112 in the customized first control layout 1200) is customized by repositioning downward, rotating counterclockwise, and scaling to expand the first (e.g., left) control group 1508 and repositioning upward, rotating clockwise, and scaling to expand the second control group 1512.

[0137] The embodiments may also be implemented in processes or methods. For example, Figure 16A flow chart of an example method for providing a control socket template according to an example embodiment is shown. The embodiments disclosed herein and other embodiments can operate according to example method 1600. Method 600 includes step 1602. However, other embodiments can operate according to other methods. Based on the foregoing discussion of each embodiment, other structural and operational embodiments will be apparent to those skilled in the relevant art. Unless explicitly stated or inherently required, the order of steps is not required. It is not required that the method embodiments implement all the steps illustrated in Figure 1600. Figure 16 This is just one of many possible embodiments. Various embodiments may implement fewer, more, or different steps.

[0138] Method 600 includes step 1602. In step 1602, a UI socket template having a grouping (e.g., cluster) of sockets (e.g., a socket grouping) can be provided. Controls can be selectively assigned to portions and / or the entirety of the socket. The socket (e.g., and the assigned controls) can be selectively customized (e.g., as a grouping). For example, Figure 1 As shown, the platform developer 102 may provide a UI socket template having at least one socket group (e.g., Figure 2 3 ), an application developer 104 can use the UI socket template to create a UI control layout for an application (e.g., a game) by selectively assigning controls to the entirety and / or separate portions of the socket (e.g., as shown in the example in FIG. 3 ). App developers 104 can provide, for example, a content application (content application) 140 having a UI to one or more computing devices 106, 136. Users 134 can interact with the content application 140 using a UI that can be managed by the UI layout manager 108. Users 134 can customize the UI by, for example, changing the position, rotation, and / or scale of one or more control groups created using one or more socket groups.

[0139] Figure 17 A flowchart of an example method for providing a control layout for an application according to an example embodiment is shown. The embodiments disclosed herein and other embodiments may operate according to example method 1700. Method 1700 includes step 1702. However, other embodiments may operate according to other methods. Based on the foregoing discussion of the various embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant art. Unless explicitly stated or inherently required, the order of steps is not required. It is not required that the method embodiments implement all the steps illustrated in diagram 1700. Figure 17 This is just one of many possible embodiments. Various embodiments may implement fewer, more, or different steps.

[0140] Method 1700 includes step 1702. In step 1702, at least one UI control layout may be created (e.g., using at least one UI control socket layout template) for each of N applications. Each UI control layout may include UI control groups based on UI socket groups that may be customized as groups. For example, Figure 1 As shown, the application developer 104 can utilize a UI socket template having at least one socket group (e.g., Figure 2 3 ) to create one or more UI control layouts for an application (e.g., a game) by selectively assigning controls to the entirety and / or separate portions of a socket (e.g., as shown in the example in FIG. 3 ). App developer 104 can provide a content application (content application) 140 to one or more computing devices 106, 136. App development can provide a UI separately or together with a content application. User 134 can interact with content application 140 using a UI that can be managed by UI layout manager 108. User 134 can customize the UI by, for example, changing the position, rotation, and / or scaling of one or more control groups created using one or more socket groups.

[0141] Figure 18 A flowchart of an example method for providing a control layout manager according to an example embodiment is shown. Embodiments disclosed herein and other embodiments may operate according to example method 1800. Method 1800 includes step 1802. However, other embodiments may operate according to other methods. Based on the foregoing discussion of the various embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant art. Unless expressly stated or inherently required, the order of steps is not required. Method embodiments are not required to implement all of the steps illustrated in diagram 1800. Figure 18 This is just one of many possible embodiments. Various embodiments may implement fewer, more, or different steps.

[0142] Method 1800 includes step 1802. In step 1802, a UI manager may be provided to manage UI control layouts for applications 1-K. The UI manager may have a menu for selecting a UI control layout for an application, a library for organizing, browsing, and selecting UI control layouts, and a layout editor for customizing layouts and user preferences to apply customizations for one UI control layout to multiple UI control layouts. For example, Figure 1 As shown, the platform developer 102 can provide a UI layout manager 108 to the user device to manage the UI developed by the application developer 104. The UI layout manager 108 can include a UI layout menu 110 (e.g., Figure 5) to select a UI control layout (e.g., standard and custom 1). The UI layout manager 108 may include a UI layout library 114 (e.g., Figure 13 ) to organize, browse, and select UI control layouts (e.g., UI control layouts in layout group 1307). The UI layout manager 108 may include a layout editor 112 (e.g., Figure 8-10 ) to customize the layout (e.g., standard layout 401 to customized standard layout 1100). The UI layout manager 108 may include UI layout preferences 116 to apply customizations for one UI control layout to multiple UI control layouts (e.g., Figure 12 、 14 and 15 as examples).

[0143] Figure 19 A flowchart of an example method 1900 for selecting a control layout using a control layout manager according to an example embodiment is shown. The embodiments disclosed herein and other embodiments may operate according to the example method 1900. The method 1900 includes steps 1902-1908. However, other embodiments may operate according to other methods. Based on the foregoing discussion of the various embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant art. Unless expressly stated or inherently required, the order of steps is not required. It is not required that the method embodiments implement all of the steps illustrated in Figure 1900. Figure 19 This is just one of many possible embodiments. Various embodiments may implement fewer, more, or different steps.

[0144] Method 1900 includes step 1902. In step 1902, the UI control application and the layout control manager can be started. Figure 1As shown, user 134 can interact with touchscreen 124 to direct computing device 106 or computing device 136 to open a content application (e.g., by selecting a content application from content applications 140). UI layout manager 108 may, for example, already be running, invoked by a content application, or selected to run by user 134. In one example, a user may open an Xbox gaming application and access a number of different gaming application titles, which may be running locally, streamed locally, or streamed over the internet. UI layout manager 108 may, for example, be integrated into the Xbox gaming application. User 134 may select content application 140 through another application (e.g., the Xbox gaming application). Each gaming application may be associated with a (e.g., unique) UI control layout that may be displayed on display screen 118 and / or display screen 148. In another example, a user may open a flying drone control application, wherein the drone provides video, and user 134 may interact with the displayed UI control layout to control one or more virtual or physical objects associated with the drone (e.g., drone direction, speed, altitude, camera, gimbal, camera zoom or focus, etc.). In another application, the user 134 can open a robot control application. The UI control layout can be displayed to the user 134 who can personally view and control the robot (e.g., using a touch screen UI).

[0145] In step 1904, a UI control layout for the UI control application may be activated and displayed, wherein the user customization preferences are applied to the UI control layout. Figure 1 As shown, the UI control layout can be activated, for example, when the UI control application 140 and / or the UI layout manager 108 is activated. The UI control layout can be embedded in the content application 140 and activated by the content application 140 or selected and activated separately, for example, by the user 134 interacting with the UI layout manager 108. Figure 12 As shown in the example in, a UI control layout (e.g., Custom 1 layout 700) can be activated as a customized Custom 1 layout 1200 based on user customization preferences created by customizing a different UI control layout (e.g., standard layout 401 is modified to customized standard layout 1100).

[0146] In step 1906, the UI control layout manager menu may be selected. For example, Figure 1 As shown, user 134 can interact with touch screen 134 to select UI layout menu 110 shown in displayed content 122 on display screen 118 of computing device 106 (e.g., smart phone or tablet). Figure 4B As shown, user 134 may select UI layout manager 406 .

[0147] In step 1908, a selection can be made (e.g., in the UI layout manager) to return to active / use mode, select a different UI control layout for the UI application, edit the UI control layout, or browse the UI control layout library. Figure 5 As shown, user 134 can select Exit ("X") to return to the active / use mode (e.g., Figure 4B ), select Custom 1 Layout and then Exit to activate Custom 1 Layout as the UI for the application (e.g., as Figure 7 ), select edit mode 516 to edit the standard layout (e.g., as shown in the example in Figure 8-10 ) or select library mode 518 to browse a library of UI control layouts (e.g., as shown in the example in Figure 13 ).

[0148] Figure 20 A flow chart of an example method for editing a control layout according to an example embodiment is shown. The embodiments disclosed herein and other embodiments may operate according to example method 2000. Method 2000 includes steps 2002-2008. However, other embodiments may operate according to other methods. Based on the foregoing discussion of the various embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant art. Unless explicitly stated or inherently required, the order of the steps is not required. It is not required that the method embodiments implement all of the steps illustrated in diagram 2000. Figure 20 This is just one of many possible embodiments. Various embodiments may implement fewer, more, or different steps.

[0149] Method 2000 includes step 2002. In step 2002, an editor may be opened to edit a first UI control layout having at least one UI control group. Figure 1 and 5 As shown, the user 134 can select the editor 516 to edit the standard layout. An example of the UI layout editor 112 is shown in FIG. Figure 8 The example layout editor 800 shows an editable representation 400C of a first control layout (standard layout) 401. The editable representation 400C includes an editable first control group 808 and an editable second control group 812.

[0150] In step 2004, the first UI control layout may be customized by changing at least one of the position, scale, and rotation of the first UI control group and / or the second UI control group. Figure 9 As shown in the example in FIG, the rotation and scaling of the first control group 808 are changed to the customized first control group 908. Figure 10As shown in the example in , the positions of the first and second control groups 808 and 812 are changed to customized first and second control groups 1008 and 1012.

[0151] In step 2006, the first UI control layout may be edited by changing the socket of the first control in the first UI control group and / or the second UI control group. Figure 2 , 3 and 8 shown and discussed, the user 134 can move the control from the first set of interfaces to the second set of interfaces (e.g., Figure 2 3 ) to customize one or more controls (e.g., in the first editable control group 808 and / or the editable second control group 812). The user can, for example, select (e.g., touch and hold) a control (e.g., in the first or second control group) and drag it to a new location (e.g., in the first or second control group). In one example, user 134 can swap the positions of an RT control and an RB control or move an RB control from socket 0.0 (see, e.g., Figure 3A ) moves to socket 0.1.

[0152] In step 2008, the customization of the first UI control layout may be applied to other UI control layouts of other UI control applications. Figure 1 As shown, the UI layout manager 108 may include UI layout preferences 116 to apply customizations for one UI control layout to multiple UI control layouts (e.g., Figure 12 、 14 and 15). Figure 12 As shown in the example in, a UI control layout (e.g., Custom 1 layout 700) can be activated as a customized Custom 1 layout 1200 based on user customization preferences created by customizing a different UI control layout (e.g., standard layout 401 is modified to customized standard layout 1100).

[0153] Figure 21 A flow chart of an example method 2100 for browsing a control layout library according to an example embodiment is shown. The embodiments disclosed herein and other embodiments may operate according to the example method 2100. The method 2100 includes steps 2102-2108. However, other embodiments may operate according to other methods. Based on the foregoing discussion of the various embodiments, other structural and operational embodiments will be apparent to those skilled in the relevant art. Unless explicitly stated or inherently required, the order of the steps is not required. It is not required that the method embodiments implement all the steps illustrated in Figure 2100. Figure 21 This is just one of many possible embodiments. Various embodiments may implement fewer, more, or different steps.

[0154] Method 2100 includes step 2102. In step 2102, a UI control layout library may be selected and displayed. Figure 1 、 5 As shown in FIG. 13 , user 134 may select UI layout library 114 by selecting library 516. An example of a user interface for library 114 is shown in FIG. Figure 13 shown.

[0155] In step 2104, multiple UI control layout groups developed for multiple applications can be browsed. Figure 13 As shown, user 134 can browse UI control layout groups 1307, for example, by scrolling up and down, performing a search, selecting a group to view the UI control layouts in the group, and so on.

[0156] In step 2106, a UI control layout group may be selected. For example, Figure 13 and 15 As shown, user 134 may select the UI control layout group "War".

[0157] In step 2108, a second UI control layout may be selected from the selected group. For example, user 134 may select a "third person" UI control layout in Group Wars, which may be displayed according to the customization based on the customization of the first UI control layout (such as standard layout 401 being modified to customized standard layout 1100).

[0158] III. Example Mobile Device and Computing Device Embodiments

[0159] The various embodiments described herein can be implemented in hardware or hardware in combination with software and / or firmware. For example, the various embodiments described herein can be implemented as computer program code / instructions configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, the various embodiments described herein can be implemented as hardware logic / circuitry.

[0160] As mentioned herein, the described embodiments include Figure 1-21Together with any modules, components and / or subcomponents thereof, and the flowcharts / flowcharts described herein (including portions thereof) and / or other examples described herein, may be implemented in hardware or hardware with any combination of software and / or firmware, including as computer program code configured to be executed in one or more processors and stored in a computer-readable storage medium, or as hardware logic / circuitry, such as in a system on a chip (SoC), a field programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). A SoC may include an integrated circuit chip that includes one or more of the following: a processor (such as a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or further circuitry and / or embedded firmware for performing its functions.

[0161] The embodiments described herein can be implemented in one or more computing devices that are similar to the mobile systems and / or computing devices described in the stationary or mobile computer embodiments, including one or more features and alternative features of the mobile systems and / or computing devices described herein. The descriptions of the mobile systems and computing devices provided herein are provided for illustrative purposes and are not intended to be limiting. The embodiments can also be implemented in other types of computer systems known to those skilled in the relevant art.

[0162] Figure 22 2 is a block diagram of an exemplary mobile system 2202 including a mobile device 2200 that can implement various embodiments described herein. For example, in the preceding sections, the mobile device 2202 can be used to implement any system, client, or device or its components / subcomponents. Figure 22 As shown, mobile device 2202 includes various optional hardware and software components. Any component in mobile device 2202 can communicate with any other component, but for ease of illustration, not all connections are shown. Mobile device 2202 can be any of a variety of computing devices (e.g., a cellular phone, a smartphone, a handheld computer, a personal digital assistant (PDA), etc.) and can enable wireless two-way communication with one or more mobile communication networks 2204, such as cellular or satellite networks, or with a local area network or a wide area network.

[0163] The mobile device 2202 may include a controller or processor 2210 (e.g., a signal processor, microprocessor, ASIC, or other control and processing logic circuitry) for performing tasks such as signal encoding, data processing, input / output processing, power control, and / or other functions. An operating system 2212 may control the allocation and use of the mobile device 2202 and provide support for one or more application programs 2214 (also referred to as "applications," "apps," etc.). The application programs 2214 may include common mobile computing applications (e.g., email applications, calendars, contact managers, web browsers, messaging applications), or any other computing applications (e.g., word processing applications, drawing applications, media player applications).

[0164] Mobile device 2202 may include memory 2220. Memory 2220 may include non-removable memory 2222 and / or removable memory 2224. Non-removable memory 2222 may include RAM, ROM, flash memory, a hard drive, or other well-known memory devices or technologies. Removable memory 2224 may include flash memory or a Subscriber Identity Module (SIM) card, as is known in GSM communication systems, or other well-known memory devices or technologies such as "smart cards." Memory 2220 may be used to store data and / or code for running operating system 2212 and application programs 2214. Example data may include web pages, text, images, sound files, video data, or other data sets sent to and / or received from one or more network servers or other devices via one or more wired or wireless networks. Memory 2220 may be used to store subscriber identifiers, such as an International Mobile Subscriber Identity (IMSI), and device identifiers, such as an International Mobile Equipment Subscriber Identifier (IMEI). Such identifiers may be transmitted to a network server to identify the user and device.

[0165] A number of programs may be stored in the memory 2220. These programs include an operating system 2212, one or more application programs 2214, and other program modules and program data. Examples of such applications or program modules may include, for example, Figure 1 The computer program logic (e.g., computer program code or instructions) of one or more of the systems 100, together with any components and / or subcomponents thereof, and the flowcharts / flow diagrams described herein, including portions thereof, and / or other examples described herein.

[0166] The mobile device 2202 may support one or more input devices 2230 (such as a touch screen 2232, a microphone 2234, a camera 2236, a physical keyboard 2238, and / or a trackball 2240) and one or more output devices 2250 (such as a speaker 2252 and a display 2254). Other possible output devices (not shown) may include piezoelectric or other tactile output devices. Some devices may provide more than one input / output function. For example, the touch screen 2232 and the display 2254 may be combined in a single input / output device. The input device 2230 may include a natural user interface (NUI).

[0167] One or more wireless modems 2260 may be coupled to an antenna(s) (not shown) and may support two-way communication between the processor 2210 and external devices, as is well understood in the art. The modems 2260 are shown generally and may include a cellular modem 2266 and / or other radio-based modems (e.g., Bluetooth 2264 and / or Wi-Fi 2262) for communicating with the mobile communications network 2204. At least one wireless modem 2260 is typically configured to communicate with one or more cellular networks (such as a GSM network) for data and voice communications within a single cellular network, between multiple cellular networks, or between a mobile device and a public switched telephone network (PSTN).

[0168] The mobile device 2202 may further include at least one input / output port 2280, a power supply 2282, a satellite navigation system receiver 2284 such as a global positioning system (GPS) receiver, an accelerometer 2286, and / or a physical connector 2290, which may be a USB port, an IEEE 2294 (FireWire) port, and / or an RS-232 port. The components of the mobile device 2202 shown are not required or all-inclusive, as any component may be absent and other components may be present in addition, as will be understood by those skilled in the art.

[0169] In one embodiment, the mobile device 2202 is configured to implement any of the above features of the flowcharts herein. Computer program logic for performing any operations, steps, and / or functions described herein may be stored in the memory 2220 and executed by the processor 2210.

[0170] Figure 23An exemplary implementation of a computing device 2300 in which various embodiments may be implemented is depicted. For example, various embodiments described herein may be implemented in one or more computing devices that are similar to computing device 2300 in the stationary or mobile computer embodiments, including one or more features and / or alternative features of computing device 2300. The description of computing device 2300 provided herein is for illustration only and is not intended to be limiting. Embodiments may also be implemented in other types of computer systems and / or game consoles known to those skilled in the relevant art.

[0171] like Figure 23 As shown, computing device 2300 includes one or more processors (referred to as processor circuits 2302), system memory 2304, and a bus 2306 that couples various system components, including system memory 2304, to processor circuits 2302. Processor circuits 2302 are electronic and / or optical circuits implemented as central processing units (CPUs), microcontrollers, microprocessors, and / or other physical hardware processor circuits using one or more physical hardware electronic circuit device elements and / or integrated circuit devices (semiconductor material chips or dies). Processor circuits 2302 can execute program code stored in computer-readable media, such as operating system 2330, application programs 2332, and program code for other programs 2334. Bus 2306 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. System memory 2304 includes read-only memory (ROM) 2308 and random access memory (RAM) 2310. A basic input / output system 2312 (BIOS) is stored in ROM 2308 .

[0172] The computing device 2300 also has one or more of the following drives: a hard disk drive 2314 for reading and writing to a hard disk, a disk drive 2316 for reading and writing to a removable disk 2318, and an optical drive 2320 for reading and writing to a removable optical disk 2322 (such as a CD-ROM, DVD, or other optical media). The hard disk drive 2314, the disk drive 2316, and the optical drive 2320 are connected to the bus 2306 via a hard disk drive interface 2324, a disk drive interface 2326, and an optical drive interface 2328, respectively. These drives and their associated computer-readable media provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computer. Although a hard disk, a removable disk, and a removable optical disk are described, other types of hardware-based computer-readable storage media such as flash memory cards, digital video disks, RAM, ROM, and other hardware storage media may also be used to store data.

[0173] Several program modules may be stored on a hard disk, magnetic disk, optical disk, ROM, or RAM. These programs include an operating system 2330, one or more application programs 2332, other programs 2334, and program data 2336. Application programs 2332 or other programs 2334 may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the various embodiments described herein, along with any modules, components, and / or subcomponents thereof, and the flowcharts / flowcharts described herein, including portions thereof, and / or other examples described herein.

[0174] A user may enter commands and information into the computing device 2300 through input devices such as a keyboard 2338 and a pointing device 2340. Other input devices (not shown) may include a microphone, a joystick, a game controller, a satellite dish, a scanner, a touch screen and / or touch pad, a voice recognition system for receiving voice input, a gesture recognition system for receiving gesture input, and the like. These and other input devices are typically connected to the processor circuit 2302 through a serial port interface 2342 coupled to the bus 2306, but may be connected through other interfaces such as a parallel port, a game port, or a universal serial bus (USB).

[0175] A display screen 2344 is also connected to the bus 2306 via an interface, such as a video adapter 2346. The display screen 2344 can be external to or incorporated into the computing device 2300. The display screen 2344 can display information, as well as serve as a user interface for receiving user commands and / or other information (e.g., through touch, finger gestures, a virtual keyboard, etc.). In addition to the display screen 2344, the computing device 2300 may also include other peripheral output devices (not shown), such as speakers and a printer.

[0176] The computing device 2300 is connected to a network 2348 (e.g., the Internet) through an adapter or network interface 2350, a modem 2352, or other means for establishing communications over the network. The modem 2352 (which may be internal or external) may be connected to the bus 2306 via a serial port interface 2342, such as a serial bus. Figure 23 As shown, another interface type including a parallel interface may be used to connect to bus 2306.

[0177] As used herein, the terms "computer program medium", "computer-readable medium", and "computer-readable storage medium" are used to refer to physical hardware media. Examples of such physical hardware media include the hard disk associated with the hard disk drive 2314, the removable disk 2318, the removable optical disk 2322, other physical hardware media such as RAM, ROM, flash memory cards, digital video disks, zip disks, MEMs, nanotechnology-based storage devices, and other types of physical / tangible hardware storage media (including Figure 23 Memory 2320). Such computer-readable media and / or storage media are distinct from and do not overlap with communication media and propagated signals (excluding communication media and propagated signals). Communication media carry computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave. The term "modulated data signal" means a signal in which one or more characteristics are set or changed in a manner that encodes information in the signal. By way of example and not limitation, communication media include wireless media (such as acoustic, RF, infrared, and other wireless media) as well as wired media. Various embodiments also relate to such communication media that are separate from and do not overlap with the embodiments directed to computer-readable storage media.

[0178] As noted above, computer programs and modules (including application programs 2332 and other programs 2334) can be stored on a hard disk, magnetic disk, optical disk, ROM, RAM, or other hardware storage medium. Such computer programs can also be received via network interface 2350, serial port interface 2342, or any other interface type. These computer programs, when executed or loaded by an application, enable the computer 2300 to implement the features of the various embodiments discussed herein. Accordingly, such computer programs represent controllers of the computer system 2300.

[0179] The various embodiments also relate to computer program products comprising computer code or instructions stored on any computer-readable medium or computer-readable storage medium, including hard drives, optical drives, memory device packs, portable memory sticks, memory cards, and other types of physical storage hardware.

[0180] IV. Additional Example Embodiments

[0181] Methods, systems, and computer program products are provided for sharing user interface (UI) customizations across applications. Application developers can create UIs (e.g., game control layouts) using control groups that can be customized as groups. A control group can include multiple controls that are selectively assigned to a socket cluster (e.g., a socket group) in a layout template. The controls in a control group can be customized as a group, for example, by changing the position, rotation, and / or scaling of the control group. Users can customize controls for comfort, switch controls for left-handed or right-handed gaming, and share customized preferences across applications. UI customizations can be shared across applications with different UIs. For example, customizations to one or more control groups in a first UI can be (e.g., automatically) applied to one or more control groups in other UIs of other applications. Users can avoid tedious, time-consuming individual control changes for many UIs.

[0182] In one example, a method for sharing UI customization across applications may include, for example, providing a first UI configured with a first plurality of controls in a first layout; providing a second UI configured with a second plurality of controls in a second layout; and displaying the second UI with the second plurality of controls on a display device in a customized layout based on the customized layout of the first plurality of controls. The second plurality of controls may be different from the first plurality of controls.

[0183] In one example, a first UI may include a first control grouping that may include a first plurality of controls customizable as a group. A second UI may include a second control grouping that may include a second plurality of controls customizable as a group.

[0184] In an example, the method may (eg, further) include customizing the first UI by changing at least one of a position, a rotation, and a scale of the first control cluster.

[0185] In one example, a first control grouping may be based on a first socket grouping comprising a plurality of control sockets. The first plurality of controls may be selectively assigned to the plurality of control sockets. A second control grouping may be based on a second socket grouping comprising a plurality of control sockets. The second plurality of controls may be selectively assigned to the plurality of control sockets.

[0186] In one example, the first socket grouping may be part of a first template having multiple socket groups. The second socket grouping may be part of a second template having multiple socket groups.

[0187] In one example, the first UI may include multiple control groups, for example, based on multiple socket groupings in the first template.The second UI may include multiple control groups, for example, based on multiple socket groupings in the second template.

[0188] In an example, the method may (eg, further) include customizing the first UI by changing at least one of a position, rotation, and scale of each of the plurality of control groups. In an example, the customization of the first and second control groups may be simultaneous.

[0189] In one example, a first control grouping can be configured to be displayed on a first side of a display and at least one other control grouping can be configured to be displayed on a second side of the display. In one example, customizing the first UI can include moving the first control grouping to the second side of the display; and moving at least one other control grouping to the first side of the display.

[0190] In an example, the first UI may be configured for a first application; and the second UI may be configured for a second application.

[0191] In one example, the method may (e.g., further) include customizing the first UI by moving a first control in the first control group from a first socket in the first socket group to a second socket. In one example, the first control may be assigned to multiple sockets in the first socket group.

[0192] In one example, the method may (e.g., further) include receiving user input for the first application from a second UI displayed on the computing device; switching from the second UI to the first UI by browsing a library and selecting the first UI from a plurality of UIs in the library; and receiving user input for the first application from the second UI displayed on the computing device.

[0193] In an example, the method may (eg, further) include displaying the second UI on the display device as an overlay on the displayed content generated based on the first application.

[0194] In one example, a computer-readable medium may have program code recorded thereon that, when executed by at least one processor, causes the at least one processor to perform a method. In one example, the method includes, for example, providing a first user interface (UI) configured with a first control group for display on a display device, the first control group including a first plurality of controls customizable as a group by changing at least one of a position, a rotation, and a scale of the first control group.

[0195] In one example, the method may (e.g., further) include providing a second UI configured with a second control group for display on a display device, the second control group including a second plurality of controls customizable as a group; and customizing the second UI by changing at least one of a position, a rotation, and a scale of the second control group based on the customization of the first UI. The second plurality of controls may be different from the first plurality of controls.

[0196] In an example, the first UI may be configured for a first application; and the second UI may be configured for a second application.

[0197] In one example, a first control grouping may be based on a first socket grouping comprising a plurality of control sockets. The first plurality of controls may be selectively assigned to the plurality of control sockets. A second control grouping may be based on a second socket grouping comprising a plurality of control sockets. The second plurality of controls may be selectively assigned to the plurality of control sockets.

[0198] In one example, a system may include at least one computing device configured to provide a first user interface (UI) for display on a display device, the first user interface (UI) configured for a first application and configured with a first control group including a first plurality of controls, the first plurality of controls customizable as a group. The at least one computing device may be configured to apply customizations of the first control group in the first UI to a second control group in a second UI, the second control group including a second plurality of controls customizable as a group, wherein the second plurality of controls is different from the first plurality of controls.

[0199] In one example, the at least one computing device may (e.g., further) be configured to provide a UI editor for display on a display device, the UI editor being configured to provide customization of a UI configured with a control group by changing at least one of a position, rotation, and scaling of the control group.

[0200] V. Conclusion

[0201] Although various embodiments of the present application have been described above, it should be understood that they are presented by way of example only and not limitation. Those skilled in the relevant art(s) will appreciate that various modifications may be made in form and detail without departing from the spirit and scope of the present application as defined by the appended claims. Accordingly, the scope of the present application should not be limited by any of the exemplary embodiments described above, but should be defined only in accordance with the following claims and their equivalents.

Claims

1. A method comprising: providing a first user interface (UI) configured with a first plurality of controls in a first layout, the first plurality of controls including a specific first control configured for use with a first application, the specific first control being provided at a specific location in the first UI; providing a second UI configured with a second plurality of controls in a second layout, wherein the second plurality of controls includes a specific second control configured for use with a second application, the specific second control being provided at the specific position in the second UI; and In response to a customization input received while the first UI is displayed without the second UI: performing a first instance of a specific layout modification on the first layout to obtain a customized first layout for the first UI; as well as automatically performing a second instance of the specific layout modification to the second layout to obtain a customized second layout for the second UI, wherein a first instance of the specific layout modification involves performing a specific movement of the specific first control relative to other controls in the first layout in response to the custom input, and a second instance of the specific layout modification involves automatically performing the specific movement of the specific second control relative to other controls in the second layout in the absence of explicit custom input directed to the specific second control.

2. The method according to claim 1, characterized in that The first application is a first video game, the second application is a second video game, and the particular first control and the particular second control are configured to provide different user game inputs to the first video game and the second video game, respectively.

3. The method according to claim 2, characterized in that Also includes: controlling the first video game with a computing device using the first UI having the customized first layout; and The second video game is controlled with the computing device using the second UI having the customized second layout.

4. The method according to claim 1, wherein The specific first control is part of a first group of multiple controls in the first layout, the specific second control is part of a second group of multiple controls in the second layout, a first instance of the specific layout modification is performed on the first group of multiple controls, and a second instance of the specific layout modification is performed on the second group of multiple controls.

5. The method according to claim 4, characterized in that A first instance of the particular layout modification involves repositioning the first plurality of controls downward relative to another plurality of controls in the first layout, and a second instance of the particular layout modification involves repositioning the second plurality of controls downward relative to another plurality of controls in the second layout.

6. The method according to claim 4, characterized in that The first instance of the particular layout modification involves expanding the zoom of the first plurality of controls, and the second instance of the particular layout modification involves expanding the zoom of the second plurality of controls.

7. The method according to claim 1, characterized in that Also includes: displaying the first UI having the customized first layout on a computing device; and In response to user input, the first UI is removed and the second UI having the customized second layout is switched to be displayed on the computing device.

8. The method according to claim 7, characterized in that Also includes: A library of available control layouts is displayed on the computing device, the available control layouts including at least the customized first layout of the first UI and the customized second layout of the second UI.

9. A system comprising: one or more processors; as well as One or more computer-readable storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to: determining that a first instance of a particular customization has been performed on a particular first control of a first user interface (UI), while displaying the first UI without a second UI, the first UI being configured to control a first application and provided at a particular location in the first UI; accessing the second UI, wherein the second UI is configured to control a second application other than the first application, the second UI having a specific second control provided at the specific position in the second UI; as well as automatically applying a second instance of a particular customization to the particular second control in the absence of explicit customization input directed to the particular second control of the second UI, Wherein, the first instance of the specific customization involves a specific movement of the specific first control relative to other controls in the first UI in response to customization input, and the second instance of the specific customization involves performing a specific movement of the specific second control relative to other controls in the second UI.

10. The system according to claim 9, characterized in that The first UI and the second UI are separate UIs configured to be displayed on a display screen at different times and to control different applications at different times.

11. The system according to claim 9, wherein: The specific first control is part of a first group of multiple controls of the first UI, the specific second control is part of a second group of multiple controls of the second UI, a first instance of specific customization is performed on the first group of multiple controls, and a second instance of specific customization is performed on the second group of multiple controls.

12. The system according to claim 11, wherein: The first instance of the particular customization involves repositioning one of the first plurality of controls downward relative to another control in the first plurality of controls, and the second instance of the particular customization involves repositioning one of the second plurality of controls downward relative to another control in the second plurality of controls.

13. The system according to claim 11, wherein: The first instance of the particular customization involves expanding the zoom of the first plurality of controls, and the second instance of the particular customization involves expanding the zoom of the second plurality of controls.

14. The system according to claim 9, wherein: Also included is a touch screen configured to display the first UI and the second UI at different times.

15. The system of claim 14, wherein the instructions, when executed by the one or more processors, cause the one or more processors to: The first instance of the specific customization is executed in response to user input to the touch screen while the first UI is displayed and the second UI is not displayed.

16. A computer-readable medium having program code recorded thereon, wherein when the program code is executed by at least one processor, the at least one processor is caused to perform actions, the actions comprising: Displaying a first user interface (UI) configured with a first plurality of controls in a first layout, the first plurality of controls including a specific first control configured for use with a first application, the specific first control being provided at a specific location in the first UI; providing a second UI configured with a second plurality of controls in a second layout, wherein the second plurality of controls includes a specific second control configured for use with a second application, the specific second control being provided at the specific position in the second UI; and In response to a customization input received while the first UI is displayed without the second UI: performing a first instance of a specific layout modification on the first layout to obtain a customized first layout for the first UI; as well as automatically performing a second instance of the specific layout modification to the second layout to obtain a customized second layout for the second UI, wherein a first instance of the specific layout modification involves performing a specific movement of the specific first control relative to other controls in the first layout in response to the custom input, and a second instance of the specific layout modification involves automatically performing the specific movement of the specific second control relative to other controls in the second layout in the absence of explicit custom input directed to the specific second control.

17. The computer-readable medium of claim 16, wherein: The actions also include: At a first time, displaying the first UI having the customized first layout, and transmitting a first control input to the first application based on a first user input directed to the first UI; and At a second time, the first UI is replaced with the second UI having the customized second layout, and a second control input is transferred to the second application based on a second user input directed to the second UI.

18. The computer-readable medium of claim 17, It is characterized in that The first application includes a first video game, the second application includes a second video game, the specific first controls include specific first video game controls, and the specific second controls include specific second video game controls.

19. The computer-readable medium of claim 16, wherein: The first instance and the second instance of the particular layout modification move the particular first control and the particular second control from the same particular position to another new position on the first UI and the second UI, respectively.

20. The computer-readable medium of claim 16, the second UI being configured to replace the first UI and be used separately from the first UI.

Images