State-based action buttons
By dynamically adjusting the state and function of the virtual controller buttons, and combining user input and avatar state, the contradiction between the number and complexity of buttons in touch device control solutions is resolved, achieving complex input and efficient user experience within a limited screen space.
Patent Information
- Application Number
- CN202280019768.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-03-10
- Filing Date
- 2022-03-09
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-03-09
AI Technical Summary
Existing touch device control solutions fail to fully utilize the flexibility of virtualization and cannot provide the same haptic and responsive capabilities as physical controllers, especially in the limited screen space where complex input actions are difficult to achieve.
By dynamically adjusting the button states and functions of the virtual controller, and combining user input, historical behavior, and avatar states, the appearance and function of virtual buttons can be dynamically changed, reducing the number of buttons without reducing complexity.
Providing more complex input methods within limited screen space enhances the user experience, enables muscle memory-based input, and reduces the number of buttons without affecting the game.
Smart Images

Figure CN117043736B_ABST
Abstract
Description
Background Technology
[0001] Modern computer controller systems, such as those used by computers and video games, as well as by general-purpose operating systems, employ a variety of techniques to guide the movement of objects displayed on the screen. Known techniques include using external control devices such as mice, directional pads, touchpads, pens, game controllers, or joysticks to create direction vectors or specify the position for moving objects on the screen (such as pointers or markers), or to induce movement of the user's viewpoint. Some techniques employ an additional layer of complexity by measuring the movement speed of the external device to enhance the movement of objects on the screen by altering the behavior of objects in response to input parameters (e.g., pointer acceleration based on the speed of the external device's movement). Touch-enabled devices can also be configured to accept input in a manner that mimics the behavior of external control devices. However, control schemes for touch-enabled devices often fall short of the haptic and responsive capabilities achieved by physical controllers, thus necessitating further development in this area. For example, while some touch-enabled control schemes exist in existing games, they fail to take advantage of the flexibility offered by virtualization. Summary of the Invention
[0002] This document provides techniques for implementing a virtual controller, wherein input mechanisms can be dynamically updated based on the current state when those states are updated. In embodiments, actions associated with one or more buttons on the controller can change as different buttons are pressed and / or different actions are performed.
[0003] In one embodiment, a method performed by a user equipment is disclosed, the method comprising receiving, via a touchscreen display, a first touch input corresponding to at least one of a set of input mechanisms from a user, determining that a first active state is currently present, and generating, based on the first state, first activation data including an indication of a first action. The method may further comprise receiving, via a touchscreen display, a second touch input corresponding to at least one input mechanism from a user, determining that a second active state is currently present, and generating, based on the second state, second activation data including an indication of a second action, different from the first action.
[0004] The embodiment relates to a computing system comprising: a processor; and a memory including instructions that, when executed by the processor, cause the computing device to at least: receive from a user via a touchscreen display a first touch input corresponding to at least one of a set of input mechanisms; determine that a first state is currently active; generate first activation data based on the first state including an indication of a first action; receive from a user via a touchscreen display a second touch input corresponding to at least one input mechanism; determine that a second state is currently active; and generate second activation data based on the second state including an indication of a second action, the second action being different from the first action.
[0005] The embodiments relate to a non-transitory computer-readable medium that collectively stores computer-executable instructions that, when executed, cause one or more computing devices to collectively perform the following actions: receiving from a user via a touchscreen display a first touch input corresponding to at least one of a set of input mechanisms, determining that a first active state is currently present, and generating, based on the first state, first activation data including an indication of the first action. The actions further include: receiving from a user via a touchscreen display a second touch input corresponding to at least one input mechanism, determining that a second active state is currently present, and generating, based on the second state, second activation data including an indication of a second action, distinct from the first action.
[0006] The foregoing, as well as other features and embodiments, will become more apparent when referenced to the following description, claims, and drawings. The embodiments covered herein are defined by the appended claims, not by the summary of the invention. This summary is a high-level overview of various aspects of this application and introduces some concepts further described in the specific embodiments below. The summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used alone to determine the scope of the claimed subject matter. The subject matter should be understood by referring to appropriate portions of the entire specification, any or all of the drawings, and each claim. Attached Figure Description
[0007] A detailed description is provided with reference to the accompanying drawings. In the drawings, the leftmost numeral of the reference numeral indicates the drawing in which the reference numeral first appears. The same reference numerals are used in different drawings to indicate similar or identical items or features.
[0008] Figure 1 This is a simplified system diagram illustrating a service environment in which a virtual controller may be used, according to various embodiments of the present disclosure.
[0009] Figure 2 An illustrative example of an environment including a virtual controller communicating with a video game system, according to an embodiment, is depicted;
[0010] Figure 3 This is a block diagram illustrating various components of a computing system architecture that supports the implementation of a virtualized physical controller according to an embodiment;
[0011] Figure 4 A block diagram illustrating a process for implementing a dynamic-based controller according to an embodiment is depicted;
[0012] Figure 5 An illustration depicts a process for dynamically adjusting an input mechanism (i.e., a button) on a virtual controller according to an embodiment;
[0013] Figure 6 A block diagram illustrating a process for providing dynamic state changes of buttons for controlling an avatar, according to some embodiments, is depicted; and
[0014] Figure 7 A flowchart illustrating an example processing flow for generating and implementing a virtualized physical controller according to an embodiment is depicted. Detailed Implementation
[0015] In the following description, various embodiments will be described. Specific configurations and details are set forth for illustrative purposes to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that these embodiments can be practiced without specific details. Furthermore, well-known features may be omitted or simplified so as not to obscure the described embodiments.
[0016] The embodiments described herein relate to techniques for providing virtual controller configurations that can be dynamically adjusted based on state. For example, when implementing a virtual controller for a video game, this might involve dynamically changing the appearance and functionality of virtual buttons based on the state and interaction history of different virtual buttons. These methods utilize user input (if the user is currently touching a button), tracking the state of past actions, and the state of the user-controlled "avatar" in the game.
[0017] The embodiments disclosed herein offer numerous advantages over conventional systems. Screen space is extremely valuable for handheld tablet devices (such as mobile phones) that can be used as virtual controllers. For games intended to be played horizontally (or "landscape"), there is limited space to provide input to the game with both hands, but still enough space to view the actual in-game gameplay (if it's a mobile game) or to view control schemes or accessibility information (if it's a console or PC game played using a virtual controller).
[0018] This spatial constraint means that games need to be mindful of the number and size of UI elements on the screen. When combined with the physical characteristics of the human hand (e.g., the size of the player's thumb, the length of the thumbnail, the direction and angle of the thumb joint), there are hard constraints and trade-offs when considering the desire to provide more input methods (more buttons for more complex input actions), the size and placement of these buttons (affecting the ease of physical touch for the user), and the remaining "screen space" for viewing in-game actions. In contrast, typical modem console games traditionally use eight buttons for input, often expecting players to use multiple buttons simultaneously (e.g., using the right thumb and right index finger at the same time).
[0019] Finally, the goal of input design is to achieve input based on "muscle memory." After a brief training / instruction for control, if desired ("I want my avatar in the game to jump"), the user should no longer need to physically look at the position of the button on the screen to know which button to click. Their thumb should automatically and instinctively move to the correct position and touch the virtual button. Most mobile games address these issues by reducing the number of buttons (often by reducing the complexity of the game), adding a large number of transparent on-screen buttons (to avoid obstructing actions in the game), or dynamically changing the text of the on-screen buttons based on the game state.
[0020] In contrast, this disclosure utilizes the current state of the virtual button (pressed or not pressed), the previous state of the virtual button (whether it was pressed a certain amount of time ago), and the state of the player avatar (e.g., "Can my character do X now?") to minimize the number of buttons without reducing complexity or requiring changes to the game state.
[0021] Figure 1 This is a simplified system diagram illustrating a service environment 100 in which a virtual controller can be used according to various embodiments of the present disclosure. The service environment 100 includes at least one server 101, which includes at least one processor 103 and non-transitory memory 105 storing software instructions to facilitate the operation of the service environment. The server 101 is connected via a network 121 (e.g., the Internet or a local area network) to any suitable number of user-owned client devices 133, 143, which typically operate in conjunction with a corresponding local user network 131, 141 (e.g., a consumer or business LAN, a Wi-Fi network, etc.).
[0022] Server 101 can also connect to any suitable number of control services 111, such as a computing system with its own processor 113 and memory 115 and network connectivity, which monitors the network traffic to and from server 101 and client devices 133, 143. In some embodiments, server 101 can be one or more servers operating at a commercial scale, such as a data center or server cluster. Client devices 133, 143 can include, but are not limited to, consumer personal computers, video game consoles, thin client devices operable to stream video content from server 101 for display on a local screen, or mobile devices such as smartphones or tablets. Client devices 133, 143 can connect to any suitable number of controllers, such as controllers 135, 137, 145, 147.
[0023] Each controller (e.g., controller 135) may be a hardware device (e.g., a console-specific controller, a cross-compatible controller, or a virtual controller) having connectivity hardware and protocols for communicating with their respective client devices 133. According to some embodiments, controller 135 may be a virtualized controller operating on a thin client device or a touchscreen device, such as a controller emulated on a touchscreen smartphone, tablet, or a console-like controller with a touch panel. According to some other embodiments, for example, if client device 133 is a thin client device or a mobile device, controller 135 may be a touchscreen with virtualized controls built into the client device. Alternatively, even if client device 133 is a thin client device, controller 135 may be a hardware controller configured to connect physically or wirelessly to the client device. According to some embodiments, client device 133 and server 101 may operate on the same hardware, for example, the client device may run as a virtual instance on the server.
[0024] The method described in this article can be used with, for example... Figure 1 These methods combine service environments, such as service environment 100 described herein, with implementation on the client device. They can also work in the context of an arbitrarily placed virtual controller that controls the orientation and movement of the avatar on the screen.
[0025] For clarity, Figure 1 A certain number of components are shown. However, it should be understood that embodiments of this disclosure may include more than one of each component. Additionally, some embodiments of this disclosure may include fewer or more than [number missing]. Figure 1 All components are shown. Furthermore... Figure 1 The components can communicate using any suitable communication protocol via any suitable communication medium (including the Internet).
[0026] Figure 2 Illustrative examples of an environment 200 including a virtual controller 235 communicating with a video game system (e.g., a game console) 233 according to various embodiments are depicted. The virtual controller 235 includes a touchscreen 251, a frame 253, and virtualized controls, such as 255 and 257. For example, imagine two virtual buttons (“dash” is a first virtual button 259, and “jump” is a second virtual button 261). They can be triggered independently (the user can press or hold either button with their thumb).
[0027] When the "Dash" button is pressed at 257b (e.g., touch 263 appears at the edge of the "Dash" button), we can indicate a successful touch through one or more of the following actions: changing the appearance (icon texture) of the "Dash" button, providing haptic feedback, audio, and if the controlled avatar can perform a "long jump," the character's state can be changed to "Dashing," and we change the text of the "Jump" button 261 to "Long Jump" alone or in combination with a change in button color or texture. In response to the "Dash" command, the player's avatar begins to dash. At 257c, the appearance (icon texture) of the "Dash" button changes to give the user immediate feedback when the user stops touching it (the "Dash" button returns to its default state). However, even when the user stops touching the "Dash" button, we track the time the user stops pressing the button and start a timer. When a user's thumb presses the "Long Jump" button at position 257d, we indicate a successful touch by changing one or more of the following: altering the appearance (icon texture), haptic feedback, or audio of the "Long Jump" button; and responding to a command from the second button that the player's avatar now performs a "Long Jump". Therefore, the behavior of the second button dynamically changes based on a combination of the most recent button press and the player avatar's state, without altering the global game state.
[0028] When the "sprint" timer expires, the character can resume from the "sprint" state, the "long jump" button text can revert to indicating "jump," and the player's avatar stops sprinting. When the user removes their thumb from the "jump" button at 257e, the appearance (icon texture) of the "jump" button also returns to its default state, indicating the default state of both buttons. In this way, the user can now provide input to make their avatar "jump" or "long jump" without needing an additional button explicitly for "long jump," and without deviating from the intended control scheme (i.e., related tasks such as "jump" and "long jump" can depend on the context, specifically whether the character is sprinting). These methods differ from combination-based contextual cues in that, in traditional games, button presses are either simultaneous (e.g., left-hand directional control plus right-hand button control), or, when the context requirements are met, there is no way to signal a change in button behavior by altering the virtual button (e.g., right-hand "code" or "combination" entry).
[0029] Figure 3 This is a block diagram illustrating various components of a computing system architecture supporting an implementation of a virtualized physical controller according to an embodiment. The system architecture may include at least one controller 302. In some embodiments, the controller 302 may communicate with one or more servers 304, which may be as described above. Figure 1 An example of the described server 101. In some embodiments, one or more servers 101 may provide backend support for controller 302. For example, at least a portion of the processes described as being performed by controller 302 may, in some cases, be performed alternatively by server 101. In some embodiments, controller 302 may communicate with client device 306. Client device 306 may be as described above regarding... Figure 1 Examples of client devices 133 or 143 are described. In some embodiments, client device 306 may also communicate with display device 308. Each component described herein may communicate via a connection over network 310.
[0030] Controller 302 may include any suitable computing device configured to perform at least a portion of the operations described herein and configured to enable a user to interact with a software application. In some embodiments, the controller may be a mobile device with touchscreen functionality (e.g., a smartphone or tablet). Controller 302 may include a communication interface 312, one or more processors 314, memory 316, and hardware 318. Communication interface 312 may include wireless and / or wired communication components that enable controller 302 to send data to and receive data from other networked devices. Hardware 318 may include additional user interface, data communication, or data storage hardware. For example, the user interface may include at least one output device 320 (e.g., a visual display, audio speaker, and / or haptic feedback device) and one or more data input devices 322. Data input devices 322 may include, but are not limited to, combinations of one or more of a keypad, keyboard, mouse device, gesture-receiving touchscreen, microphone, voice or speech recognition device, and any other suitable device.
[0031] The memory 316 may be implemented using a computer-readable medium (e.g., a computer storage medium). Computer-readable media include at least two types: computer storage media and communication media. Computer storage media include any suitable volatile and non-volatile, removable and non-removable media implemented using any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, DRAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, cassette tape, magnetic tape, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. Conversely, communication media may embody computer-readable instructions, data structures, program modules, or other data in modulated data signals, such as carrier waves or other transmission mechanisms.
[0032] One or more processors 314 and memory 316 of the controller can implement functionality including one or more software modules and data storage. Such software modules may include routines, program instructions, objects, and / or data structures executed by processor 314 to perform specific tasks or implement specific data types. More specifically, memory 316 may include modules configured to determine the current state of the controller and implement input mechanism configuration based on such state (e.g., state management module 324), and modules configured to maintain and implement configuration information of the input mechanisms of the virtualized controller (e.g., configuration management module 326).
[0033] Additionally, memory 316 may include various data storage. For example, memory 316 may maintain data regarding the virtualization controller configuration based on context (e.g., configuration data 328). In some embodiments, such configuration data may include indications of one or more aspects of input mechanisms that should be implemented based on a state. For example, configuration data may include indications of the size, location, shape, appearance (e.g., color, shading, and / or text) of each input mechanism associated with a particular state. In some cases, configuration data may indicate which input mechanisms should or should not be presented during a specific state.
[0034] The state management module 324 can be configured, in conjunction with the processor 314, to implement one or more input mechanisms based on detected state and configuration data. In some cases, the state management module is configured to monitor changes in state. Such a state change can be caused when an input mechanism is activated or when certain other conditions are met. Upon detecting a state change, the state management module can be configured to determine the current state (or multiple states) of the controller or the software application communicating with the controller and implement each appropriate input mechanism based on that state. This may involve providing instructions on which input mechanisms should be presented and how each of these input mechanisms should be presented.
[0035] Configuration management module 326 can be configured, in conjunction with processor 314, to generate and manage configuration information related to the arrangement of one or more input mechanisms within the user interface presented on controller 302. In some embodiments, according to certain examples, configuration management module facilitates customization of the input mechanism layout. It should be noted that such customization is described in Gregory Peng's relevant Patent Cooperation Treaty (PCT) application US2022 / 019240 entitled "Virtualized Physical Controller," which is incorporated herein by reference in its entirety.
[0036] In some embodiments, configuration data can be customized by the user to indicate the layout / appearance of the input mechanisms presented by the virtual controller. Additionally, this layout / appearance information stored in the configuration data for each input mechanism can be customized by the user on a state-by-state basis. In other words, how each input mechanism (including its appearance, location, size, shape, or other suitable factors) is presented relative to each state can be specified at least in part based on user-provided input.
[0037] Server 304 may include any computing device configured to perform at least a portion of the operations attributed to it. Server 304 may consist of one or more general-purpose computers, special-purpose server computers (including, for example, PC (personal computer) servers), Servers (including servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server groups, server clusters, or any other suitable arrangement and / or combination. Server 304 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization, such as one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for computers. For example, server 304 may include virtual computing devices in the form of virtual machines or software containers hosted in the cloud.
[0038] Client device 306 may include any suitable computing device configured to receive input from controller 302 and perform actions based on that input. In some embodiments, the client device may be a gaming system, such as a game console that can receive input from multiple controllers, each of which may be used to control an avatar or character within a software application (e.g., a computer game).
[0039] Figure 4 A block diagram illustrating a process for implementing a dynamic-based controller according to an embodiment is depicted. Process 400 can be executed on a user device, where a virtual physical controller is implemented, such as those described above. Figure 3 The controller 302 is described.
[0040] At 402, process 400 may involve detecting the current state applicable to an operating virtual controller. In some embodiments, state data may be provided to the virtual controller by a software application that interacts with the virtual controller. In some embodiments, multiple states may be applied at a given time. For example, in the case of a video game, one state may correspond to the currently played character, while a second state may correspond to the action that character is currently performing.
[0041] At 404, process 400 may involve depicting a button (or other input mechanism) based on the detected current state. In some embodiments, a set of configuration data for the virtual controller may include indications of how each button should be depicted based on different states. In the configuration data, each button may be associated with a default appearance. Additionally, each button may be assigned a different appearance based on different detected states. In some cases, different sizes, positions, and / or shapes may be assigned to each button based on the currently determined state. In some embodiments, one or more buttons may be implemented only during a specific state (e.g., visible and interactive).
[0042] At 406, process 400 may involve monitoring changes in state. In some embodiments, a change in state may be detected when an input mechanism is activated. For example, if a player presses a button intended to trigger an action, a state corresponding to that action may be initiated. In some embodiments, one or more states may be temporary, as they are set to expire after a predetermined amount of time has elapsed since the current state has been in effect.
[0043] At 408, process 400 may involve determining whether any state change has occurred (e.g., as detected at 406). If such a change has been detected in the current state, the process may return to 404, and the depiction of one or more buttons of the virtual controller may be implemented for the newly detected state. In some cases, the appearance of one or more buttons may be updated based on the newly detected state. For example, the text on the buttons may be changed or otherwise updated based on information included in the configuration data.
[0044] If no change in state is detected, process 400 may involve monitoring user input (e.g., touch input) at 410. In various embodiments, user input may be detected when a user touch originating from one or more areas associated with an implemented button is detected. Specifically, when user touch input is detected, it may be determined whether the touch input is located within one or more areas associated with one or more buttons among the currently implemented buttons. Based on whether touch input from the user is detected and whether the touch input falls within one or more areas of the currently implemented button, it may be determined whether activation of at least one button has been detected at 412. If no activation is detected, the process may involve continuing to monitor for changes in state at 406.
[0045] At 414, if activation of at least one button has been detected, process 400 may involve generating activation data based on the button activation. The activation data may include indications of one or more buttons that have been detected as active. It may be determined which actions should be initiated by activating the buttons based on which states or states are currently active. In some embodiments, the virtual controller's configuration data may maintain information about which actions should be initiated when each button is activated and in what states. The generated activation data may include indications of actions associated with button activation in the currently detected states.
[0046] In some cases, activation data may also include information about the degree or amount associated with activation, such as the duration of the user's touch on the button, the direction and / or length of the user's swipe relative to the button, or any other suitable information. Once activation data is generated, the process can return to monitoring for changes in state at point 406.
[0047] Figure 5 An illustration depicts a process for dynamically adjusting input mechanisms (i.e., buttons) on a virtual controller according to an embodiment. Process 500 is depicted on a series of images of a user equipment 502 (AC) on which the virtual controller can be implemented. As shown, the virtual controller can be implemented via a graphical user interface (GUI) 504 presented on a touchscreen display.
[0048] At 502(A), a first state of the virtual controller can be detected. In the depicted example, configuration data associated with the virtual controller can indicate information for implementing one or more input mechanisms on the virtual controller in each state. In some embodiments, when the implementation of a particular input mechanism is conflicting for each of two different states determined to be active, the configuration data can further indicate which implementation takes precedence when the two states are detected.
[0049] In some embodiments, a user can provide touch input to the input mechanism, as shown at 506. Upon detecting touch input, a first action can be determined to be initiated based on the detected first state. Activation data can then be generated to initiate the action. When the action is initiated, a second state can be detected.
[0050] For example, consider a scenario where a virtual controller is implemented on a mobile device running a video game. In this scenario, the character in the video game can be in a "standing" state (i.e., the first state). The user can touch a button (e.g., an input mechanism) corresponding to the execution of a "jump" action. In this example, once the user touches the jump button, the character can initiate a jump action and a second "jump" state (i.e., the second state) can be detected. In this case, if the implementation of a specific button in the configuration data contradicts the "standing" and "jump" states, it can be resolved in a way that favors the "jump" state.
[0051] In 502(B), the virtual controller can detect a second state. In some cases, this state can be attached to the first state (i.e., both states can be active simultaneously). In other cases, the states can be mutually exclusive, as activation of a state can override the current state. Upon detecting a second state, the virtual controller can determine whether one or more of the input mechanisms need to be updated. To do this, configuration data can be consulted to determine the implementation data for each input mechanism. Based on such implementation data, the virtual controller can update the position, size, shape, appearance (e.g., text), and / or action associated with the input mechanism.
[0052] As depicted, upon detecting a second state, the virtual controller can update the implemented input mechanisms. In some cases, this may involve adding or displaying input mechanisms to be activated when the second state is active (e.g., input mechanism 508). In some cases, this involves removing, deactivating, or hiding input mechanisms that are not activated when the second state is active (e.g., input mechanism 510). In some cases, this involves changing the appearance of one or more input mechanisms (e.g., input mechanism 512).
[0053] At 502(C), it can be determined that the second state is no longer active. In some cases, this may be due to the second state being replaced by another state. In some cases, this may be because the second state has expired after a predetermined amount of time associated with it has expired (e.g., if the second state is a temporary state). In some embodiments, the second state can persist as long as the user continues to provide touch input to the button. For example, the "Sprint" state can persist as long as the user holds down the "Sprint" button.
[0054] As an example, consider a continuation of the example scenario given above. When the "jump" state is detected to be active, the virtual controller can update the button configuration so that each button in the current button configuration is implemented in a way associated with the jump state. Touching one or more of these buttons may result in the execution of an action different from the action that would be taken during the standing state. In this example, the jump state can persist as long as the character is jumping, which could depend on the set amount of time the character spends jumping, the extent to which the user presses the jump button, or any other suitable factor. Continuing with this example scenario, once the jump is complete, the state can return to the first "standing" state. Each button can then be reimplemented based on the first state.
[0055] Figure 6 A block diagram illustrating a process for providing dynamic state changes of buttons for controlling an avatar, according to some embodiments, is depicted. Process 600 can be executed on any suitable service environment, including but not limited to... Figure 1The service environment 100 is shown. According to various embodiments, process 600 includes sensing a first touch input at 601 at a location on the touchscreen device corresponding to a first button. Then, at 602, the system can cause a player avatar to perform a first action based on the first touch input and assign a status to the player avatar based on the first touch input, wherein the status persists for a non-zero time period after the first touch input is released at 603, for example, the player avatar "dashes" on the screen and is assigned the status "dash," which persists for a period of time after the "dash" button is released.
[0056] In process 600, at point 604, in response to the assigned state, the behavior of the second button is temporarily modified from the default behavior to the modified behavior, for example, changing "jump" to "long jump" or "attack" to "charge attack," etc. At point 605, the display state of the second button, i.e., the name displayed to the user, can also be changed from the initial state to the modified state during the duration of the assigned state. Therefore, not only does the button's behavior change, but its visibility attributes also change.
[0057] At point 606, when the system senses a second touch input at a second position corresponding to the second button, assuming the behavior of the second button is still modified (i.e., for a period of time after the first button is released), at point 607, the system reads the second input and causes the player avatar to perform the modified action of the second button instead of the default action based on the second touch input. Conversely, if the modified behavior times out, the system will instead cause the player avatar to perform the default action corresponding to the second button.
[0058] Figure 7 A flowchart illustrating an example processing flow 700 for generating and implementing a virtualized physical controller according to an embodiment is depicted. Process 700 can be executed by a computing device configured to generate and provide a product strategy. For example, process 700 can be executed by a controller (e.g., as described above). Figure 3 The controller 302 described herein is used to perform this action, and the controller is capable of facilitating interaction between the user and the software application. In some embodiments, such a software application is a video game played by the user.
[0059] At 702, process 700 includes receiving a first touch input corresponding to one of a set of input mechanisms implemented on the controller. As noted elsewhere, the controller may include a touchscreen display and may receive the first touch input (and a second touch input) from the user via the touchscreen display. In some embodiments, the user equipment is one of a smartphone or a tablet device.
[0060] At 704, process 700 includes determining a first state currently applicable to the controller. In some embodiments, the first state may be a default state or a state in which the software application is started.
[0061] At 706, process 700 includes generating first activation data based on the activation and first state of the input mechanism. The first activation data may include information about one or more actions triggered by the input mechanism.
[0062] At 708, process 700 includes receiving a second touch input corresponding to an input mechanism implemented on the controller. Similar to the first touch, the second touch can be received via touch input provided on a touchscreen display of the controller.
[0063] At 710, process 700 includes determining a second state currently applicable to the controller. In some embodiments, the second state may be initiated when a different input mechanism in the group of input mechanisms is activated. For example, a user may press or touch different buttons, which then causes the second state to be initiated. In some cases, the state may be additional, because the first state and the second state may be active simultaneously. In some cases, the state may be exclusive, because the second state replaces the first state such that no more than one of the first or second states may be active simultaneously.
[0064] In some embodiments, the controller may detect when the second state has been initiated. Furthermore, when it is determined that the second state is currently active, the process may involve updating at least one of the size, position, shape, or appearance of the input mechanism. In some embodiments, the process also involves updating the group of input mechanisms by adding at least one additional input mechanism during the duration the second state is active, when it is determined that the second state is currently active. For example, one or more additional buttons (or other suitable input mechanisms) may be added to the virtual controller when the second state is active. In some embodiments, the process also involves updating the group of input mechanisms by removing at least one additional input mechanism from the group of input mechanisms during the duration the second state is active, when it is determined that the second state is currently active. For example, one or more buttons (or other suitable input mechanisms) may be removed or otherwise deactivated when the second state is active. In some embodiments, each of the first and second states relates to the state of the avatar played by the user in a video game.
[0065] At 712, process 700 includes generating second activation data based on the activation and second state of the input mechanism. In an embodiment, one or more actions indicated in the second activation data differ from one or more actions indicated in the first activation data.
[0066] In some embodiments, the second state can be temporary, as it lasts for a non-zero amount of time. In some cases, the second state lasts for a predetermined amount of time. In some cases, the second state persists as long as certain conditions are met (e.g., as long as the user continues to press a button, etc.). When the second state expires, the virtual controller can revert to making the second state inactive and the first state active. In this case, the process may also involve receiving a third touch input, determining that the second state is no longer active, and generating third activation data that includes an indication of the first action.
[0067] The methods described herein are for virtual controllers, i.e., controllers that use a touchscreen or similar touchscreen functionality to provide an easily customizable controller button layout. According to some embodiments, the touchscreen is at least a portion of a physical, handheld controller that engages with a gaming device such as a game console, personal computer, tablet, smartphone, or thin client device (e.g., a USB or HDMI device plugged into a screen). According to some embodiments, the touchscreen is a primary feature of the controller, which engages with a gaming device such as a game console, personal computer, tablet, smartphone, or thin client device (e.g., a USB or HDMI device plugged into a screen). According to some embodiments, the controller comprises a mobile device or tablet along with enabling software that connects the mobile device or tablet to the gaming device, such as a game console, personal computer, thin client device (e.g., a USB or HDMI device plugged into a screen), or other suitable gaming device. According to some other embodiments, the touchscreen is a touch-enabled touchscreen of a gaming device such as a game console, personal computer, tablet, or smartphone.
[0068] The description and drawings should be considered illustrative rather than restrictive. However, it will be apparent that various modifications and changes can be made thereto without departing from the broader spirit and scope of this disclosure as set forth in the claims.
[0069] Other variations are all within the spirit of this disclosure. Therefore, while the disclosed technology is readily adaptable to various modifications and alternative constructions, certain illustrated embodiments are shown in the accompanying drawings and have been described in detail above. However, it should be understood that the invention is not intended to be limited to the specific one or more forms disclosed, but rather is intended to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention, as defined in the appended claims.
[0070] In the context of describing the disclosed embodiments (especially in the context of the appended claims), the terms “a,” “an,” and “the,” and similar designations, should be interpreted to cover both the singular and plural, unless otherwise stated herein or obviously contradicted by the context. The terms “comprising,” “having,” “including,” and “containing” should be interpreted as open-ended terms (i.e., meaning “including but not limited to”), unless otherwise stated. The term “connected” should be interpreted as being partially or wholly contained therein, attached to, or joined together, even if something else is involved. The enumeration of numerical ranges herein is intended only as a shorthand method for individually referring to each individual value falling within that range, unless otherwise indicated herein, and each individual value is incorporated into the specification as if it were individually referenced herein. All methods described herein may be performed in any suitable order unless otherwise indicated herein or obviously contradicted by the context. The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate embodiments of the invention and does not constitute a limitation on the scope of the invention, unless otherwise required. No language in the specification should be construed as indicating that any unclaimed element is necessary for the practice of the invention.
[0071] This document describes preferred embodiments of the present disclosure, including the best modes known to the inventors for carrying out the invention. Variations of those preferred embodiments will be apparent to those skilled in the art upon reading the foregoing description. The inventors expect those skilled in the art to appropriately employ such variations, and the inventors intend to practice the invention in ways other than those specifically described herein. Therefore, the invention includes all modifications and equivalents of the subject matter recited in the appended claims as permitted by applicable law. Furthermore, unless otherwise stated herein or clearly contradicted by the context, the invention covers any combination of the foregoing elements in all possible variations.
[0072] Further examples are described below to facilitate understanding of various aspects of the invention:
[0073] Example A, a method including:
[0074] The user receives a first touch input via a touchscreen display, corresponding to at least one of a set of input mechanisms;
[0075] This indicates that the current state is the first state of the activity;
[0076] First activation data, including an instruction for a first action, is generated based on the first state;
[0077] Receives a second touch input from the user via a touchscreen display, corresponding to at least one input mechanism;
[0078] It is determined that the current state is the second state of the activity; and
[0079] A second activation data is generated based on the second state, which includes an instruction for a second action that is different from the first action.
[0080] Example B, any of the methods in the preceding examples, further includes updating at least one of the following of at least one input mechanism when it is determined that the second state is currently active: size, position, shape, or appearance.
[0081] Example C, any of the methods in the preceding examples, further includes updating the group of input mechanisms by adding at least one additional input mechanism to the group of input mechanisms while the second state is active, when it is determined that the second state is currently active.
[0082] Example D, any of the preceding examples, further includes updating the group of input mechanisms by removing at least one additional input mechanism from the group of input mechanisms when it is determined that the second state is currently active.
[0083] Example E, any of the methods in the preceding examples, wherein both the first state and the second state are determined to be active simultaneously.
[0084] Example F, any of the methods in the preceding examples, wherein the second state replaces the first state such that no more than one of the first state or the second state is active at the same time.
[0085] Example G, any of the methods in the preceding examples, wherein the second state is initiated when different input mechanisms in the group of input mechanisms are activated.
[0086] Example H, any of the methods in the preceding examples, wherein the second state lasts for a predetermined amount of time.
[0087] Example I, any of the methods in the preceding examples, also includes:
[0088] Receive third touch input;
[0089] It is determined that the second state is no longer active; and
[0090] Generate third activation data that includes instructions for the first action.
[0091] Example J, a user equipment, comprising:
[0092] Processor; and
[0093] The memory includes instructions that, when executed by the processor, enable the user equipment to at least:
[0094] The user receives a first touch input corresponding to at least one of a set of input mechanisms;
[0095] This indicates that the current state is the first state of the activity;
[0096] First activation data, including an instruction for a first action, is generated based on the first state;
[0097] Receive a second touch input from the user corresponding to at least one input mechanism;
[0098] It is determined that the current state is the second state of the activity; and
[0099] A second activation data is generated based on the second state, which includes an instruction for a second action that is different from the first action.
[0100] Example K, the user equipment in the aforementioned examples, also includes a touch screen display, wherein the first touch input and the second touch input are received from the user via the touch screen display.
[0101] Example L, a user device of any of the preceding examples, wherein the user device is a smartphone or a tablet device.
[0102] Example M, a user device of any of the preceding examples, wherein the instruction further causes the user device to initiate a second state upon receiving a third touch input corresponding to a second input mechanism in the group of input mechanisms, wherein the second input mechanism is different from the at least one input mechanism.
[0103] Example N, a user device of any of the preceding examples, wherein the instructions include a virtual controller capable of facilitating interaction between the user and a software application.
[0104] Example O, a user device of any of the preceding examples, wherein the software application includes a video game played by the user.
[0105] Example P, any of the user devices in the preceding examples, wherein each of the first and second states relates to the state of the avatar the user is playing in the video game.
[0106] Example Q, a user device of any of the preceding examples, wherein the instruction further causes the user device to update at least one of the following of at least one input mechanism when it is determined that the second state is currently active: size, position, shape, or appearance.
[0107] Example R: A non-transitory computer-readable medium that jointly stores computer-executable instructions, which, when executed, cause one or more computing devices to jointly perform the following actions:
[0108] The user receives a first touch input via a touchscreen display, corresponding to at least one of a set of input mechanisms;
[0109] This indicates that the current state is the first state of the activity;
[0110] First activation data, including an instruction for a first action, is generated based on the first state;
[0111] Receives a second touch input from the user via a touchscreen display, corresponding to at least one input mechanism;
[0112] It is determined that the current state is the second state of the activity; and
[0113] A second activation data is generated based on the second state, which includes an instruction for a second action that is different from the first action.
[0114] Example S, the non-transitory computer-readable medium in the preceding examples, also includes updating at least one of the following of at least one input mechanism when it is determined that the second state is currently active: size, position, shape, or appearance.
[0115] Example T, a non-transitory computer-readable medium of any of the preceding examples, wherein the second state is initiated when different input mechanisms in the group of input mechanisms are activated.
[0116] in conclusion
[0117] Although the subject matter has been described in language specific to features and method actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described herein. Rather, specific features and actions are disclosed as exemplary forms for implementing the claims.
Claims
1. A method comprising: The user receives a first touch input via a touchscreen display, corresponding to at least one of a set of input mechanisms; The current state is determined to be the first active state, where the first active state corresponds to the active state of an object within the interactive video game, and the object is controlled by the user using a virtual controller; First activation data, including an instruction for a first action, is generated based on the first state; The first action is performed based on the first activation data; Receive a second touch input from the user via the touchscreen display, corresponding to the at least one input mechanism; The current state is determined to be a second active state, wherein the second active state corresponds to a different active state of the object within the interactive video game, and wherein the first state changes to the second active state in response to performing the first action; as well as Based on the second state, second activation data is generated that includes an instruction for a second action, which is different from the first action.
2. The method of claim 1, further comprising, when determining that the second state is currently active, updating at least one of the following of the at least one input mechanism: size, position, shape, or appearance.
3. The method of claim 1, further comprising, when determining that the second state is currently active, updating the set of input mechanisms by adding at least one additional input mechanism to the set of input mechanisms while the second state is active.
4. The method of claim 1, further comprising, when determining that the second state is currently active, updating the set of input mechanisms by removing at least one additional input mechanism from the set of input mechanisms while the second state is active.
5. The method according to claim 1, wherein, Both the first state and the second state are determined to be active simultaneously.
6. The method according to claim 1, wherein, The second state replaces the first state, such that no more than one of the first state or the second state is active at the same time.
7. The method according to claim 1, wherein, The second state is initiated when different input mechanisms in the set of input mechanisms are activated.
8. The method according to claim 1, wherein, The second state lasts for a predetermined amount of time.
9. The method according to claim 1, further comprising: Receive third touch input; It has been determined that the second state is no longer active; as well as Generate third activation data that includes an instruction for the first action.
10. A user equipment, comprising: processor; as well as The memory includes instructions that, when executed using the processor, cause the user equipment to at least: The user receives a first touch input corresponding to at least one of a set of input mechanisms; The current state is determined to be the first active state, where the first active state corresponds to the active state of an object within the interactive video game, and the object is controlled by the user using a virtual controller; First activation data, including an instruction for a first action, is generated based on the first state; The first action is performed based on the first activation data; Receive a second touch input from the user corresponding to the at least one input mechanism; The current state is determined to be a second active state, wherein the second active state corresponds to a different active state of the object within the interactive video game, and wherein the first state changes to the second active state in response to performing the first action; as well as Based on the second state, second activation data is generated that includes an instruction for a second action, which is different from the first action.
11. The user equipment of claim 10, further comprising a touchscreen display, wherein, The first touch input and the second touch input are received from the user via the touchscreen display.
12. The user equipment according to claim 10, wherein, The user equipment is either a smartphone or a tablet.
13. The user equipment according to claim 10, wherein, The instruction also causes the user equipment to activate the second state upon receiving a third touch input corresponding to the second input mechanism in the set of input mechanisms, wherein the second input mechanism is different from the at least one input mechanism.
14. The user equipment according to claim 10, wherein, The instructions include a virtual controller that facilitates interaction between the user and the software application.
15. The user equipment according to claim 14, wherein, The software application includes video games played by the user.
16. The user equipment according to claim 15, wherein, Each of the first state and the second state relates to the state of the avatar played by the user in the video game.
17. The user equipment according to claim 10, wherein, The instructions also cause the user equipment to update at least one of the following: size, position, shape, or appearance of the at least one input mechanism when it is determined that the second state is currently active.
18. A non-transitory computer-readable medium that jointly stores computer-executable instructions, which, when executed, cause one or more computing devices to jointly perform the following actions, said actions including: The user receives a first touch input via a touchscreen display, corresponding to at least one of a set of input mechanisms; The current state is determined to be the first active state, where the first active state corresponds to the active state of an object within the interactive video game, and the object is controlled by the user using a virtual controller; First activation data, including an instruction for a first action, is generated based on the first state; The first action is performed based on the first activation data; Receive a second touch input from the user via the touchscreen display, corresponding to the at least one input mechanism; The current state is determined to be the second state of the activity; as well as Based on the second state, second activation data is generated including an instruction for a second action, which is different from the first action, wherein the second state corresponds to a different activity state of the object within the interactive video game, and wherein the first state changes to the second state in response to performing the first action.
19. The non-transitory computer-readable medium of claim 18, further comprising, upon determining that the second state is currently active, updating at least one of the following: size, position, shape, or appearance of the at least one input mechanism.
20. The non-transitory computer-readable medium according to claim 18, wherein, The second state is initiated when different input mechanisms in the set of input mechanisms are activated.
Citation Information
Patent Citations
System and method of assisted or automated grain unload synchronization
US20220019240A1
Device and method for a dynamically configurable user interface in a game of chance
US20060252542A1
