Waiting time adjustment for user input devices

Abstract

Provided herein are enhanced user input devices and user input interface systems that can reduce perceived interaction latency. In one example, a method of operating a user input interface on a host system includes: identifying a target pace for delivery of a user input state to an application, and determining one or more timing parameters for delivery of the user input state from the user input device based at least on the target pace. The method also includes indicating the one or more timing parameters to the user input device, wherein the user input device responsively delivers the user input state based on the one or more timing parameters.

Description

Background Art

[0001] The user input device may be employed to interface with a variety of interactive systems, such as game consoles, computing environments, computer systems, virtual / augmented reality devices, and other devices and systems, which may also include aerospace, military, or industrial control systems. Microsoft A series of game consoles are example systems that may employ user input devices (commonly referred to as game controllers). A game controller is configured to communicate with the game console via a wired or wireless interface. In such game systems, the game controller may employ a joystick, thumb stick, control lever, proportional trigger, gamepad, adaptive control mechanism, or other input mechanism. Various computing systems may also employ touch screens, keyboards, mice, trackpads, adaptive controllers, virtual / augmented reality controllers, or other similar equipment.

[0002] These user input devices allow user manipulation of on-screen user interface elements, which may be graphical in nature. A common complaint among users of gaming systems employing user interface devices is the perceived input latency between user manipulation of the input device and the change of on-screen user interface elements. For example, when a user presses a button on a game controller, the user expects to see the user interface elements of the corresponding game react to the button press. However, due to input and rendering latency, sometimes the displayed content seems less responsive than what the user would expect.

[0003] Overview

[0004] Provided herein are enhanced user input devices and user input interface systems that can reduce perceived interaction latency. In one example, a method of operating a user input interface on a host system includes: identifying a target pace for delivery of a user input state to an application, and determining one or more timing parameters for delivery of the user input state from the user input device based at least on the target pace. The method also includes indicating the one or more timing parameters to the user input device, wherein the user input device responsively delivers the user input state based on the one or more timing parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Many aspects of the present disclosure may be better understood with reference to the following drawings. Although several implementations are described in conjunction with these drawings, the present disclosure is not limited to the implementations disclosed herein. On the contrary, it is intended to cover all alternatives, modifications and equivalents.

[0007] Figure 1 is a system diagram of a user input environment in an implementation.

[0008] Figure 2A method of operating a user input interface in an implementation is explained.

[0009] Figure 3 is a system diagram of a user input device in an implementation.

[0010] Figure 4 is a system diagram of a host system in an implementation.

[0011] Figure 5 A method of operating a user input interface in an implementation is explained.

[0012] Detailed Description

[0013] User input devices can be employed in a computing environment to interface with game consoles, computing platforms, computer systems, virtual / augmented reality devices, and other devices and systems. For example, Microsoft Series game systems can communicate with one or more game controllers to control user interface elements on the screen. When using games or user applications on the game system, user interaction with the game controller can affect the graphics and text user interface elements rendered on the associated display. These user input devices usually include both mechanical elements and electronic components to allow user manipulation of the user interface elements on the screen. The system using the user interface device can experience the input latency perceived by the user between the user manipulation of the input device and the change of the user interface elements on the screen. For example, when the user presses the button on the game controller, the user may experience a small amount of delay in the corresponding change of the rendered user interface element. This is particularly problematic in games and applications (such as first-person shooting (FPS) style games or accurate computer-aided drawing applications) with precise or fast on-screen movement and interaction. Although the increased sampling rate of the input device mechanism can be achieved, this does not ensure that the perceived input latency is reduced or the outdated user input state is occasionally delivered to the application.

[0014] Input devices typically send information about the current input device state to the host system at their own pace. When an application goes to consume this input device state, it is typically out of date on the order of tens of milliseconds due to a combination of hysteresis or latency from factors including input device sampling delays, transfer delays on one or more communication links, and processing or handling delays within software elements of the host system. This hysteresis or latency in the reception of input device state by an application on the host system results in input latency that is perceptible to the user. Discussed herein are enhanced systems, architectures, and methods for application request synchronization for input device state delivery to application-defined epochs. Advantageously, this ensures that the application has a more recent input device state than is typically obtained.

[0015] As a first example of an enhanced user input device and a user interface element of a host system, Figure 1 is presented. Figure 1 1 is a system diagram illustrating a user input environment 100. Environment 100 includes input device 110, host system 120, and display device 130. Input device 110 and host system 120 communicate via link 140. Host system 120 and display device 130 communicate via link 141. Input device 110 is an example of any suitable user input device, such as Figure 1 101-103. Links 140-141 may include wireless or wired links and are discussed in further detail below. Although game controllers and gaming systems may be employed in environment 100, it should be understood that this discussion may also apply to other user input devices and user system types.

[0016] The input device 110 includes a timing adjustment client 115, and the host system 120 includes an application 122 and a timing adjustment system 125. In operation, a user will employ the input device 110 to interact with the application 122 executed by the host system 120. The timing adjustment client 115 and the timing adjustment system 125 operate to reduce user input latency for the application 112 in the delivery of information indicative of user manipulation of the input device 110. This information indicative of user manipulation is referred to herein as input device state, and generally includes a digital representation of the user manipulation, which is communicated by the input device 110 for delivery to the host system 120. In some examples, the timing adjustment system 125 adjusts the delivery of the input device state from the input device 110 according to input state timing requests issued by the application 122 or other elements of the host system 120.

[0017] Figure 2 is a flow chart illustrating an example method of operation of the elements included in environment 100. Figure 2 Operation. Figure 2 , an input state delivery adjustment process is described in detail, wherein the input state delivery timing is adjusted between the input device 110 and the host system 120. The host system 120 identifies (201) a target pace or target cadence for the delivery of user input state from the user input device 110 to the application 122. The target pace or target cadence refers to a repetitive or cyclic time interval that separates individual transfers of the user input state. The timing adjustment system 125 may receive an indication of the target pace from the application 122. The indication of the target pace may include an indication of a recurring event within the host system 120 executing the application 122 and a timing offset from the recurring event. The recurring system event may be used as a baseline event from which timing adjustments are made using at least a timing offset.

[0018] Recurring system events related to graphics rendering or display refresh may be referenced by application 122 at a target pace. This may be due in part to the fact that these recurring system events are associated with user interface elements that are affected by the user input state of input device 110. Example recurring system events include a display refresh indicator, a display vertical synchronization indicator (VSYNC), or a graphics frame rate timing characteristic of a graphics process of host system 120. For example, a VSYNC signal may be employed to indicate the timing for a display refresh operation or a display rendering operation with respect to display 130. The VSYNC signal (referring to the vertical synchronization refresh of display 130) is cyclically repeated according to a refresh interval (such as 60 Hertz (Hz), 120 Hz, or other refresh interval). Another recurring system event may include a frame rate for rendering graphics associated with application 122 by a graphics processing portion of host system 120. The frame rate may correspond to a frames per second (FPS) metric.

[0019] The timing offset may be used to offset the delivery of the user input state to the application 122 from recurring system events (such as those discussed above). For example, the timing offset may be an indication of time in microseconds (μs) used to delay the delivery of the user input state relative to the recurring system event. Thus, the expected delivery time of the user input state to the application 122 may correspond to a number of microseconds after each occurrence of the system event. The timing offset may allow the application 122 to handle the time to integrate each new instance of the user input state into rendering operations, user interface change / update operations, game logic operations, and other user interface triggers based on changes in the user input state.

[0020] Although the timing adjustment system 125 may receive an indication of a target pace from the application 122, in further examples, the timing indication may be determined by the timing adjustment system 125 without the direct participation of the application 122. This may be referred to as predictive, adaptive, or automatic user input state timing adjustment. In these adaptive timing adjustment examples, the timing adjustment system 125 monitors the timing associated with one or more system calls or requests for user input states from the application. The timing adjustment system 125 may then determine the target pace based at least on the timing associated with one or more requests. For example, the timing adjustment system 125 may determine that a request for user input states from the application 122 exhibits a specific pace or rate that is periodic or quasi-periodic. The timing adjustment system 125 then determines a timing offset from the adaptively determined pace or rate from which the timing adjustment discussed below is made. In this way, the application does not need to be specifically configured to receive the benefits of the enhancement in the user input state delivery discussed herein.

[0021] Once the target pace has been identified, the host system 120 determines (202) one or more timing parameters for the transfer of the user input state from the input device 110 based on the target pace. The one or more timing parameters may include a transfer interval and a transfer offset for adjusting the transfer of consecutive user input states from the input device 110. The transfer interval indicates a periodic or cycle time for the user input device 110 to transfer the user input state via the link 140. The transfer interval may include a cycle time corresponding to the recurring system event determined in operation 201, or may include a different periodic interval. The transfer offset may be based at least in part on a hysteresis between the transmission of the user input state by the input device 110 and the receipt of the user input state by the user input interface of the host system 120.

[0022] In order to determine the transfer offset, the timing adjustment system 125 determines the hysteresis or delay between the transfer of the user input state by the input device 110 and the subsequent reception of the user input state by the host system 120. The timing adjustment system 125 can determine the hysteresis in various ways. For example, the estimation of the hysteresis can be made based on the estimated link delay and other delays in the transfer of the user input state by the input device 110. However, a more direct measurement can be made by the timing adjustment system 125 instead. The direct measurement may include instructing the input device 110 to generate a timing indicator corresponding to when the input device 110 transfers the user input state, and the timing indicator may be included in the signaling, packet or frame accompanying or including the user input state. In other examples, a separate calibration process is performed between the timing adjustment client 115 in the input device 110 and the timing adjustment system 125 in the host system 120. The calibration process may include transmitting the timing indicator in the traffic from the input device 110 with the host system 120 as the destination. The timing indicator may also be included with each instance or portion of the user input state delivered to the host system 120. In a further example, the timing adjustment system 125 periodically requests the inclusion of a timing indicator to accompany the user input state to make adjustments due to operational drift.

[0023] The timing indicator may include a timestamp or a monotonically changing count generated by the timing adjustment client 115 of the input device 110 and included in communications transmitted by the input device 110. The monotonically changing count may include an increasing count or a decreasing count that increases or decreases according to a clock signal, a time granularity, or a periodic event of the input device 110. A predetermined number of count digits or a predetermined count size may be used, and when a count limit or maximum count is reached, the count may be reset or rolled over to an initial value to restart the increasing or decreasing count. The count size is typically set to account for one or more instances of user input state transmission or to account for time delays greater than a maximum expected delay to ensure proper calculation of the actual delay.

[0024] Once received by the timing adjustment system 125, the timing indicator may be used by the timing adjustment system 125 to determine a delivery offset. The delivery offset is determined to compensate for the hysteresis or delay between the input device 110 delivering the user input state and the interface element of the host system 120 receiving the user input state. Thus, the timing adjustment system 125 determines one or more timing parameters indicated to the input device 110 based on the target pace and the hysteresis.

[0025] As described above, the first timing parameter in the timing parameters may include a transfer interval. The transfer interval corresponds to a repetitive system event used as a timing baseline in operation 201. Since the input device 110 may not have visibility of a specific repetitive system event of the host system 120, the timing adjustment system 125 may indicate the transfer interval to the input device 110 with another metric or symbol. For example, the repetitive system event of the host system 120 may include a VSYNC signal or other repetitive signals mentioned herein. The timing adjustment system 125 may determine an equivalent transfer interval corresponding to the VSYNC signal or other repetitive system event, and pass an indication of the timing interface to the input device 110. The indication of the transfer interval may be converted into an interval with a time granularity that can be interpreted by the input device 110 (such as an absolute granularity in microseconds or a relative granularity associated with a clock or repetitive event to which the user device 110 has visibility).

[0026] The second timing parameter in the timing parameters may include a transfer offset. The transfer offset indicates an offset in time relative to the transfer interval, which compensates for the hysteresis or delay when the user input state is transferred from the input device 110. However, the transfer offset also takes into account the timing offset for the application 122 related to the repetitive system event. Hysteresis and timing offset are taken into account when calculating the transfer offset to ensure that the delivery of the user input state to the application 122 is consistent with the expected delivery time-as indicated by the repetitive system event and the timing offset. In one example, the hysteresis and timing offset are converted into a common time granularity and then added together to generate the transfer offset. In other examples, the transfer interval is taken into account in this calculation to ensure that any offset in timing between the transfer interval and the repetitive system event is taken into account. Other delays, hysteresis, offsets, intervals and timing parameters may also be taken into account.

[0027] Once the one or more timing parameters are determined by the timing adjustment system 125, the timing parameters are indicated (203) to the input device 110 via the link 140. The timing adjustment client 115 receives the timing indicators and makes adjustments to the operation of the input device 110 based on the timing indicators. Specifically, the timing adjustment client 115 makes adjustments to the input device 110 so that the input device 110 communicates (204) the user input state based on the one or more timing parameters. These adjustments may include adjusting when to sample user input from the input mechanism, when to determine that the user input state is updated or changed by the input device 110, and when the communication interface element of the input device 110 communicates the user input state for delivery via the link 140.

[0028] During operation, the relative timing between the input device 110 and the host system 120 may drift and reintroduce latency or other significant effects, even with timing adjustments. Periodic timing calibrations for adjustments can be performed using the process described above. These periodic timing calibrations can correct or compensate for drifts in various timing parameters, timing offsets, or other signals. Moreover, when a wireless link is used for link 140, changes in the hysteresis between the input device 110 and the host system 120 may change over time due to changes in link characteristics (e.g., wireless signal delays). Among other considerations, these periodic timing calibrations may be made in response to a periodic timer, in response to power-up of the input device 110 or the host system 120, at the start-up of the application 122, in response to a user request or user command, or when a drift above a certain threshold level is experienced.

[0029] In addition to the initial or periodic calibration or adjustment process, the application 122 may request different timing intervals or offsets based on activity in the application. For example, during a non-interactive portion of the application (such as a game cut-scene or animation), user input may not be used. However, once the non-interactive portion of the application is completed, user input may be desirable again. The application 122 may anticipate such transitions between non-interactive and interactive architectures and issue one or more requests for user input state timing adjustments. These timing adjustments may be made to ensure that outdated user input data is not presented to the application 122 at the transition between the architectures. Moreover, the timing offset indicated by the application 122 can be used to realign user input state delivery with respect to recurring system events to provide an enhanced transition from a non-interactive architecture and into an interactive architecture.

[0030] Return to Figure 1 In the detailed discussion of the elements of FIG. 1 , input device 110 is shown as an exemplary input device including game controller 101, mouse 102, or keyboard 103. However, input device 110 may include any type of user input device. In a computer system, the user input device may include a game controller, which uses a joystick, a thumb stick, a control rod, a proportional trigger, a game handle, an adaptive control mechanism, or other input mechanisms. Various computing systems may also use touch screens, keyboards, mice, touch pads, adaptive controllers, virtual reality controllers, augmented reality controllers, or other similar equipment. However, various types and forms of user input devices generally share common elements for receiving user manipulation and user input. These are referred to as input mechanisms, and may include joysticks, d-pads, buttons, keys, triggers, paddles, wheels, or other input elements. Other input mechanisms may be used, such as touch pads, touch screens, voice control, or other input mechanisms, including combinations thereof. The input mechanism generally has corresponding electromechanical elements and structural elements for receiving user manipulations, and also includes a measuring element for interpreting user manipulations into electrical signals. Measuring elements (such as position sensing elements and transducer elements) may be included in the input device 110 and may include potentiometers, position determination circuit systems, angle resolvers, photoelectric position sensors, magnetic sensors (such as Hall effect sensors or magnetoresistive sensors), encoders, capacitive sensors, inductive sensors, or other position measurement elements and position sensing elements. Analog-to-digital (A / D) conversion elements may also be included in the control circuit system of the input device 110 to convert electrical signaling from analog measurement or positioning signals into digital representations of input data. These digital representations can be transmitted to the host system 120 via the link 140 for controlling user interface elements associated with the application 122 or other software. However, as discussed herein, the timing adjustment client 115 is also included in the input device 110 to perform adjustments to the sampling and delivery of user input states.

[0031] The host system 120 includes an end-user system, such as a gaming system, an augmented / virtual reality system, a terminal, a computer system, a computing device, a tablet device, a smart phone, a personal computer, a server, a cloud-based system, a distributed computing platform, and the like. A user (such as a user of a gaming system, an entertainment platform, or a computing system) can interact with the user interface elements of the host system 120 via a user interface device. The host system 120 may include a processing system, a memory system, a storage system, a network interface device, a user interface element, an audio and graphics processing system, a video handling system, and a user input state timing adjustment service (such as a timing adjustment system 125). The host system 120 may include one or more microcontrollers, microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or discrete logic and associated circuit systems, including combinations thereof. The host system 120 may include additional hardware and software elements, such as a video processor, a graphics core, a video card, a graphics interface card, a video memory, a graphics engine, an execution unit, a rendering unit, a memory interface unit, or other elements, including combinations and variations thereof.

[0032] Display device 130 may include one or more video monitors, televisions, projectors, touch screens, transmission video interfaces, virtualization interfaces, or other video or graphic viewing elements, including combinations thereof. Display device 130 may have one or more input interfaces for receiving graphics or video from a source (such as from host system 120) via link 141. Display device 130 may include various display technologies. For example, display device 130 may include a light emitting panel or a light projection display, etc. Display panel generally includes displays such as liquid crystal (LCD), light emitting diode (LED), or organic light emitting diode (OLED). Light projection display generally includes digital light projection (DLP) display, or micro-electromechanical system (MEMS) display, or laser scanning display, etc.

[0033] Link 140 includes a user input device link, which may include one or more wired or wireless communication links. Link 140 may include various logical, physical or application programming interfaces (APIs). Example links may use metal, glass, optics, air, space or some other material as a transmission medium. Link 140 may use various protocols and formats, such as Universal Serial Bus (USB), Bluetooth (such as IEEE 802.15.1), WiFi (IEEE 802.11), infrared communication, Internet Protocol (IP), Ethernet, Transmission Control Protocol (TCP), Near Field Communication (NFC), other wired or wireless data interfaces, dedicated communication signaling, or some other communication formats, including combinations, improvements or variants thereof. Link 140 may include a direct link or may include an intermediate network, system, hub, router or device, and may include a logical network link transmitted through multiple physical links.

[0034] Link 141 may include one or more communication links, such as one or more video links including wireless or wired links. Link 141 may include various logical, physical or application programming interfaces (APIs). Example links may use metal, glass, optics, air, space or some other materials as transmission media. Link 141 may use various video protocols and formats, such as High Definition Multimedia Interface (HDMI), Video Electronics Standards Association Explicit Port (DisplayPort), Mobile Industry Processor Interface (MIPI) Alliance Display Serial Interface (DSI), Digital Video Interface (DVI), Wireless Display (WiDi), Miracast, Video Graphics Array (VGA) Interface, Wireless Display Interface, or other digital and analog video links, including interface types transmitted through other types of links. In some examples, video data may be transmitted through link 141 using various data protocols, such as IP, Ethernet, Hybrid Fiber Coaxial (HFC), WiFi, Bluetooth, other wired or wireless data interfaces, or some other communication formats, including combinations, improvements or variants thereof. Link 141 may include a direct link or may include an intermediate network, system or device, and may include a logical network link transmitted through multiple physical links. Link 141 may include routers, switches, bridges, traffic handling nodes, hubs, etc. for transmitting traffic among endpoints.

[0035] To further illustrate an enhanced system for making timing adjustments to reduce latency in a user input device, a Figure 3 . Figure 3 is a system diagram illustrating an input device 310 as an example of the input device 110 in an implementation. Figure 3 Including user input environment 300. Figure 5 discuss Figure 3 Example operation of the element.

[0036] First turn Figure 3 The environment 300 includes an input device 310 that communicates via a link 360. The input device 310 is shown as an exemplary input device including a game controller 301, a mouse 302, or a keyboard 303. However, the input device 310 may include the above-mentioned input devices. Figure 1 110, but variations are possible. Input device 310 includes input mechanism 320, control processor 330, firmware 331, storage system 340, and communication interface 350. Communication interface 350 communicates via link 360, which includes any of the link and protocol types discussed above for link 140, but variations are possible.

[0037] Input device 310 may be coupled to a host system via link 360, such as Figure 1 The host system 120 or Figure 4 The input device 310 may be a computer program product that is configured to display a display of the host system 410. The user may manipulate one or more input mechanisms 320 to control user interface elements presented by the host system. These user interface elements may include graphical and textual elements presented on a display of the host system. The control processor 330 may monitor signals associated with user manipulation of one or more input mechanisms 320 to generate user input states based on the user manipulations. As will be discussed below, the input device 310 includes enhanced circuitry, firmware, and processing for reducing latency of an application or game executed by a host system to which the input device is communicatively coupled.

[0038] The input mechanism 320 includes various user-oriented controls with corresponding electromechanical elements for receiving user manipulation and user input. The input mechanism 320 may include actuators, keys, buttons, direction pads (d-pads), triggers, rocker controls, paddle controls, wheel controls, pad controls, or other types of controls discussed herein. The input mechanism 320 has corresponding measuring elements for interpreting user manipulation into electrical signaling. Measuring elements (such as position sensing elements and transducer elements) may be included in the input mechanism 320, and may include potentiometers, position determination circuit systems, angle rotary transformers, photoelectric position sensors, magnetic sensors (such as Hall effect sensors or magnetoresistive sensors), encoders, capacitive sensors, inductive sensors, or other position measurement elements and position sensing elements. A / D conversion circuit systems may also be included to convert electrical signaling from analog measurement or positioning signals into digital representations of input data for input to the control processor 330. In some examples, the control processor 330 includes an A / D conversion circuit system and receives analog signals from the input mechanism 320. Sampling of the analog signal from the input mechanism 320 may occur according to a predetermined sampling rate, or according to a dynamic or adjustable sampling rate based on timing adjustments made by the control processor 330 .

[0039] The control processor 330 may include one or more microcontrollers, microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or discrete logic and associated circuitry, including combinations thereof. The control processor 330 may include or be paired with one or more analog-to-digital conversion elements to receive user input associated with the input mechanism 320. The control processor 330 may include data processing elements, data storage elements, and other elements. In some examples, the control processor 330 includes discrete logic and circuitry, while in other examples, the control processor 330 includes one or more portions of various microprocessors, software, firmware, storage, and related elements. Figure 3 In the embodiment, the control processor 330 executes the firmware 331, which may be stored in the storage system 340 until it is transferred to the memory element of the control processor 330 for execution.

[0040] Firmware 331 includes a mechanism interface 332 and a timing adjustment system 333. The mechanism interface 332 includes one or more digital interfaces configured to receive digital representations of movement data encoded by elements of the input mechanism 320. The mechanism interface 332 may be coupled to a logical or physical interface of the input mechanism 320, which may include one or more buffers, signal conditioners, signal de-jitter features, transceivers, wired or wireless interfaces, serial or parallel interfaces, and other elements.

[0041] The timing adjustment system 333 comprises a software or firmware service executed by the control processor and configured to adjust the delivery timing for the user input state provided via the link 360. The timing adjustment system 333 may comprise one or more computer executable instructions stored on a non-transitory storage medium, such as the storage system 340 or execution memory of the control processor 330, although variations are possible. Figure 3 In FIG. 3 , the timing adjustment system 333 includes an input sampler 334, a counter 335, and an output interface 336. Although a brief discussion of these elements of the timing adjustment system 333 is now discussed, in the following description, Figure 5 Further details can be found in the discussion of .

[0042] The input sampler 334 includes an interface for receiving digital representations of user manipulations of the input mechanism 320. These digital representations may be sampled at a particular sampling rate (such as by an A / D circuit system or other similar circuit system) and provided to the control processor 330 for delivery to the input sampler 334. In some examples, the input sampler 334 may adjust or control the sampling rate using one or more signals provided to the sampling circuit system (such as an enable signal or a trigger signal provided to the A / D circuit system). When the input sampler 334 adjusts the sampling rate or sampling period, the input sampler 334 may base the sampling rate on the transfer interval received via the link 360, and the timing offset on the transfer offset received via the link 360.

[0043] Counter 335 includes one or more processes for counting time cycles with respect to counting granularity. Counter 335 can maintain counts in one or more registers, memory locations, or other storage elements. These counts can include a timestamp indicating the current system time, or can instead include a monotonically increasing or decreasing count with counting granularity. Counter 335 can provide an indication of the current count for inclusion in the communication transmitted by link 360. Counter 335 can also track or generate a periodic timing signal, which is used to trigger the delivery of the output interface 336 to the user input state. The periodic timing signal can be based on the transmission interval received by link 360.

[0044] Output interface 336 includes an adjustable interface process for providing user input status via link 360 using communication interface 350. Output interface 336 may also adjust the delivery time of the user input status according to one or more timing parameters received via link 360. Although the sampling rate associated with the sampling circuit system may be different from the delivery interval and delivery offset provided in the one or more timing parameters, output interface 336 may compensate for these differences to ensure delivery of the user input status according to the one or more timing parameters. This may require buffering the user input status until it is delivered according to the one or more timing parameters, or indicating to input sampler 334 that the sampling circuit system is to be controlled according to the one or more timing parameters.

[0045] Communications interface 350 includes a transceiver, interface circuitry, control logic, and other elements for communicating digital representations of user input status and other information over link 360. Communications interface 350 may include circuitry, processing elements, and software for communicating over various protocols and formats, such as USB, Bluetooth, IP, Ethernet, TCP, WiFi, NFC, infrared communications, other wired or wireless data interfaces, or some other communications format, including combinations, modifications, or variations thereof.

[0046] As yet another example of adjustable user input state delivery techniques and implementations, the following is provided: Figure 4 . Figure 4 4 is a system diagram illustrating an adjustable user input state delivery environment 400. Environment 400 includes a host system 410, a display 461, and a game controller 301. Host system 410 and display 461 communicate via link 460. Host system 410 and game controller 301 communicate via link 360. Game controller 301 is Figure 3 An example of an input device 310. Figure 5 discuss Figure 4 Example operation of the element.

[0047] Host system 410 illustrates a control system that represents any system or collection of systems in which the various operational architectures, scenarios, and processes disclosed herein may be implemented. For example, host system 410 may be used to implement Figure 1 Any control element of the environment 400, such as the host system 120. Taking the example that the elements of the environment 400 include user systems such as a gaming system or a virtual / augmented reality system (which may include a virtual / augmented reality headset), the user may couple the example user system to Figure 4460. The selected display device is labeled as exemplary display 461 in FIG. The selected display device may include any associated physical monitor, television, projector, or may include a virtualized or networked display system. The display device may be coupled to the video interface circuitry 413 of the host system 410 via an associated link, which may include associated connectors, cables, interface electronics, and other circuitry represented by link 460.

[0048] Host system 410 includes processor 411, graphics core 412, video interface 413, memory system 414, communication interface 415 and storage system 416. The elements of host system 410 may include system on chip (SoC) elements, peripheral elements, discrete circuit systems and various other forms of computing equipment. Host system 410 may also include a housing, a shell or a chassis and any associated power supply elements, cooling elements and integrated user indicators, such as power and status indicators. The elements of host system 410 can communicate through an associated communication bus or interface and can receive power from an associated power system.

[0049] The host system 410 includes various software that instructs the operation of the processing elements of the host system 410 to operate as discussed herein. In one implementation, the software 420 is stored on a non-transient computer-readable medium (such as a storage system 416). The processor 411 loads and executes the software 420 from the storage system 416. The software 420 includes a user operating system 430, a device driver 431, a game application 432, a user application 433, and a user interface system 440, which represent the processes, services, and platforms discussed with reference to the previous figures. The user interface system 440 further includes an application programming interface (API) 441, an adjustment system 442, and an input interface 443. When executed by the processor 411 to provide enhanced user input services and other services, the software 420 instructs the processor 411 to operate as described herein at least for the various processes, operation scenarios, and sequences discussed in the aforementioned implementations. The host system 410 may optionally include additional devices, features, or functionality that are not discussed for the purpose of simplicity.

[0050] Still reference Figure 4 , processor 411 may include a microprocessor and processing circuitry that retrieves and executes firmware 420 from storage system 416. Processor 411 may be implemented in a single processing device, but may also be distributed across multiple processing devices, processing cores, or processing subsystems that cooperate in executing program instructions. Examples of processor 411 include general-purpose central processing units, special-purpose processors, and logic devices, as well as any other type of processing device, combination, or variation thereof.

[0051] Graphics core 412 includes various hardware components, such as processing stages, memory elements, and other pipeline processing elements for processing graphics data into rendered graphics presented on a display. Graphics core 412 can be provided in a graphics processing unit (GPU) having one or more graphics cores and associated memory elements. Graphics core 412 may include one or more levels or internal components that can process content / data into rendered images for display. These elements may include shaders, texture memories, texture units, tile maps, residency maps, and other various elements. Moreover, video interface 413 may be included in each part of graphics core 412, or may include elements for connecting to specific signaling and protocols adopted by link 460 and display 461 outside graphics core 412.

[0052] Graphics core 412 may generate / render video data in response to operations of host system 410, such as user interactions with user interface elements, media content playback, augmented / virtual reality operations, and other activities. This video data, including multiple time-organized video frames, may be stored in frame buffers for output to a display device. These frame buffers may be filled and cleared according to various recurring events associated with graphics core 412 and display 461, such as according to the VSYNC timing of display 461 or the frame rate of graphics generated by graphics core 412. Figure 4 In FIG. 4 , the recurring events may correspond to system events 450 including VSYNC 451 and frame timing 452 , but other events may be employed.

[0053] Memory system 414 may include random access memory (RAM) or other memory types that allow software 420 to be executed by processor 411. Memory system 414 and storage system 416 together may include a non-transitory data storage system, but variations are possible. Memory system 414 may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Memory systems 414 may each include additional elements capable of communicating with processor 411, such as a controller.

[0054] The communication interface 415 may include communication connections and devices that allow communication with various operating elements, user interface devices, peripheral devices, display devices, or communication networks. Examples of connections and devices that together allow inter-system communication may include system management interfaces, network interfaces, network interface cards, communication buses, antennas, RF circuit systems, transceivers, and other communication circuit systems. The connections and devices may communicate over a communication medium such as metal, glass, air, or any other suitable communication medium to exchange communications with other computing systems or system networks.

[0055] The storage system 416 may include any computer-readable storage medium that is readable by the processor 411 and capable of storing the software 420. The storage system 416 may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Examples of storage media include random access memory, read-only memory, magnetic disks, optical disks, flash memory, virtual and non-virtual memory, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other type of storage media.

[0056] In addition to computer-readable storage media, in some implementations, storage system 416 may also include computer-readable communication media through which at least some software 420 can communicate internally and externally. Storage system 416 may be implemented as a single storage device, but may also be implemented across multiple storage devices or subsystems that are co-located or distributed relative to each other. Storage system 416 may include additional elements, such as a controller, that can communicate with processor 411 or possibly other systems.

[0057] The software 420 may be implemented with program instructions and, among other functions, when executed by the processing system 411, may instruct the processor 411 to operate as described with reference to the various operating scenarios, sequences, and processes explained herein. For example, the software 420 may include program instructions for implementing enhanced user input services and other services.

[0058] Specifically, program instructions can include collaboration or otherwise interact to perform various components or modules of various processes and operation scenarios described herein. Various components or modules can be embodied in the instruction compiled or interpreted, or embodied in some other variant or combination of instruction. Various components or modules can be executed in a synchronous or asynchronous manner, serially or in parallel, in a single thread environment or a multithread environment, or according to any other suitable execution paradigm, variant or its combination. Except user interface system 440 or including user interface system 440, software 420 can include additional process, program or component, such as operating system software or other application software. Software 420 can also include the firmware that can be executed by processor 411 or some other form of machine-readable processing instruction.

[0059] In general, firmware 420, when loaded into processor 411 and executed, can transform the entirety of a suitable device, system, or apparatus (represented by host system 410) from a general purpose computing system to a specialized computing system customized to provide enhanced user input services and other services. In fact, the encoded software 416 on storage system 420 can transform the physical structure of storage system 416. In different implementations of this specification, the specific transformation of the physical structure can depend on various factors. Examples of such factors may include, but are not limited to, the technology of the storage medium used to implement storage system 416 and whether the computer storage medium is characterized as primary storage or secondary storage, as well as other factors.

[0060] For example, if the computer-readable storage medium is implemented as a semiconductor-based memory, then when encoding program instructions therein, the software 420 can transform the physical state of the semiconductor memory, such as by transforming the state of transistors, capacitors, or other discrete circuit elements that make up the semiconductor memory. A similar transformation can occur with respect to magnetic or optical media. Other transformations of the physical medium are also possible without departing from the scope of this specification, and the examples provided above are only for the convenience of this discussion.

[0061] The operating system 430 includes one or more operating elements for supporting the execution of user applications by the processor 411 of the host system 410. The operating system 430 may include various kernel units, task execution units, instruction fetch units, resource allocation units, resource sharing units, memory management units, and other various functions and units provided by a typical operating system. Example operating systems include Linux, Unix, And other similar operating systems that can provide a graphical user interface or a command line user interface. Along with these functions, the operating system 410 may include a device driver 431 or interwork with a device driver 431.

[0062] Device drivers 431 are each configured to dock with a specific type of communication interface or peripheral device. For example, a network interface may communicate via a network interface card (NIC) having a specific device driver, which is configured to communicate with the hardware-level circuit system of the NIC and provides an interface to the hardware for the operating system 430 and the application executed in the operating system 430. Graphics core 412 may have one or more device drivers that allow the operating system 430 to dock with graphics core 412. When game controller 301 or other user input devices are coupled by link 360, the link may have a specific device driver assigned to it, such as a USB or Bluetooth device driver, etc. The device driver communicates signals from the physical layer of the device or interface upwards through various software stack layers to ultimately present to the operating system 430 and one or more user applications.

[0063] Game applications 432 and user applications 433 each include user-provided applications that execute on processor 411 within operating system 430. Game applications 432 may include entertainment applications such as first-person shooter games (FPS), real-time strategy (RTS) games, side-scrolling games, puzzle games, card games, board games, etc. User applications 433 may include office productivity applications, system management applications, email applications, browser applications, media player applications, connectivity applications, utility programs, drawing / painting applications, and other various applications. It should be understood that game applications 432 and user applications 433 are merely representative of many different types of user applications that may be employed.

[0064] However, both the game application 432 and the user application 433 may receive input from one or more user input devices (such as Figure 4 The user input system 440 may be used to receive user input in the form of user input states (eg, the game controller 301 in the example embodiment of the present invention, and any other input device discussed herein). These applications may desire to have a reduced impact or influence on the user input state latency. The user interface system 440 is provided to reduce the impact or influence of the user input state latency at least with respect to the game application 432 and the user application 433.

[0065] The user interface system 440 is included in the software 440 and provides user input and output related operations of the host system 410. The user interface system 440 may receive input from an input device through an input interface 443, and may receive input from an application through an API 441. However, it should be understood that other interface systems may be provided, such as for controlling an input device, the host system 410, or an application, game, or platform executed by the host system 410.

[0066] Input interface 443 includes a circuit system or program instruction configured to receive the input device state transmitted by input device and other peripheral devices. Input interface 443 may include a data interface customized for a specific input device or peripheral device, and one or more buffers for storing the received input device state before being delivered to applications, games and other operating system parts. In the case of adopting a multi-layer protocol to transmit data between a device and a host system 410, input interface 443 may include parts of a network stack or a protocol stack. In some examples, parts of input interface 443 may be integrated into device driver 431 or operating system 430.

[0067] The adjustment system 442 includes a circuit system or program instructions that are configured to receive a request to adjust the delivery of an input device state to an application (such as a game application 442) and determine an adjustment instruction for delivery to the input device to change the delivery timing of the user input state. The adjustment system 442 may include a link hysteresis or latency determination unit that determines the link hysteresis between the user input device and the application. The adjustment system 442 may anticipate or adaptively formulate timing or delivery adjustments based on the user input request timing detected for one or more applications. In this way, the application does not need to request a change in input delivery for enhanced user input state delivery. The adjustment system 442 may also periodically adjust the user input state delivery timing to reduce the impact of drift over time between the device / host clock or input device interface.

[0068] API 441 includes circuitry or program instructions configured to interface with applications to request synchronization associated with various system events. This typically takes the form of an application indicating an event identifier (event ID) and a time offset from the event, but other configurations are possible. API 441 may also receive count information or a timing indicator initially transmitted by the application indicating when the input device initially transmitted the user input state. In other examples, API 441 may receive an indication of the time of receipt of the user input state from which the adjustment system 442 may calculate the link hysteresis between the user input device and the application.

[0069] Now turn to Figure 3 and Figure 4 A detailed description of example operation of the elements of the timing diagram 500 is provided in Figure 5 For Figure 5 In the example of FIG. 3 , game controller 301 is coupled to host system 410 via link 360. It should be understood that other user input devices and associated links may be alternatively employed using similar operations as described below. In many examples, host system 410 includes many concurrent user input devices, and each of these user input devices may be individually adjusted or calibrated for user input state delivery.

[0070] Figure 5 There are several activity phases indicated on the right side of the diagram. These activity phases can be Figure 5 The first activity phase includes a hysteresis determination phase, in which the user interface system 440 determines the hysteresis or latency between an input device (such as a game controller 301) and the host system 410. The second activity phase includes a game activity phase, in which the game application 432 and the graphics core 412 interact to process user input and render graphics for delivery to the display 461. This second phase generally runs through Figure 5 4. The third activity stage comprises a delivery adjustment stage, in which the user interface system 440 may adjust the communication / delivery of the user input state from the game controller 301 to the host system 410. The fourth activity stage comprises an enhanced delivery stage, in which adjustments to the communication / delivery of the user input state are made by the game controller 310. The fourth activity stage will generally coincide with the in-game activity indicated by the second activity stage.

[0071] exist Figure 5 4, an application executing in operating system 430, such as game application 432, determines when it wants to receive user input from a user input device, which may vary from application to application. For example, game application 432 may want to read a game controller state two milliseconds after the start of each new graphics frame being rendered by graphics core 412 so game application 432 can run an associated game world simulation and prepare the next graphics frame for rendering. In another example, an application controlling an animation on a large LED array display may want to read user input in a manner and at a rate associated with the animation.

[0072] During the execution of the game application 432 in the second activity stage, the graphics data or rendering instructions are provided by the game application 432 to the graphics core 412 for rendering into a graphics frame that can be displayed on the display 461. The graphics core 412 or other elements of the host system 410 will generally have one or more repetitive system events that can correspond to the activity of the graphics data rendering or display. As mentioned above, these may include a loop frame rendering indicator, a VSYNC indicator or other similar indicators. Typically, the game application 432 will have visibility to these various indicators, and various game events or graphics rendering activities can be triggered based on the appearance of these indicators for repetitive system events. The game application 432 is thus aware of the rendering epochs and associated scheduling of game events and graphics events. The game application 432 can pre-render frames or buffer frames to anticipate game events that occur in response to user control. These pre-rendered or buffered frames may or may not be displayed, depending on the actions taken by the user regarding user input.

[0073] However, game applications 432 and other similar applications generally expect to associate changes in user input states with specific frames on which user-initiated actions occur. For example, a user pressing a button on a joystick or selecting a specific direction may affect the on-screen elements rendered for the user. The delay from the user actuating a specific user control (such as a button) to the on-screen elements being displayed to the user may cause a perceptible hysteresis. This may reduce the usability of the game and the enjoyment of the user and other disadvantages. Thus, game applications 432 want to synchronize user inputs with the graphics frames on which the user inputs are reflected on the screen at least in part, thereby reducing the user's perception of input hysteresis or waiting time. The rendering or pre-rendering of graphics frames is generally temporally related to one or more repetitive system events, and game applications 432 may use these one or more repetitive system events as a baseline or trigger from which user input states may be delivered to game applications 432. Game applications 432 may determine an increment or offset from one or more repetitive system events, and indicate that the user input state should be delivered according to this offset from the repetitive system events. This increment or offset can advantageously provide consistent application of user input state to rendered frames by allowing the game application 432 to at least synchronize the user input state with specific game activities (such as elements in the rendered frame). Thus, the game application 432 can be provided with the "freshest" user input state available not only by triggering the delivery of user input state to the game application 432 based on repeated system events, but also by triggering the sampling and transmission of user input state at the input device level. The upstream handling of such triggering by the user input device is coordinated by the user input system 440.

[0074] In order to indicate such timing, the delivery adjustment stage may occur, wherein the game application 432 may indicate a repeated system event via API 441. This repeated system event may be used to trigger the delivery of the user input state to the game application 432. The user interface system 440 may also have visibility to these repeated system events in the host system 410, and may determine the timing adjustment relative to the same event. In addition to the indication of the repeated system event, the offset with the repeated system event may be indicated by the game application 432, which may allow the game application 432 to have time to process the user input state and apply the user input state to a specific rendered frame or initiate a game-related activity. Thus, the game application 432 indicates that the game application 432 wants to receive the user input state to better synchronize the user input state with the target pace or target rhythm of the elements on the screen. The game application 432 may advantageously optimize the waiting time of the user input state to be consistent with the rendering event, the display screen electrical timing or other repeated system events. Once the API 441 receives one or more indications of repeated system events and offsets from the game application 432, the user interface system 440 may responsively initiate the timing adjustment stage. The one or more indications of recurring system events and offsets include a target pace or a target cadence that also indicates offsets from the recurring system events.

[0075] In an alternative scheme for receiving one or more indications of a repetitive system event or an offset from a repetitive system event, the adjustment system 442 can perform a predictive or adaptive adjustment process. In this adaptive adjustment process, the adjustment system 442 checks the calling pattern of the game application 432, and determines whether the game application 432 is requesting the user input state in a predictable pattern. If such a pattern can be found, the adjustment system 442 requests input synchronization on behalf of the game application 432, and the game application 432 does not have to request such input synchronization. This pattern can be used to determine the delivery of the game application 432 from its baseline user input state and the related repetitive system events of any offset from the repetitive system event.

[0076] The user interface system 440 determines the timing and scheduling for user input transmission / delivery based on the indication received from the game application 432 or based on adaptive or predictive determination. However, a part of this timing and scheduling for user input transmission / delivery may depend on the transmission hysteresis or transmission waiting time experienced between transmitting the user input state from the game controller 301 and being received by the game application 432. The hysteresis determination stage can thus be initiated to determine the transmission hysteresis or transmission waiting time. Moreover, the hysteresis determination stage can be periodically initiated to account for drift and link changes over time. When adopting a wireless link, this periodic drift adjustment may be particularly useful.

[0077] In the hysteresis determination stage, different hysteresis determination processes may occur. In the first example hysteresis determination process, the game application 432 receives the timing indicator corresponding to the time when the game controller 301 transmits the user input state transmitted by the game controller 301. The timing indicator may include a count generated by the game controller 301 and accompanied or included in the communication including the monotonically changed user input state transmitted by the user input device. Once the game application 432 receives the user input state with the accompanying count, the game application 432 can indicate the count and time of the reception of the user input state to the user interface system 440. In an example, the game application 432 marks or updates the data structure or data cookie visible to the user input system 440. The user input system 440 can read the data structure to determine the count or timestamp associated with the time when the game controller 301 initially transmits the user input state and the game application 432 receives the user input state. The adjustment system 442 determines the time difference indicated by the count and timestamp to determine the transmission hysteresis or waiting time between the transmission of the user input state by the game controller 301 and the reception of the game application 432.

[0078] In a second example hysteresis determination process, a timing indicator (such as the count mentioned above) may be passed by the game controller 301 for delivery to the user interface system 440 or the adjustment system 442. The adjustment system 442 determines a time or timestamp associated with the receipt of the timing indicator passed by the game controller 301. The adjustment system 442 may derive a delivery hysteresis or wait time from the time or timestamp. In this second example process, the adjustment system 442 may also determine a further wait time between the receipt of the timing indicator by the user interface system 440 and the game application 432 to determine the total delivery hysteresis between the game controller 301 and the game application 432. When the count discussed above is employed, the adjustment system 442 may be configured to handle rollovers that occur in the counters received from the game controller 432.

[0079] Once the delivery hysteresis is determined, the adjustment system 442 may proceed to a delivery adjustment phase. In the delivery adjustment phase, the adjustment system 442 processes one or more indications of repetitive system events and offsets received via the API 441 and the delivery hysteresis to determine one or more timing parameters for delivery of the user input state. The one or more timing parameters include a delivery offset and a delivery interval for adjustment of delivery of consecutive user input states from the game controller 432. The delivery offset is determined based at least in part on the offsets from the repetitive system events received via the API 441 and the delivery hysteresis determined during the hysteresis determination process.

[0080] In one example, the transfer offset includes the offset of the repeated system events indicated by the game application 432 and the transfer hysteresis for the game controller 301. The transfer offset includes the indication that the game controller 301 is informed to accelerate or delay the delivery of its next input state in units of + / - microseconds. This allows the real-time correction of the clock drift on the game controller 301, and allows to reduce the influence of the transfer hysteresis to ensure that the game application 432 receives the user input state about the appropriate timing of the repeated system events. Adjustment system 442 can calculate two values, until the microsecond number of the next input state should be sent from the input device and the interval in microseconds that the subsequent user input state traffic should be sent from the device. These values ​​are sent to the game controller 301 via link 360 subsequently.

[0081] The timing adjustment system 333 of the game controller 360 then responsively changes the delivery pace / rhythm and timing for the user input state. The enhanced delivery phase includes delivering the user input state from the game controller 301 to the host system 410 once the game controller adjusts the user input state traffic as described above. Figure 5 As can be seen in FIG, the game controller 301 uses the delivery offset to adjust the delivery time of the first user input state, and the delivery interval is used to perform a cyclic time delivery of subsequent user input states. Figure 5 4, but it should be understood that such cycles of user input state delivery may continue during operation of the host system 410 and the game application 432. Periodic iterations of the hysteresis determination phase and the delivery adjustment phase may be performed to correct for drift in the clock and link timing among the game controller 301 and the game application 432.

[0082]

[0066] Certain inventive aspects can be appreciated from the foregoing disclosure, and following are various examples of these inventive aspects.

[0083] Example 1: An apparatus comprising one or more computer-readable storage media, a processing system operably coupled to the one or more computer-readable storage media, and a user input interface comprising program instructions stored on the one or more computer-readable storage media. Upon being read and executed by the processing system, the program instructions instruct the processing system to at least identify a target pace for delivery of a user input state to an application, determine one or more timing parameters for delivery of the user input state from a user input device based at least on the target pace, and indicate the one or more timing parameters to the user input device, wherein the user input device responsively delivers the user input state according to the one or more timing parameters.

[0084] Example 2: The user input interface of Example 1, wherein the one or more timing parameters include a delivery offset and a delivery interval for adjusting the delivery of continuous user input states from a user input device, wherein the delivery offset is based at least in part on a hysteresis between transmission of the user input state by the user input device and receipt of the user input state by the user input interface.

[0085] Example 3: The user input interface of Example 1-2, comprising further program instructions, which, based on execution by the processing system, instruct the processing system to: receive an indication of a target pace at least from an application, wherein the indication of the target pace includes an indication of repetitive events within a host system executing the application and a timing offset from the repetitive events.

[0086] Example 4: The user input interface of Examples 1-3, wherein the recurring event comprises at least one of: a display refresh indicator, a display vertical synchronization indicator, or a graphics frame rate timing characteristic of a graphics process of the host system.

[0087] Example 5: The user input interface of Examples 1-4, comprising further program instructions, which, upon execution by the processing system, instruct the processing system to at least receive a timing indicator transmitted by the user input device corresponding to the time at which the user input device transmits a user input state.

[0088] Example 6: The user input interface of Examples 1-5, comprising further program instructions, which, based on execution by the processing system, instruct the processing system to: at least determine the hysteresis between the transmission of the user input state by the user input device and the reception of the user input state by the user input interface; and determine the one or more timing parameters based on the target pace and the hysteresis.

[0089] Example 7: The user input interface of Examples 1-6, wherein the timing indicator comprises a monotonically changing count generated by the user input device and included in the communication comprising the user input state communicated by the user input device.

[0090] Example 8: The user input interface of Examples 1-7, comprising further program instructions that, upon execution by the processing system, instruct the processing system to: at least monitor the timing associated with one or more requests for user input status from an application, and determine a target pace based at least on the timing associated with the one or more requests, wherein the target pace includes at least a timing offset from recurring events within a host system executing the application.

[0091] Example 9: The user input interface of Examples 1-8, wherein the user input device comprises at least one of: a game controller, a keyboard, a mouse, a trackpad, an adaptive controller, a virtual reality controller, an augmented reality controller, and a touch screen.

[0092] Example 10: A method of operating a user input interface on a host system, the method comprising: identifying a target pace for delivery of user input state to an application, determining one or more timing parameters for transfer of the user input state from a user input device based at least on the target pace, and indicating the one or more timing parameters to the user input device, wherein the user input device responsively transfers the user input state according to the one or more timing parameters.

[0093] Example 11: The method of Example 10, wherein the one or more timing parameters include a delivery offset and a delivery interval for adjusting the delivery of continuous user input states from a user input device, wherein the delivery offset is based at least in part on a hysteresis between transmission of the user input state by the user input device and receipt of the user input state by the user input interface.

[0094] Example 12: The method of Examples 10-11, further comprising receiving an indication of a target pace from the application, wherein the indication of the target pace comprises an indication of a recurring event within a host system executing the application and a timing offset from the recurring event.

[0095] Example 13: The method of Examples 10-12, wherein the recurring event comprises at least one of: a display refresh indicator, a display vertical synchronization indicator, or a graphics frame rate timing characteristic of a graphics process of the host system.

[0096] Example 14: The method of Examples 10-13, further comprising receiving a timing indicator communicated by the user input device corresponding to a time at which the user input device communicated the user input state.

[0097] Example 15: The method of Examples 10-14, further comprising determining a hysteresis between transmission of a user input state by a user input device and receipt of the user input state by a user input interface, and determining the one or more timing parameters based on the target pace and the hysteresis.

[0098] Example 16: The method of Examples 10-15, wherein the timing indicator comprises a monotonically changing count generated by the user input device and included in the communication comprising the user input state communicated by the user input device.

[0099] Example 17: The method of Examples 10-16 further includes monitoring the timing associated with one or more requests for user input state from the application, and determining a target pace based at least on the timing associated with the one or more requests, wherein the target pace corresponds at least to repetitive events within a host system executing the application and to timing offsets with the repetitive events.

[0100] Example 18: The method of Examples 10-17, wherein the user input device includes at least one of the following: a game controller, a keyboard, a mouse, a trackpad, an adaptive controller, a virtual reality controller, an augmented reality controller, and a touch screen.

[0101] Example 19: A user interface system, comprising an application interface configured to identify a preferred delivery timing of a user input state to an application, wherein the preferred delivery timing corresponds to at least a recurring event within a host system executing the application and a timing offset to the recurring event. The user interface system comprises a timing adjustment system configured to determine one or more timing parameters for transfer of the user input state from a user input device based at least on the preferred delivery timing. The user interface system comprises an input interface that indicates the one or more timing parameters to a user input device, wherein the user input device responsively transfers the user input state according to the one or more timing parameters.

[0102] Example 20: The user interface system of Example 19, wherein the timing adjustment system is configured to determine a hysteresis between transmission of a user input state by a user input device and receipt of the user input state by a user input system, and to determine the one or more timing parameters based on the hysteresis and the preferred delivery timing, the one or more timing parameters including a delivery offset and a delivery interval for adjusting the delivery of continuous user input states from the user input device.

[0103] The functional block diagrams, operation scenarios and sequences and flow charts provided in the accompanying drawings represent exemplary systems, environments and methods for executing the novel aspects of the present disclosure. Although for the purpose of explaining concisely, the methods included herein can be shown in the form of functional diagrams, operation scenarios or sequences, or flow charts and can be described as a series of actions, it can be understood and appreciated that each method is not limited by the order of these actions, because according to the present invention, some actions can occur in a different order from that shown and described herein and / or concurrently with other actions. For example, those skilled in the art will understand and appreciate that the method can be alternatively represented as a series of mutually related states or events, such as in the form of a state diagram. In addition, not all actions explained in the method are necessary for novel implementation.

[0104] The description and drawings included herein depict specific implementations for teaching those skilled in the art how to make and use the best options. For the purpose of teaching creative principles, some traditional aspects have been simplified or ignored. Those skilled in the art will appreciate that variations from these implementations also fall within the scope of the present invention. Those skilled in the art will also appreciate that the various features described above can be combined in various ways to form multiple implementations. As a result, the present invention is not limited to the specific implementations described above, but is only limited by the claims and their equivalents.

Claims

1. A device comprising: one or more computer-readable storage media; a processing system operably coupled to the one or more computer-readable storage media; as well as The user input interface includes program instructions stored on the one or more computer-readable storage media, and the program instructions instruct the processing system to perform at least the following operations upon being read and executed by the processing system: Identifying a target pace for delivery of user input status to the application; determining one or more timing parameters for communication of the user input state from a user input device based at least on the target pace; as well as The one or more timing parameters are indicated to the user input device, wherein the user input device responsively communicates the user input state in accordance with the one or more timing parameters.

2. The device according to claim 1, wherein: The one or more timing parameters include a delivery offset and a delivery interval for adjusting delivery of consecutive user input states from the user input device, wherein the delivery offset is based at least in part on a hysteresis between transmission of the user input state by the user input device and receipt of the user input state by the user input interface.

3. The apparatus of claim 1, comprising further program instructions, wherein upon execution by the processing system, the further program instructions instruct the processing system to at least perform the following operations: receiving an indication of the target pace from the application; The indication of the target pace includes an indication of repetitive events in a host system executing the application and a timing offset from the repetitive events.

4. The device as claimed in claim 3, wherein: The recurring event includes at least one of: a display refresh indicator, a display vertical synchronization indicator, or a graphics frame rate timing characteristic of a graphics process of the host system.

5. The apparatus of claim 1, comprising further program instructions, wherein upon execution by the processing system, the further program instructions instruct the processing system to at least perform the following operations: A timing indicator communicated by the user input device is received corresponding to a time at which the user input device communicated the user input state.

6. The apparatus of claim 5, comprising further program instructions, which upon execution by the processing system instruct the processing system to at least perform the following operations: determining a hysteresis between transmission of the user input state by the user input device and receipt of the user input state by the user input interface; and The one or more timing parameters are determined based on the target pace and the hysteresis.

7. The device according to claim 5, wherein: The timing indicator comprises a monotonically changing count generated by the user input device and included in communications comprising the user input state communicated by the user input device.

8. The apparatus of claim 1, comprising further program instructions, which upon execution by the processing system instruct the processing system to at least perform the following operations: monitoring a timing associated with one or more requests from the application for a user input state; and The target pace is determined based at least on the timing associated with one or more requests, wherein the target pace includes at least a timing offset from a recurring event within a host system executing the application.

9. The device of claim 1, wherein: The user input device includes at least one of the following: a keyboard, a mouse, a touchpad, an adaptive controller, a virtual reality controller, an augmented reality controller, and a touch screen.

10. The device of claim 1, wherein: The user input device comprises a game controller.

11. A method of operating a user input interface on a host system, the method comprising: Identifying a target pace for delivery of user input status to the application; determining one or more timing parameters for communication of the user input state from a user input device based at least on the target pace; as well as The one or more timing parameters are indicated to the user input device, wherein the user input device responsively communicates the user input state in accordance with the one or more timing parameters.

12. The method of claim 11, wherein: The one or more timing parameters include a delivery offset and a delivery interval for adjusting delivery of consecutive user input states from the user input device, wherein the delivery offset is based at least in part on a hysteresis between transmission of the user input state by the user input device and receipt of the user input state by the user input interface.

13. The method of claim 11, further comprising: receiving an indication of the target pace from the application; The indication of the target pace includes an indication of repetitive events within the host system executing the application and a timing offset from the repetitive events.

14. The method of claim 13, wherein: The recurring event includes at least one of: a display refresh indicator, a display vertical synchronization indicator, or a graphics frame rate timing characteristic of a graphics process of the host system.

15. The method of claim 11, further comprising: A timing indicator communicated by the user input device is received corresponding to a time at which the user input device communicated the user input state.

16. The method of claim 15, further comprising: determining a hysteresis between transmission of the user input state by the user input device and receipt of the user input state by the user input interface; as well as The one or more timing parameters are determined based on the target pace and the hysteresis.

17. The method of claim 15, wherein: The timing indicator comprises a monotonically changing count generated by the user input device and included in communications comprising the user input state communicated by the user input device.

18. The method of claim 11, further comprising: monitoring a timing associated with one or more requests for a user input state from the application; as well as The target pace is determined based at least on the timing associated with one or more requests, wherein the target pace corresponds to at least recurring events within the host system executing the application and a timing offset to the recurring events.

19. The method of claim 11, wherein: The user input device includes at least one of the following: a keyboard, a mouse, a touchpad, an adaptive controller, a virtual reality controller, an augmented reality controller, and a touch screen.

20. The method of claim 11, wherein: The user input device comprises a game controller.

21. A user interface system, comprising: an application interface configured to identify a delivery timing of a user input state to an application, wherein the delivery timing corresponds to at least a recurring event within a host system executing the application and a timing offset to the recurring event; a timing adjustment system configured to determine one or more timing parameters for communication of the user input state from a user input device based at least on the delivery timing; as well as An input interface indicates the one or more timing parameters to the user input device, wherein the user input device responsively communicates the user input state according to the one or more timing parameters.

22. The user interface system of claim 21, wherein the timing adjustment system is configured to: determining a hysteresis between transmission of the user input state by the user input device and receipt of the user input state by a user input system; and The one or more timing parameters are determined based on the hysteresis and the delivery timing, the one or more timing parameters comprising a delivery offset and a delivery interval for adjusting delivery of consecutive user input states from the user input device.

23. A computer-readable storage medium having instructions, which, when executed, cause a machine to perform the method of any one of claims 11-20.

24. A computer system comprising means for executing the method as claimed in any one of claims 11 to 20.

Images