Method and system for generating and sending auxiliary output

Abstract

Aspects of the present disclosure relate to systems and methods for generating auxiliary outputs and sending the auxiliary outputs from a server to a client device, the method comprising generating an auxiliary output based on data including measurements of time between user inputs generated at a client device, wherein the auxiliary output utilizes the measurements of time between the user inputs to implement the inputs with a server that uses the same input time differences as the inputs generated at the client device; and sending the auxiliary output from the server to the client device.

Description

[0001] This application is a divisional application of the Chinese invention patent application with the application date of March 4, 2016, application number 201680012906.2, invention name “Cloud Input Channel Management”, and applicant being Sony Interactive Entertainment USA LLC. Technical Field

[0002] The present disclosure relates to data transmission via a network. In particular, aspects of the present disclosure relate to systems and methods for reducing the impact of latency experienced between a server and a client device configured to operate on a network. Background Art

[0003] With the increasing popularity of digital streaming services and various cloud-based computing solutions, the ability to quickly and accurately transfer large amounts of data between remote devices is an important task. Sending digital data to a destination system via a shared resource of a network, such as the Internet, a wide area network (WAN), or a local area network (LAN), typically involves configuring the data into formatted blocks called packets, which may have fixed or variable lengths. Each data packet typically includes: a payload or body, which has the underlying client data to be delivered to the destination; and certain supplemental information for routing and control purposes, which is typically at least partially contained within the header of the data packet. Broadly speaking, the network, the sending system, and the receiving system can use this supplemental information to ensure proper routing and delivery of the payload to the intended destination.

[0004] However, digital streaming services and cloud-based computing solutions may experience limitations in the quality and bandwidth of the network established or used during the transfer of data between remote devices when utilizing latency-sensitive applications (such as video games). These limitations may cause delays in data transmission and may therefore cause latency, which often causes discontinuities during the use of the application. Although client devices will attempt to achieve the lowest latency through various methods, inevitably each client device will experience different amounts of latency due to differences in factors such as the decoding speed of the transmitted data, the rendering rate, input polling, or even the client's network connection.

[0005] In addition, the usually inevitable result of transmitting data on a packet switching network is packet loss, which occurs when one or more data packets fail to correctly reach their destination. Packet loss can occur due to various factors, including channel congestion, signal degradation and other reasons. In order to prevent certain network conditions that cause packet loss from occurring while also effectively using available bandwidth in the network channel, various congestion control techniques have been developed. In addition, there are a series of transport protocols that can incorporate tools to handle packet loss, and the specific method for handling packet loss when packet loss occurs depends on the specific transport protocol used during data transfer. In general, these transport protocols can be divided into two types, namely, reliable protocols and unreliable protocols, each of which has certain tradeoffs, and the specific choice of the protocol used in any case can depend on the nature of the data transfer.

[0006] Reliable protocols incorporate the following guarantees: each data packet is delivered to its destination in order, and lost packets are retransmitted in the event of packet loss. Reliable protocols are often, but not always, connection-oriented protocols, and delivery guarantees are usually accomplished by establishing a reverse channel from the receiver back to the sender for a specific communication session, which the receiver can use to send some type of confirmation receipt to verify that the packet was delivered correctly. The sender can use these confirmations to guide the retransmission process when it indicates that the data packet failed to arrive at its destination correctly. A popular and well-known example of a reliable protocol is the Transmission Control Protocol (TCP), which is also connection-oriented. Reliable protocols (such as TCP) are well suited for tasks where accurate delivery of data is a primary concern and a certain amount of delay can be tolerated in order to verify that the data packets are delivered correctly, such as sending text-based emails, digital content downloads, and media streaming services where audio / video can be buffered at the destination system. Unfortunately, the data verification properties and data retransmission introduce significant overhead, making many reliable protocols undesirable for time-critical applications, including real-time data delivery, such as real-time audio and / or video streaming, online video gaming, and Internet telephony.

[0007] In contrast, unreliable protocols typically forgo a certain type of data delivery verification for a particular packet as described above, and are typically characterized by the fact that they neither guarantee that each packet reaches its destination nor that the packets are delivered in the correct order. Unreliable protocols are typically, but not always, connectionless, and typically do not establish fixed channels during any particular communication session. Instead, each data packet may be routed independently based on supplementary information contained in each data packet. A popular and well-known example of an unreliable protocol is the User Datagram Protocol (UDP), which is also connectionless. Since unreliable protocols such as UDP relatively reduce overhead by abandoning the reliability properties mentioned above, unreliable protocols are better suited for time-sensitive applications where minimizing latency is a primary concern, such as real-time applications mentioned above.

[0008] Importantly, network conditions typically vary over time, causing the maximum bit rate available to a sender on a network channel to vary based on the current load on the channel. When a sender system attempts to send data packets at a bit rate that exceeds the currently available bandwidth of the channel, it may cause a congestion condition that triggers severe packet loss in response. This may be tolerable in less time-sensitive applications involving reliable data transmission (such as TCP) because retransmission of lost data is guaranteed; however, this may be unacceptable in many real-time applications and other applications involving unreliable transmission because packet loss may reach a certain level that the receiver cannot reconstruct the lost data, thereby causing undesirable results, such as loss of signal. On the other hand, when the maximum available bit rate is changed to far exceed the bit rate provided by the sender, this is also undesirable because the full transmission capacity of the network channel is inefficiently utilized, and therefore the signal quality at the receiving side may be unnecessarily weak.

[0009] Unfortunately, it is a significant challenge to use unreliable protocols to deliver data in a way that effectively utilizes the available bandwidth of a network channel without causing congestion conditions that result in unacceptable packet loss. Traditional congestion control techniques are generally only suitable for reliable protocols, such as TCP, which build feedback to the sender into the transport layer; but are inefficient for many unreliable protocols (such as UDP), which generally lack the necessary feedback unless added independently at the transport layer by the client. In addition, congestion control or congestion avoidance algorithms designed for TCP or other reliable protocols are generally not fast real-time streaming applications or and may not be suitable for many data delivery applications involving unreliable protocols, because the exponential reduction in bit rate caused by congestion may therefore affect the quality of the real-time signal too much. In addition, although packet loss caused by increasing the bit rate to the level of congestion may be tolerable in less time-sensitive applications that use TCP or other reliable protocols to retransmit data, packet loss may be unacceptable in many real-time applications because it will cause the receiver to be unable to reconstruct the data.

[0010] Therefore, there is a need in the art to find alternative means for reducing the unique latency constraints of the client, which means are also suitable for use with UDP and other unreliable transport protocols when the data being communicated is encrypted.Aspects of the present disclosure arise against this background. Summary of the invention

[0011] According to certain implementations of the present disclosure, a method performed on a server configured to operate on a network may include receiving client device information from a client device via a network connection by the server. The client device information may include latency data. The method may include determining an auxiliary output that takes into account the client device latency information. The method may also include sending the auxiliary output from the server to the client device. The client device information or the auxiliary output may include an encrypted data packet stream.

[0012] According to certain specific implementations of the present disclosure, a sender computing system may include at least one processor unit and at least one memory unit coupled to the at least one processor unit. The at least one processor unit and the at least one memory unit may be configured to perform a method. The method may include receiving client device information from a client device through the server via a network connection. The client device information may include latency data. The method may include determining an auxiliary output that takes into account the client device latency information. The method may also include sending the auxiliary output from the server to the client device. The client device information or the auxiliary output may include an encrypted data packet stream.

[0013] According to certain implementations of the present disclosure, a non-transitory computer-readable medium may contain computer-readable instructions included therein. The computer-readable instructions may be configured to implement a method when executed. The method may receive client device information from a client device through the server via a network connection. The client device information may include latency data. The method may include determining an auxiliary output that takes into account the client device latency information. The method may also include sending the auxiliary output from the server to the client device. The client device information or the auxiliary output may include an encrypted data packet stream.

[0014] According to certain specific implementations of the present disclosure, a method in a server configured to operate on a network may include generating an auxiliary output based on data including a measurement of time between user inputs generated at a client device, wherein the auxiliary output utilizes the measurement of time between user inputs to implement input with the server, and the server uses the same input time difference as the input generated at the client device; and sending the auxiliary output from the server to the client device.

[0015] According to some specific implementations of the present disclosure, a system may include: a processor, and a memory, the memory coupled to the processor, wherein the processor is configured to execute a method, the method may include generating an auxiliary output based on data including a measurement of time between user inputs generated at a client device, wherein the auxiliary output utilizes the measurement of time between user inputs to implement input with a server, the server using the same input time difference as the input generated at the client device; and sending the auxiliary output from the server to the client device.

[0016] According to certain specific implementations of the present disclosure, a non-transitory computer-readable medium having processor-executable instructions therein, wherein execution of the instructions by the processor causes the processor to implement a method that may include generating an auxiliary output based on data including a measurement of time between user inputs generated at a client device, wherein the auxiliary output utilizes the measurement of time between user inputs to implement inputs with a server, the server using the same input time difference as the input generated at the client device; and sending the auxiliary output from the server to the client device.

[0017] According to certain specific implementations of the present disclosure, a method may include sending client device information from a client device to a server via a network connection, wherein the client device information includes a measurement of time between user inputs; receiving an auxiliary output from the server, wherein the auxiliary output takes into account the measurement of time between user inputs, wherein the auxiliary output utilizes the measurement of time between user inputs to implement input with the server, and the server uses the same input time difference as the input generated at the client device; and implementing assistance corresponding to the auxiliary output by the client device.

[0018] According to some specific implementations of the present disclosure, a system may include: a processor, and a memory, the memory being coupled to the processor. The processor is configured to execute a method, the method may include sending client device information from a client device to a server via a network connection, wherein the client device information includes a measurement of time between user inputs; receiving an auxiliary output from the server, wherein the auxiliary output takes into account the measurement of time between user inputs, wherein the auxiliary output utilizes the measurement of time between user inputs to implement input with the server, the server using the same input time difference as the input generated at the client device; and implementing assistance corresponding to the auxiliary output by the client device.

[0019] According to some specific implementations of the present disclosure, a non-transitory computer-readable medium includes processor-executable instructions, wherein the execution of the instructions by the processor causes the processor to implement a method, which may include sending client device information from a client device to a server via a network connection, wherein the client device information includes a measurement of time between user inputs; receiving an auxiliary output from the server, wherein the auxiliary output takes into account the measurement of time between user inputs, wherein the auxiliary output uses the measurement of time between user inputs to implement input with the server, and the server uses the same input time difference as the input generated at the client device; and implementing assistance corresponding to the auxiliary output by the client device. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The teachings of the present disclosure may be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:

[0021] Figure 1 Flowchart of an exemplary auxiliary output technique from a client device according to certain aspects of the present disclosure.

[0022] Figure 2 is a block diagram of an exemplary system according to certain aspects of the present disclosure.

[0023] Figure 3 Flowchart of an exemplary auxiliary output technique from the client side according to certain aspects of the present disclosure.

[0024] Figure 4 Flowchart of an exemplary auxiliary output technique from a server side according to certain aspects of the present disclosure. DETAILED DESCRIPTION

[0025] Although the following detailed description contains many specific details for the purpose of illustration, those skilled in the art will appreciate that many changes and modifications to the following details are within the scope of the invention. Therefore, the illustrative specific implementations of the present disclosure described below are set forth, but in no way lose the generality of the claimed invention and will not impose limitations on the claimed invention.

[0026] Introduction

[0027] Aspects of the present disclosure relate to systems and methods for reducing the impact of latency experienced between a server and client devices configured to operate on a network.

[0028] According to certain aspects, one or more servers configured to operate on a network may receive client device information from one or more client devices via a network connection through the server. The client device information may include latency data. The one or more servers may determine an auxiliary output that takes the client device latency information into account. The server may also send the auxiliary output from the server to the client device. The client device information or the auxiliary output may include an encrypted data packet stream.

[0029] Further details

[0030] Now turn to Figure 1 , depicts an illustrative example of how auxiliary output may be provided to a client device in response to received client device latency data according to certain aspects of the present disclosure. Specifically, Figure 1 Depicted is an exemplary process flow for providing auxiliary output to a client device in response to received client device latency data according to certain embodiments of the present disclosure. It should be noted that Figure 1 An exemplary method of providing an auxiliary output in may have the following Figure 2 and Figure 3 It is also important to point out that Figure 1 The examples of are merely simplified examples for the purpose of illustrating only certain aspects of how auxiliary outputs may be provided in accordance with the present disclosure.

[0031] At the outset, it is important to point out that in this example, the auxiliary output is provided to a client device that is experiencing latency. However, in alternative implementations, the auxiliary output may be sent or triggered under other circumstances. Figure 1 As shown in , how auxiliary output 148 is provided in response to client device information 132 received by the server from one or more client devices may generally depend on whether the client device information 132 indicates that latency exists (as indicated at 146).

[0032] According to certain aspects, in the case where the client device information 132 indicates that the client device is experiencing latency 146, the server may determine that an auxiliary output should be provided to the client device 144 in order to help the user overcome the effects of latency. This can be illustrated by the following example. Assume that data is transmitted between the device and the server via a network that is experiencing jitter (e.g., latency changes measured by the variability of packet latency over time on the network). Once the client device has detected the jitter, the client device may report to the server that it is experiencing latency 146. Upon receiving this information, the server may determine that the severity of the latency requires providing an auxiliary output to the client device or implementing the auxiliary output 144 by the client device. The type of auxiliary output may depend on the severity of the latency. Therefore, as an example of one possibility of auxiliary output 148, the server may then provide auxiliary instructions to the client device. Alternatively, these instructions may be pre-loaded on the client device or stored elsewhere, and the server may send a trigger signal that causes the client device to execute the pre-loaded instructions or obtain instructions from elsewhere and execute these instructions. Such instructions may be in the form of a script. As used herein and as generally understood by those skilled in the art, a "script" refers to a program written for a particular execution time environment that can interpret (rather than compile) tasks and automate the execution of tasks that can be performed one by one by a human operator. In an embodiment where auxiliary instructions have been loaded on the client device, the auxiliary output 148 can be a signal that triggers the client device to implement the pre-loaded auxiliary instructions. The signal can contain information indicating the amount of latency so that the client device can select an appropriate auxiliary to execute based on the degree of latency.

[0033] Whether preloaded on the client device, sent to the client from a server, or obtained by the client from elsewhere, the auxiliary instructions can take many different forms. By way of example and not limitation, some input combinations from the client device involve several button presses and these buttons must be pressed within 16ms of each other. The auxiliary input script can map this combination of button presses to a single button press.

[0034] In another example of auxiliary output 148, the server may provide or trigger auxiliary instructions to cause the client device to record the time difference (ΔT) between user inputs. Assume that the user's next three inputs are three button presses in the order of square, square, circle; each input is separated by 16ms. However, due to network jitter, the server receives these inputs as square, square, circle with a lag of 16ms between the first and second inputs and a lag of 33ms between the second and third inputs. However, by executing the auxiliary output instructions (such as a script), the client device can record the ΔT between the executed inputs and send it to the server. Therefore, in addition to these inputs, the server also receives data about the ΔT inputs at the client device. Therefore, the server then enters the user command with an appropriate ΔT of 16ms between each input, contrary to the input timing originally received by the server.

[0035] In some implementations, the client device can automatically record the ΔT between inputs that have been executed and send it to the server as part of the client device information 132. The server can then compare the ΔT value received from the client device with the difference in the time that the inputs arrived at the server (e.g., by taking the difference) and determine the latency. The server can then use the determined latency to select the appropriate auxiliary output 148.

[0036] According to certain aspects, the client device information may also be sent via encrypted data packet 134. Figure 1 As shown in , the server may receive an encrypted data stream 134 via a secure or unsecure network (e.g., UDP). If the encrypted data stream is received out of order, traditional methods such as forward error correction (FEC) will need to be implemented in order to correctly decrypt the data stream. However, according to certain aspects of the present disclosure, if the server determines that the data stream requires reassembly 140 due to out-of-order transmission, as shown in 142, the server can use the header information included with each packet to maintain the correct decryption order and reassemble the data stream before decryption.

[0037] It should be emphasized that providing Figure 1 The exemplary techniques described in the examples are for illustration purposes only, in order to highlight certain aspects of the present disclosure. In fact, embodiments of the present disclosure may be implemented without Figure 1 Examples of additional or alternative considerations are taken into account and may be more Figure 1 The simplified scheme depicted in is complex.

[0038] from Figure 1It will be appreciated from the examples of FIG. 1 that certain aspects of the present disclosure include various methods of providing auxiliary output to a client device. This may be by changing the user experience through the client device. This may also be by providing various scripts to the client device. In either case, the auxiliary output allows the user to overcome the effects of waiting time. Figure 1 It will also be appreciated that certain aspects of the present disclosure include various data streams that will be encrypted and decrypted via the use of packet header information. Thus, data sent from a server to a client device may also be encrypted, and the client device may use the encrypted packet header information to determine the appropriate decryption order for the received data stream.

[0039] Certain specific implementations of aspects of the present disclosure include systems configured for auxiliary outputs. By way of example and not limitation, according to certain aspects of the present disclosure, Figure 2 A distributed computing system is depicted, the distributed computing system including at least one server computing system 202 and at least one client device computing system 204, and the computing systems 202 and 204 are configured to communicate data via a network. In some implementations, the server 202 may be configured to execute instructions having to do with Figure 1 and / or Figure 4 In some implementations, the client device 204 may be configured to execute instructions that have the same characteristics as described above with respect to Figure 1 Or below about Figure 3 The server 202 and the client device 204 may be configured with suitable software and / or hardware to implement various aspects of the methods described herein. The server 202 and the client device 204 may be a server, an embedded system, a mobile phone, a personal computer, a laptop computer, a tablet computer, a portable gaming device, a workstation, a gaming console, a wearable device (such as a smart watch), and the like.

[0040] According to certain embodiments, the server 202 may be a cloud computing server, and the client device 204 may be a cloud computing client of the server 202, and the client server 202 may be configured to provide a data stream to the client device via a network 299 using an unreliable protocol. By way of example and not limitation, the server 202 or the client device 204 may be a computing system configured to provide a real-time data stream (such as a real-time audio stream, a real-time video stream, or a real-time audio / video stream) to at least one server 202 or at least one client device 204 via a network 299, which may be a wide area network (WAN), such as the Internet. The computing system (server 202 or client device 204) may be configured to send data via the network in the form of encrypted data packets containing data to be delivered to a recipient system. For example, according to various aspects of the present disclosure, the client device 204 may send these encrypted packets to the recipient device 202 simultaneously. The recipient system 202 may decrypt these encrypted data packets, and these encrypted data packets decrypted may include a header indicating the position of the packet in the decryption order.

[0041] Either of the systems 202 and 204 (i.e., the server 202, the client device 204, or both) may include one or more processor units 270, which may be configured according to well-known architectures, such as single-core, dual-core, quad-core, multi-core, processor-coprocessor, cell processor, and the like. Either of the systems 202 and 204 may also include one or more memory units 272 (e.g., RAM, DRAM, ROM, and the like). The processor unit 270 may execute one or more programs 274, which may be stored in the memory 272, and the processor 270 may be operatively coupled to the memory 272, such as by accessing the memory via a data bus 276. The memory unit 272 may include data 277, and the processor unit 270 may utilize the data 277 to implement the program 274. According to various aspects of the present disclosure, data 277 for either of systems 202 and 204 may include, for example, auxiliary output data 256 transmitted from server 202 to client device 204, and optionally client device information 252 including client device latency data 254 from client device 204 to server 202 or from server 202 to client device 204. Program 274 may optionally include instructions that, when executed by a processor, perform one or more operations associated with decryption of packets over an unsecured network or delivery of auxiliary output 256, such as, for example, having instructions associated with Figure 1 , Figure 3 and / or Figure 4 methods share one or more characteristics. For example, according to Figure 1In order to implement the server-side aspects of the method depicted in the foregoing and / or the transmission of the client device latency data, the program 274 of the server 202 may include instructions that, when executed by the processor 270, cause the server to transmit data to at least one receiving device 204 and / or provide an auxiliary output 256 in response to the latency data 254 from the client device 204. The program 274 of the client device 204 may include instructions that, when executed by the processor 270, cause the client device to execute the auxiliary output instructions 256 that are preloaded or provided by the server 202.

[0042] Either of the systems 202 and 204 may also include well-known support circuits 278, such as input / output (I / O) circuits 279, power supplies (P / S) 280, clocks (CLK) 281, and cache memory 282, which may communicate with other components of the system, for example, via bus 276. Either of the systems 202 and 204 may optionally include a mass storage device 284, such as a tape drive, CD-ROM drive, tape drive, flash memory, or the like, and the mass storage device 284 may store programs and / or data. Either of the systems 202 and 204 may also optionally include a display unit 286. The display unit 286 may be in the form of a cathode ray tube (CRT), a flat screen, a touch screen, or other device that displays text, numbers, graphic symbols, or other visual objects. Either of the systems 202 and 204 may also include a user interface 288 to facilitate interaction between the system 202 / 204 and a user. The user interface 288 may include a keyboard, mouse, light pen, game control pad, touch interface or other device. The user interface may also include audio I / O devices such as speakers and / or microphones.

[0043] The user can interact with any of the computer systems via the user interface 288. For example, the server 202 can be a cloud game server, and the client device 204 can be a cloud game client, and the video game user can interact with the video game executed by the server 202 and streamed to the client 204 via the user interface 288. The rate at which data is transmitted from the server to the client can be optimized according to aspects of the present disclosure to enhance the user's experience and maintain the quality of the signal received at the client side. Part of the user interface 288 may include a graphical user interface (GUI), which can be displayed on the display unit 286 to facilitate the user's interaction with the system 202 / 204. The system 202 / 204 may include a network interface 290, which is configured to implement the use of Wi-Fi, Ethernet ports or other communication methods. According to certain aspects of the present disclosure, the network interface 290 can incorporate suitable hardware, software, firmware or a combination thereof to facilitate communication via a telecommunications network, and can support data transmission using an unreliable protocol. The network interface 290 can be configured to implement wired or wireless communication via a local area network and a wide area network (such as the Internet). Either of systems 202 and 204 may send a request for received data and / or a file via one or more data packets 299 over the network.

[0044] like Figure 3 As shown in , a set of client device platform instructions 371 may be implemented, for example, by the client device platform 204. The client device platform instructions 371 may be formed on a non-transitory computer-readable medium, such as the memory 272 or the mass storage device 284. The client device platform instructions 371 may also be part of the process control program 274. As indicated at 373, the client device platform instructions 371 may include instructions for determining latency data 254 to be included in the client device information 252 that is to be sent to the server 202. The instructions 371 may include instructions 374 for delivering the client device information 252 to the server 202. If the client device information 252 is to be sent in the form of encrypted packets, these latter instructions may optionally include instructions for inserting packet encryption order information into the packet headers so that the server 202 can decrypt the packets even if the packets are received out of order. Thereafter, at 376, the instructions may include instructions for receiving auxiliary output 356 and subsequently may include instructions for executing auxiliary instructions triggered by or included in the received auxiliary output 256. If the auxiliary output 256 is to be received in the form of encrypted packets, these latter instructions may optionally include instructions for extracting packet encryption order information from the packet headers so that the client device 204 can decrypt the packets even if the packets are received out of order.

[0045] like Figure 4 As shown in , a set of server instructions 481 may be implemented, for example, by the server 202. The server instructions 481 may be formed on a non-transitory computer-readable medium, such as the memory 272 or the mass storage device 284. The server instructions 481 may also be part of the process control program 274. As indicated at 483, the instructions 481 may include instructions for receiving the client device information 252 from the client device 204. If the client device information 252 is received in the form of encrypted packets, these latter instructions may optionally include instructions for retrieving packet encryption order information from the packet headers so that the server 202 can decrypt these packets even if the packets are received out of order. Thereafter, at 484, these instructions may include instructions for determining whether the client device information 252 contains latency data 254. In addition, as indicated at 486, if latency data 254 is received with the client device information 252, the instructions 481 may contain instructions for generating an auxiliary output 256 to be delivered to the client device 204. Thereafter, at 487, the instructions may include instructions for sending the auxiliary output 256 to the client device 204. If the auxiliary output 256 is to be sent in the form of encrypted packets, these latter instructions may optionally include instructions to insert packet encryption order information into the packet headers so that the client device 204 can decrypt the packets even if the packets are received out of order.

[0046] The above components may be implemented in hardware, software, firmware or some combination thereof.

[0047] Although the above is a complete description of various illustrative embodiments of the present disclosure, various substitutions, modifications and equivalents can be used. Therefore, the scope of the present invention should not be understood to be limited by the above description, but should be determined by reference to the full scope of the attached claims and their equivalents instead. Any feature described herein (whether or not it is preferred) can be combined with any other feature described herein (whether or not it is preferred). In the attached claims, except for the case where it is clearly specified otherwise, the indefinite article "a (a / an)" refers to the number of one or more of the items following the article. The attached claims are not interpreted as including component or step plus function restrictions, except in the given claims using the phrase "component for..." or "step for..." Such restrictions are explicitly stated.

Claims

1. A method for sending data, in a server configured to operate on a network, the method comprising: determining an output based on data including a measurement of time between user inputs generated at a client device, wherein determining the output comprises: utilizing the measurement of time between the user inputs to implement the inputs by the server using the same input time differences as used when the inputs were generated at the client device; sending the output from the server to the client device, The output comprises a signal triggering execution of auxiliary instructions loaded onto the client device, or the output comprises instructions to be executed on the client device. The method of claim 1 , wherein the instructions are scripts. The method of claim 2 , wherein the script is configured to assist a user in the completion of a task. The method of claim 1 , wherein the manner in which the user experiences is altered by allowing more time for user input.

5. The method of claim 1, wherein the way the user experiences is changed by reducing the difficulty level of the artificial intelligence asset to attract the user. The method according to claim 1 , wherein the way the user experiences is changed by presenting different scenarios to the user.

7. The method of claim 1, wherein the client device information or the output comprises an encrypted data packet stream.

8. The method of claim 7, wherein each data packet in the stream of encrypted data packets includes a header indicating the position of the packet in a decryption order.

9. The method of claim 8, further comprising reassembling, by the server, the stream of encrypted data packets using the decryption order of the encrypted data packets when the packets are received out of order.

10. The method of claim 7, wherein the data packet stream is sent via an unreliable protocol. The method according to claim 10 , wherein the unreliable protocol is a User Datagram Protocol (UDP).

12. A system for transmitting data, comprising: processor, and a memory coupled to the processor; The processor is configured to execute a method comprising: determining an output based on data including a measurement of time between user inputs generated at a client device, wherein determining the output comprises: utilizing the measurement of time between the user inputs to implement the inputs by a server using the same input time differences as used when the inputs were generated at the client device; sending the output from the server to the client device, The output comprises a signal triggering execution of auxiliary instructions loaded onto the client device, or the output comprises instructions to be executed on the client device.

13. The system of claim 12, wherein the client device information or the output comprises an encrypted data packet stream.

14. A non-transitory computer-readable medium having processor-executable instructions embodied therein, wherein execution of the instructions by a processor causes the processor to perform a method comprising: determining an output based on data including a measurement of time between user inputs generated at a client device, wherein determining the output comprises: utilizing the measurement of time between the user inputs to implement the inputs by a server using the same input time differences as used when the inputs were generated at the client device; sending the output from the server to the client device, The output comprises a signal triggering execution of auxiliary instructions loaded onto the client device, or the output comprises instructions to be executed on the client device.

15. The non-transitory computer-readable medium of claim 14, wherein the client device information or the output comprises an encrypted data packet stream.

16. A method for sending data, comprising: sending client device information from the client device to the server via the network connection, wherein the client device information includes a measurement of time between user inputs; receiving an output from the server, wherein the output takes into account the measurement of time between the user inputs, wherein determination of the output comprises: utilizing the measurement of time between the user inputs to implement the inputs by the server using the same input time differences as used when generating the inputs at the client device; and Implementing assistance corresponding to the output through the client device, The output comprises a signal triggering execution of auxiliary instructions loaded onto the client device, or the output comprises instructions to be executed on the client device. The method of claim 16 , wherein the instructions are scripts.

18. The method of claim 17, wherein the script is configured to assist a user in the completion of a task.

19. The method of claim 16, wherein the manner in which the user experiences is altered by allowing more time for user input.

20. The method of claim 16, wherein the manner in which the user experiences is altered by reducing the difficulty level of the artificial intelligence asset to engage the user.

21. The method of claim 16, wherein the way a user experiences is changed by presenting different scenarios to the user.

22. The method of claim 16, wherein the client device information or the output comprises an encrypted data packet stream.

23. The method of claim 22, wherein each data packet in the stream of encrypted data packets includes a header indicating the position of the packet in a decryption order.

24. The method of claim 23, further comprising reassembling, by the server, the stream of encrypted data packets using the decryption order of the encrypted data packets when the packets are received out of order.

25. The method of claim 22, wherein the data packet stream is sent via an unreliable protocol.

26. The method of claim 25, wherein the unreliable protocol is a User Datagram Protocol (UDP).

27. A system for transmitting data, comprising: processor, and a memory coupled to the processor; The processor is configured to execute a method comprising: sending client device information from the client device to the server via the network connection, wherein the client device information includes a measurement of time between user inputs; receiving an output from the server, wherein the output takes into account the measurement of time between the user inputs, wherein determination of the output comprises: utilizing the measurement of time between the user inputs to implement the inputs by the server using the same input time differences as used when generating the inputs at the client device; and Implementing assistance corresponding to the output through the client device, The output comprises a signal triggering execution of auxiliary instructions loaded onto the client device, or the output comprises instructions to be executed on the client device.

28. A non-transitory computer-readable medium having processor-executable instructions embodied therein, wherein execution of the instructions by a processor causes the processor to perform a method comprising: sending client device information from the client device to the server via the network connection, wherein the client device information includes a measurement of time between user inputs; receiving an output from the server, wherein the output takes into account the measurement of time between the user inputs, wherein determination of the output comprises: utilizing the measurement of time between the user inputs to implement the inputs by the server using the same input time differences as used when generating the inputs at the client device; and Implementing assistance corresponding to the output through the client device, The output comprises a signal triggering execution of auxiliary instructions loaded onto the client device, or the output comprises instructions to be executed on the client device.

Images