Method of emitting a digital signal by a control means
By predicting future movements and transmitting digital signals ahead of actual positions, the method addresses high latency in digital signal transmission, enhancing user experience in cloud gaming.
Patent Information
- Application Number
- FR2021005222
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-05-19
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-05-19
AI Technical Summary
Existing digital signal transmission methods, particularly in cloud gaming, suffer from high latency between user interaction with control means and image display on the screen, leading to user fatigue, nausea, and decreased performance.
A method for transmitting digital signals by control means that involves predicting future movements based on kinematic and biomechanical laws, using a learning phase to calculate hypothetical movements, and transmitting signals ahead of actual positions to reduce latency.
Significantly reduces latency by anticipating user input, providing a smoother and more responsive user experience in video game services, even with high-latency connections.
Smart Images

Figure 00000026_0000 
Figure 00000026_0001 
Figure 00000027_0000
Abstract
Description
Title of the invention: Method for transmitting a digital signal by a control means Technical field of the invention
[0001] The invention relates to a method for transmitting a digital signal by a control means such as a computer mouse. The invention also relates to a method for interacting a control means with a remote server comprising the implementation of such a method for transmitting a digital signal. The invention further relates to a computer program product comprising program code instructions for implementing the steps of such methods. The invention finally relates to a data recording medium on which is recorded a computer program comprising program code instructions for implementing such methods. State of the prior art
[0002] Information processing machines such as computers or video game consoles are generally controlled by control means, or peripherals, including a keyboard, a computer mouse, a game controller or even a virtual reality headset. These control means are provided with interaction means that can be activated by a user. When a user activates an interaction means, a digital signal is emitted by the control means to the information processing machine and interpreted by the latter to display an image on a screen connected to the information processing machine. There is a certain time lag, commonly called latency, between the moment when the user interacts with the interaction means and the moment when an image taking this interaction into account is displayed on the screen.
[0003] For example, when a user moves a computer mouse on a support surface, a sensor detects the movement of the mouse. Then, the mouse transmits to a computer a digital signal relating to this movement. This digital signal is received by the computer and interpreted in order to calculate an instruction to move a pointer and / or a new image to be displayed on the screen. These instructions are then sent from the computer to the screen and then projected on the screen during a next refresh of the image. All of these operations can take a duration of the order of a hundred milliseconds. Users are generally accustomed to such latency.
[0004] At the same time, new video game services, called "cloud gaming", are now offered and aim to execute calculations relating to a video game on a remote server to which a data processing machine is connected. One advantage of such a service is that it allows video games requiring a high level of computing power to be played even with relatively simple data processing machines. Data processing machines do not perform the calculations necessary for the video game to function, but simply transmit the digital signals they receive from the control means to the remote server, and transmit the display instructions calculated by the remote server to the screen.
[0005] With such video game services, the latency between the movement of an interaction means and the display on the screen of an image calculated following this movement is penalized by the time taken to transfer the data to the remote server, then back to the information processing machine. Excessive latency can lead to greater user fatigue, nausea, loss of performance, or even an inability to use the computer system. To make this latency as low as possible, users resort to more efficient means of connecting to the Internet, such as for example optical fiber. However, even with such equipment the latency is still too high and the user experience is not completely satisfactory. Presentation of the invention
[0006] The aim of the invention is to provide a method for transmitting a digital signal by a control means remedying the above drawbacks and improving the methods for transmitting digital signals known from the prior art.
[0007] More specifically, a first object of the invention is a method of transmitting a digital signal by a control means making it possible to reduce the latency between the moment when a user interacts with the control means and the moment when an image taking this interaction into account is displayed on the screen, in particular for video game services on a remote server. Summary of the invention
[0008] The invention relates to a method of transmitting a digital signal by a control means capable of cooperating with an information processing machine, the method comprising: - a first phase of effective movement of a means of interaction of the control means, - detection by a sensor integrated into the control means of a set of effective positions of the interaction means during the first phase of movement, - calculation by a microprocessor integrated into the control means of a hypothetical movement relating to a second phase of movement of the interaction means following the first phase of movement of the interaction means, said hypothetical movement being calculated as a function of the set of effective positions of the first phase of movement, then - an emission by said control means of a digital signal defined as a function of said hypothetical displacement.
[0009] Said hypothetical displacement may be defined by a function of the time of a set of hypothetical positions of the interaction means during the second displacement phase, said digital signal may comprise at least a part of said set of hypothetical positions of the interaction means.
[0010] The digital signal associated with a given hypothetical position may be emitted by said control means at a time prior to the time at which the given hypothetical position is reached according to said function of the time of the set of hypothetical positions.
[0011] The method may comprise a learning phase comprising: - a learning movement of the means of interaction of the means of control, - detection by the sensor integrated into the control means of a set of learning positions of the interaction means during its learning movement, - storage in an integrated memory by means of controlling said learning movement or a reference movement calculated as a function of the learning movement, and the calculation of said hypothetical displacement may comprise a comparison of the set of effective positions of the interaction means during the first displacement phase with respectively said learning displacement or said reference displacement.
[0012] The calculation of said hypothetical displacement may comprise the determination of an extrapolation function as a function of said set of effective positions of the first displacement phase.
[0013] Said digital signal may comprise data defining the extrapolation function.
[0014] The extrapolation function may be a polynomial function, and said digital signal may comprise coefficients of said polynomial function.
[0015] The method may comprise a step of calculating a space of physically plausible movements comprising: - a maximum amplitude of the interaction means during its second phase of movement, - and / or a maximum speed of the means of interaction during its second phase of movement, - and / or maximum acceleration of the means of interaction during its second phase of movement, said hypothetical displacement being calculated as a function of the physically plausible control space.
[0016] The method may comprise an emission by said control means of a first digital signal describing the first phase of movement of the interaction means, then an emission by said control means of a second digital signal defined as a function of said hypothetical movement relating to the second phase of movement of the interaction means.
[0017] The invention also relates to a method of interaction of a control means with a remote server comprising the implementation of a method of transmitting a digital signal by a control means to an information processing machine as defined previously, then the transmission of a third digital signal by the information processing machine to the remote server of the information processing machine, the third digital signal being defined as a function of said hypothetical movement.
[0018] The interaction method may comprise transmitting a fourth digital signal from the remote server to the information processing machine, the fourth digital signal comprising display data calculated based on the third digital signal, and the display data may be transmitted to a screen connected to the information processing machine.
[0019] The display data can be transmitted to the screen only if said hypothetical movement satisfies a comparison test with a second phase of movement of the interaction means actually carried out.
[0020] The control means may be a computer mouse, or a game controller, or a virtual reality headset.
[0021] The invention also relates to a computer program product comprising program code instructions recorded on a computer-readable medium for implementing the steps of the method as defined above when said program operates on a computer.
[0022] The invention also relates to a data recording medium, readable by a computer, on which is recorded a computer program comprising program code instructions for implementing the method as defined above.
[0023] The invention finally relates to a signal of a data medium, carrying the program product as defined previously. Presentation of figures
[0024] These objects, characteristics and advantages of the present invention will be explained in detail in the following description of a particular embodiment made without limitation in relation to the attached figures among which:
[0025] [Fig.l] is a schematic view of a computer system according to one embodiment of the invention.
[0026] [Fig. 2] is a first graph illustrating on the one hand the position along an X axis of a computer mouse as a function of time, and on the other hand the position along an X' axis of a pointer displayed on a screen as a function of time.
[0027] [Fig. 3] is a block diagram of a method for transmitting a digital signal according to a first embodiment of the invention.
[0028] [Fig.4] is a second graph illustrating on the one hand the position along an X axis of a computer mouse as a function of time, and on the other hand the position along an X' axis of a pointer displayed on a screen as a function of time.
[0029] [Fig. 5] is a third graph illustrating on the one hand the position along an X axis of a computer mouse as a function of time, and on the other hand the position along an X' axis of a pointer displayed on a screen as a function of time. Detailed description
[0030] [Fig.l] schematically illustrates a computer system 1 according to an embodiment of the invention. The computer system 1 comprises an information processing machine 20 connected on the one hand to a control means 10 and on the other hand to a remote server 30.
[0031] The information processing machine 20 may be, for example, a desktop computer, a laptop, a game console, a television, a tablet, a smartphone, and more generally any computer equipment with which a user is likely to interact. The information processing machine 20 is connected to a screen 21, or monitor, capable of displaying images. The screen 21 may refresh the images at a given frequency, for example 50 Hz or 60 Hz. The information processing machine 20 further comprises means 22 for receiving a digital signal from the control means 10, and communication means 23 capable of transmitting and receiving digital signals to the remote server 30. In particular, the communication means 23 may be means for connecting to the Internet.
[0032] The control means 10 is a peripheral of the information processing machine 20. It allows a user to transmit commands to the information processing machine. For this purpose, it comprises at least one interaction means 11. The interaction means 11 is an element of the control means movable between at least two distinct positions. The control means 10 comprises a sensor 12 configured to detect the movement of the interaction means 11. In particular, the sensor 12 can detect a plurality of intermediate positions defined between two extreme positions between which the control means is movable. The sensor 12 is therefore distinguished from a simple binary sensor, such as a sensor assigned to a button or a keyboard key, and which is only capable of detecting two distinct positions of the interaction means. The sensor 12 can be, for example, a position sensor, a potentiometer, or an accelerometer. The sensor 12 can detect at least one characteristic linked to the movement of the interaction means IL. This characteristic can be an absolute position of the interaction means and / or a relative position of the interaction means, i.e. a position defined relative to a previous position of the interaction means.
[0033] The control means 10 also comprises a means 13 for transmitting a digital signal to the information processing machine 20. The digital signal transmitted by the control means can convey information on a relative position and / or on an absolute position of the interaction means. The transmission means 13 can be connected to the information processing machine 20 via its reception means 22. The connection between the control means 10 and the information processing machine can be either a wired connection or a wireless connection, such as a Bluetooth connection. The control means is capable of transmitting digital signals to the information processing machine at a frequency significantly higher than the refresh rate of the screen 21, for example at a frequency of 500 Hz or 1000 Hz.
[0034] The control means 10 further comprises a microprocessor 14, a memory 15 for storing data and an internal clock 16. The microprocessor 14 is connected to the memory 15, to the sensor 12, to the transmission means 13 and to the internal clock 16. The internal clock 16 makes it possible to associate a time base with a position detected by the sensor 12. The memory 15 is a data recording medium on which is recorded a computer program comprising program code instructions for implementing a method for transmitting a digital signal according to an embodiment of the invention. The microprocessor 14 is capable of executing at least part of this method.
[0035] The control means 10 may be, for example, a computer mouse, a game controller, or even a virtual reality headset.
[0036] If the control means is a computer mouse, it preferably comprises a housing, movable with the hand of a user on a support plane. The housing then constitutes a means of interaction of the mouse. The sensor 12 may for example be an optical sensor integrated into a sole of the mouse to detect its movements on the support plane. A pointer in the form of an arrow is displayed on the screen 21. The de Pointer placements are correlated with mouse movements. The computer mouse may include other means of interaction such as, for example, a joystick attached to the housing and adjustable in at least one direction.
[0037] If the control means is a game controller, the interaction means may be, for example, a joystick that can be tilted more or less in at least one given direction, a trigger whose position can be controlled with a user's finger, or a controller housing whose movements can be detected by one or more accelerometers integrated into the housing. The position of the joystick and / or the trigger may be detected by means of a sensor integrated into the controller.
[0038] If the control means is a virtual reality headset, the different movements of the head can be detected by means of one or more accelerometers integrated into the virtual reality headset.
[0039] Generally speaking, the control means are intended to be controlled with a part of the human body, in particular the fingers, the hand, or the head of the user. Alternatively, a control means could also be formed by a means for observing the movements of a user such as a camera.
[0040] The remote server 30 is connected to the information processing machine 20 via the communication means 23. The connection between the remote server 30 and the information processing machine 20 can be established via an Internet connection box (not shown). This Internet connection box can be on the one hand in a wireless connection, for example in a Wi-Fi connection, with the information processing machine 20, and on the other hand it can be in a wired connection, for example in an ADSL connection or by optical fiber to the remote server 30. The remote server 30 comprises a microprocessor 31, a memory 32, and communication means 33 allowing it to be connected to the information processing machine 20. The remote server 30 can be configured to run a video game. The control instructions are received from the information processing machine 20 by the remote server 30.Sequences of the video game are calculated by the remote server and images to be displayed on the screen 21 are sent to the information processing machine 20. Thus, the remote server 30 is configured to execute a video game service, called "cloud gaming". It is further assumed that the images to be displayed on the screen 21 depend on the position and / or movement of the interaction means 11 of the control means 10. For example, if the pointer reaches a specific area such as the edge of the screen, a background displayed on the screen can scroll automatically as long as the pointer remains positioned in this specific area.
[0041] In the remainder of the description, the example of a computer mouse will be used as a control means. The invention may be transposed to the use of any other control means. In practice, the mouse is freely movable in a plane. Mouse movements can therefore be represented in a two-dimensional frame of reference. In order to simplify the presentation, we will subsequently consider only mouse movements along a first X axis. The invention can be transposed to mouse movement along a second Y axis perpendicular to the first axis. More generally, the invention can be transposed to any control means as defined above.
[0042] [Fig.2] is a graph representing, by a first curve Cl in solid line, the actual position of the mouse relative to the X axis as a function of time t during a movement. The movement is temporally delimited by two instants t1 and t5 where the mouse speed is zero. The movement of the mouse is characterized by an acceleration phase followed by a holding phase where the mouse is moved at a relatively stable speed and then a slowing down phase to zero speed. The duration of a mouse movement can typically be between a hundred milliseconds for a very fast movement, and several seconds for a long movement.
[0043] This same graph shows a second curve C2 in solid lines corresponding to the position along an axis X' of a pointer displayed on the screen 21 as a function of time t, without implementing the transmission method according to the invention. It can be seen that there is a time lag DT between the physical movement of the mouse and the movement of the pointer displayed on the screen. This time lag DT is called latency. To simplify the presentation, it is considered that this latency is identical throughout the movement of the mouse. Thus, curve C2 is identical to curve Cl but translated on the time axis by a value equal to the latency DT.
[0044] The DT latency can be, for example, of the order of 100 ms, or even 200 ms. It is mainly linked to the time required to transmit digital signals from the information processing machine to the remote server, for the execution of calculations by the remote server, and finally to transmit digital signals from the remote server to the information processing machine.
[0045] The invention proposes to reduce the latency delay DT by means of hypotheses, or in other words predictions, on a future movement of the mouse. The future movement of the mouse can be anticipated by considering kinematic and biomechanical laws to which the user holding the mouse in his hand is subject. More generally, any means of interaction is constrained by kinematic and biomechanical laws of the human body.
[0046] Once the hypothetical movement has been calculated, the mouse emits a digital signal determined as a function of this hypothetical movement. This digital signal deceives the information processing machine to which it is connected. In [Fig.2], the dotted curve C3 represents the mouse movement information emitted by the mouse to the information processing machine as a function of time t. The pointer displayed on the screen, which is represented by the dotted curve C4, is moved with the latency DT relative to the movement signal emitted by the mouse. The curve C4 is therefore shifted by the latency delay DT relative to the curve C3. A latency DT' is defined as equal to the time difference between the actual movement of the mouse (curve Cl) and the movement of the pointer on the screen (curve C4). This gives a latency DT' lower than the latency DT. This latency can even be zero.
[0047] The transmission method according to the invention can be repeated according to a predefined frequency, in particular a frequency corresponding to the frequency at which the mouse is able to transmit new signals to the information processing machine, i.e. 500 Hz, or even 1000 Hz.
[0048] With reference to [Fig. 3], an embodiment of a method for transmitting a digital signal according to the invention is now described in more detail.
[0049] The invention proposes several approaches to achieve prediction of mouse movement.
[0050] According to a first approach, the method begins with a first phase PI, called the learning phase. During this phase, a user uses his mouse in a conventional manner and therefore performs a set of movements of the mouse, called learning movements, the characteristics of which are memorized. More precisely, the learning phase may comprise a step E01 of moving the mouse, a step E02 of detecting by means of the sensor 12 the positions of the mouse, and a step E03 of storing in the memory 15 the positions and of the mouse as a function of the time during each of these movements. Thus, the learning movements may be memorized in the form of a set of pairs comprising the time and the position.A database is thus created, recorded in memory 15, which characterizes the movements of the mouse, in particular the trajectory followed by the mouse, its amplitude of movement, its speed of movement, and its acceleration.
[0051] According to a first variant, all of the learning movements can be memorized by memorizing each position of the mouse during each learning movement. This solution is simple to implement but requires a large capacity memory to store a large number of positions.
[0052] According to a second variant, one can use a method called principal component analysis (also known by the acronym PCA for "principal component analysis"). In particular, one can define a given number of reference movements whose characteristics are refined as the different learning movements are executed. The number of reference movements can, for example, be between ten and twenty. A reference movement can be defined by a set of reference positions as a function of time. A reference move can be calculated, for example, by averaging all training moves that satisfy a test of similarity to the reference move. If a training move does not satisfy any test of similarity to a reference move, a new reference move can be created. If the predefined maximum number of reference moves is reached, a reference move for which the lowest number of training moves has been performed can be deleted. This frees up storage space in the memory.
[0053] According to a third variant, the learning phase may be based on artificial intelligence processes, in particular deep learning processes (commonly called deep learning) and / or on the use of a neural network based on an architecture called LSTM (English acronym for Long Short Term Memory).
[0054] This learning phase is then used to calculate a hypothesis on a future movement of the mouse.
[0055] The movement of the mouse as represented by the curve C1 in [Fig. 2] can be broken down into two successive phases: a first movement phase DI is directly followed by a second movement phase D2. The first phase DI is defined between the times t1 and t2 and the second phase is defined between the times t2 and t5. In a step E1, a set of positions of the mouse is detected by means of the sensor 12 during the first movement phase DI. As seen previously, this set of positions can be a set of relative positions or a set of absolute positions. This set of positions is then recorded at least temporarily in the memory 15 of the mouse in the form of a function of time.
[0056] In a step E2, the mouse can transmit to the information processing machine 20 first digital signals relating to the positions of the mouse during the first movement phase DI previously detected.
[0057] In parallel with step E2, a hypothetical movement relating to the second movement phase of the mouse is calculated during a step E3, by means of the microprocessor 14 integrated into the mouse. This hypothetical movement is calculated as a function of the positions of the mouse during the first movement phase D1, in particular by comparing the set of effective positions of the interaction means during the first movement phase D1 with the learning movements or the reference movements.
[0058] Advantageously, the sensor 12 integrated into the mouse is capable of providing the microprocessor 14 with a large quantity of information on the first phase of movement. The sensor 12 can typically acquire data at a frequency of 2000Hz. All of this information is not transmitted to the information processing machine by the first digital signals because it is useless for the information processing machine or for the remote server to know the position of the mouse with very high precision. In addition, this would unnecessarily clutter the bandwidth between the mouse and the information processing machine and / or between the information processing machine and the remote server. Thus, the frequency of sending information from the mouse to the information processing machine may be lower than the frequency of data acquisition by the sensor 12, typically of the order of 125 Hz. On the other hand, the large quantity of information detected by the sensor 12 can be used by the microprocessor 14 to determine with great precision the movement of the mouse during its first movement phase.This allows the hypothetical displacement to be calculated more accurately and reliably.
[0059] Said hypothetical displacement can be defined by a function of the time of a set of hypothetical positions of the interaction means during the second displacement phase. It can be considered that this hypothetical displacement is calculated instantaneously or almost instantaneously at time t2.
[0060] Concretely, to calculate this hypothetical displacement, one can search in the database storing the learning displacements or the reference displacements a particular displacement whose first phase most closely resembles the first displacement phase DI. One can for example calculate a score equal to the result of a correlation function between on the one hand the first displacement phase and on the other hand each learning displacement or each reference displacement stored in the memory 15. Then, one selects the learning displacement or the reference displacement which obtains the highest score. Finally, one defines the hypothetical displacement as equal to the second displacement phase of the selected displacement.
[0061] According to one embodiment of the invention, the method may comprise a step of identifying by the mouse a software program executed by the remote server, in particular a video game executed by the remote server. The learning phase PI and / or the calculation step E3 may then be made specific to the use of this software. In particular, learning movements and / or reference movements may be assigned to the use of a particular software program. Indeed, the movements of the mouse necessary to issue commands may be specific to the software program executed. For example, when the software program is a so-called "first-person" video game, that is to say a video game in which the images are displayed according to the point of view of a character, a command to turn the character is carried out by a very specific movement of the mouse. A good identification of this de mouse placement allows for rapid execution of the turnaround and therefore a successful video game experience. Identifying the software being executed allows for a more accurate prediction of the upcoming mouse movement. Similarly, the method may also include a step of identifying a mouse user, with the learning movements and / or reference movements being specific to each user.
[0062] In a step E4, the mouse transmits to the information processing machine 20 second digital signals relating to the previously calculated hypothetical displacement. From time t2, the mouse transmits hypothetical positions in place of the positions actually reached. The second digital signals therefore convey hypothetical positions of the mouse. Advantageously, these hypothetical positions are transmitted by the mouse before they are actually reached: this simulates a faster movement of the mouse than its actual movement. In other words, the digital signal associated with a given hypothetical position is transmitted by said control means at a time prior to the time at which the given hypothetical position is reached according to the previously calculated hypothetical displacement. In [Fig. 2], this results in a slope of the curve C3 greater than the slope of the curve C1 from time t2.Advantageously, the movement of the pointer is faster but is not instantaneous. This avoids producing a jerky image which would be unpleasant for the user. The transient increase in the speed of the pointer and / or the speed of scrolling images in the background may be imperceptible to the user.
[0063] In [Fig. 2], curve C4 describes the movement of the pointer with the implementation of the invention. The first hypothetical position is emitted at time t2 by the mouse. The movement of the pointer following the emission of the first hypothetical position occurs at time t3. From this time t3, the speed of movement of the pointer on the screen is therefore greater than the speed it would have had in the absence of implementation of the invention. The latency DT' defined between the actual movement of the mouse and the actual movement of the pointer is thus a decreasing function of time. According to the example of [Fig. 2] the latency DT' reaches a value of zero or close to zero at time t4. The second digital signals can be emitted so as to obtain a linear decrease in the latency DT'.
[0064] The example in [Fig.2] is constructed by considering that the hypothetical movement corresponds exactly to the actual movement of the mouse. In practice, the hypothetical movement does not always correspond exactly to the movement actually carried out. However, as the process is repeated several times during the movement of the mouse, it is possible to recalculate and correct the hypothetical movement as the mouse actually moves. A possible im accuracy on the calculated hypothetical movement may result in a smaller reduction in the latency delay. However, the latency delay obtained thanks to the invention is never greater than the latency delay that would be obtained in the absence of the method according to the invention. [Fig. 4] illustrates as an example a case where the hypothetical movement differs from the movement actually carried out. It can be seen that curve C3 describing the hypothetical movement deviates slightly from the actual movement of the mouse. Consequently, curve C4 representing the movement of the pointer on the screen also undergoes these slight variations. However, thanks to recognition of the phase of movement of the mouse during which its speed decreases, it is still possible to anticipate the position at which the mouse stops.
[0065] When the signal emitted by the mouse is received by the information processing machine, the latter forwards it to the remote server 30 via a third digital signal in a step E5. The remote server interprets this signal as a command from the user and calculates an image to be displayed on the screen during a step E6. Once this calculation has been executed, it transmits in return, in a step E7, a fourth digital signal to the information processing machine. This fourth digital signal comprises an instruction to display an image on the screen 21. The information processing machine therefore acts as a simple gateway between the mouse, the remote server and the screen.
[0066] In parallel with steps E5, E6 and E7, the actual movement of the mouse continues. At the moment when the information processing machine receives the display instructions, it can be confirmed whether the hypothetical movement is consistent with the movement actually carried out. The display instructions can be transmitted to the screen only if said hypothetical movement satisfies a comparison test with a second phase of movement of the interaction means actually carried out. To this end, the mouse can transmit to the information processing machine and in parallel with the position signals, a confirmation signal indicating whether the actual movement of the mouse was consistent with the hypothetical movement. Alternatively, the mouse can transmit in parallel digital signals conveying the predictive position of the mouse, digital signals conveying the actual position of the mouse.This double information can be transferred to the remote server which then calculates two image instructions corresponding to these two digital signals. The information processing machine can then arbitrate, when it receives these two instructions, which one it should actually display on the screen, depending on a movement actually made by the mouse.
[0067] In the embodiments illustrated in Figures 2 and 3, the method is implemented so that the latency delay is close to zero. However, the method can be implemented simply to reduce the latency delay. The more one anticipates in the future movement, the greater the risk that the hypothetical movement is far from the actual movement. Thus, it may be interesting to implement the method only for the purpose of anticipating the movement of the mouse by about ten milliseconds or a few tens of milliseconds. It is thus possible to make a video game experience using a remote server close to that of a video game directly executed by the information processing machine and for which a reasonable latency delay is accepted. [Fig.5] illustrates as an example a method in which the latency delay is reduced from a value DT to a value DT' which is not zero but is strictly less than the latency delay DT.
[0068] Advantageously, the invention also provides for the mouse to receive information relating to an actual latency delay between the mouse and the information processing machine and / or the remote server. Then, a more or less distant prediction can be selected and emitted by the mouse to replace the original data. This makes it possible to smooth out the perceived latency and therefore makes it possible to correct jerky operation linked to erratic latency.
[0069] A number of variant embodiments of the method just described are now described. These different variants can be freely combined with each other.
[0070] According to an alternative embodiment of the method according to the invention, the method could base the calculation of the hypothetical displacements on the basis of an extrapolation function. The extrapolation functions may be, for example, polynomial functions and in particular Bézier functions and / or splines, i.e. piecewise polynomial functions. For example, the extrapolation function may be a third-degree polynomial defined by the following formula:
[0071] [Math 1] Pfy = at3 + bi2 + and + dwhere a, b, c and d are parameters of the extrapolation function to be determined. All actual mouse positions during the first movement phase are then used to determine these parameters. These parameters can be determined by the least squares method or by an equivalent method.
[0072] Prior to determining the parameters of the extrapolation function, a space of physically plausible movements of the mouse can be determined. This movement space characterizes the plausible movements of the mouse taking into account the kinematic and biomechanical laws of the human body. For example, a mouse intended to be held in a user's hand cannot exceed a certain maximum movement amplitude, a certain maximum speed, or a certain maximum acceleration.
[0073] These maximum values can be predefined, for example, as follows: - The maximum distance traveled by the mouse is 3m. - The maximum mouse speed is 3m.s - The maximum acceleration of the mouse is 10g, or about 98 lm.s2. These same maximum values could apply to any other means of control intended to be held in the hand. A biomechanical study of each means of interaction allows these values to be adjusted for different means of control of a mouse.
[0074] Alternatively, the maximum values can be calculated on the basis of different learning movements carried out during a learning phase. This makes it possible in particular to define maximum values adapted to a particular user. They can also be assigned to a particular user, previously identified.
[0075] The space of physically plausible displacements is then used to express equality or inequality constraints on the parameters of the extrapolation function. These constraints must be respected when determining the parameters of the extrapolation function. Mathematical methods such as constrained Lagrangian optimization can then be used.
[0076] Once the extrapolation function is determined, one can easily calculate a hypothetical displacement by calculating the result of the extrapolation function at a given time t.
[0077] In addition, the space of physically plausible movements can be used to filter signals that are clearly independent of the user's will. For example, if the mouse hits an obstacle during its movement, the impact against the mouse can result in a very significant acceleration of the mouse. The method can then be implemented to predict the user's intentional movement, that is, the movement that the mouse would have followed if it had not hit the obstacle. According to another example, the mouse is sometimes lifted by the user relative to the support on which it rests in order to generate a large amplitude movement of the pointer displayed on the screen. Although the mouse is lifted, the sensor can detect a movement of the mouse.Lifting the mouse and moving it out of the holder then causes the detection of an unintentional movement resulting in an unwanted movement of the pointer. Such a lift can be detected and filtered. Instead, the mouse can emit a signal of movement of the pointer in the same direction as that which it followed before the mouse was lifted.
[0078] According to an advantageous embodiment of the invention, a communication protocol can be defined between the mouse, the information processing machine and the remote server so that the parameters defining the extrapolation function are sent in replacement of or in addition to the mouse position information. Sending the parameters of the extrapolation function in replacement of the mouse positions The mouse allows to reduce the amount of information exchanged with the remote server, and thus frees up bandwidth that can be usefully used to transfer other operating data of a video game and thus improve the user experience. Sending the parameters of the extrapolation function in addition to the mouse positions could allow to outsource the calculation of the hypothetical movement to the data processing machine or even to the remote server.
[0079] Particular examples of implementation are now described.
[0080] According to a first example, a system comprising a computer, a screen and a peripheral is considered, the entire system operating without the intervention of an external internet network, i.e. without a remote server. The screen display frequency is 60Hz, i.e. the image displayed on the screen is refreshed approximately every 16 ms. The user command capture frequency is 500Hz. The duration of a movement of the control means is 180ms. The inherent latency of the system considered is 50ms. The target latency is 2ms. The following table illustrates the latency observed during the display of each image during the movement of the control means, with and without implementation of the method according to the invention.
[0081] [Tables 1] Image No. DT Latency without implementation of the invention Observations DT' Latency with implementation of the invention 1 8ms The first image is randomly placed 0 to 16ms after the start of the movement 8ms 2 24ms The second image is placed 16ms after the first 24ms 3 40ms The third image is placed 16ms after the second 40ms 4 50ms If the first image was placed 8ms or less after the start of the movement, the replacement could not be done and the latency remains the same. Otherwise, the user commands of this fourth image are replaced by their prediction and the latency becomes 2ms on average. 26ms average (average of 50ms and 2ms) 5 50ms Latency becomes target latency 2ms 6 50ms Latency becomes target latency 2ms 7 50ms Latency becomes target latency 2ms 8 50ms Latency becomes target latency 2ms 9 50ms Latency becomes target latency 2ms 10 50ms Latency becomes target latency 2ms average 42.2ms - 11ms
[0082] In a second example, the initial conditions are identical to the first example, and the latency is also smoothed so that it decreases more gradually.
[0083] [Tables2] Image No. DT Latency without implementation of the invention Observations DT' Latency with implementation of the invention 1 8ms The first image is randomly placed 0 to 16ms after the start of the movement 8ms 2 24ms The second image is placed 16ms after the first 24ms 3 40ms The third image is placed 16ms after the second 40ms 4 50ms Case 1: the first image was placed 8ms or less after the start of the movement. The replacement could not be done and the latency remains at 50ms. Case 2: the latency drops to its lower limit starting from 40ms, i.e.: 40 - 2 x 16 = 8ms.On average 29ms (average of 50ms and 8ms) 5 50ms Case 1: the latency drops to its lower limit starting from 50ms, so 50 - 2 x 16 = 18ms Case 2: the latency drops to its lower limit starting from 8ms, i.e. becomes the target latency of 2ms On average 10ms (average of 18ms and 2ms) 6 50ms The latency becomes the target latency 2ms 7 50ms The latency becomes the target latency 2ms 8 50ms The latency becomes the target latency 2ms 9 50ms The latency becomes the target latency 2ms 10 50ms The latency becomes the target latency 2ms average 42.2ms - 12.1ms .
[0084] As a note, the lower bound of the latency is inherent to the system. It has been assumed that the system latency is 2 ms, however this value depends on many parameters such as the hardware configuration of the information processing machine, the load of the remote server, the network congestion Internet, possible interference in connection cables. The lower limit of latency can also fluctuate over time.
[0085] According to a third example, we consider a system comprising a computer, a screen, a peripheral, the computer being connected to a remote server. The display frequency of the screen is 60Hz, that is to say that the image displayed on the screen is refreshed approximately every 16 ms. The frequency of capturing user commands is 500Hz. The duration of a movement of the control means is 180ms. The inherent latency of the system considered is 80ms. The target latency is 2ms.
[0086] [Tables3] Image No. DT Latency without implementation of the invention Observations DT' Latency with implementation of the invention 1 8ms The first image is randomly placed 0 to 16ms after the start of the movement 8ms 2 24ms The second image is placed 16ms after the first 24ms 3 40ms The third image is placed 16ms after the second 40ms 4 56ms The fourth image is placed 16ms after the third 56ms 5 72ms The fifth image is placed 16ms after the fourth 72ms 6 80ms If the first image was placed 8ms or less after the start of the movement, the replacement could not be done and the latency remains the same. Otherwise, the user commands of this fourth image are replaced by their prediction and the latency becomes 2ms on average. 36ms average (average of 80ms and 2ms) 7 80ms Latency becomes target latency 2ms 8 80ms Latency becomes target latency 2ms 9 80ms Latency becomes target latency 2ms 10 80ms Latency becomes target latency 2ms average 60ms - 24.9ms
[0087] In a fourth example, the initial conditions are identical to the third example, and the latency is also smoothed so that it decreases more gradually.
[0088] [Tables4] Image No. DT Latency without implementation of the invention Observations DT' Latency with implementation of the invention 1 8ms The first image is randomly placed 0 to 16ms after the start of the movement 8ms 2 24ms The second image is placed 16ms after the first 24ms 3 40ms The third image is placed 16ms after the second 40ms 4 56ms The fourth image is placed 16ms after the third 56ms 5 72ms The fifth image is placed 16ms after the fourth 72ms 6 80ms Case 1: the first image was placed 8ms or less after the start of the movement. The replacement could not be done and the latency becomes 80ms. Case 2: the latency drops to its lower limit starting from 72ms, i.e.: 72 - 2 x 16 = 40ms. 60ms on average (average of 80ms and 40ms) 7 80ms Case 1: the latency drops to its lower limit starting from 80ms, i.e. 80 - 2 x 16 = 48ms. Case 2: the latency drops to its lower limit starting from 40ms, i.e. 40 - 2 x 16 = 8ms.28ms on average (average of 28ms and 8ms) 8 80ms Case 1: the latency drops to its lower limit starting from 48ms, i.e. 48 - 2 x 16 = 16ms Case 2: the latency becomes the target latency. 9ms on average (average of 16ms and 2ms) 9 80ms The latency becomes the target latency 2ms 10 80ms The latency becomes the target latency 2ms average 60ms - 24.9ms .
[0089] Finally, thanks to the invention, a reduced latency delay is obtained between a user's command and its effects displayed on a screen. The use of the software is more fluid and more pleasant. The use of software or video games implemented by a remote server is improved.
Claims
Claims
1. Method for transmitting a digital signal by a control means (10) capable of cooperating with an information processing machine (20), the control means (10) being intended to be controlled with the fingers or the hand, the method comprising: - a first phase of effective displacement with the fingers or the hand of an interaction means (11) of the control means (10), - a detection (El) by a sensor (12) integrated in the control means of a set of effective positions of the interaction means during the first displacement phase, characterized in that it comprises: - a calculation (E3) by a microprocessor (14) integrated in the control means of a hypothetical displacement relating to a second phase of displacement of the interaction means following the first phase of displacement of the interaction means, said hypothetical displacement being calculated as a function of the set of effective positions of the first displacement phase,then - a transmission (E4) by said control means of a digital signal defined as a function of said hypothetical displacement.,
2. Method according to the preceding claim, characterized in that said hypothetical displacement is defined by a function of the time of a set of hypothetical positions of the interaction means during the second displacement phase, and in that said digital signal comprises at least a part of said set of hypothetical positions of the interaction means.
3. Method according to the preceding claim, characterized in that the digital signal associated with a given hypothetical position is emitted by said control means at an instant prior to the instant at which the given hypothetical position is reached according to said function of the time of the set of hypothetical positions.
4. Method according to one of the preceding claims, characterized in that it comprises a learning phase (PI) comprising: - a learning movement (E01) of the interaction means (11) of the control means (10), - a detection (E02) by the sensor (12) integrated in the control means of a set of learning positions of the interaction means during its learning movement, - a storage (E03) in a memory (15) integrated into the means of controlling said learning movement or a reference movement calculated as a function of the learning movement, and in that the calculation of said hypothetical movement comprises a comparison of the set of effective positions of the interaction means during the first movement phase with respectively said learning movement or said reference movement.
5. Method according to one of the preceding claims, characterized in that the calculation of said hypothetical displacement comprises the determination of an extrapolation function as a function of said set of effective positions of the first displacement phase.
6. Method according to the preceding claim, characterized in that said digital signal comprises data defining the extrapolation function.
7. Method according to claim 5 or 6, characterized in that the extrapolation function is a polynomial function, and in that said digital signal comprises coefficients of said polynomial function.
8. Method according to one of the preceding claims, characterized in that it comprises a step of calculating a space of physically plausible movements comprising: - a maximum amplitude of the interaction means during its second phase of movement, - and / or a maximum speed of the interaction means during its second phase of movement, - and / or a maximum acceleration of the interaction means during its second phase of movement, said hypothetical movement being calculated as a function of the physically plausible control space.
9. Method according to one of the preceding claims, characterized in that it comprises an emission by said control means of a first digital signal describing the first phase of movement of the interaction means, then an emission by said control means of a second digital signal defined as a function of said hypothetical movement relating to the second phase of movement of the interaction means.
10. Method for interaction of a control means with a remote server, characterized in that it comprises the implementation of a method of transmitting a digital signal by a control means (10) to an information processing machine (20) according to one of the preceding claims, then transmitting (E5) a third digital signal by the information processing machine to the remote server of the information processing machine, the third digital signal being defined as a function of said hypothetical displacement.
11. Interaction method according to the preceding claim, characterized in that it comprises the transmission (E7) of a fourth digital signal by the remote server to the information processing machine, the fourth digital signal comprising display data calculated as a function of the third digital signal, and in that the display data are transmitted to a screen (21) connected to the information processing machine (20).
12. Method according to the preceding claim, characterized in that the display data are transmitted to the screen only if said hypothetical movement satisfies a comparison test with a second phase of movement of the interaction means actually carried out.
13. Method according to one of the preceding claims, characterized in that the control means (10) is a computer mouse or a game controller.
14. A computer program product comprising program code instructions recorded on a computer-readable medium for implementing the steps of the method according to any one of claims 1 to 13 when said program is running on a computer.
15. A computer-readable data storage medium on which is recorded a computer program comprising program code instructions for implementing the method according to one of claims 1 to 13.
