Data transmission method and device based on cloud game, electronic equipment and storage medium

Abstract

The invention relates to the field of data processing, and particularly discloses a data transmission method and device based on a cloud game, electronic equipment and a storage medium. The method comprises the following steps: acquiring a transmission frequency of transmitting device state data to the client device by an input device; under the condition that a comparison result between the transmission frequency and the game frame rate of the cloud game meets a preset adjustment condition, an adjustment time window matched with the comparison result is determined; and according to the adjustment time window, processing the device state data transmitted by the input device, and sending the processed device state data to a cloud server of the cloud game. According to the mode, the client equipment can flexibly adjust the transmission frequency of the equipment state data of the input equipment according to the requirements of the cloud game.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of data processing technology, and specifically to a data transmission method, device, electronic device, and storage medium based on cloud gaming. Background Art

[0002] Cloud gaming is a gaming method based on cloud computing. It allows users to run games on the server side through the network without the need for high-end processors and graphics cards, and compresses the rendered game images and transmits them to the user-side device, achieving a smooth experience of high-quality gaming.

[0003] Cloud gaming is based on cloud computing technology. All game resources are run on the server side, and the user-side device only needs to have basic video decompression capabilities. Users do not need to install games, and are not restricted by device configuration. As long as there is an Internet connection, they can enjoy high-quality gaming experience anytime and anywhere.

[0004] In the related art, the frequency of data transmission between the client device and the cloud game is mainly determined by the input device connected to the client device, while the device performance of the input device is fixed and therefore cannot be adapted to various types of cloud games. Summary of the invention

[0005] In view of the above problems, the present disclosure is proposed to provide a cloud gaming-based data transmission method, device, electronic device and storage medium that overcome the above problems or at least partially solve the above problems.

[0006] According to one aspect of an embodiment of the present disclosure, a cloud gaming-based data transmission method is provided, which is applied to a client device, and the method includes:

[0007] Acquire a transmission frequency of the input device transmitting device status data to the client device;

[0008] When a comparison result between the transmission frequency and the game frame rate of the cloud game satisfies a preset adjustment condition, determining an adjustment time window adapted to the comparison result;

[0009] According to the adjustment time window, the device status data transmitted by the input device is processed; wherein the processed device status data is used to be sent to the cloud server of the cloud game, so that the cloud server can generate user control instructions based on the received device status data.

[0010] In an optional implementation, the preset adjustment condition includes: a first type of adjustment condition corresponding to a first type of comparison result, and / or a second type of adjustment condition corresponding to a second type of comparison result; wherein the first type of comparison result is used to characterize that the transmission frequency is lower than the game frame rate of the cloud game, and the second type of comparison result is used to characterize that the transmission frequency is higher than the game frame rate of the cloud game;

[0011] Wherein, the processing of the device status data transmitted by the input device comprises: processing the device status data transmitted by the input device according to a processing strategy corresponding to the condition type of the preset adjustment condition;

[0012] Among them, the processing strategies corresponding to the first type of adjustment conditions include: an interpolation type processing strategy for increasing the transmission frequency between the client device and the cloud server; the processing strategies corresponding to the second type of adjustment conditions include: a merging type processing strategy for reducing the transmission frequency between the client device and the cloud server.

[0013] In an optional implementation, when the comparison result between the transmission frequency and the game frame rate of the cloud game satisfies the first type of adjustment condition, the adjustment time window adapted to the comparison result includes: an interpolation sending time window, and / or an interpolation backtracking time window;

[0014] The interpolation sending time window is used to characterize the sending timing of the device status data obtained according to the interpolation type processing strategy; the interpolation backtracking time window is used to characterize the amount of device status data for interpolation prediction.

[0015] In an optional implementation, the processing of the device status data transmitted by the input device according to the adjustment time window, and sending the processed device status data to the cloud server of the cloud game includes:

[0016] Acquire a preset number of device status data matching the interpolation backtracking time window;

[0017] Performing interpolation prediction based on the preset number of device status data, and sending the device status data obtained by interpolation prediction to the cloud server of the cloud game when the interpolation sending time window arrives;

[0018] The interpolation backtracking time window is determined according to the transmission frequency of the device status data transmitted by the input device to the client device, and the interpolation sending time window is determined according to the interpolation multiple adapted to the comparison result.

[0019] In an optional implementation, when the comparison result between the transmission frequency and the game frame rate of the cloud game satisfies the adjustment condition of the second type, the adjustment time window adapted to the comparison result includes: merging the sending time window;

[0020] The combined sending time window is used to indicate the sending timing of the device status data obtained according to the combined processing strategy.

[0021] In an optional implementation manner, the combined sending time window is determined by:

[0022] Determine a combined sending time window in an initial state according to the game frame rate;

[0023] The combined sending time window in the initial state is dynamically adjusted according to the currently detected network state.

[0024] In an optional implementation, the processing of the device status data transmitted by the input device according to the adjustment time window, and sending the processed device status data to the cloud server of the cloud game includes:

[0025] When the combined sending time window has not arrived, combining the multiple device status data received from the input device to obtain combined device status data;

[0026] When the merged sending time window is reached, the merged device status data is sent to the cloud server of the cloud game, so that the cloud server can determine the game screen frame corresponding to the merged device status data and generate user control instructions corresponding to the game screen frame.

[0027] In an optional implementation, the input device includes: a mouse; a transmission frequency of the input device transmitting device status data to the client device is determined according to a return rate of the mouse;

[0028] Furthermore, the device status data includes: a click-related status parameter for characterizing a mouse click status, and / or a position-related status parameter for characterizing a mouse movement position.

[0029] In an optional implementation, the input device includes: a keyboard; and the method further includes:

[0030] In response to receiving a key press message sent by the keyboard, querying the current state of the key corresponding to the key press message in the key state table; if the current state of the key is not pressed, updating the current state of the key to pressed, and sending the received key press message to the cloud server; if the current state of the key is pressed, ignoring the key press message;

[0031] When a key-up message is received from the keyboard, the current state of the key corresponding to the key-up message in the key state table is updated from pressed to not pressed, and the received key-up message is sent to the cloud server.

[0032] According to another aspect of the present disclosure, a data transmission method based on cloud gaming is provided, which is applied to a cloud server, and the method includes:

[0033] Receiving device status data transmitted by the client device through the above method;

[0034] A user control instruction is generated according to the device status data, and the user control instruction is sent to the client device.

[0035] According to another aspect of the present disclosure, a data transmission device based on cloud gaming is provided, comprising:

[0036] An acquisition module, adapted to acquire a transmission frequency of the input device transmitting device status data to the client device;

[0037] a determination module, adapted to determine an adjustment time window adapted to the comparison result between the transmission frequency and the game frame rate of the cloud game if the comparison result between the transmission frequency and the game frame rate of the cloud game meets a preset adjustment condition;

[0038] The processing module is adapted to process the device status data transmitted by the input device according to the adjustment time window, and send the processed device status data to the cloud server of the cloud game.

[0039] According to another aspect of the present disclosure, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0040] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the data transmission method based on cloud gaming as described above.

[0041] According to another aspect of the present disclosure, a computer storage medium is provided, wherein the storage medium stores at least one executable instruction, and the executable instruction enables a processor to execute the above-mentioned cloud gaming-based data transmission method.

[0042] In an embodiment of the present disclosure, the transmission frequency of the device status data transmitted by the input device to the client device can be compared with the game frame rate of the cloud game, and when the comparison result satisfies the preset adjustment condition, the device status data transmitted by the input device can be processed according to the adjustment time window adapted to the comparison result, so as to send the processed device status data to the cloud server. It can be seen that in the present disclosure, according to the comparison result between the transmission frequency of the device status data transmitted by the input device to the client device and the game frame rate, the device status data transmitted by the input device can be processed according to the adjustment time window, so as to facilitate the adjustment of the data transmission frequency between the client device and the cloud game according to the processing result, avoid the problem that the input device cannot adapt to various types of cloud games due to the fixed device performance of the input device, and enable the client device to flexibly adjust the transmission frequency of the device status data of the input device according to the needs of the cloud game.

[0043] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present disclosure. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0045] Figure 1 A flow chart of a cloud gaming-based data transmission method provided by an embodiment of the present disclosure is shown;

[0046] Figure 2 A flowchart of a cloud gaming-based data transmission method provided by an example of the present disclosure is shown;

[0047] Figure 3 A schematic diagram of a data transmission device based on cloud gaming provided by another embodiment of the present disclosure is shown;

[0048] Figure 4 A schematic structural diagram of an electronic device provided by yet another embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0049] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0050] Figure 1 FIG. 1 is a flow chart of a data transmission method based on cloud gaming provided by an embodiment of the present disclosure. Figure 1 As shown, the method is applied to a client device and specifically includes:

[0051] Step S110: Acquire the transmission frequency of the input device transmitting device status data to the client device.

[0052] The input device refers to an external device used to input user control data to the client device. For example, the input device may include various devices such as a mouse, a keyboard, and a mouse. Among them, the mouse may include an air mouse, an eye-movement mouse, a gesture detector, etc. In short, any device that can detect the user's control state can be used as an input device in the present disclosure. In addition, the input device can also be various virtual input devices, such as a touch screen containing virtual buttons in a terminal device.

[0053] The device status data transmitted by the input device to the client device is used to characterize the status information presented by the input device under the control of the user. For example, the device status data may include at least one of the following: a click-type status parameter for characterizing the mouse click status, a position-type status parameter for characterizing the mouse movement position, and a key-type status parameter for characterizing whether a key in the keyboard is pressed or lifted. In short, the device status data is used to characterize the user's control status, and the present disclosure does not limit the specific connotation of the device status data.

[0054] The transmission frequency of the input device to the client device for transmitting the device status data can be characterized according to the device interaction performance parameters of the input device, for example, it can be characterized by the mouse's polling rate. The polling rate refers to the frequency of sending data from the mouse to the device to which it is connected (i.e., the client device), that is, the frequency at which the mouse updates its position to the client device, usually expressed in Hz. Usually, the polling rate of an office mouse is 125Hz, the polling rate of a gaming mouse is 1,000Hz, and the polling rate of some high-precision gaming mice can be as high as 8,000Hz.

[0055] It can be seen from this that the transmission frequency of the input device transmitting device status data to the client device can be directly determined based on the device interaction performance parameters of the input device, or the frequency of the input device transmitting device status data in a historical period can be dynamically detected, thereby dynamically determining the current transmission frequency based on the detection results.

[0056] Step S120: When the comparison result between the transmission frequency and the game frame rate of the cloud game meets the preset adjustment condition, determine an adjustment time window that matches the comparison result.

[0057] Among them, the transmission frequency of the device status data transmitted by the input device to the client device is compared with the game frame rate of the cloud game, for example, the two are interpolated and the ratio is calculated to obtain a comparison result. Determine whether the comparison result meets the preset adjustment condition. Among them, the preset adjustment condition can be set from the perspective of the size of the difference or the ratio. In short, the preset adjustment condition is intended to measure whether the transmission frequency matches the game frame rate, and to perform subsequent adjustment processing if the two do not match. For example, the preset adjustment condition may be: the absolute value of the difference between the transmission frequency and the game frame rate is greater than a preset threshold, or the ratio between the transmission frequency and the game frame rate belongs to the preset threshold interval.

[0058] When it is determined that the comparison result satisfies the preset adjustment condition, an adjustment time window adapted to the comparison result is further determined. The adjustment time window is used to limit the adjustment timing and / or adjustment method of adjusting the device status data transmitted by the input device. In short, the adjustment time window is used to limit the specific implementation method of the subsequent adjustment processing.

[0059] As the name implies, the adjustment time window refers to a time window that is limited according to the actual business situation, and the time window is used to adjust the transmission frequency of the device status data. Among them, without adjusting the time window, the client device will directly forward the device status data from the input device, that is, the frequency at which the client device transmits device status data to the cloud server is the same as the frequency at which the input device transmits device status data to the client device. It can be seen that in the related art, since there is no adjustment process of adjusting the time window, the frequency at which the client device transmits device status data to the cloud server is usually consistent with the return rate of the mouse used by the user, and cannot be adapted according to the business needs of the cloud game.

[0060] When the time window is adjusted, the client device adjusts the device status data from the input device according to the adjustment time window, thereby changing the transmission frequency of the device status data, so that the frequency of the client device transmitting the device status data to the cloud server is different from the frequency of the input device transmitting the device status data to the client device, and thus the frequency of the client device transmitting the device status data to the cloud server does not need to be limited by the return rate of the mouse used by the user, but can be flexibly adapted according to the business needs of the cloud game.

[0061] Among them, the transmission frequency of the device status data after adjustment by the adjustment time window may be greater than the original transmission frequency of the input device, or may be less than the original transmission frequency of the input device. Accordingly, the adjustment time window may include at least the following two types: the first type of adjustment time window is used to increase the transmission frequency of the device status data, so that the frequency of the client device transmitting the device status data to the cloud server is greater than the frequency of the input device transmitting the device status data to the client device. For example, the first type of adjustment time window may include the interpolation sending time window mentioned below, and / or the interpolation backtracking time window. The second type of adjustment time window is used to reduce the transmission frequency of the device status data, so that the frequency of the client device transmitting the device status data to the cloud server is less than the frequency of the input device transmitting the device status data to the client device. For example, the second type of adjustment time window may include the merged sending time window mentioned below.

[0062] Step S130: Processing the device status data transmitted by the input device according to the adjusted time window; wherein the processed device status data is used to be sent to the cloud server of the cloud game, so that the cloud server can generate user control instructions according to the received device status data.

[0063] According to the adjustment time window, the device status data transmitted by the input device is adjusted to adjust the transmission frequency of the device status data transmitted by the client device to the cloud server, so that the transmission frequency of the device status data transmitted by the client device to the cloud server is different from the transmission frequency of the device status data transmitted by the input device to the client device.

[0064] Normally, the transmission frequency of the device status data transmitted by the client device to the cloud server is equal to the transmission frequency of the device status data transmitted by the input device to the client device, that is, after the client device receives the device status data from the input device, it immediately synchronizes the received device status data to the cloud server. The adjustment process in the present disclosure is intended to change the transmission frequency of the device status data transmitted by the client device to the cloud server, so as to flexibly adapt to the business needs of cloud games. Among them, the adjustment process may include: an interpolation-type adjustment process for increasing the transmission frequency of the device status data transmitted by the client device to the cloud server (specifically implemented by the first type of adjustment time window mentioned above), and / or a merging-type adjustment process for reducing the transmission frequency of the device status data transmitted by the client device to the cloud server (specifically implemented by the second type of adjustment time window mentioned above).

[0065] In addition, it should be noted that the device state data in the present disclosure is used to characterize various interactive operations performed by the user through the input device, and accordingly, the cloud server is used to generate user control instructions corresponding to the device state data. For example, the device state data is used to characterize various operations such as clicks, slides, and drags performed by the user on the game elements in the cloud game. Among them, the game elements may include various types such as game props, virtual characters, and interactive elements. It can be seen that the device state data in the present disclosure is not only used to characterize the operating state of the input device itself, but also can be combined with the current game screen to characterize the interactive operations triggered by the user for the current game screen. Accordingly, after receiving the device state data, the cloud server needs to combine the game screen content corresponding to the device state information to determine the interactive operation triggered by the user, and then generate a user control instruction (also called an interactive instruction), and calculate the update result of the game screen based on the user control instruction, so as to transmit the game screen updated according to the user control instruction to the client device. Therefore, in the unoptimized related scheme, since the transmission frequency of the device status data transmitted from the client device to the cloud server is equal to the transmission frequency of the device status data transmitted from the input device to the client device, for input devices with a higher transmission frequency, the cloud server will frequently generate user control instructions corresponding to the device status information. If the generation frequency of user control instructions exceeds the game frame rate, the screen update operation executed by the background for the user control instructions will not be presented due to the untimely screen refresh, which will lead to a huge waste of computing resources. For input devices with a lower transmission frequency, the generation frequency of user control instructions will be lower, and the screen cannot be driven to update in time.

[0066] In the method disclosed herein, the transmission frequency of the device status data transmitted by the input device to the client device can be compared with the game frame rate of the cloud game, and when the comparison result meets the preset adjustment condition, the device status data transmitted by the input device is processed according to the adjustment time window adapted to the comparison result, so as to send the processed device status data to the cloud server, so that the cloud server can generate corresponding user control instructions based on the processed device status data.

[0067] It can be seen that in the present disclosure, according to the comparison result between the transmission frequency of the device status data transmitted by the input device to the client device and the game frame rate, the device status data transmitted by the input device can be processed according to the adjustment time window, so as to facilitate the adjustment of the data transmission frequency between the client device and the cloud game according to the processing result, and avoid the problem that the device performance of the input device is fixed and cannot be adapted to various types of cloud games, so that the client device can flexibly adjust the transmission frequency of the device status data of the input device according to the needs of the cloud game. In addition, this method can effectively avoid the generation of unnecessary user control instructions in the cloud, and avoid the execution of unnecessary screen update calculations. This method can flexibly adapt to input devices with various transmission frequencies, and no matter what the performance of the input device used by the user is, it will not affect the operation process of the cloud game, and can provide stable cloud game image quality for users of input devices with different performance.

[0068] In addition, in order to achieve better technical effects, those skilled in the art may also make various changes and modifications to the above embodiments:

[0069] In an optional implementation, the preset adjustment conditions include: a first type of adjustment condition corresponding to a first type of comparison result, and / or a second type of adjustment condition corresponding to a second type of comparison result; wherein the first type of comparison result is used to characterize a transmission frequency lower than the game frame rate of the cloud game, and the second type of comparison result is used to characterize a transmission frequency higher than the game frame rate of the cloud game. It can be seen that the present disclosure can flexibly set different adjustment conditions according to the size relationship between the transmission frequency and the game frame rate. Accordingly, the device status data transmitted by the input device can be processed in the following manner: the device status data transmitted by the input device is processed according to the processing strategy corresponding to the condition type of the preset adjustment condition; wherein the processing strategy corresponding to the first type of adjustment condition includes: an interpolation type processing strategy for increasing the transmission frequency between the client device and the cloud server; the processing strategy corresponding to the second type of adjustment condition includes: a merging type processing strategy for reducing the transmission frequency between the client device and the cloud server.

[0070] In an optional implementation, when the comparison result between the transmission frequency and the game frame rate of the cloud game meets the first type of adjustment condition, it means that the transmission frequency is lower than the game frame rate of the cloud game. Accordingly, the adjustment time window adapted to the comparison result may include: an interpolation sending time window, and / or an interpolation backtracking time window.

[0071] Among them, the interpolation sending time window is used to characterize the sending timing of the device status data obtained according to the interpolation type processing strategy. For example, after the interpolated device status data is generated, it is determined whether the interpolated device status data needs to be sent to the cloud server based on whether the interpolation sending time window has been reached. By setting the interpolation sending time window, the transmission frequency of the interpolated device status data when it is transmitted to the cloud server can be controlled so that the transmission frequency of the interpolated device status data when it is transmitted to the cloud server matches the game frame rate.

[0072] The interpolation backtracking time window is used to characterize the number of device status data for interpolation prediction. In order to increase the transmission frequency of device status data from the client device to the cloud server, it is necessary to perform interpolation processing on the acquired device status data. In each interpolation processing, it is necessary to perform interpolation prediction based on multiple device status data acquired before this interpolation processing. Accordingly, the interpolation backtracking time window is used to characterize the number of multiple device status data based on which the interpolation prediction is performed.

[0073] In a specific implementation, the interpolation backtracking time window can be represented by a fixed time length, which can be determined based on the return rate of the mouse. Alternatively, the interpolation backtracking time window can also be represented by the amount of device status data transmitted by the input device. For example, the interpolation backtracking time window can be fixedly set to the time length for the input device to transmit a preset number of device status data, which can be dynamically adjusted based on factors such as the transmission delay of the input device.

[0074] In an optional implementation, processing the device status data transmitted by the input device according to the adjustment time window, and sending the processed device status data to the cloud server of the cloud game may include: obtaining a preset number of device status data that match the interpolation backtracking time window; performing interpolation prediction based on the preset number of device status data, and when the interpolation sending time window is arrived, sending the device status data obtained by the interpolation prediction to the cloud server of the cloud game.

[0075] Among them, the preset number of device state data matching the interpolation backtracking time window refers to: device state data used for interpolation prediction. In an optional implementation, the preset number of device state data matching the interpolation backtracking time window can be obtained from the device state data sent to the cloud server of the cloud game. In the scenario where the mouse return rate is low, in order to improve the fluency of the operation, it is not only necessary to send the device state data transmitted by the input device to the cloud server in a timely manner, but also to further perform interpolation prediction so that the device state data obtained by the interpolation prediction is also sent to the cloud server, thereby increasing the frequency of data transmission from the client device to the cloud server. Among them, the device state data sent to the cloud server of the cloud game can include two types, one type is the device state data transmitted by the input device, which can also be called the original device state data; the other type is the device state data obtained by the interpolation prediction, which can also be called the predicted device state data. It can be seen that both the original device state data and the predicted device state data can be used as a reference for interpolation prediction. Optionally, the interpolation backtracking time window can be determined according to the transmission frequency of the device state data transmitted by the input device to the client device. The interpolation sending time window may be determined according to an interpolation multiple adapted to the comparison result.

[0076] In an optional implementation, when the comparison result between the transmission frequency and the game frame rate of the cloud game satisfies the second type of adjustment condition, it means that the transmission frequency is higher than the game frame rate of the cloud game, and accordingly, the adjustment time window adapted to the comparison result may include: a combined sending time window; wherein the combined sending time window is used to characterize the timing of sending the device status data obtained according to the combined processing strategy. For example, the larger the window length of the combined sending time window, the lower the frequency of the client device sending the device status data to the cloud server; the smaller the window length of the combined sending time window, the higher the frequency of the client device sending the device status data to the cloud server.

[0077] In an optional implementation, the combined sending time window can be determined in the following manner: First, the combined sending time window in the initial state is determined according to the game frame rate. For example, the window length of the combined sending time window can be determined according to twice the game frame rate. Then, according to the currently detected network state, the combined sending time window in the initial state is dynamically adjusted. For example, when the network state is poor, the combined sending time window is increased; when the network state is good, the combined sending time window is reduced. By dynamically adjusting the combined sending time window according to the network state, the window length of the combined sending time window can be dynamically adapted to various network conditions.

[0078] In an optional implementation, according to the adjustment time window, the device state data transmitted by the input device is processed, and the processed device state data is sent to the cloud server of the cloud game, including: when the merged sending time window has not arrived, the multiple device state data received from the input device are merged and processed to obtain the merged device state data; when the merged sending time window arrives, the merged device state data is sent to the cloud server of the cloud game, so that the cloud server can determine the game screen frame corresponding to the merged device state data, and generate the user control instruction corresponding to the game screen frame. Among them, in the related art, since it has not been merged, the cloud server will receive more device state data, and it is necessary to process each received device state data one by one. Among them, for each device state data, the cloud server needs to determine the game screen frame corresponding to the device state data, and generate the user control instruction corresponding to the game screen frame. However, if the transmission frequency of the device state data is too high, it will cause the cloud server to repeatedly generate multiple user control instructions when the game screen frame remains unchanged, thereby greatly increasing the processing overhead of the cloud server and reducing the transmission performance of the cloud game. In this application, the amount of device status data after merging is greatly reduced. Therefore, the cloud server only needs to determine the corresponding game screen frame and generate game control instructions for the merged device status data, which greatly reduces the processing overhead of the cloud server and improves the transmission performance of cloud games. In addition, through reasonable configuration, the transmission frequency of the merged device status data can be matched with the update frequency of the game screen frame (i.e., the game frame rate), thereby avoiding the cloud server from generating unnecessary game control instructions and improving the operating efficiency of the cloud server.

[0079] Among them, when the merged sending time window has not yet arrived, when the multiple device status data received from the input device are merged and processed, various types of merge processing operations such as deduplication operation and update operation can be performed on the multiple device status data, as long as the number of device status data after the merged processing can be reduced. For example, if multiple device status data with the same content are received in the same merged sending time window, only one device status data can be retained through deduplication operation. For another example, if multiple device status data with different contents are received in the same merged sending time window, for each device status data currently received, combined with the previously received device status data, the latest device status data is calculated, so that only the latest device status data is retained through iterative update. In short, through the merge processing operation, the number of device status data can be greatly reduced, and accordingly, the transmission frequency of the merged device status data (i.e., the frequency of transmission from the client device to the cloud server) is less than the device status data before the merge (i.e., the frequency of transmission from the input device to the client device). It can be seen that the merge processing is used to reduce the transmission frequency of the device status data, thereby reducing the processing overhead of the cloud server.

[0080] Through the above method, the merged device state data can be sent to the cloud server when the merge sending time window arrives, thereby reducing the frequency of data transmission between the client device and the cloud server, and avoiding the problems of excessive load and high traffic consumption caused by the high frequency of data transmission between the client device and the cloud server. Among them, when performing the merge processing, it is necessary to merge the data according to the relative position relationship between each device state data and the game element in the corresponding game screen (i.e., the screen content in the game screen frame). For example, if the relative position relationship between multiple adjacent device state data and the corresponding game element has not changed, only the latest device state data among the multiple device state data can be retained. For example, if the relative position relationship between multiple adjacent device state data and the corresponding game element has changed, the relative position relationship between the corresponding game element and the corresponding device state data can be retained. In short, the specific implementation method of the merge processing needs to be determined in combination with the interaction state between the device state data and the game elements in the game screen, and the device state data whose interaction state has changed with the game element is retained during the merge processing, so that the user control instructions generated by the cloud server based on the received device state data are more accurate.

[0081] In an optional implementation, the input device may include: a mouse; the frequency at which the input device transmits device status data to the client device is determined based on the return rate of the mouse; and the device status data includes: click-type status parameters for characterizing the mouse click status, and / or position-type status parameters for characterizing the mouse movement position.

[0082] In an optional implementation, the input device may include: a keyboard; and the method further includes the following operations: in response to receiving a key press message sent by the keyboard, querying the current state of the key corresponding to the key press message in the key state table; when the current state of the key is not pressed, updating the current state of the key to pressed, and sending the received key press message to the cloud server; when the current state of the key is pressed, ignoring the key press message; wherein, in the case of receiving a key lift message sent by the keyboard, updating the current state of the key corresponding to the key lift message in the key state table from pressed to not pressed, and sending the received key lift message to the cloud server. It can be seen that in this method, the key state table can be dynamically maintained, and the key press state and lift state can be updated to the key state table, so that with the help of the key state table, it is possible to avoid repeatedly sending multiple key press messages to the cloud server for continuously pressed keys, thereby improving processing efficiency.

[0083] In addition, in order to further optimize the performance of cloud games, when processing the device status data transmitted by the input device, the processing method can be dynamically adjusted in combination with the interaction status between the device status data and the game elements. For example, when interpolation prediction processing is required, the interaction status between the device status data and the game elements can be further monitored, and when it is detected that the device status data and the game elements are in an interactive state, the interpolation prediction processing is performed. Alternatively, when it is detected that the device status data and the game elements are in an interactive state, the window length of the interpolation sending time window is reduced (i.e., the interpolation multiple is increased) to achieve better optimization effects. Among them, the device status data and the game elements are in an interactive state means that the device status data is used to characterize that the user is interacting with the game elements through the input device, for example, the user is controlling the input device to click on a certain interactive game element. At this time, in order to achieve a better control effect, the interpolation multiple can be increased to make the user operation smoother.

[0084] Similarly, when it is necessary to perform merge processing, the interaction status between the device status data and the game elements can be further monitored, and when it is detected that the device status data and the game elements are in an interactive state, the merge processing is performed through a merge sending time window of the first length; when it is not detected that the device status data and the game elements are in an interactive state, the merge processing is performed through a merge sending time window of the second length. Among them, the first length is less than the second length. In this way, in the scenario where the input device is not performing an interactive operation (for example, the mouse is just sliding randomly), the resource consumption of network transmission can be reduced as much as possible through a larger merge sending time window; and in the scenario where the input device is performing an interactive operation (for example, the mouse is dragging or clicking on a certain interactive game element), the real-time performance of the user operation can be improved as much as possible through a smaller merge sending time window.

[0085] It can be seen that the window length of the adjustment time window in the present disclosure can be dynamically adjusted according to whether the device status data is in an interactive state with the game element: when the device status data is in an interactive state with the game element, the window length of the adjustment time window is configured to increase the transmission frequency of the client device transmitting the device status data to the cloud server; when the device status data is not in an interactive state with the game element, the window length of the adjustment time window is configured to reduce the transmission frequency of the client device transmitting the device status data to the cloud server.

[0086] In addition, the window length of the adjustment time window can be further determined according to the element priority of the game element in the interactive state. For example, when the device state data interacts with a game element with a higher priority (such as a higher interactive real-time requirement), the window length of the adjustment time window is configured to increase the transmission frequency of the client device transmitting the device state data to the cloud server; when the device state data interacts with a game element with a lower priority (such as a lower interactive real-time requirement), the window length of the adjustment time window is configured to reduce the transmission frequency of the client device transmitting the device state data to the cloud server.

[0087] To facilitate understanding, an example is used below to introduce the implementation of the above embodiment.

[0088] The following is an explanation of some of the relevant concepts that may be involved in this example:

[0089] Cloud Gaming: Cloud gaming is a game mode in which the game is run and rendered on a remote server, and the player receives the video stream and sends control signals in real time through the network. The advantage of cloud gaming is that it reduces the performance requirements of the player's device, making high-quality gaming experience possible for more users. However, cloud gaming has high requirements for network latency and bandwidth to ensure a good gaming experience.

[0090] Delay: Delay refers to the time it takes for data to reach the receiver from the sender in network communications. Delay includes propagation delay (the time it takes for data to propagate in the channel), processing delay (the time it takes for a network device to process a data packet), queuing delay (the time it takes for a data packet to wait for transmission in a network device), and transmission delay (the time it takes for a data packet to actually be transmitted between the sender and the receiver). Lower latency helps achieve high-quality real-time communications and online gaming experiences. Latency is usually affected by factors such as network signal strength, transmission distance, and device performance.

[0091] Game Control Data: also called game control data or user control instructions, is data used to transmit player operations and game status information in online games. Game control data needs to be transmitted in real time and accurately to ensure game synchronization and interaction between players. The transmission performance of game control data directly affects the quality of the game experience. The device status data in this disclosure is used to generate game control data. For example, mouse clicks, movements, and other operations will trigger corresponding game control data.

[0092] Frame rate: Frame rate refers to the number of frames updated per second in a video or game. For example, the frame rate of common games is currently set at 30, 60, 90, and 120 frames, with 60 frames being the most common.

[0093] RTC (Real-Time Communication): Real-Time Communication (RTC) refers to the communication technology that transmits voice, video and data in real time over the network. RTC technology requires a low-latency, high-reliability network environment and is often used in online conferences, video chats, real-time online games and other scenarios.

[0094] With the rapid development and popularization of Internet technology, cloud gaming has gradually attracted widespread attention as a new gaming model. Cloud gaming transfers game running and rendering operations from local devices to remote servers. Players receive video streams transmitted by the server and send control data in real time through the network. This model reduces the performance requirements of players' devices, allowing more users to easily enjoy high-quality gaming experience. However, cloud gaming has high requirements for network latency and bandwidth to ensure real-time performance and gaming experience. Therefore, how to effectively reduce the transmission latency of control data in cloud gaming has become an urgent problem that needs to be solved.

[0095] In related technologies, the device status data transmitted by the user to the client device through the input device will be synchronously sent to the cloud server, so that the cloud server can generate corresponding user control instructions based on the received device status data, and calculate the game logic of the cloud game based on the generated user control instructions, so as to obtain the next frame of the game screen.

[0096] At present, with the development of technology, many users use professional gaming mice, and their mouse return rate generally reaches 1000Hz or even higher. When the device status data corresponding to all mouse operations are sent to the server, the cloud server will frequently receive the device status data and frequently generate corresponding user control instructions, which will inevitably cause network congestion, increase player operation delays, reduce user experience, and even increase power consumption of the client and server. It can be seen that in the relevant technology, cloud games need to generate corresponding user control instructions for each operation information of input devices such as mice, without optimizing the processing based on factors such as user devices, network conditions, and game conditions.

[0097] In order to solve the above problems, this example can flexibly adapt to various situations and select the best processing method to improve the player's user experience. Figure 2 A schematic diagram of the process of the data transmission method based on cloud gaming in this example is shown. Figure 2 As shown, the specific steps include:

[0098] Step S201: Compare the transmission frequency of the acquired input device to transmit the device status data to the client device with the game frame rate of the cloud game to obtain a comparison result.

[0099] After the game is started, the mouse return rate can be continuously detected, and the highest mouse return rate can be refreshed in real time, and the transmission frequency of the input device to the client device to transmit the device status data can be determined according to the highest mouse return rate. In addition, the game frame rate can be obtained from the server, and after a small window period, the highest mouse return rate detected within the current small window period can be compared with the game frame rate. The length of the above-mentioned small window period can be determined according to the game frame rate.

[0100] Step S202: determining whether the comparison result satisfies a preset adjustment condition.

[0101] Specifically, the client device can determine whether the preset adjustment condition is met based on the value of the comparison result. The client device can be a computer device of the client, or a device such as a mouse controller of the client. For example, the mouse controller can determine whether adjustment processing, such as interpolation processing or deduplication processing, is required based on the comparison result between the game frame rate and the mouse return rate.

[0102] Wherein, when the comparison result satisfies the first type of adjustment condition, step S203 and its subsequent steps are executed; when the comparison result satisfies the second type of adjustment condition, step S205 and its subsequent steps are executed.

[0103] Step S203: when the comparison result satisfies the first type of adjustment condition, determining an interpolation sending time window and an interpolation backtracking time window that match the comparison result.

[0104] If the comparison result satisfies the adjustment condition of the first type, it means that the transmission frequency of the device status data transmitted by the input device to the client device is lower than the game frame rate, which may cause problems such as unsmooth operation and freezing. Therefore, in this example, when it is detected that the transmission frequency of the device status data is lower than the game frame rate, the interpolation sending time window and the interpolation backtracking time window need to be determined according to the comparison result.

[0105] Among them, the interpolation sending time window is used to determine the sending timing of the device status data obtained according to the interpolation type processing strategy, which can be specifically determined according to the interpolation multiple. For example, assuming that the transmission frequency of the input device to the client device to transmit the device status data is half of the game frame rate, the interpolation multiple is determined to be twice, that is: insert a predicted device status data between every two original device status data, and accordingly, the window length of the interpolation sending time window can be half of the time interval between the two original device status data, so that the frequency of the client device transmitting the device status data to the cloud server is twice the frequency of the input device transmitting the device status data to the client device. In addition, if the interpolation multiple is three times, that is: insert two predicted device status data between every two original device status data, and accordingly, the window length of the interpolation sending time window can be one-third of the time interval between the two original device status data, so that the frequency of the client device transmitting the device status data to the cloud server is three times the frequency of the input device transmitting the device status data to the client device.

[0106] Among them, the interpolation backtracking time window is used to determine the number of device state data for interpolation prediction. For example, if the interpolation backtracking time window corresponds to the duration of the input device transmitting N device state data to the client device, then for any predicted device state data, it is necessary to perform interpolation prediction based on the input device transmitting N device state data to the client device. Optionally, for any predicted device state data, in addition to performing interpolation prediction based on the input device transmitting N original device state data to the client device, interpolation prediction can be further performed in combination with the predicted device state data generated within the corresponding interpolation backtracking time window. For example, when the interpolation multiple is three times, that is, two predicted device state data are inserted between every two original device state data, the subsequent predicted device state data can be obtained based on the previous predicted device state data and the original device state data.

[0107] Step S204: Obtain a preset number of device status data that match the interpolation backtracking time window, perform interpolation prediction based on the preset number of device status data, and when the interpolation sending time window arrives, send the device status data obtained by the interpolation prediction to the cloud server of the cloud game.

[0108] Among them, according to the current time point and the window length of the interpolation backtracking time window, a preset number of device status data matching the interpolation backtracking time window are obtained. Specifically, the current time point can be used as the end time point, and the time point before the current time point and the time interval with the current time point equal to the window length of the interpolation backtracking time window can be used as the start time point, and the device status data between the start time point and the end time point can be determined as a preset number (for example, 5-8) of device status data matching the interpolation backtracking time window, wherein the preset number of device status data can include both the original device status data and the predicted device status data. In addition, the arrival of the interpolation sending time window means that the time interval between the current time point and the time point when the client device last sent device status data to the cloud server reaches the window length of the interpolation sending time window.

[0109] Among them, predictive interpolation operations can be performed for some mice with low return rates to improve the operation experience of such users. For example, a mouse with a return rate lower than 125Hz (for example, an operation message is received once every 8ms) can be regarded as a mouse with a low return rate, and its return rate can be doubled by predictive interpolation through the method in this example. Specifically, when it is detected that the frequency of user reporting mouse messages is too low, the interpolation prediction function of the client will be triggered. The prediction function can use linear fitting as a prediction algorithm. This function will take 5-8 device status data points in the historical time as a prediction reference. Among them, 8 device status data points can be used for linear fitting first. When the residual sum is lower than the set threshold, it is considered that the fitting is successful, and a device status data point after interpolation prediction is obtained, and the position of the next device status data point is predicted, and the device status data point obtained by the interpolation prediction is sent at half of its historical average return interval (such as half of the historical average return interval of a 125Hz device is 4ms). Among them, half of the historical average return interval is a form of interpolation sending time window. In addition, if 8 device status data points cannot be well fitted, 7 device status data points can be used, and so on, and at least 5 device status data points can be used for fitting. The set thresholds when using different numbers of device status data points for fitting can be different to improve the rationality of the prediction.

[0110] It can be seen that the window length of the interpolation backtracking time window can be dynamically adjusted according to the fitting results. If the fitting results are not ideal (that is, the residual sum of the fitting results is greater than the set threshold), the window length of the interpolation backtracking time window can be shortened to use fewer device status data points for fitting, avoiding the fitting failure caused by the influence of some irregular interference data points, and improving the prediction success rate.

[0111] In a specific example, the number of mouse positions (i.e., device status data points) that need to be traced back for prediction can be determined based on the mouse return rate, and the maximum number can be 8. Assuming that N mouse positions need to be traced back forward, linear fitting is performed on the first N mouse positions. If the fitting difference is lower than the set threshold, the fitting is considered successful, and the next point is predicted and sent. On the contrary, N-1 mouse positions are traced back for prediction until 5 points are not predicted successfully. In statistics, increasing the number of data points can usually improve the accuracy of the fitting model. Using 8 points for fitting can provide enough information so that the fitting model can more accurately capture the trend and change of the data. Through experimental and theoretical analysis, the inventor found that 8 points is a balance point, which can provide enough historical information without causing excessive computational complexity. Therefore, preferably, this example selects 8 device status data points for fitting, which corresponds to a longer time window to reflect the trend and change of user operations. Compared with using fewer data points, 8 points can capture the user's operation mode more comprehensively, thereby improving the accuracy of the prediction. At the same time, the historical data of the 8 points will not be too outdated, and can better reflect the current operating status, avoiding the prediction lag problem caused by too long historical data.

[0112] Among them, the interpolation prediction operation is based on the past mouse operation status to fit the operation status that may occur in the future. Based on the continuity of the operation action, the interpolation prediction can effectively improve the smoothness of the mouse operation.

[0113] Step S205: When the comparison result satisfies the second type of adjustment condition, a combined sending time window is determined according to the game frame rate.

[0114] Among them, the higher the game frame rate, the smaller the combined sending time window; the lower the game frame rate, the larger the combined sending time window. It can be seen that the combined sending time window is adapted to the game frame rate. Preferably, the combined sending time window can be determined based on the game frame rate and the network status. For example, the combined sending time window in the initial state can be determined based on the game frame rate, and then the combined sending time window in the initial state can be dynamically adjusted based on the currently detected network status to obtain the adjusted combined sending time window.

[0115] For example, the current frame rate of cloud gaming applications can be 30FPS and 60FPS, that is, the screen image is updated once every 33.3ms and 16.6ms. In theory, it is only necessary to send control data to the game at a frequency less than this interval to meet the game's calculation requirements for the next frame. However, considering that the timing of the game's access to control data is uncertain, the control data can be sent as frequently as possible to ensure that the data content is closer to the user's actual operation when the game accesses this data next time. Therefore, this example dynamically sets the merged sending time window according to the remote game frame rate. For example, at a game frame rate of 60FPS, 4ms can be set as the merged sending time window (that is, more than twice the time interval of the game screen). Similarly, at a game frame rate of 30FPS, 8ms can be set as the merged sending time window. For example, for low-frame rate games, sending device status data to the cloud server at a frequency that is too high will result in a waste of resources. However, if the frequency is too low, the game may miss the opportunity to process the control data, resulting in processing in the next frame, which increases the player's operation delay. Therefore, setting the combined sending time window to more than twice the time interval of the game screen can effectively ensure the timely processing of control data.

[0116] In addition, the combined sending time window can be dynamically set according to the existing network conditions. The combined sending time window is to reduce network congestion and reduce operation delays under normal network conditions. However, in weak networks, especially networks with large delays, even sending an operation message once every 4ms will still cause network congestion and reduce the player's operation delay. Therefore, this example can dynamically adjust the combined sending time window in different ways under different network conditions based on network conditions, mainly based on RTT. For example, several network status indicators can be set as interval dividing lines, and the combined sending time window can be increased or adjusted for client devices that reach a certain network status indicator to adapt to various network environments and improve user experience.

[0117] Step S206: When the merged sending time window has not yet arrived, merge the multiple device status data received from the input devices to obtain merged device status data; when the merged sending time window arrives, send the merged device status data to the cloud server of the cloud game.

[0118] The arrival of the merged sending time window means that the time interval between the current time and the last time the client device sent device status data to the cloud server reaches the window length of the merged sending time window. Through the merge processing, the network congestion between the client device and the cloud server can be reduced.

[0119] Specifically, for mouse events, the client device detects the reporting frequency of the current mouse operation event and infers the user's mouse reporting rate. For a mouse with a high reporting rate, the client device of the cloud game will combine multiple control data (i.e., device status data) into one copy and delay it to send at a fixed interval. For example, in the related art, for a mouse with a reporting rate of 1000Hz, the client device needs to send 1000 copies of control data to the cloud server per second, which causes a lot of transmission pressure between the two ends.

[0120] In this example, the client device saves the cumulative relative offset of the mouse message within the window period of the merged sending time window, and saves the latest absolute coordinates of the mouse, so that after the window period arrives, these messages are combined into a new mouse message to be sent (that is, multiple device status data are combined into one device status data to be sent). Among them, the mouse message is a message used to carry device status data. Correspondingly, when the cloud server generates user control instructions based on the received device status data, it is implemented in the following way: determine the game screen frame corresponding to the merged device status data, and generate user control instructions corresponding to the game screen frame.

[0121] For example, the client device can set the merged sending time window to 4ms or 8ms, that is, reduce the frequency of transmission from the client device to the cloud server to 250Hz or 125Hz. This method can significantly reduce the amount of data transmission between the two ends, thereby reducing the load on the client and server, reducing the possibility of network congestion, and shortening operation delays. It can be seen that when it is necessary to deduplicate multiple device status data, the merged sending time window will be obtained by combining network indicators and game frame rate, so that multiple device status data can be aggregated according to the time window and sent to the server to complete the merge operation within a window period.

[0122] It can be seen that, if the optimization method in this application is not adopted, the multiple device status data before the merge process will be directly transmitted to the cloud server, that is, the transmission frequency of the multiple device status data before the merge process is high, which will increase the processing overhead of the cloud server. In the case of adopting the optimization method in this application, only the device status data after the merge process is transmitted to the cloud server. Since the number of device status data is reduced after the merge operation is performed, the transmission frequency of the device status data after the merge process is low, which can reduce the processing overhead of the cloud server and improve processing efficiency.

[0123] Of course, if the frame rate and the mouse reporting rate match, no special processing is required. It can be seen that the device status data in the present disclosure can be represented by various forms such as mouse messages and user control instructions, and the present disclosure does not limit this.

[0124] Among them, the Nyquist sampling theorem points out that in order to accurately reconstruct a signal, the sampling frequency must be at least twice the highest frequency of the signal. Therefore, in order to ensure that the control data can be adopted in time, this example needs to send the control data multiple times in each screen update interval. According to the Nyquist sampling theorem, in order to ensure that the control data can be adopted in time in each screen update interval, a sufficiently high sampling frequency needs to be selected. Specifically, it is necessary to send the control data at least twice in each screen update interval. In order to further improve the timeliness and accuracy of the control data, it is possible to choose to send more control data in each screen update interval. Through experience and theoretical analysis, the inventor found that sending 4 control data (i.e., device status data) in each screen update interval is a more reasonable choice. That is: the frequency of the client device transmitting device status data to the cloud server can be 4 times the screen update frequency. Through experiments and user feedback, the inventor verified the impact of different window sizes on the game experience. The results show that the 4ms and 8ms window settings can significantly improve the responsiveness and smoothness of the game at game frame rates of 60FPS and 30FPS.

[0125] In addition, in this example, in addition to being able to optimize for mouse messages, keyboard messages can also be optimized: only the keyboard control data that has changed is sent to the cloud server, while the keyboard control data that has not changed is ignored. For example, for keyboard events, when the user keeps pressing a key, the system will continue to receive the key press message, and this message is redundant for the remote game. In related technologies, the cloud game client will continue to send the key press message to the cloud server multiple times when it keeps receiving the key press message, thereby causing data redundancy and network congestion.

[0126] To solve the above problem, in this example, the client device only sends keyboard operation data with state changes to the cloud server. Accordingly, the client device will maintain a data structure (such as a data table, queue, etc.) for saving the keyboard state. When a key is pressed, it will first check whether the information of this key already exists in the data structure. If so, it proves that this state has been sent to the cloud server before and does not need to be processed repeatedly, so the message is discarded. When the key is released, the keyboard state in the data structure is deleted, and the release event is sent to the server, thereby reducing the amount of data transmission.

[0127] In summary, no matter in any scenario, the operating experience of users using the processing strategy in this example can be improved, mainly reflected in lower operating delays and smoother operations. For example, for users of high-return rate mice, the operating delay can be reduced, the client and server loads can be reduced, and the network traffic consumption can be reduced. For users in poor network conditions, the operating delay can be reduced, and the deterioration of the experience caused by network problems can be reduced. For users with low mouse return rates, the operating experience will be smoother and more fluid.

[0128] In addition, another embodiment of the present disclosure also provides a data transmission method based on cloud gaming, which is applied to a cloud server, and the method includes: receiving device status data transmitted by a client device; generating a user control instruction according to the device status data, and sending the user control instruction to the client device. The device status data transmitted by the client device is the device status data adjusted by the adjustment method mentioned above.

[0129] Figure 3 A schematic diagram of a data transmission device based on cloud gaming provided by another embodiment of the present disclosure is shown, and the device specifically includes:

[0130] An acquisition module 31, adapted to acquire a transmission frequency of the input device transmitting device status data to the client device;

[0131] A determination module 32, adapted to determine an adjustment time window adapted to the comparison result when a comparison result between the transmission frequency and the game frame rate of the cloud game satisfies a preset adjustment condition;

[0132] The processing module 33 is adapted to process the device status data transmitted by the input device according to the adjustment time window, and send the processed device status data to the cloud server of the cloud game.

[0133] In an optional implementation, the preset adjustment condition includes: a first type of adjustment condition corresponding to a first type of comparison result, and / or a second type of adjustment condition corresponding to a second type of comparison result; wherein the first type of comparison result is used to characterize that the transmission frequency is lower than the game frame rate of the cloud game, and the second type of comparison result is used to characterize that the transmission frequency is higher than the game frame rate of the cloud game;

[0134] Wherein, the processing of the device status data transmitted by the input device comprises: processing the device status data transmitted by the input device according to a processing strategy corresponding to the condition type of the preset adjustment condition;

[0135] Among them, the processing strategies corresponding to the first type of adjustment conditions include: an interpolation type processing strategy for increasing the transmission frequency between the client device and the cloud server; the processing strategies corresponding to the second type of adjustment conditions include: a merging type processing strategy for reducing the transmission frequency between the client device and the cloud server.

[0136] In an optional implementation, when the comparison result between the transmission frequency and the game frame rate of the cloud game satisfies the first type of adjustment condition, the adjustment time window adapted to the comparison result includes: an interpolation sending time window, and / or an interpolation backtracking time window;

[0137] The interpolation sending time window is used to characterize the sending timing of the device status data obtained according to the interpolation type processing strategy; the interpolation backtracking time window is used to characterize the amount of device status data for interpolation prediction.

[0138] In an optional implementation, the processing module is specifically adapted to:

[0139] Acquire a preset number of device status data matching the interpolation backtracking time window;

[0140] Performing interpolation prediction based on the preset number of device status data, and sending the device status data obtained by interpolation prediction to the cloud server of the cloud game when the interpolation sending time window arrives;

[0141] The interpolation backtracking time window is determined according to the transmission frequency of the device status data transmitted by the input device to the client device, and the interpolation sending time window is determined according to the interpolation multiple adapted to the comparison result.

[0142] In an optional implementation, when the comparison result between the transmission frequency and the game frame rate of the cloud game satisfies the adjustment condition of the second type, the adjustment time window adapted to the comparison result includes: merging the sending time window;

[0143] The combined sending time window is used to indicate the sending timing of the device status data obtained according to the combined processing strategy.

[0144] In an optional implementation manner, the combined sending time window is determined by:

[0145] Determine a combined sending time window in an initial state according to the game frame rate;

[0146] The combined sending time window in the initial state is dynamically adjusted according to the currently detected network state.

[0147] In an optional implementation, the processing module is specifically adapted to:

[0148] When the combined sending time window has not arrived, combining the multiple device status data received from the input device to obtain combined device status data;

[0149] When the combined sending time window arrives, the combined device status data is sent to the cloud server of the cloud game.

[0150] In an optional implementation, the input device includes: a mouse; a transmission frequency of the input device transmitting device status data to the client device is determined according to a return rate of the mouse;

[0151] Furthermore, the device status data includes: a click-related status parameter for characterizing a mouse click status, and / or a position-related status parameter for characterizing a mouse movement position.

[0152] In an optional implementation, the input device includes: a keyboard; and the apparatus further includes:

[0153] a key response module, adapted to query the current state of the key corresponding to the key press message in the key state table in response to receiving the key press message sent by the keyboard; if the current state of the key is not pressed, update the current state of the key to pressed, and send the received key press message to the cloud server; if the current state of the key is pressed, ignore the key press message;

[0154] Wherein, when a key-up message sent by the keyboard is received, the current state of the key corresponding to the key-up message in the key state table is updated from pressed to not pressed, and the received key-up message is sent to the cloud server. The above device can be applied to a client device. The specific working principle of each of the above modules can refer to the description of the corresponding part of the method embodiment, which will not be repeated here.

[0155] In addition, another embodiment of the present disclosure also provides a data transmission device based on cloud gaming, which can be applied to a cloud server and specifically includes: a receiving module, suitable for receiving device status data transmitted by a client device through the above method; a sending module, suitable for generating user control instructions based on the device status data, and sending the user control instructions to the client device.

[0156] Figure 4A schematic structural diagram of an electronic device according to another embodiment of the present disclosure is shown. The specific embodiment of the present disclosure does not limit the specific implementation of the electronic device.

[0157] like Figure 4 As shown, the electronic device may include: a processor (processor) 502 , a communication interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0158] in:

[0159] The processor 502 , the communication interface 504 , and the memory 506 communicate with each other via a communication bus 508 .

[0160] The communication interface 504 is used to communicate with other devices such as clients or other servers.

[0161] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above-mentioned video image detection method embodiment.

[0162] Specifically, the program 510 may include program codes, which include computer operation instructions.

[0163] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure. The one or more processors included in the electronic device may be processors of the same type, such as one or more CPUs. They may also be processors of different types, such as one or more CPUs and one or more ASICs.

[0164] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0165] The program 510 can be specifically used to enable the processor 502 to execute the corresponding operations in the above-mentioned video image detection method embodiment.

[0166] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device or other device. Various general purpose devices may also be used together with the teachings based thereon. According to the above description, it is apparent that the structure required for constructing such devices is. In addition, the present disclosure is not directed to any particular programming language either. It should be understood that various programming languages ​​may be utilized to implement the disclosure described herein, and the description of the specific languages ​​above is intended to disclose the best mode of implementation of the present disclosure.

[0167] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0168] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various disclosed aspects, in the above description of exemplary embodiments of the present disclosure, various features of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed disclosure requires more features than are expressly recited in each claim.

[0169] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0170] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present disclosure and form different embodiments. For example, any one of the claimed embodiments may be used in any combination.

[0171] The various component embodiments of the present disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the device according to the embodiments of the present disclosure. The present disclosure may also be implemented as a device or device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present disclosure may be stored on a computer-readable medium, or may have the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0172] It should be noted that the above embodiments illustrate the present disclosure rather than limit the present disclosure, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present disclosure may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A data transmission method based on cloud gaming, applied to a client device, the method comprising: Acquire a transmission frequency of the input device transmitting device status data to the client device; When a comparison result between the transmission frequency and the game frame rate of the cloud game satisfies a preset adjustment condition, determining an adjustment time window adapted to the comparison result; According to the adjustment time window, the device status data transmitted by the input device is processed; wherein the processed device status data is used to be sent to the cloud server of the cloud game, so that the cloud server can generate user control instructions based on the received device status data.

2. The method according to claim 1, wherein the preset adjustment condition comprises: a first type of adjustment condition corresponding to a first type of comparison result, and / or a second type of adjustment condition corresponding to a second type of comparison result; wherein the first type of comparison result is used to characterize that the transmission frequency is lower than a game frame rate of the cloud game, and the second type of comparison result is used to characterize that the transmission frequency is higher than a game frame rate of the cloud game; Wherein, the processing of the device status data transmitted by the input device comprises: processing the device status data transmitted by the input device according to a processing strategy corresponding to the condition type of the preset adjustment condition; Among them, the processing strategies corresponding to the first type of adjustment conditions include: an interpolation type processing strategy for increasing the transmission frequency between the client device and the cloud server; the processing strategies corresponding to the second type of adjustment conditions include: a merging type processing strategy for reducing the transmission frequency between the client device and the cloud server.

3. The method according to claim 2, wherein: When the comparison result between the transmission frequency and the game frame rate of the cloud game satisfies the first type of adjustment condition, the adjustment time window adapted to the comparison result includes: an interpolation sending time window and / or an interpolation backtracking time window; The interpolation sending time window is used to characterize the sending timing of the device status data obtained according to the interpolation type processing strategy; the interpolation backtracking time window is used to characterize the amount of device status data for interpolation prediction.

4. The method according to claim 3, wherein: The step of processing the device status data transmitted by the input device according to the adjustment time window and sending the processed device status data to the cloud server of the cloud game comprises: Acquire a preset number of device status data matching the interpolation backtracking time window; Performing interpolation prediction based on the preset number of device status data, and sending the device status data obtained by interpolation prediction to the cloud server of the cloud game when the interpolation sending time window arrives; The interpolation backtracking time window is determined according to the transmission frequency of the device status data transmitted by the input device to the client device, and the interpolation sending time window is determined according to the interpolation multiple adapted to the comparison result.

5. The method according to claim 2, wherein: In the case where the comparison result between the transmission frequency and the game frame rate of the cloud game satisfies the adjustment condition of the second type, the adjustment time window adapted to the comparison result includes: a combined sending time window; The combined sending time window is used to indicate the sending timing of the device status data obtained according to the combined processing strategy.

6. The method according to claim 5, wherein: The combined sending time window is determined by: Determine a combined sending time window in an initial state according to the game frame rate; The combined sending time window in the initial state is dynamically adjusted according to the currently detected network state.

7. The method according to claim 5 or 6, wherein: The step of processing the device status data transmitted by the input device according to the adjustment time window and sending the processed device status data to the cloud server of the cloud game comprises: When the combined sending time window has not arrived, combining the multiple device status data received from the input device to obtain combined device status data; When the merged sending time window is reached, the merged device status data is sent to the cloud server of the cloud game, so that the cloud server can determine the game screen frame corresponding to the merged device status data and generate user control instructions corresponding to the game screen frame.

8. The method according to any one of claims 1 to 7, wherein: The input device comprises: a mouse; the transmission frequency of the input device transmitting the device status data to the client device is determined according to the reporting rate of the mouse; The device status data includes: a click-related status parameter for characterizing a mouse click status, and / or a position-related status parameter for characterizing a mouse movement position.

9. The method according to any one of claims 1 to 8, wherein: The input device comprises: a keyboard; and the method further comprises: In response to receiving a key press message sent by the keyboard, querying the current state of the key corresponding to the key press message in the key state table; if the current state of the key is not pressed, updating the current state of the key to pressed, and sending the received key press message to the cloud server; if the current state of the key is pressed, ignoring the key press message; When a key-up message is received from the keyboard, the current state of the key corresponding to the key-up message in the key state table is updated from pressed to not pressed, and the received key-up message is sent to the cloud server.

10. A data transmission method based on cloud gaming, applied to a cloud server, the method comprising: Receiving device status data transmitted by a client device through any method of claims 1-9; A user control instruction is generated according to the device status data, and the user control instruction is sent to the client device.

11. A data transmission device based on cloud gaming, comprising: An acquisition module, adapted to acquire a transmission frequency of the input device transmitting device status data to the client device; a determination module, adapted to determine an adjustment time window adapted to the comparison result between the transmission frequency and the game frame rate of the cloud game if the comparison result between the transmission frequency and the game frame rate of the cloud game meets a preset adjustment condition; The processing module is adapted to process the device status data transmitted by the input device according to the adjustment time window, and send the processed device status data to the cloud server of the cloud game.

12. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the cloud gaming-based data transmission method as described in any one of claims 1-10.

13. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute the cloud gaming-based data transmission method as described in any one of claims 1-10.

Images