Cloud game terminal device and cloud game server and related method

Abstract

The present application relates to a cloud game terminal device, a cloud game server and related methods. The cloud game terminal device comprises a playback quality assessment unit and a feedback engine client. The playback quality assessment unit is configured to generate at least one playback quality assessment parameter based on information provided by at least one of a network interface control unit, a media decoding unit and a media rendering unit in the cloud game terminal device. The feedback engine client is configured to transmit the at least one playback quality assessment parameter to a cloud game server via a network. The cloud game server is configured to adjust at least one media encoding parameter of a media stream based on the received at least one playback quality assessment parameter.

Description

Technical Field

[0001] This invention relates to cloud gaming technology, and more particularly to a cloud gaming terminal device, a cloud gaming server, and related methods for adjusting encoder parameters based on terminal playback quality. Background Technology

[0002] In traditional video streaming services, servers primarily adjust streaming parameters based on two factors: the hardware specifications of the terminal device and network conditions. Hardware specifications include display capabilities such as the maximum resolution supported by the monitor, color depth, and high dynamic range, as well as the computing power of the terminal device's central processing unit (CPU) and graphics processing unit (GPU). Regarding network conditions, the server can monitor the currently available bandwidth and dynamically adjust the video stream's resolution and bitrate accordingly. However, this adjustment mechanism mainly focuses on the continuity of the streaming service, rather than its real-time nature.

[0003] Cloud gaming, as a special type of streaming application, has stricter requirements for latency performance. While the Transmission Control Protocol (TCP), commonly used in traditional streaming services, ensures reliable data transmission, its data retransmission mechanism and large buffer design lead to high end-to-end latency, making it unsuitable for cloud gaming applications. Therefore, current cloud gaming services mostly use the User Datagram Protocol (UDP) to transmit game media streams. UDP lacks a data retransmission mechanism, which can reduce transmission latency, but it also cannot automatically recover data lost during transmission. More importantly, UDP's unidirectional transmission characteristic means the server cannot directly know the actual reception and playback status of the terminal device. This problem is particularly pronounced in wireless network environments, as wireless transmission (Wi-Fi) is easily affected by environmental interference, which may lead to packet loss or delay, causing video decoding errors, screen tearing, audio interruptions, and other problems, severely impacting the gaming experience.

[0004] This reveals a lack of effective mechanisms in UDP-based video streaming applications to collect and analyze actual playback quality information from terminal devices. This absence of real-time feedback makes it difficult for cloud gaming services to maintain stable service quality in complex network environments. Summary of the Invention

[0005] In view of this, the present invention provides a dynamic encoder parameter adjustment mechanism based on the playback quality of terminal devices. This invention establishes a complete playback quality evaluation mechanism in cloud gaming terminal devices. This mechanism can monitor and analyze the transmission status of network interfaces, error events in the media decoding unit, and processing anomalies in the media rendering unit in real time, thereby generating accurate playback quality evaluation parameters. These parameters are transmitted to the cloud gaming server through a specially designed feedback engine mechanism using a reliable network transmission protocol, enabling the server to promptly grasp the actual playback status of the terminal device. Based on these playback quality evaluation parameters and combined with intelligent algorithms, the cloud gaming server dynamically adjusts the key parameters of the video encoder and audio encoder, thereby maintaining a stable gaming experience under different network environments and terminal device conditions. This innovative mechanism not only effectively improves the problem of unstable playback quality in traditional cloud gaming systems but also provides a reliable quality assurance solution for future real-time interactive multimedia applications.

[0006] An embodiment of the present invention provides a cloud gaming terminal device, comprising: a playback quality evaluation unit and a feedback engine client. The playback quality evaluation unit is used to generate at least one playback quality evaluation parameter based on information provided by at least one of a network interface control unit, a media decoding unit, and a media rendering unit in the cloud gaming terminal device. The feedback engine client is used to transmit the at least one playback quality evaluation parameter to a cloud gaming server via a network. The cloud gaming server adjusts at least one media encoding parameter of a media stream based on the received at least one playback quality evaluation parameter.

[0007] An embodiment of the present invention provides a cloud game server, comprising: a feedback engine server, an encoder parameter calculation unit, and a media encoding unit. The feedback engine server is configured to receive at least one playback quality evaluation parameter transmitted by a cloud game terminal device via a network. The encoder parameter calculation unit is configured to adjust at least one media encoding parameter based on the at least one playback quality evaluation parameter. The media encoding unit is configured to generate a media stream for providing to the cloud game terminal device based on the adjusted at least one media encoding parameter.

[0008] An embodiment of the present invention provides a method for a cloud gaming system, the method comprising: generating at least one playback quality evaluation parameter based on information provided by at least one of a network interface control unit, a media decoding unit, and a media rendering unit of a cloud gaming terminal device; transmitting the at least one playback quality evaluation parameter to a cloud gaming server via a network; and adjusting at least one media encoding parameter of a media stream generated by the cloud gaming server based on the received at least one playback quality evaluation parameter. Attached Figure Description

[0009] Figure 1 This is an architecture diagram of a cloud gaming system according to an embodiment of the present invention.

[0010] Figure 2 This is an architecture diagram of a cloud gaming system according to another embodiment of the present invention.

[0011] Figure 3 This is an architecture diagram of a cloud game server and terminal device according to an embodiment of the present invention.

[0012] Figure 4 This is a flowchart of a method for a cloud gaming system according to an embodiment of the present invention. Detailed Implementation

[0013] Numerous specific details are described below to provide the reader with a thorough understanding of embodiments of the invention. However, those skilled in the art will appreciate how the invention can be implemented in the absence of one or more specific details, or by utilizing other methods, elements, or materials. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring the core concepts of the invention.

[0014] The phrase "an embodiment" in this specification means that a particular feature, state, or characteristic described in that embodiment may be included in at least one embodiment of the invention. Therefore, the phrase "in an embodiment" appearing throughout this specification does not necessarily mean the same embodiment. Furthermore, the aforementioned particular features, states, or characteristics may be combined in any suitable form in one or more embodiments.

[0015] Please refer to Figure 1 This figure illustrates a cloud gaming system according to an embodiment of the present invention. As shown, the cloud gaming system 10 includes a cloud gaming terminal device 100 and a cloud gaming server 200. The cloud gaming system 10 provides a game computing architecture capable of achieving a high-performance and low-latency gaming experience. The cloud gaming server 200 is mainly used to perform game engine calculations and game logic processing, including but not limited to complex calculations such as physics calculations, artificial intelligence calculations, scene rendering, and sound effect processing.

[0016] On the other hand, the cloud gaming terminal device 100 primarily establishes a two-way real-time communication connection with the cloud gaming server 200 via network 50. Network 50 may include a network transmission link composed of user terminal devices, internet service provider equipment, and cloud service provider network equipment, ensuring the real-time transmission of game data. In different embodiments of the present invention, the cloud gaming terminal device 100 can be various terminal devices with network connectivity, such as game consoles, personal computers, laptops, tablets, smartphones, smart TVs, in-vehicle entertainment systems, and virtual reality headsets.

[0017] The cloud gaming terminal device 100 includes: a network interface control unit 103, a media decoding unit 105 (which includes a video decoder 110 and an audio decoder 120), a media rendering unit 107, a playback quality evaluation unit 130, and a feedback engine client 140. The network interface control unit 103 is used to receive a media stream provided by the cloud gaming server 200 through the network 50, wherein the media stream includes at least one video stream and at least one audio stream.

[0018] The video decoder 110 in the media decoding unit 105 is used to decode the video stream to obtain the decoded original video frame sequence. The video decoder 110 may employ a hardware-accelerated architecture and support multi-threaded decoding. In various embodiments of the present invention, the video decoder 110 supports multiple video coding standards, including but not limited to H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), VP9, ​​AV1, etc.

[0019] The audio decoder 120 in the media decoding unit 105 is used to decode the audio stream provided by the cloud game server 200 to obtain the original audio data in Pulse Code Modulation (PCM) format. The audio decoder 120 can support multi-channel audio processing and complex audio streams including spatial effects. In various embodiments of the present invention, the audio decoder 120 supports multiple audio coding standards, such as Advanced Audio Coding (AAC), MP3 (MPEG-1 / 2 Audio Layer III), and AC3 (Dolby Digital).

[0020] Furthermore, the video and audio streams are encapsulated in a media stream using a standardized multimedia container format (such as MPEG-TS, WebM, MP4, RTP, etc.). This container format provides the necessary synchronization information and metadata to ensure synchronized transmission and playback of audio and video. The cloud gaming terminal device 100 can then de-multiplex and demux the media stream to separate and extract the video and audio streams for subsequent decoding.

[0021] The media rendering unit 107 is used to render the original video frame sequence generated by the video decoder 110 and the original audio data generated by the audio decoder 120, and finally output video display signals and audio playback signals that meet the specifications of the display device 150 (e.g., a television or monitor) and audio device 160 (e.g., a sound system). The display device 150 and audio device 160 can be configured in two ways: built-in integration or external expansion. In built-in integration mode, these devices are directly integrated into the hardware system of the cloud gaming terminal device 100; in external expansion mode, they are connected to the cloud gaming terminal device 100 through standardized interfaces to provide users with a complete gaming interactive experience.

[0022] The playback quality assessment unit 130 can generate at least one playback quality assessment parameter based on information provided by at least one of the network interface control unit 103, the media decoding unit 105, and the media rendering unit 107. In one embodiment, the playback quality assessment unit 130 is used to determine a playback quality assessment parameter associated with at least one of a network latency and a network packet loss rate based on information provided by the network interface control unit 103, such as packet latency or packet continuity, so that the playback quality assessment unit 130 reflects anomalies such as excessively high network latency or excessively high network packet loss rate in the playback quality assessment parameter.

[0023] In one embodiment, the playback quality assessment unit 130 is used to determine a playback quality assessment parameter associated with a video decoding error based on information provided by the video decoder 110. For example, when the video decoder 110 detects an error block during decoding (which may produce visual anomalies such as mosaic or tearing in the image), it can report this to the playback quality assessment unit 130, so that the playback quality assessment unit 130 reflects the video decoding error in the playback quality assessment parameter. Alternatively, when the video decoder 110 detects an error or discontinuity in the transmission sequence number of the packet to be decoded, or an error or discontinuity in the presentation timestamp of the video frame, it can report this to the playback quality assessment unit 130, thereby reflecting the video decoding error in the playback quality assessment parameter.

[0024] In one embodiment, the playback quality evaluation unit 130 is used to determine a playback quality evaluation parameter associated with an audio decoding error based on information provided by the audio decoder 120. For example, when the audio decoder 120 detects insufficient data to be decoded in the buffer during decoding (which may cause audio interruption or distortion), it can report this to the playback quality evaluation unit 130, so that the playback quality evaluation unit 130 reflects the audio decoding error in the playback quality evaluation parameter.

[0025] In one embodiment, the playback quality evaluation unit 130 is used to determine a playback quality evaluation parameter associated with a rendering anomaly state based on information provided by the media rendering unit 107. For example, when the media rendering unit 107 detects that the number of video frames to be rendered in the buffer is insufficient during the rendering process (which may lead to duplicate frames or dropped frames), it can report this to the playback quality evaluation unit 130, so that the playback quality evaluation unit 130 reflects the rendering anomaly in the playback quality evaluation parameter. Alternatively, when the media rendering unit 107 detects that the video frame number or display timestamp is inconsistent with the expected display sequence during the rendering process, it can report this to the playback quality evaluation unit 130, thereby reflecting the rendering anomaly in the playback quality evaluation parameter.

[0026] After the above operations are completed, the playback quality evaluation unit 130 will hand over the generated at least one playback quality evaluation parameter to the feedback engine client 140, so as to transmit it back to the cloud game server 200 via the network 50.

[0027] On the other hand, the cloud game server 200 includes a network interface control unit 203, a media encoding unit 205 (which includes a video encoder 210 and an audio encoder 220), an encoder parameter calculation unit 230, and a feedback engine server 240. The video encoder 210 encodes the source stream 201 generated by the cloud game server 200 executing a game program, thereby generating a video stream. The audio encoder 220 encodes the source stream 201 to generate an audio stream. The video stream and the audio stream may be based on a specific multimedia container format, encapsulated in a media stream, and further transmitted to the cloud game terminal device 100 via the network 50 through the network interface control unit 203.

[0028] In one embodiment, the encoder parameter calculation unit 230 is used to adjust at least one media encoding parameter based on the at least one playback quality evaluation parameter returned by the cloud gaming terminal device 100. Further, the feedback engine server 240 receives the at least one playback quality evaluation parameter returned by the feedback engine client 140 via the network interface control unit 203. Based on the at least one playback quality evaluation parameter, the encoder parameter calculation unit 230 dynamically adjusts at least one video encoder parameter used by the video encoder 210 and dynamically adjusts at least one audio encoder parameter used by the audio encoder 220.

[0029] In one embodiment, the at least one video encoding parameter includes at least one of a group of pictures length parameter, a forced instantaneous decoder refresh frame parameter, a bitrate parameter, a frame rate parameter, and a resolution parameter. Furthermore, when the at least one playback quality evaluation parameter returned by the cloud gaming terminal device 100 reflects high latency or playback errors (e.g., excessively high network latency, excessively high network packet loss rate, video decoding errors, and / or abnormal rendering state), at least one of the group of pictures length parameter, forced instantaneous keyframe refresh frame parameter, bitrate parameter, frame rate parameter, and resolution parameter will be adjusted accordingly.

[0030] In one embodiment, if high latency or playback error occurs, the image group length parameter is adjusted to reduce the image group length (e.g., from 300 to 100), and conversely, the image group length parameter is adjusted to increase the image group length. In one embodiment, if high latency or playback error occurs, a forced keyframe insertion parameter is set to cause the video encoder 210 to immediately insert a decoding refresh frame into the video frame sequence, suspending inter-frame reference for video decoding. In one embodiment, if high latency or playback error occurs, the bitrate parameter is adjusted to reduce the bitrate of the video stream (e.g., from 5-7 Mbps to 2-4 Mbps), and conversely, the bitrate parameter is adjusted to increase the bitrate of the video stream. In one embodiment, if high latency or playback error occurs, the frame rate parameter is adjusted to reduce the frame rate of the video stream (e.g., from 60 fps to 30 fps), and conversely, the frame rate parameter is adjusted to increase the frame rate of the video stream. In one embodiment, if a high latency or playback error occurs, the resolution parameter is adjusted to reduce the resolution of the video stream (e.g., from 1080p to 720p), and conversely, the resolution parameter is adjusted to increase the resolution of the video stream.

[0031] In one embodiment, the at least one audio encoder parameter dynamically adjusted by the encoder parameter calculation unit 230 includes at least one of a channel number parameter, a bitrate parameter, and a sampling rate parameter. In one embodiment, if high latency or playback errors occur, the channel number parameter is adjusted to reduce the number of channels (e.g., reducing the number of channels from 5.1 channels to 2 channels), and conversely, the channel number parameter is adjusted to increase the number of channels. In one embodiment, if high latency or playback errors occur, the bitrate parameter is adjusted to reduce the bitrate (e.g., reducing the bitrate from 320kbps to 192kbps), and conversely, the bitrate parameter is adjusted to increase the bitrate. In one embodiment, if high latency or playback errors occur, the sampling rate parameter is adjusted to reduce the sampling rate (e.g., reducing the sampling rate from 48kHz to 44.1kHz), and conversely, the sampling rate parameter is adjusted to increase the sampling rate.

[0032] Please refer to Figure 2 This figure illustrates a cloud gaming system according to another embodiment of the present invention. As shown, the cloud gaming system 20 has multiple cloud gaming terminal devices 300_1 to 300_N and a cloud gaming server 400. The cloud gaming server 400 provides cloud gaming services to the cloud gaming terminal devices 300_1 to 300_N. The cloud gaming terminal devices 300_1 to 300_N mainly communicate with the cloud gaming server 400 through a network 50. The network 50 includes a network transmission link composed of user terminal devices, internet service provider devices, and cloud service provider network devices. In different embodiments of the present invention, each of the cloud gaming terminal devices 300_1 to 300_N can be a game console, personal computer, laptop, tablet computer, smartphone, smart TV, in-vehicle entertainment system, virtual reality head-mounted display, or other terminal devices with network connectivity capabilities.

[0033] Furthermore, each of the cloud gaming terminal devices 300_1 to 300_N includes a network interface control unit 303_1 to 303_N, a media decoding unit 305_1 to 305_N (which includes video decoders 310_1 to 310_N and audio decoders 320_1 to 320_N), a media rendering unit 307_1 to 307_N, a playback quality evaluation unit 330_1 to 330_N, a feedback engine client 340_1 to 340_N, a display device 350_1 to 350_N, and an audio device 360_1 to 360_N. Since the cloud gaming terminal devices 300_1 to 300_N have a similar architecture and operation to the previously described cloud gaming terminal device 100, further details will not be provided here.

[0034] On the other hand, the cloud game server 400 includes multiple cloud game service modules 400_1 to 400_N, each used to provide cloud game services to cloud game terminal devices 300_1 to 300_N on a one-to-one basis (note that in some embodiments, the number of cloud game service modules that the cloud game server 400 can implement may be greater than the actual number of connected cloud game terminal devices). Each of the cloud game service modules 400_1 to 400_N performs video and audio encoding on the source streams 401_1 to 401_N through media encoders 405_1 to 405_N, thereby generating corresponding video and audio streams, which are then provided to the corresponding cloud game terminal devices 300_1 to 300_N through the network interface control unit 403. The game execution module 402 in the cloud game server 400 executes a specific game program according to the user selection provided by the cloud game terminal devices 300_1 to 300_N, thereby generating source streams 401_1 to 401_N corresponding to the terminal devices, so as to provide the corresponding game content to the corresponding game service modules 400_1 to 400_N.

[0035] Furthermore, each of the cloud gaming terminal devices 300_1 to 300_N transmits at least one playback quality evaluation parameter generated by its respective playback quality evaluation unit 330_1 to 330_N back to the feedback engine server 440 of the cloud gaming server 400 via network 50 through its feedback engine clients 340_1 to 340_N. The feedback engine server 440 receives the playback quality evaluation parameter through the network interface control unit 403 and distributes it to the corresponding one of the cloud gaming service modules 400_1 to 400_M.

[0036] The encoder parameter calculation units 430_1 to 430_N in the cloud gaming service modules 400_1 to 400_M will adjust the parameters of at least one video encoder used by the video encoder 410_1 to 410_N and / or adjust the parameters of at least one audio encoder used by the audio encoder 420_1 to 420_N according to the method described above and based on at least one playback quality evaluation parameter provided by one of the corresponding cloud gaming terminal devices 300_1 to 300_N.

[0037] It is worth noting that the playback quality assessment and feedback mechanism and encoder parameter adjustment mechanism described in this invention are not only applicable to cloud gaming systems, but also, in other embodiments of this invention, can be applied to real-time multimedia streaming systems. For example, in live streaming platforms, terminal devices can send playback quality assessment parameters back to the streaming source, enabling the encoder to dynamically adjust the streaming encoding parameters. Similarly, in a multimedia projection system within a local area network environment, the receiving device can provide real-time playback quality feedback to the projection source device through the same mechanism, thereby adjusting the streaming encoding and ensuring the best user viewing experience.

[0038] Figure 3A schematic diagram of a computing device according to an embodiment of the present invention is shown. This computing device can be used to implement the aforementioned cloud game server and / or cloud game terminal device. As shown, the computing device 500 includes a central processing unit 501, a graphics processing unit 502, a memory 503, a storage unit 504, a network interface control unit 505, and a media encoding / decoding processing unit 506. The central processing unit 501 can be an x86 architecture processor, an ARM architecture processor, or other general-purpose processor. In the embodiment of the cloud game server, the central processing unit 501 is mainly used to perform game engine calculations, process player input, control game logic, and coordinate the operation of various functional modules. In the embodiment of the cloud game terminal device, the central processing unit 501 is responsible for managing the multimedia decoding process, processing playback quality evaluation, and controlling the operation of the feedback mechanism. The graphics processing unit 502 can be a discrete graphics card or an integrated graphics chip, supporting graphics application programming interfaces (APIs) such as OpenGL and DirectX. In the embodiment of the cloud game server, the graphics processing unit 502 is mainly used to perform real-time rendering calculations of game scenes and supports hardware-accelerated video encoding. In an embodiment of a cloud gaming terminal device, the graphics processing unit 502 assists in video decoding and image display processing. Memory 503 may be dynamic random access memory or other high-speed memory. In an embodiment of a cloud gaming server, memory 503 stores game runtime state data, caches player input data, and provides a buffer required for video encoding. In an embodiment of a cloud gaming terminal device, memory 503 stores decoded video frames, audio data, and temporary data required for playback quality evaluation. Storage unit 504 may be a solid-state drive, hard disk, or other non-volatile storage device. In an embodiment of a cloud gaming server, storage unit 504 stores game programs, player data, and system configuration data. In an embodiment of a cloud gaming terminal device, storage unit 504 stores client applications, playback settings, and necessary system data. Network interface control unit 505 may be a wired network interface card or a wireless network interface card. In an embodiment of a cloud gaming server, network interface control unit 505 is responsible for handling a large number of concurrent player connections, transmitting video streams, and receiving quality evaluation parameters from the terminal device. In an embodiment of a cloud gaming terminal device, the network interface control unit 505 is responsible for receiving video streaming data and sending playback quality evaluation parameters back to the server.

[0039] Furthermore, the aforementioned media encoding / decoding units can all be implemented in software or using additional hardware encoding / decoding circuitry, for example, using... Figure 3The media encoding / decoding unit 506 is implemented in the middle, which can be a standalone or integrated audio or video processing chip, or one or more functional modules implemented in software. Furthermore, the playback quality evaluation unit, feedback engine client, feedback engine server, and encoder parameter calculation unit can also be implemented in software or hardware as needed.

[0040] Figure 4 A flowchart illustrating a method for a cloud gaming system according to an embodiment of the present invention is shown. As shown, the method includes the following steps: S101: Generate at least one playback quality evaluation parameter based on information provided by at least one of a network interface control unit, a media decoding unit, and a media rendering unit of a cloud gaming terminal device. S102: Transmit the at least one playback quality assessment parameter to a cloud gaming server via a network; and S103: Adjust at least one media encoding parameter of a media stream generated by the cloud game server based on the received at least one playback quality assessment parameter.

[0041] Since the principles and details of the above steps have been explained in detail through the above embodiments, they will not be repeated here. It is worth noting that the above process can be improved by adding other additional steps or making appropriate modifications and adjustments to better enhance the gaming experience of cloud gaming services.

[0042] Embodiments of the present invention can be embodied as apparatus, method, or computer program product. Accordingly, embodiments of the present invention can take the form of an entity entirely implemented in hardware, an entity entirely implemented in software (including firmware, resident software, microcode, etc.), or an entity combining software and hardware aspects, which can be collectively referred to as a "module" or "system". Furthermore, embodiments of the present invention can take the form of a computer program product embodied in any tangible medium, and the medium has computer-usable program code embodied therein. In terms of hardware, the present invention can be implemented by applying any of the following techniques or related combinations: individual operating logic of a logic gate capable of executing logic functions according to data signals, and application-specific integrated circuits (ASICs), programmable gate arrays (PGAs), or field-programmable gate arrays (FPGAs) with appropriate combinational logic.

[0043] Flowcharts and block diagrams illustrate the architecture, functionality, and operation of systems, methods, and computer program products that represent different possible implementations of embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of program code, including one or more executable instructions to implement a specific logical function. It should also be noted that each block in a block diagram and / or flowchart, as well as combinations of blocks in block diagrams and / or flowcharts, can be implemented by a special-purpose hardware infrastructure system, or a combination of special hardware and computer program instructions. These computer program instructions may be stored in readable computer media to command a computer or other programmable data processing device to operate in a specific manner so that the instructions stored in the readable computer media produce the functions / operations specified in the blocks or combinations of blocks in the flowcharts and / or block diagrams.

[0044] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be included in the scope of the present invention.

[0045] [Symbol Explanation]

[0046] 10, 20 Cloud Gaming Systems

[0047] 50 Networks

[0048] 100, 300_1~300_N Cloud gaming terminal devices

[0049] 103, 203, 303_1~303_N, 403 Network Interface Control Unit

[0050] 105, 305_1~305_N Media Decoding Units

[0051] 110, 310_1~310_N video decoders

[0052] 120, 320_1~320_N audio decoders

[0053] 107, 307_1~307_N Media Rendering Units

[0054] 130, 330_1~330_N Playback Quality Evaluation Unit

[0055] Feedback Engine Clients 140, 340_1~340_N

[0056] 150, 350_1~350_N display devices

[0057] 160, 360_1~360_N audio devices

[0058] 200, 400 cloud gaming servers

[0059] 201, 401_1~401_N Source Stream

[0060] Media coding units 205, 405_1~405_N

[0061] 210, 410_1~410_N video encoders

[0062] 220, 420_1~420_N audio encoders

[0063] 230, 430_1~430_N Encoder Parameter Calculation Unit

[0064] 240, 440 Feedback Engine Server

[0065] 400_1~400_N Cloud gaming service module

[0066] 402 Game Execution Module

[0067] 500 computing device

[0068] 501 Central Processing Order

[0069] 502 Graphics Processing Unit

[0070] 503 Memory

[0071] 504 storage unit

[0072] 505 Network Interface Control Unit

[0073] 506 Media Encoding / Decoding Processing Unit.

Claims

1. A cloud gaming terminal device, comprising: A playback quality assessment unit is used to generate at least one playback quality assessment parameter based on information provided by at least one of a network interface control unit, a media decoding unit, and a media rendering unit in the cloud gaming terminal device. A feedback engine client for transmitting at least one playback quality assessment parameter to a cloud gaming server via a network; The cloud gaming server adjusts at least one media encoding parameter of a media stream based on the received at least one playback quality assessment parameter.

2. The cloud gaming terminal device according to claim 1, wherein the playback quality assessment unit is configured to determine a playback quality assessment parameter associated with at least one of a network latency time and a network packet loss rate based on information provided by the network interface control unit.

3. The cloud gaming terminal device according to claim 1, wherein the playback quality assessment unit is configured to determine a playback quality assessment parameter associated with a video decoding error based on information provided by a video decoder in the media decoding unit.

4. The cloud gaming terminal device according to claim 1, wherein the playback quality assessment unit is configured to determine a playback quality assessment parameter associated with an audio decoding error based on information provided by an audio decoder in the media decoding unit.

5. The cloud gaming terminal device according to claim 1, wherein the playback quality assessment unit is configured to determine a playback quality assessment parameter associated with a rendering anomaly state based on information provided by the media rendering unit.

6. A cloud gaming server, comprising: A feedback engine server for receiving at least one playback quality evaluation parameter transmitted from a cloud gaming terminal device via a network; An encoder parameter calculation unit is used to adjust at least one media encoding parameter based on the at least one playback quality evaluation parameter; and A media encoding unit is configured to generate a media stream for providing to the cloud gaming terminal device based on the adjusted at least one media encoding parameter.

7. The cloud gaming server of claim 6, wherein the encoder parameter calculation unit is used to adjust at least one video encoding parameter according to the at least one playback quality evaluation parameter.

8. The cloud gaming server of claim 7, wherein the at least one video encoding parameter includes at least one of a group of pictures length parameter, a force instantaneous decoder refresh frame parameter, a bitrate parameter, a frame rate parameter, and a resolution parameter.

9. The cloud gaming server of claim 6, wherein the encoder parameter calculation unit is used to adjust at least one audio encoding parameter according to the at least one playback quality evaluation parameter.

10. A method for a cloud gaming system, comprising: Based on information provided by at least one of a network interface control unit, a media decoding unit, and a media rendering unit of a cloud gaming terminal device, at least one playback quality evaluation parameter is generated. The playback quality assessment parameter is transmitted via a network to a cloud gaming server; and Adjust at least one media encoding parameter of a media stream generated by the cloud gaming server based on the received at least one playback quality assessment parameter.

Images