Data sending and receiving method of cloud gaming and related equipment

Abstract

The present application discloses a method for sending and receiving data of a cloud game and related equipment; the game data to be transmitted can be obtained, and the game data includes at least one frame of game screen; for each frame of game screen in the game data, the game screen is divided into blocks to obtain at least one game screen block corresponding to the game screen; each game screen block of the game screen is encoded in parallel to obtain the encoded game screen data corresponding to the game screen; the encoded game screen data is processed to obtain at least two game screen data streams; different transmission links are used to transmit the at least two game screen data streams in parallel to the game terminal. The embodiment of the present application can reduce the delay of game data encoding and transmission as a whole based on parallel encoding and parallel transmission.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for sending and receiving data for a cloud game and related equipment. Background Art

[0002] With the rapid development of computer technology, cloud gaming has become increasingly important in the gaming market. Cloud gaming is a new gaming method based on cloud computing. Cloud gaming technology mainly transmits the audio and video media data generated by the game running on the server to the player's client through a computer network, and transmits the operation instructions generated by the player on the client back to the server.

[0003] In the cloud gaming scenario, the data latency requirement is relatively high, because generally speaking, if the game operation data delay exceeds 100 milliseconds, it will seriously affect the gaming experience; while in traditional audio and video transmission, a short delay is acceptable. However, in current related technologies, due to the immaturity of cloud gaming related technologies, there is often a problem of high game data latency. Summary of the invention

[0004] An embodiment of the present application provides a method for sending and receiving data for a cloud game and related equipment. The related equipment may include a data sending device for a cloud game, a data receiving device for a cloud game, an electronic device, a computer-readable storage medium, and a computer program product, which can reduce the waiting delay of game data as a whole.

[0005] The present application provides a method for sending data of a cloud game, including:

[0006] Acquire game data to be transmitted, wherein the game data includes at least one frame of game screen;

[0007] For each frame of the game screen in the game data, the game screen is divided into blocks to obtain at least one game screen block corresponding to the game screen;

[0008] Parallel encoding is performed on each game screen block of the game screen to obtain encoded game screen data corresponding to the game screen;

[0009] Processing the encoded game screen data to obtain at least two game screen data streams;

[0010] The at least two game screen data streams are respectively transmitted in parallel to the game terminal using different transmission links.

[0011] The present application also provides a method for receiving data of a cloud game, including:

[0012] Receiving at least two game screen data streams transmitted in parallel by a server through different transmission links, wherein the game screen data streams include encoded game screen blocks corresponding to respective game screen blocks of the game screen;

[0013] Decoding the encoded game screen block in the game screen data stream to obtain a decoded game screen block;

[0014] Rendering the decoded game screen block to obtain a target game screen block;

[0015] The target game screen blocks belonging to the same frame of the game screen are merged to obtain and display the target game screen.

[0016] Accordingly, an embodiment of the present application provides a data sending device for a cloud game, comprising:

[0017] An acquisition unit, used to acquire game data to be transmitted, wherein the game data includes at least one frame of game screen;

[0018] A block division unit, configured to divide the game screen into blocks for each frame of the game screen in the game data, and obtain at least one game screen block corresponding to the game screen;

[0019] An encoding unit, used for encoding each game screen block of the game screen in parallel to obtain encoded game screen data corresponding to the game screen;

[0020] A data stream division unit, used for processing the encoded game screen data to obtain at least two game screen data streams;

[0021] The transmission unit is used to use different transmission links to transmit the at least two game screen data streams in parallel to the game terminal.

[0022] Accordingly, the embodiment of the present application further provides a data receiving device for a cloud game, comprising:

[0023] A receiving unit, configured to receive at least two game screen data streams transmitted in parallel by a server through different transmission links, wherein the game screen data streams include encoded game screen blocks corresponding to respective game screen blocks of the game screen;

[0024] A decoding unit, configured to decode the encoded game screen block in the game screen data stream to obtain a decoded game screen block;

[0025] A rendering unit, used for rendering the decoded game screen block to obtain a target game screen block;

[0026] The fusion unit is used to fuse the target game screen blocks belonging to the same frame of the game screen to obtain and display the target game screen.

[0027] Optionally, in some embodiments of the present application, the encoding unit may include a sampling parameter determination subunit and a parallel encoding subunit, as follows:

[0028] The sampling parameter determination subunit is used to determine the encoding sampling parameters corresponding to the game screen according to the current network transmission parameters;

[0029] The parallel encoding subunit is used to perform parallel encoding on each game screen block of the game screen according to the encoding sampling parameters to obtain encoded game screen data corresponding to the game screen.

[0030] Optionally, in some embodiments of the present application, the encoded game screen data includes encoded game screen blocks corresponding to each game screen block;

[0031] The data stream dividing unit may include a position determining subunit and a first dividing subunit, as follows:

[0032] The position determination subunit is used to determine the position information corresponding to the encoded game screen block in the game screen;

[0033] The first division subunit is used to process the encoded game screen block according to the position information to obtain at least two game screen data streams.

[0034] Optionally, in some embodiments of the present application, the data stream division unit may include a frame identifier acquisition subunit and a second division subunit, as follows:

[0035] The frame identifier acquisition subunit is used to acquire the game screen frame identifier corresponding to the encoded game screen data;

[0036] The second division subunit is used to process the encoded game screen data according to the game screen frame identifier to obtain at least two game screen data streams.

[0037] Optionally, in some embodiments of the present application, the game screen frame identifier includes the frame number of the game screen; the second division sub-unit can be specifically used to divide the encoded game screen data into even-numbered frame game screen data streams when the frame number corresponding to the encoded game screen data is an even number; and to divide the encoded game screen data into odd-numbered frame game screen data streams when the frame number corresponding to the encoded game screen data is an odd number.

[0038] Optionally, in some embodiments of the present application, the data stream division unit may include a redundancy construction subunit and a third division subunit, as follows:

[0039] The redundancy construction subunit is used to construct redundant data for the encoded game screen data according to the current network transmission parameters to obtain target game screen data, wherein the target game screen data includes the encoded game screen data and redundant data corresponding to the encoded game screen data;

[0040] The third division subunit is used to process the target game screen data to obtain at least two game screen data streams.

[0041] Optionally, in some embodiments of the present application, the data sending device of the cloud game may further include a link switching unit, as follows:

[0042] The link switching unit is used to perform real-time detection of the link transmission parameters of the transmission link corresponding to the game screen data stream; when the detected link transmission parameters do not meet the preset transmission conditions, the game screen data stream is switched to the target transmission link that meets the preset transmission conditions for transmission to the game terminal.

[0043] Optionally, in some embodiments of the present application, the transmission unit may include an acceleration subunit and a transmission subunit, as follows:

[0044] The acceleration sub-unit is used to perform data transmission acceleration processing on the at least two game screen data streams respectively;

[0045] The transmission subunit is used to use different transmission links to transmit at least two game screen data streams after acceleration processing in parallel to the game terminal.

[0046] Optionally, in some embodiments of the present application, the data transmission device of the cloud game may further include a link determination unit, which is used to determine the transmission link corresponding to the game screen data stream from each candidate transmission link. Specifically, the link determination unit may include an extraction subunit, a prediction subunit, and a link determination subunit, as follows:

[0047] The extraction subunit is used to extract features of the current network transmission parameters to obtain transmission feature information corresponding to the current network transmission parameters;

[0048] A prediction subunit, configured to determine, based on the transmission characteristic information, the predicted transmission delay information corresponding to the game screen data stream on each candidate transmission link;

[0049] The link determination subunit is used to determine the transmission link corresponding to the game screen data stream from each candidate transmission link according to the predicted transmission delay information.

[0050] Optionally, in some embodiments of the present application, the current network transmission parameter includes a network transmission sub-parameter of at least one dimension;

[0051] The extraction subunit can be specifically used to extract features from network transmission sub-parameters of each dimension to obtain transmission sub-feature information corresponding to the network transmission sub-parameters of each dimension; and fuse the transmission sub-feature information corresponding to the network transmission sub-parameters of each dimension to obtain the transmission feature information corresponding to the current network transmission parameters.

[0052] Optionally, in some embodiments of the present application, the data stream division unit may include a position acquisition subunit, a selection subunit, and a fourth division subunit, as follows:

[0053] The location acquisition subunit is used to acquire the location information of the game terminal;

[0054] A selection subunit, configured to select a target edge server from the candidate edge servers according to the location information;

[0055] The fourth division subunit is used to process the encoded game screen data through the target edge server to obtain at least two game screen data streams.

[0056] Optionally, in some embodiments of the present application, the data stream division unit may further include a copy subunit and a sending subunit, as follows:

[0057] The copy subunit is used to copy the encoded game screen data to the kernel network card interface layer by bypassing the kernel processor through a data processing tool;

[0058] The sending subunit is used to send the encoded game screen data to the target edge server through the kernel network card interface layer.

[0059] Optionally, in some embodiments of the present application, the receiving unit may be specifically configured to receive at least two game screen data streams transmitted in parallel by the server through different transmission links in a preset transmission order;

[0060] The decoding unit may be specifically configured to, when receiving a target encoded game screen block in the game screen data stream, decode an adjacent encoded game screen block corresponding to the target encoded game screen block to obtain a decoded game screen block.

[0061] Optionally, in some embodiments of the present application, the decoding unit may be specifically configured to decode the encoded game screen block in the game screen data stream according to a preset decoding order to obtain a decoded game screen block;

[0062] The rendering unit can be specifically used to render the decoded game screen block of the adjacent encoded game screen block corresponding to the target encoded game screen block when decoding the target encoded game screen block in the game screen data stream to obtain the target game screen block.

[0063] An electronic device provided in an embodiment of the present application includes a processor and a memory, wherein the memory stores multiple instructions, and the processor loads the instructions to execute the steps in the cloud game data sending method or the cloud game data receiving method provided in the embodiment of the present application.

[0064] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps in the cloud game data sending method or the cloud game data receiving method provided in the embodiment of the present application are implemented.

[0065] In addition, an embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps in the cloud game data sending method or cloud game data receiving method provided in the embodiment of the present application.

[0066] The embodiment of the present application provides a method for sending and receiving data of a cloud game and related equipment, which can obtain game data to be transmitted, wherein the game data includes at least one frame of game screen; for each frame of game screen in the game data, the game screen is divided into blocks to obtain at least one game screen block corresponding to the game screen; each game screen block of the game screen is encoded in parallel to obtain the encoded game screen data corresponding to the game screen; the encoded game screen data is processed to obtain at least two game screen data streams; different transmission links are used to transmit the at least two game screen data streams in parallel to the game terminal. The embodiment of the present application can reduce the delay of game screen encoding by encoding each game screen block of the game screen in parallel; and the game screen data stream can also be transmitted in parallel through multiple transmission links, which reduces the delay of data transmission and improves the efficiency of data transmission; based on the parallel encoding and parallel transmission method, the waiting delay of game data is reduced as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1a It is a scenario diagram of a method for sending and receiving data for a cloud game provided in an embodiment of the present application;

[0069] Figure 1b is a flow chart of a method for sending and receiving data for a cloud game provided in an embodiment of the present application;

[0070] Figure 1c This is an overall architecture diagram of the cloud game data sending and receiving method provided in an embodiment of the present application;

[0071] Figure 1d is another flow chart of the method for sending and receiving data of a cloud game provided in an embodiment of the present application;

[0072] Figure 2a is another flow chart of the method for sending and receiving data of a cloud game provided in an embodiment of the present application;

[0073] Figure 2b It is a framework diagram of a method for sending and receiving data of a cloud game provided in an embodiment of the present application;

[0074] Figure 3 It is a structural diagram of a data sending and receiving device for a cloud game provided in an embodiment of the present application;

[0075] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0077] The embodiments of the present application provide a method for sending and receiving data of a cloud game and related equipment, and the related equipment may include a data sending device for a cloud game, a data receiving device for a cloud game, an electronic device, a computer-readable storage medium, and a computer program product. The data sending and receiving device for a cloud game may be integrated in an electronic device, and the electronic device may be a terminal or a server.

[0078] Specifically, an embodiment of the present application provides a data transmission device for cloud games applicable to a first electronic device, and the first electronic device may be a server or other device. An embodiment of the present application also provides a data receiving device for cloud games applicable to a second electronic device, and the second electronic device may be a terminal or other device.

[0079] Among them, the terminal can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal computer (PC), an intelligent voice interaction device, a smart home appliance, a car terminal, etc., but is not limited to this. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application is not limited here. The data sending and receiving method or device of the cloud game disclosed in this application, wherein multiple servers can be composed of a blockchain, and the server is a node on the blockchain.

[0080] The embodiment of the present application will take the first electronic device as a server and the second electronic device as a terminal as an example to introduce the data sending and receiving method of cloud games.

[0081] like Figure 1a As shown, a method for sending and receiving data of a cloud game performed jointly by a terminal and a server is taken as an example. The data transmission system of a cloud game provided in an embodiment of the present application includes a terminal 10 and a server 11; the terminal 10 and the server 11 are connected via a network, for example, via a wired or wireless network connection. Among them, a client may be integrated in the terminal 10, and the client may specifically be an application installed in the terminal or a browser.

[0082] Among them, the server 11 can be used to: obtain game data to be transmitted, the game data including at least one frame of game screen; for each frame of game screen in the game data, divide the game screen into blocks to obtain at least one game screen block corresponding to the game screen; encode each game screen block of the game screen in parallel to obtain encoded game screen data corresponding to the game screen; process the encoded game screen data to obtain at least two game screen data streams; and use different transmission links to transmit the at least two game screen data streams in parallel to the game terminal (i.e., terminal 10).

[0083] Among them, the terminal 10 can be used to: receive at least two game screen data streams transmitted in parallel by the server 11 through different transmission links, the game screen data stream including encoded game screen blocks corresponding to each game screen block of the game screen; decode the encoded game screen blocks in the game screen data stream to obtain decoded game screen blocks; render the decoded game screen blocks to obtain target game screen blocks; merge the target game screen blocks belonging to the same frame of game screen to obtain and display the target game screen.

[0084] The cloud game data transmission and reception method provided in the embodiment of the present application relates to computer vision technology and machine learning in the field of artificial intelligence. The embodiment of the present application can reduce the delay of game data encoding and transmission as a whole based on parallel encoding and parallel transmission.

[0085] Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. Among them, artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0086] Among them, Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, automatic driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0087] Among them, machine learning (ML) is a multi-disciplinary interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0088] The cloud gaming data sending and receiving method provided in the embodiments of the present application also relates to the cloud gaming direction in the field of cloud technology.

[0089] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0090] Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud gaming scenario, the game is not played on the player's game terminal, but on a cloud server, which renders the game scene into a video and audio stream and transmits it to the player's game terminal over the network. The player's game terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input commands and send them to the cloud server.

[0091] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0092] This embodiment will be described from the perspective of a data sending device for cloud gaming. The data sending device for cloud gaming may be specifically integrated into a first electronic device, which may be a server or other device.

[0093] like Figure 1b As shown, the specific process of the data sending method of the cloud game can be as follows:

[0094] 101. Obtain game data to be transmitted, where the game data includes at least one frame of game screen.

[0095] It is understandable that the game data may also include other relevant game data such as audio data and game operation instructions.

[0096] This embodiment can be applied in cloud gaming scenarios. For cloud gaming, the game is not installed in the game terminal, but is run on a server in the cloud. The server renders the game scene into an audio and video stream (i.e., game data), which is transmitted to the game terminal through the network. The game screen can be displayed on the display of the game terminal. At the same time, the terminal operation of the game player is converted into instructions and sent back to the server.

[0097] Among them, the server can be an independent physical server or a server cluster composed of multiple physical servers. Specifically, in some embodiments, the server cluster may include a central server and an edge server. The game terminal may be integrated with a cloud game client, which may be a browser client or an application client, which is not limited in this embodiment. The cloud game client supports multiple operating system platforms, such as Windows operating system (Windows) and Apple operating system (macOS).

[0098] Compared with traditional RTC transmission, cloud gaming scenarios have higher requirements for data waiting delay. Generally speaking, if the game operation delay exceeds 100 milliseconds, it will seriously affect the gaming experience. In addition, cloud gaming scenarios have higher requirements for image quality and resolution. In ordinary audio and video communication, 360P is already relatively clear; in gaming scenarios, if the quality of the picture is much lower than that of local games, users may be lost; game pictures usually use higher bit rates (lower compression rates) and higher resolutions (1080P, or even 4K). In addition, in terms of frame rate, traditional communication only requires 25 to 30 frames to meet communication needs; while cloud gaming scenarios require at least 60fps (Frames Per Second, frames transmitted per second) to ensure the smoothness of the game, and even 120fps is required in scenarios such as VR (Virtual Reality). Therefore, compared with traditional RTC transmission, cloud gaming application scenarios have higher requirements for transmission bandwidth, quality, frame rate, delay, etc.

[0099] Among them, RTC (Real-time Communications) refers to real-time communication, which can generally refer to the real-time transmission of various data, including real-time transmission of media and non-media data such as audio, video, text, and pictures.

[0100] Among them, 360P, 1080P and 4K represent a video display format. The letter p stands for progressive scan, 360 represents a vertical resolution of 360, that is, there are 360 ​​horizontal scanning lines in the vertical direction; similarly, 1080 represents a vertical resolution of 1080, that is, there are 1080 horizontal scanning lines in the vertical direction; 4K means a pixel resolution of 4096×2160.

[0101] The present application can meet the requirements of high bandwidth, high image quality, high resolution, and low latency in cloud gaming scenarios through a series of optimization methods. It can achieve smooth transmission of audio and video at a resolution of 1080P+60fps and a resolution of 4K+120fps in RTC scenarios of cloud gaming environments with high bandwidth and low latency. It can also quickly detect network jitter and quickly adjust the bit rate and frame rate of the game screen according to the network jitter, specifically by reducing the bit rate and frame rate. This can effectively improve the ability to resist network jitter, so that the game screen data can be smoothly transmitted even when the network is jittering; and it can also quickly restore the bit rate and frame rate of the game screen when the network is normal.

[0102] 102. For each frame of game screen in the game data, divide the game screen into blocks to obtain at least one game screen block corresponding to the game screen.

[0103] There are many ways to divide the game screen into blocks, which are not limited in this embodiment. For example, in some embodiments, the number of game screen blocks to be divided can be determined first, and then the game screen can be equally divided based on the number to obtain game screen blocks of the same size. In other embodiments, the game screen can also be unequally divided, that is, the block sizes of the divided game screen blocks are different.

[0104] In a specific embodiment, for each frame of the game screen, the game screen can be equally divided into four blocks, specifically, a game screen block corresponding to the upper left corner, a game screen block corresponding to the lower left corner, a game screen block corresponding to the upper right corner, and a game screen block corresponding to the lower right corner.

[0105] 103. Perform parallel encoding on each game screen block of the game screen to obtain encoded game screen data corresponding to the game screen.

[0106] Among them, splitting a frame of game screen into multiple game screen blocks and encoding each game screen block in parallel can reduce the delay of game screen encoding.

[0107] Specifically, the game screen blocks can be encoded by the graphics processing unit (GPU) on the server. The server may include multiple GPUs. If the encoder on a GPU is busy, the game screen blocks can be placed on another GPU for encoding through the GPU-TO-GPU copy method; among them, the GPU-TO-GPU copy method is to copy the game screen blocks from one GPU to another GPU.

[0108] Optionally, in this embodiment, the step of “encoding each game screen block of the game screen in parallel to obtain encoded game screen data corresponding to the game screen” may include:

[0109] Determine encoding sampling parameters corresponding to the game screen according to current network transmission parameters;

[0110] According to the encoding sampling parameters, each game screen block of the game screen is encoded in parallel to obtain encoded game screen data corresponding to the game screen.

[0111] The current network transmission parameters may include region information, time information, transmission data size, channel transmission speed, network service provider, transmission delay information, packet loss rate, etc., which are not limited in this embodiment. The current network transmission parameters may reflect the current network transmission situation. For example, according to the time information, it can be confirmed whether the current time is the peak period of network use. If it is the peak period, the bit rate and frame rate of the game screen may be relatively reduced.

[0112] The coding sampling parameters may include the bit rate and frame rate corresponding to the game screen, etc., which are not limited in this embodiment. The coding sampling parameters are associated with the quality of the game screen. The higher the bit rate and frame rate, the higher the quality of the game screen, but the higher the requirements for network transmission.

[0113] This embodiment can adjust the encoding sampling parameters corresponding to the game screen in real time through the current network transmission parameters, such as the bit rate and frame rate. Specifically, when the network is congested, the bit rate and frame rate can be adjusted to a lower level, sacrificing the bit rate and frame rate in exchange for fluency. Once the network returns to normal, the game screen data will be sent again at a higher bit rate and frame rate.

[0114] Generally speaking, the game screen in the game data has a lot of repeated information, such as the picture information contained in the game screen of adjacent frames is generally not much different. In order to reduce the amount of data transmission, the game screen can be compressed to remove the repeated information of the game screen. The compression processing of the game screen can use the difference frame method, key frame method, forward reference frame method, etc. It should be noted that in the cloud game scenario of this embodiment, the forward reference frame method is generally not used. Because cloud games are very sensitive to delays, if you wait until the subsequent frames are received before decoding the previous frames, it will cause the waiting time to be too long.

[0115] Among them, the difference frame method is based on the difference operation of adjacent frames, specifically, the grayscale difference is obtained by subtracting the corresponding pixels of adjacent frames, and the grayscale difference is encoded. The key frame method is a method of completely retaining the game screen information of the key frame and appropriately compressing the game screen information of other frames. The forward reference frame method is based on the difference between the frame to be encoded and the previous frame. When decoding the game screen, the previous frame needs to refer to the subsequent frame to be decoded.

[0116] 104. Process the encoded game screen data to obtain at least two game screen data streams.

[0117] Among them, processing the encoded game screen data may specifically be performing data stream division processing on the encoded game screen data; through the data stream division processing, the encoded game screen data can be divided into multiple data streams, and different data streams can be transmitted in parallel through different transmission links, thereby improving the efficiency of data transmission.

[0118] The encoded game screen data includes encoded game screen blocks corresponding to each game screen block of at least one frame of game screen.

[0119] In some embodiments, each game screen block generated by each frame of the game screen can be distributed to different game screen data streams for transmission. After each game screen block of the same frame of the game screen is transmitted to the game terminal, the game terminal will then synthesize the various game screen blocks belonging to the same frame of the game screen into a complete game screen. Specifically, the division method of each frame of the game screen is consistent, so the number of game screen blocks corresponding to each frame of the game screen is the same. In a specific scenario, each frame of the game screen is divided into four game screen blocks, which may correspond to four game screen data streams. For each frame of the game screen, its four game screen blocks are distributed in these four game screen data streams for transmission.

[0120] Optionally, in this embodiment, the encoded game screen data includes the encoded game screen blocks corresponding to each game screen block;

[0121] The step of “processing the encoded game screen data to obtain at least two game screen data streams” may include:

[0122] Determine the position information corresponding to the encoded game screen block in the game screen;

[0123] The encoded game screen block is processed according to the position information to obtain at least two game screen data streams.

[0124] The position information specifically refers to the position information of the game screen block corresponding to the encoded game screen block in the game screen.

[0125] In some specific embodiments, the game screen is equally divided into four game screen blocks, namely, a game screen block corresponding to the upper left corner, a game screen block corresponding to the lower left corner, a game screen block corresponding to the upper right corner, and a game screen block corresponding to the lower right corner, and then each game screen block is encoded to obtain an encoded game screen block corresponding to each game screen block; then, according to the position information of each encoded game screen block in the game screen, each encoded game screen block is processed. Specifically, if the position information of the encoded game screen block in the game screen is the upper left corner, it can be The encoded game screen block is divided into a game screen data stream corresponding to the upper left corner; similarly, if the position information of the encoded game screen block in the game screen is the upper right corner, the encoded game screen block can be divided into the game screen data stream corresponding to the upper right corner; the position information of the encoded game screen block in the game screen is the lower left corner, the encoded game screen block can be divided into the game screen data stream corresponding to the lower left corner; the position information of the encoded game screen block in the game screen is the lower right corner, the encoded game screen block can be divided into the game screen data stream corresponding to the lower right corner.

[0126] Optionally, in this embodiment, the step of “processing the encoded game screen data to obtain at least two game screen data streams” may include:

[0127] Obtaining a game screen frame identifier corresponding to the encoded game screen data;

[0128] The encoded game screen data is processed according to the game screen frame identifier to obtain at least two game screen data streams.

[0129] Among them, the encoded game screen data corresponding to the game screen may specifically include the encoded game screen blocks corresponding to each game screen block of the game screen, and the game screen frame identifier corresponding to the encoded game screen data may specifically be the game screen frame identifier of the game screen to which the encoded game screen block belongs; the game screen frame identifier can identify each game screen, and may specifically be the frame number of each game screen.

[0130] Optionally, in this embodiment, the game screen frame identifier includes a frame number of the game screen; the step of "processing the encoded game screen data according to the game screen frame identifier to obtain at least two game screen data streams" may include:

[0131] When the frame number corresponding to the encoded game screen data is an even number, dividing the encoded game screen data into an even-numbered frame game screen data stream;

[0132] When the frame number corresponding to the encoded game screen data is an odd number, the encoded game screen data is divided into odd-numbered frame game screen data streams.

[0133] The frame number corresponding to the encoded game screen data is specifically the frame number of the game screen corresponding to the encoded game screen data. In this embodiment, based on the frame number corresponding to the encoded game screen data, it can be divided into two game screen data streams, namely, an even-numbered frame game screen data stream and an odd-numbered frame game screen data stream. By dividing the encoded game screen data into two data streams for parallel transmission, the data transmission efficiency can be improved.

[0134] Optionally, in this embodiment, the step of “processing the encoded game screen data to obtain at least two game screen data streams” may include:

[0135] According to the current network transmission parameters, redundant data is constructed for the encoded game screen data to obtain target game screen data, where the target game screen data includes the encoded game screen data and redundant data corresponding to the encoded game screen data;

[0136] The target game screen data is processed to obtain at least two game screen data streams.

[0137] The current network transmission parameters may include region information, time information, transmission data size, channel transmission speed, network service provider, ACK packet response time, transmission delay information, packet loss rate, etc., which are not limited in this embodiment. The current network transmission parameters may reflect the current network transmission situation.

[0138] There are many ways to construct redundant data for the encoded game screen data, which are not limited in this embodiment. For example, the redundant data construction method may be to construct redundant data only for key game screens.

[0139] In some embodiments, the current bandwidth carrying capacity can also be determined based on the current network transmission parameters, and then the redundancy ratio corresponding to the redundant data to be constructed can be determined based on the bandwidth carrying capacity, so that redundant data can be constructed for the encoded game screen data based on the redundancy ratio; for example, if the redundancy ratio is determined to be 30% based on the bandwidth carrying capacity, redundant data can be constructed for 30% of the encoded game screen data.

[0140] In this embodiment, the transmission status of the network can be determined based on the current network transmission parameters, and then it can be determined whether to transmit redundant data packets; specifically, it can be determined whether to transmit redundant data packets based on the response time of the ACK packet during the transmission process. For example, during the peak period of network transmission, if the response time of the ACK packet is long or there is no response, it can be considered that the packet loss rate is high, and redundant data is constructed for the encoded game screen data, and the anti-packet loss capability during network transmission is improved by increasing redundancy.

[0141] Specifically, when a network is congested, it does not start to become congested immediately, but there is a process of congestion accumulation, which will inevitably lead to a slower frequency of ACK replies. Therefore, we generally do not wait until there is serious network packet loss to send redundant data packets, but when we find that the reply time of ACK packets starts to slow down, we can start sending redundant data packets, which can alleviate the jamming caused by network packet loss with a certain probability.

[0142] Among them, ACK stands for Acknowledge character, which means confirmation character. It is a transmission control character sent by the receiving station to the sending station in data communication, used to indicate that the sent data has been confirmed to be received correctly. In the Transmission Control Protocol (TCP, Transmission Control Protocol), if the receiver successfully receives the data, it will reply an ACK data to the sender. Usually, the ACK signal has its own fixed format, which can be replied by the receiver to the sender.

[0143] Among them, for the redundant transmission of data packets, it should be noted that if a data packet of a TCP sequence number corresponding to the encoded game screen data is lost during the data transmission process, as long as the corresponding redundant data packet is successfully sent to the game terminal, the game terminal can decode the corresponding game screen data according to the redundant data packet, and it will not affect the display of the screen during the game. If the data packet corresponding to the encoded game screen data is not lost, and the data packet corresponding to the encoded game screen data and the redundant data packet are both sent to the game terminal, the redundant data packet will be discarded by the protocol stack of the game terminal due to the TCP sequence number conflict.

[0144] Optionally, in this embodiment, the cloud game server may include a central server and an edge server, etc. Specifically, steps such as "dividing the game screen into blocks and encoding each game screen block of the game screen in parallel" may be performed on the central server. After the central server completes encoding the game screen block, the central server may send the encoded game screen data to the edge server, so that the edge server processes the encoded game screen data, and then transmits the processed game screen data stream in parallel to the game terminal.

[0145] Optionally, in this embodiment, the step of “processing the encoded game screen data to obtain at least two game screen data streams” may include:

[0146] Obtaining location information of the game terminal;

[0147] Selecting a target edge server from the candidate edge servers according to the location information;

[0148] The encoded game screen data is processed by the target edge server to obtain at least two game screen data streams.

[0149] In a specific embodiment, the physical transmission distance between each candidate edge server and the game terminal can be determined based on the location information of the game terminal and the location information corresponding to each candidate edge server. According to the physical transmission distance, a target edge server can be selected from the candidate edge servers. For example, the candidate edge server with the shortest physical transmission distance can be determined as the target edge server. By selecting the edge server with the shortest physical transmission distance, the data transmission delay can be shortened.

[0150] In some embodiments, the target edge server may be selected from the candidate edge servers in combination with the physical transmission distance between each candidate edge server and the game terminal and the network transmission parameters corresponding to each candidate edge server.

[0151] Among them, the edge server can specifically be a proxy server. Since it is deployed at the edge node, it can provide high-bandwidth and low-latency data transmission capabilities. In addition, the edge server can also detect the network transmission status in real time and send the current network transmission parameters obtained by the detection to the central server, so that the central server can determine the encoding sampling parameters corresponding to the game screen based on the current network transmission parameters.

[0152] In this embodiment, the edge server is used as a transit station for data transmission instead of directly transmitting the encoded game screen data from the central server to the game terminal. This is because when data is transmitted to the game terminal through the edge server, the transmitted data will not generate a large bandwidth in the backbone network, and the bandwidth charges are lower, thereby reducing the operating costs of cloud games. In addition, it can also avoid competing for broadband in the backbone network and reduce the probability of packet loss.

[0153] In some embodiments, when the central server transmits the encoded game screen data to the target edge server, the high-speed packet transmission technology can be used to reduce the data transmission delay in the cloud game scene. The high-speed packet transmission technology can be implemented through the user-mode protocol stack and DPDK.

[0154] Optionally, in this embodiment, before the step of “processing the encoded game screen data through the target edge server to obtain at least two game screen data streams”, the following may also be included:

[0155] The encoded game screen data is copied to the kernel network card interface layer by a data processing tool, bypassing the kernel processor;

[0156] The encoded game screen data is sent to the target edge server through the kernel network card interface layer.

[0157] Among them, a data processing tool can be used to copy the encoded game screen data directly from the application storage layer of the central server to the kernel network card interface layer, wherein the data processing tool can be used to control the encoded game screen data to bypass the kernel processor in the central server to reach the kernel network card interface layer.

[0158] Among them, optionally, the data processing tool can be a data plane development kit (DPDK, Data Plane Development Kit); it mainly runs based on the Linux operating system, can be used for fast processing of data packets, and can greatly improve the performance and throughput of data processing. Specifically, it can bypass the core processor of the central server through the user-mode protocol stack, reduce the copy of the encoded game screen data from the application storage layer of the central server to the kernel processor, and directly send the encoded game screen data from the application storage layer of the central server to the kernel network card interface layer, so as to achieve high-speed network packet sending and reduce the data transmission delay in the cloud game scene. Moreover, this can make full use of the multi-core IO (Input / Output) capabilities of the central server, improve the throughput of the central server, and give full play to the network transmission capabilities of the central server's 10G network card.

[0159] 105. Use different transmission links to transmit the at least two game screen data streams in parallel to the game terminal respectively.

[0160] Among them, this embodiment can transmit the game screen data stream in parallel through multiple transmission links, which can reduce data transmission delay, improve data transmission efficiency, and make full use of the network bandwidth between the server and the game terminal. In addition, through the parallel transmission method, a task pipeline can be formed in the stages of receiving the game screen data stream, decoding, rendering, etc. at the game terminal.

[0161] Specifically, the amount of game screen data transmitted by different transmission links may be different, that is, the amount of data corresponding to different game screen data streams may be different. In some embodiments, the amount of data to be transmitted by the transmission link may be determined based on the link transmission parameters currently corresponding to the transmission link. For example, if the bandwidth of transmission link 1 is larger than the bandwidth of transmission link 2, the amount of game screen data transmitted by transmission link 1 may be larger than the amount of game screen data transmitted by transmission link 2.

[0162] Optionally, in this embodiment, the cloud game data sending method may further include:

[0163] Performing real-time detection on the link transmission parameters of the transmission link corresponding to the game screen data stream;

[0164] When the detected link transmission parameters do not meet the preset transmission conditions, the game screen data stream is switched to a target transmission link that meets the preset transmission conditions for transmission to the game terminal.

[0165] Among them, when the link transmission parameters of a transmission link that transmits the game screen data stream do not meet the preset transmission conditions, the game screen data stream can continue to be transmitted through other transmission links that meet the preset transmission conditions.

[0166] The link transmission parameters may include packet loss rate, operators providing network services, bandwidth size, etc., which are not limited in this embodiment.

[0167] The preset transmission condition can be set according to actual conditions, and this embodiment does not limit this. If the requirement for data transmission is higher, the preset transmission condition can be set more strictly. For example, the preset transmission condition can be that the packet loss rate is less than a preset value, and the preset value can be set according to actual conditions.

[0168] Specifically, in the game terminal, users can choose to access the network such as wired, wifi or LTE for data transmission; usually wifi networks can be divided into 2.4G HZ wifi and 5G HZ wifi; and LTE networks generally have 4G or 5G. If the network environment supports it, one or more networks can be used for data transmission at the same time. For example, the game terminal can use wired, wifi and LTE networks for data transmission at the same time. By accessing different transmission links of different networks, the probability of game screen freezes can be reduced.

[0169] In some embodiments, when close to the game terminal, the game screen may freeze due to poor wifi network signals. Therefore, 5G HZ wifi, 2.4G HZ wifi and LTE networks can be used simultaneously to establish transmission links. If a transmission link generates data delay due to weak signal (or signal interference), the game screen data stream can be switched to another transmission link with better signal for transmission. By means of redundant transmission links, the probability of game screen freezes during the game is reduced.

[0170] Among them, wifi is a wireless local area network technology that can connect network-connectable devices to each other wirelessly. 4G refers to the fourth generation of mobile communication technology network, and 5G refers to the fifth generation of mobile communication technology network. LTE (Long Term Evolution) is a communication system and communication standard.

[0171] Optionally, in this embodiment, the step of “using different transmission links to transmit the at least two game screen data streams in parallel to the game terminal” may include:

[0172] Performing data transmission acceleration processing on the at least two game screen data streams respectively;

[0173] Different transmission links are used to transmit at least two accelerated game screen data streams in parallel to the game terminal.

[0174] Specifically, the target edge server can perform data stream division processing on the encoded game screen data, and send the obtained at least two game screen data streams to different acceleration node servers through different transmission links. The acceleration node servers respectively perform data transmission acceleration processing on the corresponding game screen data streams, and then transmit the accelerated game screen data streams from the corresponding acceleration node servers to the game terminal in parallel through different transmission links. Different transmission links can use different acceleration node servers.

[0175] In complex network conditions, accelerating the data transmission of the game screen data stream by accelerating the node server can reduce the link length of the network during the data transmission process.

[0176] Specifically, during the data transmission process, data may be transmitted on the networks of different operators. The data conversion between different operators may cause packet loss. By accelerating the node server, the conversion of the network across operators can be achieved, thereby reducing the data packet loss rate.

[0177] Optionally, in this embodiment, a machine learning method may be used to select a transmission link. Machine learning can comprehensively consider data from many dimensions, analyze the optimal transmission link based on the multi-dimensional data, and then construct an optimal transmission strategy.

[0178] Optionally, in this embodiment, the cloud game data sending method may further include:

[0179] Extracting characteristics of current network transmission parameters to obtain transmission characteristic information corresponding to the current network transmission parameters;

[0180] Determine, according to the transmission characteristic information, the predicted transmission delay information corresponding to the game screen data stream on each candidate transmission link;

[0181] According to the predicted transmission delay information, a transmission link corresponding to the game screen data stream is determined from each candidate transmission link.

[0182] Among them, the feature extraction of the current network transmission parameters may specifically include convolution processing and pooling processing of the current network transmission parameters.

[0183] Among them, the predicted transmission delay information corresponding to the game screen data stream on each candidate transmission link can be predicted by a classifier. The classifier can be a support vector machine (SVM), a recurrent neural network, a fully connected deep neural network (DNN), etc., which is not limited in this embodiment.

[0184] Specifically, the candidate transmission link with the lowest predicted transmission delay information may be determined as the transmission link corresponding to the game screen data stream.

[0185] Optionally, in this embodiment, the current network transmission parameter includes a network transmission sub-parameter of at least one dimension;

[0186] The step of “extracting features from the current network transmission parameters to obtain transmission feature information corresponding to the current network transmission parameters” may include:

[0187] Extracting features of network transmission sub-parameters in each dimension to obtain transmission sub-feature information corresponding to the network transmission sub-parameters in each dimension;

[0188] The transmission sub-characteristic information corresponding to the network transmission sub-parameters of each dimension is merged to obtain the transmission characteristic information corresponding to the current network transmission parameter.

[0189] The current network transmission parameters include at least one dimension of network transmission sub-parameters, specifically, it may include regional information, operators providing network services, time information, channel transmission speed, transmission data size, packet loss rate, etc. The regional information may specifically be the region where the game terminal is located, which can usually be obtained based on the IP address (Internet Protocol Address) of the game terminal.

[0190] There are many ways to fuse the transmission sub-feature information corresponding to the network transmission sub-parameters of each dimension, which are not limited in this embodiment. For example, the fusion method may be concatenation processing or weighted fusion.

[0191] In this embodiment, the step of “extracting features from the current network transmission parameters to obtain transmission feature information corresponding to the current network transmission parameters” may include:

[0192] By using the link delay prediction model, feature extraction is performed on the current network transmission parameters to obtain transmission feature information corresponding to the current network transmission parameters;

[0193] The step of “determining predicted transmission delay information corresponding to the game screen data stream on each candidate transmission link according to the transmission characteristic information” includes:

[0194] Through the link delay prediction model, according to the transmission characteristic information, the predicted transmission delay information corresponding to the game screen data stream on each candidate transmission link is determined.

[0195] Among them, the link delay prediction model can be a neural network model, which can include a convolutional neural network (CNN), a residual network (ResNet), a densely connected convolutional network (DenseNet), etc., but it should be understood that the link delay prediction model of this embodiment is not limited to the types listed above.

[0196] It should be noted that the link delay prediction model can be trained by other devices and then provided to the data sending device of the cloud game, or it can also be trained by the data sending device of the cloud game itself.

[0197] If the data sending device of the cloud game performs training by itself, before the step of "extracting features of the current network transmission parameters through the link delay prediction model to obtain transmission feature information corresponding to the current network transmission parameters", it may also include:

[0198] Acquire training data, the training data including sample network transmission parameters and expected transmission delay information corresponding to the sample network transmission parameters;

[0199] Extracting features of the sample network transmission parameters through a link delay prediction model to obtain transmission feature information of the sample network transmission parameters;

[0200] Determine, based on the transmission characteristic information, the actual transmission delay information corresponding to the sample game screen data stream on each candidate transmission link;

[0201] According to the actual transmission delay information and the expected transmission delay information, parameters of the link delay prediction model are adjusted to obtain a trained link delay prediction model.

[0202] Among them, the sample network transmission parameters and their corresponding expected transmission delay information can specifically come from the relevant network transmission parameters reported during the historical data transmission process. Optionally, in the data transmission process of this embodiment, relevant network transmission parameters can be reported at each stage, such as coding, data packet sending, data packet receiving, decoding, screen rendering and other links. In addition, network transmission parameters such as the region where the game terminal is located and the operator providing network services can also be reported. The training data is obtained by collecting the network transmission parameters reported during the historical data transmission process, and then the link delay prediction model is trained using these data.

[0203] Among them, the training process is to first calculate the actual transmission delay information corresponding to the sample game screen data stream on each candidate transmission link, and then use the back propagation algorithm to adjust the parameters of the link delay prediction model. Based on the actual transmission delay information and expected transmission delay information corresponding to each candidate transmission link, the parameters of the link delay prediction model are optimized so that the actual transmission delay information corresponding to the predicted candidate transmission link is close to its corresponding expected transmission delay information, thereby obtaining a trained link delay prediction model.

[0204] Specifically, the loss value between the actual transmission delay information corresponding to the predicted candidate transmission link and the expected transmission delay information corresponding to it can be made smaller than a preset value, and the preset value can be set according to actual conditions.

[0205] In some embodiments, the link delay prediction model can also be used to predict the encoding sampling parameters corresponding to the game screen, such as bit rate and frame rate. Specifically, the link delay prediction model can be used to predict the transmission delay information corresponding to the game screen data stream on the transmission link. Based on the transmission delay information, the current network transmission status can be determined, and then the encoding sampling parameters corresponding to the game screen can be determined.

[0206] In other embodiments, when encoding the game screen, the selected encoding sampling parameters - bit rate and frame rate can be determined in combination with the response time of the ACK packet during data transmission and the prediction of the link delay prediction model.

[0207] It should be noted that, in this embodiment, the training of the link delay prediction model can be real-time. When the latest reported network transmission parameters are obtained, the link delay prediction model can adjust the parameters inside the link delay prediction model in real time according to the latest reported network transmission parameters.

[0208] Optionally, in this embodiment, the transmission strategy for the game screen data stream can also be considered from the perspective of the operator providing network services. Compared with TCP (Transmission Control Protocol), UDP (User Datagram Protocol) has a better ability to grab bandwidth, but the use of UDP for data transmission is not conducive to the operator's coordination of bandwidth during peak hours; operators usually adopt a "UDP discrimination" strategy during peak hours to avoid network congestion by randomly dropping packets. Therefore, the present application can avoid using UDP during peak data transmission, and instead use the TCP protocol that is easy for operators to control, and can also control the degree of redundant packet transmission in the case of network congestion.

[0209] In addition, during peak data transmission periods, some operators will modify the value of the TCP sliding window to limit network traffic. In this case, the size of the sliding window can also be taken into consideration. When the sliding window becomes smaller, the bit rate and frame rate of the game screen can be modified accordingly to reduce the network bandwidth occupied during data transmission. The value of the sliding window can represent the amount of data allowed to be transmitted per second.

[0210] Understandably, cloud gaming can also establish cooperation with operators to provide better network quality for gamers.

[0211] In a cloud gaming scenario, Figure 1cThe figure shows the overall architecture of the cloud game data transmission system. Specifically, each frame of the game screen to be transmitted can be divided into blocks by the central server of the cloud game, and at least one game screen block corresponding to the game screen can be obtained, and then each game screen block of the game screen can be encoded in parallel to obtain the encoded game screen data corresponding to the game screen, and the encoded game screen data can be sent to the target edge server. After receiving the encoded game screen data, the target edge server can process the encoded game screen data to obtain at least two game screen data streams, and then use different transmission links to transmit the at least two game screen data streams in parallel to the game terminal.

[0212] Among them, optionally, in the process of transmitting the at least two game screen data streams from the target edge server to the game terminal through different transmission links, the game screen data streams can be accelerated through different acceleration nodes (specifically, acceleration node servers), and then different transmission links are used to transmit the at least two accelerated game screen data streams in parallel to the game terminal.

[0213] Among them, different networks can be used to establish transmission links respectively. If a transmission link generates data delay due to weak signal (or signal interference), the game screen data stream can be switched to another transmission link with better signal for transmission, which can reduce the packet loss rate and thus reduce the probability of game screen freeze during the game. For example, near the game terminal, network 1 can be a 2.4G HZ wifi network, network 2 can be a 4G LTE network, and network 3 can be a 5G HZwifi network; if it is detected that the transmission link corresponding to network 1 has signal interference, the game screen data stream on the transmission link corresponding to network 1 can be switched to the transmission link corresponding to network 3 for transmission to the game terminal.

[0214] In a specific embodiment, if Figure 1d As shown, it is another flow chart of the data sending method of the cloud game, which is specifically described as follows:

[0215] 1001. The central server of the cloud game obtains game data to be transmitted according to an operation instruction sent by a game terminal, where the game data includes at least one frame of game screen;

[0216] 1002. The central server of the cloud game divides each frame of the game screen in the game data into blocks to obtain at least one game screen block corresponding to the game screen;

[0217] 1003. The central server of the cloud game determines the encoding sampling parameters corresponding to the game screen according to the current network transmission parameters;

[0218] 1004. The central server of the cloud game performs parallel encoding on each game screen block of the game screen according to the encoding sampling parameters to obtain encoded game screen data corresponding to the game screen;

[0219] 1005. The central server of the cloud game sends the encoded game screen data to the target edge server;

[0220] 1006. The target edge server performs data stream division processing on the encoded game screen data to obtain at least two game screen data streams, wherein the target edge server is selected from the candidate edge servers based on the location information of the game terminal;

[0221] 1007. The target edge server uses different transmission links to transmit the at least two game screen data streams in parallel to the game terminal.

[0222] As can be seen from the above, the present embodiment can obtain the game data to be transmitted, the game data including at least one frame of game screen; for each frame of game screen in the game data, the game screen is divided into blocks to obtain at least one game screen block corresponding to the game screen; each game screen block of the game screen is encoded in parallel to obtain the encoded game screen data corresponding to the game screen; the encoded game screen data is processed to obtain at least two game screen data streams; different transmission links are used to transmit the at least two game screen data streams in parallel to the game terminal. The embodiment of the present application can reduce the delay of game screen encoding by encoding each game screen block of the game screen in parallel; and the game screen data stream can also be transmitted in parallel through multiple transmission links, which reduces the delay of data transmission and improves the efficiency of data transmission; based on the parallel encoding and parallel transmission method, the waiting delay of game data is reduced as a whole.

[0223] This embodiment will be described from the perspective of a data receiving device for cloud gaming. The data receiving device for cloud gaming may be specifically integrated into a second electronic device, which may be a terminal or other device.

[0224] like Figure 2a As shown, the specific process of the data receiving method of the cloud game is as follows:

[0225] 201. Receive at least two game screen data streams transmitted in parallel by a server through different transmission links, wherein the game screen data streams include encoded game screen blocks corresponding to respective game screen blocks of the game screen.

[0226] The encoded game screen block is transmitted to the game terminal in parallel through at least two game screen data streams, which can also improve the receiving efficiency of the game terminal for the encoded game screen block.

[0227] This embodiment can be applied in cloud gaming scenarios. For cloud gaming, the game is not installed in the game terminal, but is run on a server in the cloud, and the server renders the game scene into an audio and video stream (specifically, a game screen data stream) and transmits it to the game terminal through the network. The game terminal receives the game screen data stream transmitted by the server, and then displays the game screen based on the encoded game screen blocks in the game screen data stream.

[0228] The game terminal can be a desktop computer, tablet computer, mobile phone, TV, set-top box, or game console, etc. The cloud game client on the game terminal is equivalent to a player, which plays the game audio and video stream generated by the cloud game server.

[0229] In some embodiments, the server can distribute the various game screen blocks generated by each frame of game screen into different game screen data streams for transmission. After the various game screen blocks of the same frame of game screen are transmitted to the game terminal, the game terminal will then synthesize the various game screen blocks belonging to the same frame of game screen into a complete game screen.

[0230] In other embodiments, the server may also perform data stream division processing on the encoded game screen data corresponding to the game screen according to the game screen frame identifier, specifically, it may be divided into an even-numbered game screen data stream and an odd-numbered game screen data stream; wherein the game screen frame identifier may specifically be the frame number of the game screen. Then, the server transmits the even-numbered game screen data stream and the odd-numbered game screen data stream to the game terminal in parallel.

[0231] 202. Decode the encoded game screen block in the game screen data stream to obtain a decoded game screen block.

[0232] Among them, the decoding method of the encoded game screen block can be determined based on its corresponding encoding method.

[0233] In the current related technologies, the process of displaying the game screen by the game terminal can be divided into three stages, which are: waiting for a complete frame of screen data to be received, decoding the complete frame of screen data, and finally rendering and displaying the decoded screen data. These three stages are executed serially, and when one stage is executed, the other two stages are in a waiting state. This has a low utilization rate of hardware resources.

[0234] In this embodiment, each frame of the game screen can be split into at least one corresponding encoded game screen block through the server, and the encoded game screen block corresponding to each frame of the game screen can be transmitted to the game terminal. The game terminal can decode other received encoded game screen blocks while receiving a certain encoded game screen block, and render the decoded game screen block, so that the three stages of packet receiving, decoding, and rendering in the game terminal are always in a busy state, that is, the pipeline of the entire process is busy, thereby reducing the overall delay of the game screen display, and can also improve the resource utilization under the existing hardware conditions.

[0235] Optionally, in this embodiment, the step of “receiving at least two game screen data streams transmitted in parallel by the server through different transmission links” may include:

[0236] Receiving at least two game screen data streams transmitted in parallel by the server through different transmission links in a preset transmission order;

[0237] The step of “decoding the encoded game screen block in the game screen data stream to obtain a decoded game screen block” may include:

[0238] When receiving the target encoded game screen block in the game screen data stream, the adjacent encoded game screen block corresponding to the target encoded game screen block is decoded to obtain a decoded game screen block.

[0239] The preset transmission order may specifically be the transmission order of the encoded game screen blocks.

[0240] Among them, the target encoded game screen block can be any encoded game screen block in the game screen data stream, and the adjacent encoded game screen block corresponding to the target encoded game screen block can specifically be the encoded game screen block adjacent to the target encoded game screen block in the game screen data stream.

[0241] 203. Render the decoded game screen block to obtain a target game screen block.

[0242] Optionally, in this embodiment, the step of “decoding the encoded game screen block in the game screen data stream to obtain a decoded game screen block” may include:

[0243] Decoding the encoded game screen blocks in the game screen data stream according to a preset decoding order to obtain decoded game screen blocks;

[0244] The step of “rendering the decoded game screen block to obtain a target game screen block” may include:

[0245] When decoding the target encoded game screen block in the game screen data stream, the decoded game screen block of the adjacent encoded game screen block corresponding to the target encoded game screen block is rendered to obtain the target game screen block.

[0246] Among them, the target encoded game screen block can be any encoded game screen block that has been received but not rendered in the game screen data stream, and the adjacent encoded game screen block corresponding to the target encoded game screen block can specifically be the encoded game screen block adjacent to the target encoded game screen block in the game screen data stream.

[0247] 204. Merge target game screen blocks belonging to the same frame of game screen to obtain and display the target game screen.

[0248] Specifically, for target game screen blocks belonging to the same frame of game screen, the target game screen blocks belonging to the same frame of game screen can be merged into a complete screen, i.e., the target game screen, according to the corresponding position information of the target game screen blocks in the game screen. Specifically, the fusion method can be splicing processing, etc.

[0249] In some embodiments, the game terminal can also perform super-resolution optimization on the game screen through an image optimization model. Through super-resolution optimization, a low-definition image can be restored to a high-definition image. The image optimization model can specifically be a neural network model, such as a generative adversarial network (GAN, Generative Adversarial Networks), etc., which is not limited in this embodiment.

[0250] Among them, the generative adversarial network can obtain a reconstructed game screen with higher resolution by performing image reconstruction processing on the game screen. Specifically, the image reconstruction processing process may include: performing feature encoding on the game screen through the generative adversarial network to obtain the encoding information of the game screen; and then performing feature reconstruction on the encoding information to obtain the reconstructed game screen.

[0251] In a cloud gaming scenario, Figure 2b The figure shows the framework of the cloud game data transmission system. For cloud games, the game is not played on the game terminal where the game player is, but is run in the cloud server. Cloud games mainly transmit the audio and video streams generated by the game running on the server to the game terminal through the computer network, and transmit the operation instructions generated by the player on the game terminal back to the server. The game terminal can be integrated with a cloud game client, which can be an application installed in the terminal or a browser.

[0252] Specifically, game players can input game operations corresponding to the target game through the game control device. The game control device can convert the player's game operations into corresponding game control instructions and send the game control instructions to the terminal device. Then, the terminal device sends the game control instructions to the cloud gaming platform. The cloud gaming platform returns the corresponding game audio and video stream and game instruction stream to the terminal device based on the game control instructions. After receiving the game audio and video stream and game instruction stream returned by the cloud gaming platform, the terminal device can display the relevant game screen on the display screen of the terminal device based on the game audio and video stream and game instruction stream.

[0253] Among them, the game control device may include a game controller, a keyboard, a mouse, and a wearable device, etc., which is not limited in this embodiment. The game control device can be used to control the virtual character of the game or the game process. The game control device and the terminal device can be connected by wire or wirelessly. The terminal device may include a television, a personal computer, a mobile phone, a tablet computer, etc., which is not limited in this embodiment. Among them, the cloud gaming platform may be a server cluster composed of multiple physical servers, which can integrate game programs corresponding to various types of games, and the cloud gaming platform may include a central server and an edge server, etc.

[0254] Among them, the step of "the terminal device sends the game control instruction to the cloud gaming platform" may include: the terminal device can select a target edge server from multiple edge servers based on its location information, and send the game control instruction to the target edge server. The target edge server can forward the game control instruction to the central server corresponding to the target game in the cloud gaming platform based on the target game indicated in the game control instruction.

[0255] Among them, the step of "the cloud gaming platform returns the corresponding game audio and video stream and game command stream to the terminal device according to the game control instruction" may include: after receiving the game control instruction, the central server corresponding to the target game in the cloud gaming platform can send the game audio and video stream and game command stream corresponding to the target game to the target edge server according to the game control instruction, and then return the game audio and video stream and game command stream to the terminal device through the target edge server.

[0256] As can be seen from the above, this embodiment can receive at least two game screen data streams transmitted in parallel by the server through different transmission links, and the game screen data stream includes the encoded game screen blocks corresponding to each game screen block of the game screen; the encoded game screen blocks in the game screen data stream are decoded to obtain decoded game screen blocks; the decoded game screen blocks are rendered to obtain target game screen blocks; the target game screen blocks belonging to the same frame of game screen are merged to obtain and display the target game screen. The embodiment of the present application can receive game screen data streams transmitted in parallel, and then display the target game screen based on the game screen data stream, which can reduce the waiting delay of the game screen display as a whole.

[0257] In order to better implement the above method, the embodiment of the present application also provides a data transmission system for cloud games. The data transmission system for cloud games includes a data sending device 31 for cloud games and a data receiving device 32 for cloud games. Figure 3 As shown, the data sending device 31 of the cloud game may include an acquisition unit 3101, a block division unit 3102, an encoding unit 3103, a data stream division unit 3104 and a transmission unit 3105; the data receiving device 32 of the cloud game may include a receiving unit 3201, a decoding unit 3202, a rendering unit 3203 and a fusion unit 3204, as follows:

[0258] A. Data transmission device for cloud gaming 31

[0259] (1) Acquisition unit 3101;

[0260] The acquisition unit 3101 is used to acquire game data to be transmitted, where the game data includes at least one frame of game screen.

[0261] (2) block division unit 3102;

[0262] The block division unit 3102 is used to divide the game screen into blocks for each frame of the game screen in the game data, so as to obtain at least one game screen block corresponding to the game screen.

[0263] (3) Coding unit 3103;

[0264] The encoding unit 3103 is used to perform parallel encoding on each game screen block of the game screen to obtain encoded game screen data corresponding to the game screen.

[0265] Optionally, in some embodiments of the present application, the encoding unit may include a sampling parameter determination subunit and a parallel encoding subunit, as follows:

[0266] The sampling parameter determination subunit is used to determine the encoding sampling parameters corresponding to the game screen according to the current network transmission parameters;

[0267] The parallel encoding subunit is used to perform parallel encoding on each game screen block of the game screen according to the encoding sampling parameters to obtain encoded game screen data corresponding to the game screen.

[0268] (4) data stream division unit 3104;

[0269] The data stream division unit 3104 is used to process the encoded game screen data to obtain at least two game screen data streams.

[0270] Optionally, in some embodiments of the present application, the encoded game screen data includes encoded game screen blocks corresponding to each game screen block;

[0271] The data stream dividing unit may include a position determining subunit and a first dividing subunit, as follows:

[0272] The position determination subunit is used to determine the position information corresponding to the encoded game screen block in the game screen;

[0273] The first division subunit is used to process the encoded game screen block according to the position information to obtain at least two game screen data streams.

[0274] Optionally, in some embodiments of the present application, the data stream division unit may include a frame identifier acquisition subunit and a second division subunit, as follows:

[0275] The frame identifier acquisition subunit is used to acquire the game screen frame identifier corresponding to the encoded game screen data;

[0276] The second division subunit is used to process the encoded game screen data according to the game screen frame identifier to obtain at least two game screen data streams.

[0277] Optionally, in some embodiments of the present application, the game screen frame identifier includes the frame number of the game screen; the second division sub-unit can be specifically used to divide the encoded game screen data into even-numbered frame game screen data streams when the frame number corresponding to the encoded game screen data is an even number; and to divide the encoded game screen data into odd-numbered frame game screen data streams when the frame number corresponding to the encoded game screen data is an odd number.

[0278] Optionally, in some embodiments of the present application, the data stream division unit may include a redundancy construction subunit and a third division subunit, as follows:

[0279] The redundancy construction subunit is used to construct redundant data for the encoded game screen data according to the current network transmission parameters to obtain target game screen data, wherein the target game screen data includes the encoded game screen data and redundant data corresponding to the encoded game screen data;

[0280] The third division subunit is used to process the target game screen data to obtain at least two game screen data streams.

[0281] Optionally, in some embodiments of the present application, the data stream division unit may include a position acquisition subunit, a selection subunit, and a fourth division subunit, as follows:

[0282] The location acquisition subunit is used to acquire the location information of the game terminal;

[0283] A selection subunit, configured to select a target edge server from the candidate edge servers according to the location information;

[0284] The fourth division subunit is used to process the encoded game screen data through the target edge server to obtain at least two game screen data streams.

[0285] Optionally, in some embodiments of the present application, the data stream division unit may further include a copy subunit and a sending subunit, as follows:

[0286] The copy subunit is used to copy the encoded game screen data to the kernel network card interface layer by bypassing the kernel processor through a data processing tool;

[0287] The sending subunit is used to send the encoded game screen data to the target edge server through the kernel network card interface layer.

[0288] (5) Transmission unit 3105;

[0289] The transmission unit 3105 is used to use different transmission links to transmit the at least two game screen data streams in parallel to the game terminal.

[0290] Optionally, in some embodiments of the present application, the data sending device of the cloud game may further include a link switching unit, as follows:

[0291] The link switching unit is used to perform real-time detection of the link transmission parameters of the transmission link corresponding to the game screen data stream; when the detected link transmission parameters do not meet the preset transmission conditions, the game screen data stream is switched to the target transmission link that meets the preset transmission conditions for transmission to the game terminal.

[0292] Optionally, in some embodiments of the present application, the transmission unit may include an acceleration subunit and a transmission subunit, as follows:

[0293] The acceleration sub-unit is used to perform data transmission acceleration processing on the at least two game screen data streams respectively;

[0294] The transmission subunit is used to use different transmission links to transmit at least two game screen data streams after acceleration processing in parallel to the game terminal.

[0295] Optionally, in some embodiments of the present application, the data transmission device of the cloud game may further include a link determination unit, which is used to determine the transmission link corresponding to the game screen data stream from each candidate transmission link. Specifically, the link determination unit may include an extraction subunit, a prediction subunit, and a link determination subunit, as follows:

[0296] The extraction subunit is used to extract features of the current network transmission parameters to obtain transmission feature information corresponding to the current network transmission parameters;

[0297] A prediction subunit, configured to determine, based on the transmission characteristic information, the predicted transmission delay information corresponding to the game screen data stream on each candidate transmission link;

[0298] The link determination subunit is used to determine the transmission link corresponding to the game screen data stream from each candidate transmission link according to the predicted transmission delay information.

[0299] Optionally, in some embodiments of the present application, the current network transmission parameter includes a network transmission sub-parameter of at least one dimension;

[0300] The extraction subunit can be specifically used to extract features from network transmission sub-parameters of each dimension to obtain transmission sub-feature information corresponding to the network transmission sub-parameters of each dimension; and fuse the transmission sub-feature information corresponding to the network transmission sub-parameters of each dimension to obtain the transmission feature information corresponding to the current network transmission parameters.

[0301] B. Data receiving device for cloud gaming 32

[0302] (6) receiving unit 3201;

[0303] The receiving unit 3201 is used to receive at least two game screen data streams transmitted in parallel by the server through different transmission links, and the game screen data streams include encoded game screen blocks corresponding to each game screen block of the game screen.

[0304] (7) decoding unit 3202;

[0305] The decoding unit 3202 is used to decode the encoded game screen block in the game screen data stream to obtain a decoded game screen block.

[0306] Optionally, in some embodiments of the present application, the receiving unit may be specifically configured to receive at least two game screen data streams transmitted in parallel by the server through different transmission links in a preset transmission order;

[0307] The decoding unit may be specifically configured to, when receiving a target encoded game screen block in the game screen data stream, decode an adjacent encoded game screen block corresponding to the target encoded game screen block to obtain a decoded game screen block.

[0308] (8) rendering unit 3203;

[0309] The rendering unit 3203 is used to render the decoded game screen block to obtain a target game screen block.

[0310] Optionally, in some embodiments of the present application, the decoding unit may be specifically configured to decode the encoded game screen block in the game screen data stream according to a preset decoding order to obtain a decoded game screen block;

[0311] The rendering unit can be specifically used to render the decoded game screen block of the adjacent encoded game screen block corresponding to the target encoded game screen block when decoding the target encoded game screen block in the game screen data stream to obtain the target game screen block.

[0312] (9) fusion unit 3204;

[0313] The fusion unit 3204 is used to fuse the target game screen blocks belonging to the same frame of game screen to obtain and display the target game screen.

[0314] As can be seen from the above, in this embodiment, the acquisition unit 3101 can acquire the game data to be transmitted, and the game data includes at least one frame of the game screen; the block division unit 3102 divides the game screen into blocks for each frame of the game screen in the game data to obtain at least one game screen block corresponding to the game screen; the encoding unit 3103 encodes the game screen blocks of the game screen in parallel to obtain the encoded game screen data corresponding to the game screen; the data stream division unit 3104 processes the encoded game screen data to obtain at least two game screen data streams; and the transmission unit 3105 uses different transmission links to transmit the at least two game screen data streams in parallel to the game terminal respectively;

[0315] or,

[0316] The receiving unit 3201 receives at least two game screen data streams transmitted in parallel by the server through different transmission links, wherein the game screen data stream includes encoded game screen blocks corresponding to each game screen block of the game screen; the decoding unit 3202 decodes the encoded game screen blocks in the game screen data stream to obtain decoded game screen blocks; the rendering unit 3203 renders the decoded game screen blocks to obtain target game screen blocks; the fusion unit 3204 fuses the target game screen blocks belonging to the same frame of the game screen to obtain and display the target game screen;

[0317] The embodiments of the present application can reduce the delay of game screen encoding by parallel encoding of each game screen block of the game screen; and can also transmit the game screen data stream in parallel through multiple transmission links, thereby reducing the delay of data transmission and improving data transmission efficiency; based on the parallel encoding and parallel transmission methods, the waiting delay of game data is reduced as a whole.

[0318] The present application also provides an electronic device, such as Figure 4 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present application, and the electronic device may be a terminal or a server, etc. Specifically:

[0319] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0320] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the electronic device and processes data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0321] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0322] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0323] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0324] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, as follows:

[0325] Acquire game data to be transmitted, the game data including at least one frame of game screen; for each frame of game screen in the game data, divide the game screen into blocks to obtain at least one game screen block corresponding to the game screen; perform parallel encoding on each game screen block of the game screen to obtain encoded game screen data corresponding to the game screen; process the encoded game screen data to obtain at least two game screen data streams; and use different transmission links to transmit the at least two game screen data streams in parallel to the game terminal respectively;

[0326] or,

[0327] A receiving server transmits at least two game screen data streams in parallel via different transmission links, wherein the game screen data stream includes encoded game screen blocks corresponding to each game screen block of the game screen; decoding the encoded game screen blocks in the game screen data stream to obtain decoded game screen blocks; rendering the decoded game screen blocks to obtain target game screen blocks; and merging target game screen blocks belonging to the same frame of game screen to obtain and display a target game screen.

[0328] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0329] As can be seen from the above, the present embodiment can obtain game data to be transmitted, wherein the game data includes at least one frame of game screen; for each frame of game screen in the game data, the game screen is divided into blocks to obtain at least one game screen block corresponding to the game screen; each game screen block of the game screen is encoded in parallel to obtain encoded game screen data corresponding to the game screen; the encoded game screen data is processed to obtain at least two game screen data streams; and the at least two game screen data streams are respectively transmitted in parallel to the game terminal using different transmission links;

[0330] or,

[0331] A receiving server transmits at least two game screen data streams in parallel via different transmission links, wherein the game screen data stream includes encoded game screen blocks corresponding to each game screen block of the game screen; decoding the encoded game screen blocks in the game screen data stream to obtain decoded game screen blocks; rendering the decoded game screen blocks to obtain target game screen blocks; and merging target game screen blocks belonging to the same frame of game screen to obtain and display a target game screen.

[0332] The embodiments of the present application can reduce the delay of game screen encoding by parallel encoding of each game screen block of the game screen; and can also transmit the game screen data stream in parallel through multiple transmission links, thereby reducing the delay of data transmission and improving data transmission efficiency; based on the parallel encoding and parallel transmission methods, the waiting delay of game data is reduced as a whole.

[0333] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0334] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any of the cloud game data sending and receiving methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0335] Acquire game data to be transmitted, the game data including at least one frame of game screen; for each frame of game screen in the game data, divide the game screen into blocks to obtain at least one game screen block corresponding to the game screen; perform parallel encoding on each game screen block of the game screen to obtain encoded game screen data corresponding to the game screen; process the encoded game screen data to obtain at least two game screen data streams; and use different transmission links to transmit the at least two game screen data streams in parallel to the game terminal respectively;

[0336] or,

[0337] A receiving server transmits at least two game screen data streams in parallel via different transmission links, wherein the game screen data stream includes encoded game screen blocks corresponding to each game screen block of the game screen; decoding the encoded game screen blocks in the game screen data stream to obtain decoded game screen blocks; rendering the decoded game screen blocks to obtain target game screen blocks; and merging target game screen blocks belonging to the same frame of game screen to obtain and display a target game screen.

[0338] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0339] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0340] Since the instructions stored in the computer-readable storage medium can execute the steps in any one of the cloud game data sending and receiving methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the cloud game data sending and receiving methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0341] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in various optional implementations of the above-mentioned cloud game data transmission and reception.

[0342] The above is a detailed introduction to a cloud game data sending and receiving method and related equipment provided in an embodiment of the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for sending data of a cloud game, characterized in that: include: Acquire game data to be transmitted, wherein the game data includes at least one frame of game screen; For each frame of the game screen in the game data, the game screen is divided into blocks to obtain at least one game screen block corresponding to the game screen; Parallel encoding is performed on each game screen block of the game screen to obtain encoded game screen data corresponding to the game screen; Processing the encoded game screen data to obtain at least two game screen data streams; Using different transmission links to transmit the at least two game screen data streams in parallel to the game terminal respectively; The method for determining the transmission link includes: Through the link delay prediction model, feature extraction is performed on the network transmission sub-parameters of each dimension in the current network transmission parameters to obtain transmission sub-feature information corresponding to the network transmission sub-parameters of each dimension; By using the link delay prediction model, the transmission sub-feature information corresponding to the network transmission sub-parameters of each dimension is integrated to obtain the transmission feature information corresponding to the current network transmission parameter; Determine, by using the link delay prediction model and according to the transmission characteristic information, the predicted transmission delay information corresponding to the game screen data stream on each candidate transmission link; According to the predicted transmission delay information, a transmission link corresponding to the game screen data stream is determined from each candidate transmission link.

2. The method according to claim 1, characterized in that The step of encoding the game screen blocks of the game screen in parallel to obtain encoded game screen data corresponding to the game screen includes: Determine encoding sampling parameters corresponding to the game screen according to current network transmission parameters; According to the encoding sampling parameters, each game screen block of the game screen is encoded in parallel to obtain encoded game screen data corresponding to the game screen.

3. The method according to claim 1, characterized in that The encoded game screen data includes the encoded game screen blocks corresponding to each game screen block; The processing of the encoded game screen data to obtain at least two game screen data streams includes: Determine the position information corresponding to the encoded game screen block in the game screen; The encoded game screen block is processed according to the position information to obtain at least two game screen data streams.

4. The method according to claim 1, characterized in that: The processing of the encoded game screen data to obtain at least two game screen data streams includes: Obtaining a game screen frame identifier corresponding to the encoded game screen data; The encoded game screen data is processed according to the game screen frame identifier to obtain at least two game screen data streams.

5. The method according to claim 4, characterized in that The game screen frame identifier includes a frame number of the game screen; the encoded game screen data is processed according to the game screen frame identifier to obtain at least two game screen data streams, including: When the frame number corresponding to the encoded game screen data is an even number, dividing the encoded game screen data into an even-numbered frame game screen data stream; When the frame number corresponding to the encoded game screen data is an odd number, the encoded game screen data is divided into odd-numbered frame game screen data streams.

6. The method according to claim 1, characterized in that The processing of the encoded game screen data to obtain at least two game screen data streams includes: According to the current network transmission parameters, redundant data is constructed for the encoded game screen data to obtain target game screen data, where the target game screen data includes the encoded game screen data and redundant data corresponding to the encoded game screen data; The target game screen data is processed to obtain at least two game screen data streams.

7. The method according to claim 1, characterized in that The method further comprises: Performing real-time detection on the link transmission parameters of the transmission link corresponding to the game screen data stream; When the detected link transmission parameters do not meet the preset transmission conditions, the game screen data stream is switched to a target transmission link that meets the preset transmission conditions for transmission to the game terminal.

8. The method according to claim 1, characterized in that The method of using different transmission links to transmit the at least two game screen data streams in parallel to the game terminal includes: Performing data transmission acceleration processing on the at least two game screen data streams respectively; Different transmission links are used to transmit at least two accelerated game screen data streams in parallel to the game terminal.

9. The method according to claim 1, characterized in that: The processing of the encoded game screen data to obtain at least two game screen data streams includes: Obtaining location information of the game terminal; Selecting a target edge server from the candidate edge servers according to the location information; The encoded game screen data is processed by the target edge server to obtain at least two game screen data streams.

10. The method according to claim 9, characterized in that Before the step of processing the encoded game screen data through the target edge server to obtain at least two game screen data streams, the step further includes: The encoded game screen data is copied to the kernel network card interface layer by bypassing the kernel processor through a data processing tool; The encoded game screen data is sent to the target edge server through the kernel network card interface layer.

11. A method for receiving data of a cloud game, characterized in that: include: receiving at least two game screen data streams transmitted in parallel by a server through different transmission links, wherein the game screen data streams include encoded game screen blocks corresponding to respective game screen blocks of the game screen, wherein the transmission links are determined by the method for sending data of a cloud game according to claim 1, and the encoded game screen data are processed by the method for sending data of a cloud game according to claim 1; Decoding the encoded game screen block in the game screen data stream to obtain a decoded game screen block; Rendering the decoded game screen block to obtain a target game screen block; The target game screen blocks belonging to the same frame of the game screen are merged to obtain and display the target game screen.

12. The method according to claim 11, characterized in that The receiving server transmits at least two game screen data streams in parallel through different transmission links, including: Receiving at least two game screen data streams transmitted in parallel by the server through different transmission links in a preset transmission order; The decoding process of the encoded game screen block in the game screen data stream to obtain the decoded game screen block includes: When receiving the target encoded game screen block in the game screen data stream, the adjacent encoded game screen block corresponding to the target encoded game screen block is decoded to obtain a decoded game screen block.

13. The method according to claim 11, characterized in that The decoding process of the encoded game screen block in the game screen data stream to obtain the decoded game screen block includes: Decoding the encoded game screen blocks in the game screen data stream according to a preset decoding order to obtain decoded game screen blocks; The rendering of the decoded game screen block to obtain a target game screen block includes: When decoding the target encoded game screen block in the game screen data stream, the decoded game screen block of the adjacent encoded game screen block corresponding to the target encoded game screen block is rendered to obtain the target game screen block.

14. A data transmission device for cloud gaming, characterized in that: include: An acquisition unit, used to acquire game data to be transmitted, wherein the game data includes at least one frame of game screen; A block division unit, configured to divide the game screen into blocks for each frame of the game screen in the game data, and obtain at least one game screen block corresponding to the game screen; An encoding unit, used for encoding each game screen block of the game screen in parallel to obtain encoded game screen data corresponding to the game screen; A data stream division unit, used for processing the encoded game screen data to obtain at least two game screen data streams; A transmission unit, used to use different transmission links to transmit the at least two game screen data streams in parallel to the game terminal respectively; wherein the method for determining the transmission link includes: extracting features of network transmission sub-parameters of each dimension in the current network transmission parameters through a link delay prediction model to obtain transmission sub-feature information corresponding to the network transmission sub-parameters of each dimension; fusing the transmission sub-feature information corresponding to the network transmission sub-parameters of each dimension through the link delay prediction model to obtain the transmission feature information corresponding to the current network transmission parameters; determining the predicted transmission delay information corresponding to the game screen data stream on each candidate transmission link according to the transmission feature information through the link delay prediction model; and determining the transmission link corresponding to the game screen data stream from each candidate transmission link according to the predicted transmission delay information.

15. A data receiving device for a cloud game, characterized in that: include: a receiving unit, configured to receive at least two game screen data streams transmitted in parallel by a server through different transmission links, wherein the game screen data streams include encoded game screen blocks corresponding to respective game screen blocks of the game screen, wherein the transmission links are determined by the method for sending data of a cloud game according to claim 1, and the encoded game screen data are processed according to the method for sending data of a cloud game according to claim 1; A decoding unit, configured to decode the encoded game screen block in the game screen data stream to obtain a decoded game screen block; A rendering unit, used for rendering the decoded game screen block to obtain a target game screen block; The fusion unit is used to fuse the target game screen blocks belonging to the same frame of the game screen to obtain and display the target game screen.

16. An electronic device, characterized in that: It includes a memory and a processor; the memory stores an application, and the processor is used to run the application in the memory to perform the operations in the cloud game data sending method described in any one of claims 1 to 10 or the cloud game data receiving method described in any one of claims 11 to 13.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, which are suitable for loading by a processor to execute the steps in the cloud game data sending method described in any one of claims 1 to 10 or the cloud game data receiving method described in any one of claims 11 to 13.

18. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps in the cloud game data sending method described in any one of claims 1 to 10 or the cloud game data receiving method described in any one of claims 11 to 13 are implemented.

Images