A data processing method, device, server and system

Abstract

The present application discloses a method, device, server and system for data processing, in which the first server receives the control flow information of the terminal, parses the control flow information, starts the game according to the game ID in the control flow information, and intercepts the instruction flow information, wherein the first server is a server for deploying an Android emulator. The instruction flow information is then sent to the second server, which is a server in the GPU resource pool of a graphics processor. The second server selects a target GPU in the GPU resource pool to process the above instruction flow information, completes image rendering and encoding, generates video stream information, and finally, sends the video stream information to the terminal, which is displayed to the user by the terminal. As a result, the requirements for terminal hardware in cloud gaming scenarios are reduced, the processing efficiency of image rendering is improved, the processing time is reduced, and the user experience of the game is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing, and in particular to a method, device, server and system for data processing in a cloud gaming scenario. Background Art

[0002] With the popularity and rapid growth of terminal games, more and more game manufacturers are deploying their businesses in data centers to provide cloud services to end users.

[0003] Cloud gaming is an emerging terminal gaming application scenario. Figure 1 The system architecture diagram of this scenario is shown in Figure 1. The game service is provided by the servers of the business cluster and the advanced reduced instruction set computer machine (ARM) server cluster, wherein the business cluster includes a database server that runs a database application and a business server that records game data information. The ARM server in the ARM server cluster is mainly used to run the Android emulator (Android Virtual Device, AVD) of the cloud game. Generally, the separated rendering instruction stream solution is used in the cloud game scenario to provide users with a smooth gaming experience. This solution creates an AVD in the ARM server of the ARM server cluster, starts the game process in the AVD, and the AVD intercepts the image processing instruction stream and forwards it to the terminal, and the terminal completes the image rendering and display. The essence of this solution is to control the game process and image drawing resource consumption in the cloud service end, that is, to control the game process and image drawing resource consumption in the AVD of the ARM server cluster. However, due to the limitation of the image capability of the ARM server, the image rendering performance under multiple AVD concurrent transmission cannot meet the requirements, so that the image rendering has to be performed in the terminal, which cannot fully meet the business scenario requirements of cloud games. The graphics processing unit (GPU) of the terminal needs to be used to render the image. At this time, the hardware configuration of the terminal has high requirements. When the hardware configuration of the terminal cannot meet the requirements, it will affect the user's cloud gaming experience. On the other hand, if the AVD is directly deployed on the X86 server of the business cluster to render the image, since the data processing process between the X86 system and the Android system requires the translation of the operating system's instruction set, it will greatly consume the server performance, and it is impossible to achieve high-density concurrency of AVD, and the cost is high.

[0004] Therefore, how to realize the game image rendering function on the server side without relying on the terminal GPU in the current cloud gaming technology solution has become an urgent problem to be solved. Summary of the invention

[0005] The present application provides a data processing method, device, server and system, which can solve the problem of truly completing the image rendering function on the server side without relying on the terminal GPU in the current cloud gaming technology solution, thereby improving the user experience.

[0006] In the first aspect, a method for data processing is provided, in which a first server receives control flow information of a terminal, parses the control flow information and the IP address of the terminal, starts a game according to the game ID in the control flow information, and intercepts instruction flow information. Then, the instruction flow information is sent to a second server, which selects a target GPU in a GPU resource pool to process the instruction flow information, completes image rendering and encoding, and generates video stream information. The video stream information is then sent to the terminal, which is displayed to the user by the terminal. Among them, the instruction flow information includes the content of the knowledge game identification ID and the touch input information, and an emulator is running in the first server, which is used to assist the terminal in completing game processing. The first server is a server in an ARM server cluster where AVD is deployed, and the GPU resource pool includes a second server and at least one GPU.

[0007] Through the above method, the image rendering process of the game is completed through the GPU resource pool, which reduces the requirements for terminal hardware in the cloud game scene, improves the processing efficiency of image rendering, reduces the processing time, and thus improves the user experience of the game.

[0008] In a possible implementation, the GPU resource pool may be formed in any of the following ways:

[0009] Method 1: The GPU resource pool includes only one second server, and at least one GPU is deployed in the second server.

[0010] Method 2: The GPU resource pool includes only at least one second server, and each second server is deployed with at least one GPU.

[0011] Method 3: The GPU resource pool includes a second server and other servers, and at least one GPU is deployed in the other servers.

[0012] In another possible implementation, the terminal is installed with an Android operating system, and an Android emulator AVD is pre-installed in the first server. Then, the AVD calls the operating system command according to the game ID to start the game process, so that the game process operates the game according to the touch input information.

[0013] In another possible implementation, the process of the first server intercepting the instruction stream information includes: monitoring the game process ID, and when a new game process ID is detected, obtaining the instruction stream information corresponding to the game process ID.

[0014] Through the description of the above process, in the data processing method provided by the present application, the first server directly sends the intercepted instruction stream information to the second server in the GPU resource pool, and the second server selects the target GPU to complete the processing of the instruction stream information, and generates a video stream and transmits it to the terminal via the TCP / IP protocol, and the terminal completes the display and presents it to the user. Compared with the traditional method, the first server no longer sends the intercepted instruction stream information to the terminal for processing, which reduces the terminal load and the requirements for the terminal hardware configuration. At the same time, since the image rendering is completed by the GPU in the GPU resource pool, the processing power of the GPU is higher than the image processing power of the terminal and the configuration of the GPU can be upgraded at any time, which further improves the image rendering processing efficiency, reduces the processing time, and completely cloudifies the processing process of cloud games, that is, the image rendering and processing are completed by the server, and the terminal is only used to issue touch information and display, which fundamentally solves the problem of the game image rendering function that is truly on the server side without relying on the terminal GPU in the current cloud game technical solution, and improves the user experience.

[0015] In a second aspect, the present application provides a method for data processing, the method comprising: first, a second server receives instruction stream information sent by a first server, wherein the GPU resource pool includes the second server and at least one GPU, the first server is a server running an emulator, and the instruction stream information includes a game identification ID, touch input information, and an IP address of a terminal. Then, the second server selects a target GPU in the GPU resource pool to parse the instruction stream information, and performs image rendering and encoding to generate video stream information. Finally, the video stream information is sent to the terminal according to the IP address of the terminal.

[0016] Compared with the traditional practice of the terminal completing the image rendering and image encoding process to generate a video stream, in this application, the target GPU encodes the rendered image to generate a video stream, and then the CPU of the server where the target GPU is located transmits the video stream information to the terminal. The terminal is only used to display the content of the video stream information, which reduces the terminal's processing process and resource consumption. Moreover, the GPU in the GPU resource pool completes the above-mentioned image encoding and the process of generating video stream information, which can reduce the processing time. The terminal is only used to display the above-mentioned video stream information, which further improves the user experience.

[0017] In a possible implementation, the second server may select a target GPU in any of the following ways:

[0018] Method 1: Based on the load of each GPU in the GPU resource pool, select the GPU with the lightest load as the target GPU.

[0019] Method 2: Select a GPU as the target GPU according to the identification ID of each GPU in the GPU resource pool.

[0020] Method 3: Randomly select a GPU in the GPU resource pool as the target GPU.

[0021] In another possible implementation, the composition of the GPU resource pool may be implemented in any of the following ways:

[0022] Method 1: The GPU resource pool includes a second server, and at least one GPU is deployed on the second server.

[0023] Method 2: The GPU resource pool includes at least one server, each server is deployed with at least one GPU, and the second server is one of the servers.

[0024] Mode 3: The GPU resource pool includes at least one server and a second server, and at least one GPU is deployed in the servers except the second server.

[0025] In another possible implementation, the second server selects a target GPU in the GPU resource pool to parse instruction stream information, including: the target GPU converts the openGLES data of the instruction stream information into openGL data according to a preset open graphics library openGL and an open graphics library openGLES for embedded devices conversion mapping table.

[0026] In this application, the GPU in the GPU resource pool completes the image rendering, which reduces the requirements for the terminal hardware configuration and reduces the processing time. Compared with the ARM server, the present embodiment makes full use of the compatibility advantages of the X86 server and the GPU in the GPU resource pool, renders and encodes the instruction stream information, and transmits it to the terminal via the TCP / IP protocol for image display, thereby improving the user experience. Furthermore, due to the powerful advantages of the compatibility of the X86 server and the GPU, the same server can deploy multiple GPUs to provide multi-channel high-performance concurrent image rendering functions, that is, the GPU resource pool can process multiple instruction stream information at the same time, and complete the encoding and generate video stream information, thereby improving the processing power of the entire cloud game scene. Moreover, different GPUs on the same server can process multiple instruction stream information, which improves the concurrent processing capability and further reduces the hardware cost of the cloud game scene.

[0027] In a third aspect, the present application provides a data processing device, which includes various modules for executing the data processing method in the first aspect or any possible implementation of the first aspect.

[0028] In a fourth aspect, the present application provides a data processing device, which includes various modules for executing the data processing method in the second aspect or any possible implementation of the second aspect.

[0029] In a fifth aspect, the present application provides a first server, comprising a processor, a memory, and a bus, wherein the processor and the memory are connected via a bus and communicate with each other, the memory being used to store computer execution instructions, and when the first server is running, the processor executes the computer execution instructions in the memory to utilize the hardware resources in the first server to perform the operating steps of the method described in the first aspect or any possible implementation of the first aspect.

[0030] In a sixth aspect, the present application provides a second server, comprising a processor, a memory, and a bus, wherein the processor and the memory are connected via a bus and communicate with each other, the memory being used to store computer execution instructions, and when the second server is running, the processor executes the computer execution instructions in the memory to utilize the hardware resources in the second server to perform the operating steps of the method described in the second aspect or any possible implementation of the second aspect.

[0031] In the seventh aspect, the present application provides a data processing system, the system comprising a first server and a second server, the first server being used to execute the operating steps of the method described in the fifth aspect and any possible implementation of the fifth aspect, and the second server being used to execute the operating steps of the method described in the sixth aspect and any possible implementation of the sixth aspect.

[0032] In an eighth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect and any possible implementation of the first aspect.

[0033] In a ninth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enables the computer to execute the method described in the second aspect and any possible implementation of the second aspect.

[0034] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of the system architecture of a cloud gaming scenario in the prior art provided by this application;

[0036] Figure 2 A schematic diagram of the system architecture of a cloud gaming scenario provided for this application;

[0037] Figure 3 A flowchart of a data processing method provided in this application;

[0038] Figure 4 A flowchart of another data processing method provided by the present application;

[0039] Figure 5 A schematic diagram of the structure of a data processing device provided by the present application;

[0040] Figure 6 A schematic diagram of the structure of another data processing device provided by the present application;

[0041] Figure 7 A schematic diagram of the structure of the first server provided for this application;

[0042] Figure 8 A schematic diagram of the structure of the second server provided in this application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0044] In order to better introduce the embodiments of the present invention, first, Figure 1 Briefly introduce the system architecture of the current cloud gaming scenario. As shown in the figure, the system architecture of the cloud gaming scenario includes a business cluster, an advanced RISC machine (ARM) server cluster and a terminal. Among them, the business cluster includes at least one server, and the servers of the business cluster mainly include at least one database server for deploying database applications and at least one business server for recording game data information. Here, the database application mainly includes saving user information and game application configuration information. The game data information includes each user's level and status, as well as the current game level of the user, equipment status, etc. The server in the business cluster is a server deployed with a central processing unit (CPU) of the X86 architecture. The ARM server cluster includes at least one server, and each server is deployed with an ARM architecture CPU for running the game server. Specifically, one or more emulators are running on the ARM server, which is specifically a virtual machine. The user's partial operation of the specified game through the terminal can be completed by the emulator, that is, the emulator can assist the terminal in completing the game operation process. Since the hardware configuration of the ARM server is better than that of the terminal and it can quickly communicate with the server in the business cluster within the local area network, the processing efficiency and user experience can be improved by using the emulator in the ARM server instead of the terminal to complete part of the game processing.

[0045] It should be understood that the main difference between the X86 architecture processor and the ARM architecture processor is that they use different instruction sets. The X86 architecture processor is widely used in servers and personal computers (PCs), while the ARM architecture processor is used in terminals, automated teller machines (ATMs), car computers, etc. Terminals include mobile phones and mobile devices (such as tablets, smart terminals, etc.). In the cloud game scenario, a thin client (TC) of the game is deployed in the terminal, and the user can start the game through the TC in the terminal. In addition, the games installed in the terminal include light-load games, medium-load games, and heavy-load games. Among them, light load, medium load, and heavy load are divided according to the computing resources of the terminal and server consumed when the game is running.

[0046] It should be understood that the above-mentioned light-loaded games, medium-loaded games, and heavy-loaded games are divided according to the computing resources of the terminal consumed by the game operation, which is a relative concept. For example, if the consumable resources of the terminal that can be used for game operation are taken as the standard, games that require 30% of computing resources to run the game can be called light-loaded games, such as Angry Birds; games that require 80% of computing resources to run the game can be called heavy-loaded games, such as Honor of Kings; games that require 50% of computing resources to run the game can be called medium-loaded games, such as Fruit Ninja.

[0047] It should also be understood that Figure 1 In the illustrated architecture, the database server and the business server in the business cluster can also be deployed together according to business needs, that is, the server that deploys the database application is also used to store the application configuration information of the game. In the following embodiments of the present invention, the database server and the business server are deployed separately as an example for explanation. In addition, the division of the server cluster is a logical concept, and the business cluster and the ARM server cluster can also be combined into one cluster.

[0048] At present, the separation rendering instruction stream solution is mainly used to process data in cloud game scenarios. This solution installs an emulator, such as an Android emulator (AVD), in the server of the ARM server cluster. In the specific implementation process, each AVD can be a software module of a virtual machine. When the user starts the game through the terminal, the image processing instruction stream is intercepted in the AVD and forwarded to the terminal, and the graphics processing unit (GPU) of the terminal performs image rendering and display. The essence of this solution is to control the game process and image drawing resource consumption in the ARM server cluster. However, due to the limitation of the image capability of the ARM server, the image rendering performance under multiple AVD concurrent transmission cannot meet the requirements, so that the image rendering has to be performed in the terminal, which cannot fully meet the business scenario requirements of cloud games. It is necessary to use the GPU of the terminal to complete the image rendering. At this time, there are higher requirements for the hardware configuration of the terminal. When the hardware configuration of the terminal (processor, GPU graphics card, memory, etc.) is general or low, it will affect the user's cloud game experience. On the other hand, if the X86 server of the business cluster is directly used to implement AVD rendering of images, since the data processing process between the X86 system and the Android system requires the translation of the operating system's instruction set, it will greatly consume server performance, fail to achieve high-density concurrency of AVD, and be costly. Therefore, in the current cloud gaming technology solution, how to achieve cloud gaming scenarios without relying on the terminal GPU to truly complete the image rendering function on the server side has become an urgent problem to be solved.

[0049] The present application provides a method for data processing. Figure 1 On the basis of the system architecture shown, a multi-GPU resource pool based on X86 servers is added. The server in the GPU resource pool receives the instruction stream information for game operations forwarded by the cloud game server, and distributes it to each GPU for image rendering processing. The game video stream is then generated after GPU encoding, and then the video stream is forwarded to each user's terminal. The terminal displays the video stream information and presents the game interface to the user, thereby reducing the terminal business processing process and improving the terminal user experience.

[0050] Figure 2 This is a system architecture diagram of a cloud gaming scenario provided by this application. As shown in the figure, the system architecture includes a business cluster, an ARM server cluster, a terminal, and a GPU resource pool. Among them, the business cluster, the ARM server cluster, and the terminal are Figure 1 The same as shown, no further description is given here. Figure 2 and Figure 1The difference is that a new GPU resource pool is added. The GPU resource pool consists of at least one server, which includes at least one GPU, and each server is deployed with an X86 architecture CPU. The GPU resource pool is used to receive image instruction stream information forwarded by the business cluster, and distribute it to each GPU for image rendering processing, and then generate a game video stream after GPU encoding, which is then forwarded by the server's processor to each user's terminal. GPU, also known as display core, visual processor, display chip, is a microprocessor dedicated to image calculations on personal computers, workstations, game consoles and mobile devices (such as tablets, smart terminals, etc.). It is the "heart" of the graphics card, similar to the CPU, except that the GPU is designed to perform complex mathematical and geometric calculations, which are necessary for graphics rendering.

[0051] It is worth noting that the way messages are transmitted between the terminal and the server in the cloud gaming scenario is in the form of a "message stream". Here, a message stream refers to a collection of multiple messages, such as the above-mentioned instruction stream information, control stream information, and video stream information.

[0052] To facilitate subsequent description, we take how a server in a GPU resource pool processes data as an example. Figure 3 A flow chart of a data processing method provided in the present application is shown in the figure, and the method includes:

[0053] S110. The second server receives instruction stream information sent by the first server.

[0054] The first server is a server running the AVD corresponding to the game to be operated by the user. When the first server intercepts the instruction stream information of the game, it will directly forward the instruction stream information to the second server so that the second server can further process the instruction stream information. The second server is any server in the GPU resource pool.

[0055] The GPU resource pool can be composed in any of the following ways:

[0056] Method 1: The GPU resource pool may include only one server, in which multiple GPUs are deployed. The multiple GPUs form the GPU resource pool. The server is responsible for receiving the instruction stream information sent by the first server and selecting a GPU for data processing.

[0057] Method 2: The GPU resource pool may include multiple servers, each server includes one or more GPUs, and the GPUs of each server together constitute the GPU resource pool. One of the servers is responsible for receiving the instruction stream information sent by the first server and selecting a GPU in the GPU resource pool for data processing. At this time, since the GPU resource pool needs to process the instruction stream information sent by the AVDs of multiple first servers in parallel, a management server is required in the GPU resource pool to monitor the load of each GPU, allocate each instruction stream information to the appropriate GPU for processing, achieve load balancing in the GPU resource pool, improve processing efficiency, and reduce processing time.

[0058] Method three: The GPU resource pool includes multiple servers. Among them, one server is only used to receive the instruction stream information sent by the first server, and selects a GPU in the GPU resource pool to complete the instruction stream information processing. The other servers are deployed with one or more GPUs, and the multiple GPUs and the server receiving the instruction stream information together constitute the GPU resource pool. The difference between method three and method two is that in method three, there is a server that is only used to receive the instruction stream information sent by the first server and distribute it to the designated GPU for processing. The GPU is not deployed on the server, and there is no need to process the instruction stream information. In the scenario of method three, one server is used alone to process the instruction stream information. When there are multiple instruction stream information that need to be processed at the same time, the efficiency of the distribution of instruction stream information processing tasks is improved and the processing time is reduced.

[0059] Then, for the above three methods, the execution subject of receiving the instruction stream information sent by the first server in step S110 is different. In method one, the execution subject of receiving the instruction stream information sent by the first server is the only server in the GPU resource pool. In method two, the execution subject of receiving the instruction stream information sent by the first server is any server in the GPU resource pool. The server can be a pre-designated server or the server that first receives the instruction stream information. At this time, the server needs to send a network message to notify other servers in the GPU resource pool that the instruction stream information is selected by the target GPU for processing. In method three, the execution subject of receiving the instruction stream information sent by the first server is a server that is only used to receive the instruction stream information sent by the first server.

[0060] In addition, multiple GPUs in the GPU resource pool can process in parallel the image rendering instruction stream information forwarded by the AVDs of multiple first servers. Therefore, a second server is required in the GPU resource pool to manage and regularly obtain the usage rate of each GPU, and allocate appropriate GPUs for processing the instruction stream information of different AVDs to achieve load balancing.

[0061] Specifically, the second server includes a monitoring module, which can be responsible for managing and regularly obtaining the usage rate of each GPU, and allocating appropriate GPUs for processing according to the instruction stream information of different AVDs. In specific implementation, the monitoring module can be a software module, or it can be made into a coprocessor, and the coprocessor is installed in the second server, so that it can accurately and timely manage each GPU in the GPU resource pool.

[0062] It is worth noting that although the servers receiving the instruction stream information in the above three methods are different, the process of the server selecting the GPU after receiving the instruction stream information is the same. The following description of this embodiment takes method 2 as an example for further description, and the server receiving the instruction stream information is called the second server.

[0063] in addition, Figure 2 In the architecture shown, data transmission between servers and between terminals and servers is based on the TCP / IP protocol. Accordingly, the instruction stream information includes game identification (ID), socket connection request, and operation information of the open graphics library for embedded devices (openGLES). Among them, the socket connection request is a connection request message carried in the Transmission Control Protocol / Internet Protocol (TCP / IP), and the socket connection request may include the IP address of the terminal. OpenGLES is an upgraded version of the graphics library based on the open graphics library (openGL). OpenGLES is mainly used in embedded devices, such as ARM servers. OpenGL is mainly used in Linux and Unix operating systems. OpenGL refers to a professional graphics program interface that defines a cross-programming language and cross-platform programming interface specification, which is used for three-dimensional or two-dimensional image processing. It is a powerful and easy-to-call underlying graphics library. Moreover, openGL is a hardware-independent software interface that can be transplanted between different platforms (such as window, Unix, Linux, MacOS, etc.). OpenGLES adds a graphics library that supports embedded devices on the basis of openGL. The operation information of openGLES is to convert the touch information of the terminal user into content that can be recognized by the system, specifically the interface parameters in the openGLES graphics library. The content of the instruction stream information is essentially the openGLES interface called by the game process and the corresponding interface parameters.

[0064] S120: The second server selects a target GPU to parse the instruction stream information, and the target GPU completes image rendering and encoding to generate video stream information.

[0065] Specifically, the second server selects a GPU to parse the instruction stream information. The GPU selected by the second server can be called a target GPU, and the target GPU is any GPU in the GPU resource pool. The second server can select the target GPU in any of the following ways:

[0066] Method 1: The second server selects the GPU with the lightest load as the target GPU according to the load of each GPU in the GPU resource pool.

[0067] The second server can record the load of each GPU in the GPU resource pool, that is, record the current state, usage rate, number of processed instruction stream information, etc. of each GPU. The current state includes idle and busy. The idle state indicates that the GPU is not currently processing instruction stream information, and the busy state indicates that the GPU is currently processing instruction stream information.

[0068] When the second server receives the instruction stream information to be processed, it selects the GPU with the lightest load to process the instruction stream information according to the load condition of the GPU resource pool recorded in the second server.

[0069] Method 2: The second server selects a GPU as a target GPU in the GPU resource pool according to the GPU ID.

[0070] When the second server receives the instruction stream information, it can select one GPU as the target GPU in order of the GPU ID from high to low or from low to high. For example, when the second server receives instruction stream information 1, GPU 1 is selected as the target GPU to process instruction stream information 1. When the second server receives instruction stream information 2, GPU 1 is selected as the target GPU to process instruction stream information 2. Similarly, the second server selects one GPU as the target GPU in order of the GPU ID from high to low or from low to high each time.

[0071] Method 3: The second server randomly selects a GPU in the GPU resource pool as the target GPU.

[0072] The purpose of the target GPU parsing the instruction stream information is to let the game process know the operation content of the specified game, and then render the image of the specified game. The GPU supports the processing of openGLES and openGLES, and stores a mapping table for conversion between openGL and openGLES. It should be understood that the instruction stream information intercepted by the first server is a graphics library instruction of the support terminal based on openGLES. The target GPU needs to convert the instruction stream information into a graphics library instruction of the support server based on openGLES according to the mapping table for conversion between openGL and openGLES, and then it can render the image of the specified game. Its image rendering process includes: vertex shading, rasterization, pixel shading, texture locking and other processes to generate images. The target GPU then encodes the image to generate video stream information. The video stream information includes the encoding result of the image. Among them, the method for encoding the image includes H.264 or H.265 encoding method.

[0073] It is worth noting that the method of GPU encoding images and the process of generating video streams are both existing technologies and are not limited by the present invention.

[0074] S130: The second server sends video stream information to the terminal.

[0075] The second server will send the video stream information to the terminal, and the terminal will complete the display of the video stream, that is, the display of the game results of the user operation. Among them, when the second server sends the video stream information to the terminal, specifically, the CPU of the second server sends the video stream information to the terminal according to the IP of the terminal in the instruction stream information. The sending process can use the TCP / IP protocol to transmit data, and specifically can be carried by the Real-time Transport Protocol (RTP) message. RTP is a network transmission protocol that provides real-time end-to-end transmission services and can transmit standard data packets of audio and video on the Internet. It is widely used in the communication and entertainment fields related to streaming media, including telephone, video conferencing, television and network-based push-to-talk services (similar to walkie-talkie calls).

[0076] As a possible implementation, when the second server sends the video stream information to the terminal, other network transmission technologies such as wireless, infrared, and Bluetooth may be used to transmit the video stream information.

[0077] It is worth noting that when the composition method of the GPU resource pool is method 3 in step S110, the target GPU and the second server are not in the same physical server. At this time, the CPU of the server where the target GPU is located sends the video stream information to the terminal. Optionally, for the GPU resource pool composition method described in method 3 in step S110, the video stream information can also be first sent by the CPU of the server where the target GPU is located to the second server, and then sent by the second server to the terminal.

[0078] Through the description of the above process, in the data processing method provided by this embodiment, the GPU in the GPU resource pool completes the image rendering, and generates a video stream which is transmitted to the terminal via the TCP / IP protocol, and the terminal completes the display and presents it to the user. The entire process does not require the terminal to complete the rendering of the image, and the image rendering process of the game is completed by the target GPU in the GPU resource pool, which reduces the terminal load and the requirements for the terminal hardware configuration. At the same time, since the image rendering is completed by the GPU in the GPU resource pool, the processing power of the GPU is higher than the image processing power of the terminal, which further improves the image rendering processing efficiency, reduces the processing time, and completely cloudifies the processing process of the cloud game, that is, the image rendering and processing are completed by the server, and the terminal is only used to issue touch information and display, which fundamentally solves the problem of the game image rendering function that is truly on the server side without relying on the terminal GPU in the current cloud game technical solution, and improves the user experience. On the other hand, compared with the traditional practice of the terminal completing the image rendering and image encoding process to generate a video stream, in this application, the target GPU encodes the rendered image to generate a video stream, and then the processor of the server where the target GPU is located transmits the video stream information to the terminal. The terminal is only used to display the content of the video stream information, which reduces the terminal's processing process and resource consumption. Moreover, the GPU in the GPU resource pool completes the above-mentioned image encoding and video stream information generation process. Since the GPU can be upgraded according to business needs, the processing time can be reduced. The terminal is only used to display the above-mentioned video stream information, which further improves the user experience.

[0079] As a possible implementation method, Figure 2 In the system architecture diagram of the cloud gaming scenario shown, the GPU resource pool can be configured based on an ARM server in addition to an X86 server. Accordingly, the configuration of the server in the GPU resource pool can also be implemented in the three ways in step S110. The working process of the ARM server and GPU is the same as the above steps S110 to S130, which will not be repeated here.

[0080] Next, combine Figure 4 The data processing method provided in this application is further described. Figure 4 and Figure 3 The difference is that Figure 4The data processing method provided by this application is introduced from the perspective of interactive processing between a terminal and a server. Figure 4 As shown, the method includes:

[0081] S401: The user starts a game on a terminal.

[0082] Specifically, the user starts the game through the game TC deployed on the terminal. At this time, TC will generate information, and the control flow information includes TC's socket connection request, game identification (ID), and touch input information and coordinates after the game is started. Among them, the socket connection request is mainly used to establish a TCP connection between the terminal and the ARM server. Touch input information refers to the operation information entered by the user through the terminal screen, such as clicking, dragging 1cm to the left, etc. The coordinates are used to indicate the location information of the game interface image that the user wants to operate.

[0083] S402: The terminal sends controller flow information to the first server.

[0084] The first server is any server in the ARM server cluster, and the first server is used to process control flow information of the terminal.

[0085] One or more AVDs are pre-installed on the first server. During the specific implementation process, one or more AVDs can be pre-specified for processing a game according to business needs. When a user starts the game through a terminal, an AVD can be selected according to the correspondence between the game ID and the AVD to process the control flow information.

[0086] Optionally, if there are multiple servers in the ARM server cluster, the first server can assume the function of resource scheduling. The specific implementation process includes: the first server can record the hardware configuration and load status of all servers in the ARM server cluster, and pre-install AVD in the servers of the ARM server cluster according to the number of games supported in the cloud gaming scenario. When the control flow information sent by the terminal is received, the control flow can be forwarded to a designated AVD for processing according to the game ID.

[0087] Data is transmitted between the terminal and the first server via TCP / IP. According to the TCP / IP protocol, a TCP / IP connection is first established between the terminal and the first server via a three-way handshake.

[0088] It should be understood that there can be multiple AVDs in the first server at the same time. When a TCP connection is established between the terminal and the first server, the first server also stores the corresponding relationship between the AVD and the terminal. When the first server receives the control flow information sent by the terminal, the first server can forward the control flow information to the designated AVD for processing according to the preset mapping relationship between the AVD and the terminal.

[0089] For example, Table 1 is an example of the mapping relationship between the AVD and the game preset in the first server provided by the present application. When the first server receives the control flow information sent by the terminal 1, the control flow information can be forwarded to AVD1 according to the game ID according to the content of Table 1, and AVD1 completes the control flow processing. Similarly, when the first server receives the control flow information sent by the terminal 2, the control flow information is forwarded to AVD2 for processing. When the first server receives the control flow information sent by the terminal 3, the control flow information is forwarded to AVD3 for processing.

[0090] Table 1 Example of mapping relationship between preset AVD and terminal

[0091] Game 1 Game 2 Game 3 AVD1 yes no no AVD2 no yes no AVD3 no no yes

[0092] S403: The first server parses the control flow information and starts the game.

[0093] Specifically, the operation of parsing the control flow information is completed by the AVD in the first server that has a mapping relationship with the game corresponding to the control flow. The AVD calls the operating system command through the game ID to start the specified game process. When the game is started, the game is operated according to the touch input information in the control flow information.

[0094] S404: The first server intercepts instruction flow information.

[0095] Specifically, an image rendering instruction stream interception process is started in the AVD corresponding to the game ID, and the interception process monitors the instruction stream information. When new instruction stream information is found, the cache (buffer) content of the openGLES instruction stream called in the AVD is directly intercepted, including the interface and interface parameters corresponding to the image information of the user's specified game operation. Among them, after the game is started on the first server, there will be a game process corresponding to the game ID, and the process is the running process of the game. The first server can store game configuration information, and the configuration information indicates the games supported by the first server. When the system receives a command to start a specified game, the preset game process is started. At this time, the first server can determine that there is new instruction stream information, and then intercept the instruction stream information. Instruction stream information refers to the specific operation of the AVD on the above game according to the touch information carried in the control stream information after the game is started, such as clicking, dragging to the left, etc. Intercepting instruction stream information means saving the instruction stream information with a specified identifier to the memory of the first server. At this time, the instruction stream information only exists in the memory in the system. For the AVD of the first server, after the instruction stream information is intercepted, the game process is still in the running state, but no other tasks are executed, the image in the AVD is still, and accordingly, the display interface of the AVD is in a white screen state.

[0096] It is worth noting that the instruction stream information is the information of multiple operation contents generating images. After the AVD parses the control stream information, the touch input information in the control stream will be converted into corresponding openGLES interface parameters by the game process and stored in a continuous storage space in the cache. When the interception process determines that there is a new game process, it can obtain the instruction stream information from the designated cache corresponding to the game process.

[0097] S405: The first server sends instruction flow information to the second server.

[0098] S406: The second server selects a GPU to parse the instruction stream information, and performs image rendering and encoding to generate video stream information.

[0099] S407: The second server sends video stream information to the terminal.

[0100] The processing process from step S405 to step S407 is the same as that from step S110 to step S130, and will not be repeated in this embodiment.

[0101] In this embodiment, the GPU in the GPU resource pool completes the image rendering, which reduces the requirements for the terminal hardware configuration and reduces the processing time. Compared with the ARM server, this implementation fully utilizes the compatibility advantages of the X86 server and the GPU in the GPU resource pool, renders and encodes the instruction stream information, and transmits it to the terminal via the TCP / IP protocol for image display, thereby improving the user experience. Furthermore, due to the powerful advantages of the compatibility of the X86 server and the GPU, the same server can deploy multiple GPUs to provide multi-channel high-performance concurrent image rendering functions, that is, the GPU resource pool can process multiple instruction stream information at the same time, and complete the encoding and generate video stream information, thereby improving the processing power of the entire cloud game scene. Moreover, different GPUs on the same server can process multiple instruction stream information, which improves the concurrent processing capability and further reduces the hardware cost of the cloud game scene.

[0102] As a possible implementation, in order for the second server to quickly send the video stream information to the terminal, the mapping relationship between the AVD and the game ID in the server can also be stored in the second server, and the second server is responsible for maintaining and managing the mapping relationship. Accordingly, step S402 is that the terminal sends the instruction stream information to the second server, and the second server forwards the control stream to the first server according to the mapping relationship between the AVD and the game ID, and then the first server completes the operation process of steps S403 to S405.

[0103] It is worth noting that the embodiments of the present application are described by taking the terminal and AVD that deploy and install the operating system as an example. The data processing method in the present application is also applicable to the terminals and cloud game architectures of other operating systems, such as iOS, Firefox OS, Windows and other terminal operating systems and emulators. It is also possible to provide a hardware resource pool that supports image rendering processing of the corresponding operating system, and the hardware in the resource pool completes the image processing process of the game, thereby reducing the requirements for terminal hardware, improving processing efficiency, reducing processing time, and thus improving user experience.

[0104] It is worth noting that, for the above method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions.

[0105] Combined with the above Figures 2 to 4 , describes in detail the data processing method provided by the embodiment of the present invention, and will be combined with Figures 5 to 8 The invention describes a data processing device, a server and a system provided according to an embodiment of the invention.

[0106] Figure 5The present application provides a structural diagram of a data processing device 500. As shown in the figure, the device 500 includes a receiving unit 501, a processing unit 502, an intercepting unit 503 and a sending unit 504;

[0107] The receiving unit 501 is used to receive control flow information sent by the terminal, wherein the control flow information includes a game identification ID, touch input information and an IP address of the terminal, and the touch input information is used to identify the operation of the user on the game through the terminal;

[0108] The processing unit 502 is used to parse the control flow information and start the game according to the game ID;

[0109] The interception unit 503 is used to intercept instruction stream information, wherein the instruction stream information includes content indicating the game ID and the touch input information;

[0110] The sending unit 504 is used to send the instruction stream information to the second server, so that the second server selects a target GPU in a graphics processor GPU resource pool to process the instruction stream information, completes image rendering and encoding, and generates video stream information to send to the terminal. The GPU resource pool includes the second server and at least one GPU, and the second server is used to manage the GPU resource pool.

[0111] The above-mentioned device 500 can intercept instruction stream information and send the instruction stream information to the second server of the GPU resource pool. The target GPU selected by the second server completes the parsing and encoding of the instruction stream and generates video stream information. It no longer relies on the terminal hardware to complete the image rendering process, thereby reducing the requirements for the terminal hardware.

[0112] Optionally, the GPU resource pool includes only one second server, and at least one GPU is deployed in the second server; or,

[0113] The GPU resource pool includes at least one second server, and each of the at least one second server is deployed with at least one GPU; or,

[0114] The GPU resource pool includes a second server and other servers, and at least one GPU is deployed in the other servers.

[0115] Optionally, the terminal is installed with an Android operating system, and an Android emulator AVD is pre-installed in the device, then the processing unit 502 parses the control flow information and starts the game according to the game ID, including:

[0116] The AVD calls an operating system command according to the game ID to start a game process, so that the game process operates the game according to the touch input information.

[0117] Optionally, the interception unit 503 intercepts the instruction stream information including: monitoring the game process ID, and when a new game process ID is detected, acquiring the instruction stream information.

[0118] It should be understood that the device 500 of the embodiment of the present invention can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. It can also be implemented by software. Figure 3 When the data processing method shown in , the device and its various modules can also be software modules.

[0119] The apparatus 500 according to the embodiment of the present invention may correspond to executing the method described in the embodiment of the present invention, and the above and other operations and / or functions of each unit in the apparatus 500 are respectively to implement Figure 3 to Figure 4 For the sake of brevity, the corresponding processes executed by the first server in each method are not repeated here.

[0120] Through the description of the above device 500, the GPU in the GPU resource pool completes the image rendering, and generates a video stream that is transmitted to the terminal via the TCP / IP protocol, and the terminal completes the display and presents it to the user. The entire process does not require the terminal to complete the rendering of the image, and the image rendering processing process of the game is completed by the target GPU in the GPU resource pool, which reduces the terminal load and the requirements for the terminal hardware configuration. At the same time, since the image rendering is completed by the GPU in the GPU resource pool, the processing power of the GPU is higher than the image processing power of the terminal, which further improves the image rendering processing efficiency, reduces the processing time, and completely cloudifies the processing process of the cloud game, that is, the image rendering and processing are completed by the server, and the terminal is only used to issue touch information and display, which fundamentally solves the problem of truly completing the image rendering function on the server without relying on the terminal GPU in the current cloud game technical solution, and improves the user experience. On the other hand, compared with the traditional practice of the terminal completing the image rendering and image encoding to generate a video stream, in this application, the target GPU encodes the rendered image, generates a video stream, and directly transmits the video stream information to the terminal. The terminal is only used to display the content of the video stream information, which reduces the processing process and resource consumption of the terminal. Moreover, the process of completing the above-mentioned image encoding and generating video stream information by the GPU in the GPU resource pool can reduce the processing time, and the terminal is only used to display the above-mentioned video stream information, further improving the user experience.

[0121] Figure 6 The structural diagram of another data processing device 600 provided for the application. As shown in the figure, the device 600 includes a receiving unit 601, a processing unit 602 and a sending unit 603.

[0122] The receiving unit 601 is used to receive instruction stream information sent by the first server, the device is any server in the GPU resource pool, the GPU resource pool includes at least one GPU, the first server is a server running an emulator, the emulator is used to assist the terminal to complete the game processing, and the instruction stream information includes a game identification ID, touch input information and an IP address of the terminal;

[0123] The processing unit 602 is used to select a target GPU in the GPU resource pool to parse the instruction stream information, perform image rendering and encoding, and generate video stream information;

[0124] The sending unit 603 is configured to send the video stream information to the terminal according to the IP address of the terminal.

[0125] Optionally, the processing unit 602 selects a target GPU in the GPU resource pool to parse the instruction stream information, including:

[0126] According to the load conditions of each GPU in the GPU resource pool, select the GPU with the lightest load as the target GPU; or,

[0127] Selecting a GPU as a target GPU according to the identification IDs of the GPUs in the GPU resource pool; or,

[0128] A GPU is randomly selected from the GPU resource pool as a target GPU.

[0129] Optionally, the GPU resource pool includes one of the devices, and at least one GPU is deployed on the device; or,

[0130] The GPU resource pool includes at least one server, and each of the at least one server is deployed with at least one GPU; the device is any one of the at least one server; or,

[0131] The GPU resource pool includes the device and at least one third server, and each of the at least one third server is deployed with at least one GPU.

[0132] Optionally, the processing unit 602 selects a target GPU in the GPU resource pool to parse the instruction stream information, including:

[0133] The target GPU converts the openGLES data of the instruction stream information into openGL data according to a preset open graphics library openGL and an open graphics library openGLES for embedded devices conversion mapping table.

[0134] It should be understood that the device 600 of the embodiment of the present invention can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. It can also be implemented by software. Figure 3 to Figure 4 When the data processing method shown in , the device and its various modules can also be software modules.

[0135] The apparatus 600 according to the embodiment of the present invention may correspond to executing the method described in the embodiment of the present invention, and the above and other operations and / or functions of each unit in the apparatus 600 are respectively to implement Figure 3 to Figure 4 For the sake of brevity, the corresponding processes executed by the second server in each method are not repeated here.

[0136] Through the description of the above device 600, the GPU in the GPU resource pool completes the image rendering, and generates a video stream that is transmitted to the terminal via the TCP / IP protocol, and the terminal completes the display and presents it to the user. The entire process does not require the terminal to complete the rendering of the image, and the image rendering processing process of the game is completed by the target GPU in the GPU resource pool, which reduces the terminal load and the requirements for the terminal hardware configuration. At the same time, since the image rendering is completed by the GPU in the GPU resource pool, the processing power of the GPU is higher than the image processing power of the terminal, which further improves the image rendering processing efficiency, reduces the processing time, and completely cloudifies the processing process of the cloud game, that is, the image rendering and processing are completed by the server, and the terminal is only used to issue touch information and display, which fundamentally solves the problem of truly completing the image rendering function on the server without relying on the terminal GPU in the current cloud game technical solution, and improves the user experience. On the other hand, compared with the traditional practice of the terminal completing the image rendering and image encoding to generate a video stream, in this application, the target GPU encodes the rendered image, generates a video stream, and directly transmits the video stream information to the terminal. The terminal is only used to display the content of the video stream information, which reduces the processing process and resource consumption of the terminal. Moreover, the process of completing the above-mentioned image encoding and generating video stream information by the GPU in the GPU resource pool can reduce the processing time, and the terminal is only used to display the above-mentioned video stream information, further improving the user experience.

[0137] Figure 7 A schematic diagram of a first server 700 provided in an embodiment of the present invention, as shown in the figure, the first server 700 includes a processor 701, a memory 702, a communication interface 703 and a bus 704, and a graphics processor 705. The processor 701, the memory 702, the communication interface 703, and the storage device 705 communicate through the bus 704, and can also communicate through other means such as wireless transmission. The memory 702 is used to store instructions, and the processor 701 is used to execute the instructions stored in the memory 702. The memory 702 stores program code, and the processor 701 can call the program code stored in the memory 702 to perform the following operations:

[0138] Receive control flow information sent by the terminal, the control flow information including a game identification ID, touch input information and an IP address of the terminal, the touch input information being used to identify a user's operation on the game through the terminal;

[0139] Parsing the control flow information, and starting the game according to the game ID;

[0140] intercepting instruction stream information, wherein the instruction stream information includes content indicating the game ID and the touch input information;

[0141] The instruction stream information is sent to the second server so that the second server selects a target GPU in a graphics processor GPU resource pool to process the instruction stream information, completes image rendering and encoding, and generates video stream information to send to the terminal, wherein the GPU resource pool includes the second server and at least one GPU.

[0142] Optionally, the GPU resource pool includes only one second server, and at least one GPU is deployed in the second server; or,

[0143] The GPU resource pool includes only at least one second server, and each of the at least one second server is deployed with at least one GPU; or,

[0144] The GPU resource pool includes a second server and at least one other server, and each of the at least one other server is deployed with at least one GPU.

[0145] Optionally, the terminal is installed with an Android operating system, an Android emulator AVD is pre-installed in the first server 700, and the first server 700 parses the control flow information and starts the game according to the game ID, including:

[0146] The AVD calls an operating system command according to the game ID to start a game process, so that the game process operates the game according to the touch input information.

[0147] Optionally, the first server 700 intercepts the instruction flow information including:

[0148] The first server monitors the game process ID, and when a new game process ID is detected, obtains the instruction stream information.

[0149] It should be understood that in the embodiment of the present invention, the processor 701 may be a CPU, and the processor 701 may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor 701 may be a processor supporting an ARM architecture.

[0150] The memory 702 may include a read-only memory and a random access memory, and provide instructions and data to the processor 701. The memory 702 may also include a non-volatile random access memory. For example, the memory 702 may also store information about the device type.

[0151] The memory 702 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0152] The bus 704 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as bus 704 in the figure.

[0153] It should be understood that the first server 700 according to the embodiment of the present invention may correspond to Figure 5 The data processing device 500 shown in FIG. 1 may correspond to executing the method according to the embodiment of the present invention. Figure 3 to Figure 4 The first server in the method shown is an operation step of the execution subject, and the above and other operations and / or functions of each module in the first server are respectively to implement Figure 3 to Figure 4 For the sake of brevity, the corresponding processes of each method in are not repeated here.

[0154] Through the description of the above process, the GPU in the GPU resource pool completes the image rendering, and generates a video stream that is transmitted to the terminal via the TCP / IP protocol, and the terminal completes the display and presents it to the user. The entire process does not require the terminal to complete the rendering of the image. The image rendering process of the game is completed by the target GPU in the GPU resource pool, which reduces the terminal load and the requirements for the terminal hardware configuration. At the same time, since the image rendering is completed by the GPU in the GPU resource pool, the processing power of the GPU is higher than the image processing power of the terminal, which further improves the image rendering processing efficiency and reduces the processing time. The cloud game processing process is completely clouded, that is, the image rendering and processing are completed by the server, and the terminal is only used to issue touch information and display, which fundamentally solves the problem of truly completing the image rendering function on the server without relying on the terminal GPU in the current cloud game technical solution, and improves the user experience. On the other hand, compared with the traditional practice of the terminal completing the image rendering and image encoding to generate a video stream, the target GPU in this application encodes the rendered image, generates a video stream, and directly transmits the video stream information to the terminal. The terminal is only used to display the content of the video stream information, which reduces the terminal's processing process and resource consumption. Moreover, the process of completing the above-mentioned image encoding and generating video stream information by the GPU in the GPU resource pool can reduce the processing time, and the terminal is only used to display the above-mentioned video stream information, further improving the user experience.

[0155] Figure 8 A schematic diagram of a second server 800 provided in an embodiment of the present invention, as shown in the figure, the second server 800 includes a processor 801, a memory 802, a communication interface 803 and a bus 804, and a graphics processor 805. The processor 801, the memory 802, the communication interface 803, and the graphics processor 805 communicate through the bus 804, and can also communicate through other means such as wireless transmission. The memory 802 is used to store instructions, and the processor 801 and the graphics processor 805 are used to execute the instructions stored in the memory 802. The memory 802 stores program code, and the processor 801 and the graphics processor 805 can call the program code stored in the memory 802 to perform the following operations:

[0156] Receiving instruction stream information sent by a first server, where the second server is any server in a GPU resource pool, the GPU resource pool includes at least one GPU, the first server is a server running an emulator, and the instruction stream information includes a game identification ID, touch input information, and an IP address of a terminal;

[0157] Selecting a target GPU in the GPU resource pool to parse the instruction stream information, perform image rendering and encoding, and generate video stream information;

[0158] The video stream information is sent to the terminal according to the IP address of the terminal.

[0159] Optionally, the second server selecting a target GPU in the GPU resource pool to parse the instruction stream information includes:

[0160] The second server selects the GPU with the lightest load as the target GPU according to the load conditions of each GPU in the GPU resource pool; or,

[0161] The second server selects a GPU as the target GPU according to the identification IDs of the GPUs in the GPU resource pool; or,

[0162] The second server randomly selects a GPU in the GPU resource pool as the target GPU.

[0163] Optionally, the GPU resource pool includes a second server 800, and at least one GPU is deployed on the second server 800; or,

[0164] The GPU resource pool includes at least one server, and each of the at least one server is deployed with at least one GPU; the second server 800 is any one of the at least one server; or,

[0165] The GPU resource pool includes a second server 800 and at least one third server, and each of the at least one third server is deployed with at least one GPU.

[0166] Optionally, the second server 800 selects a target GPU in the GPU resource pool to parse the instruction stream information, including: the target GPU converts openGLES data of the instruction stream information into openGL data according to a preset open graphics library openGL and an open graphics library openGLES for embedded devices conversion mapping table.

[0167] It should be understood that in the embodiment of the present invention, the processor 801 may be a CPU, and the processor 801 may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0168] The memory 802 may include a read-only memory and a random access memory, and provide instructions and data to the processor 801. The memory 802 may also include a non-volatile random access memory. For example, the memory 802 may also store information about the device type.

[0169] The memory 802 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0170] The bus 804 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as bus 804 in the figure.

[0171] The GPU 805 is a microprocessor that is specially used for image computing in personal computers, workstations, game consoles, and mobile devices (such as tablet computers, smart terminals, etc.).

[0172] It should be understood that the second server 800 according to the embodiment of the present invention may correspond to Figure 6 The data processing device 600 shown in FIG. 6 may correspond to executing the method according to the embodiment of the present invention. Figure 3 to Figure 4 The second server in the method shown is the operating step of the execution subject, and the above and other operations and / or functions of each module in the second server are respectively to realize Figure 3 to Figure 4For the sake of brevity, the corresponding processes of each method in are not repeated here.

[0173] Through the description of the above process, the GPU in the GPU resource pool completes the image rendering, and generates a video stream that is transmitted to the terminal via the TCP / IP protocol, and the terminal completes the display and presents it to the user. The entire process does not require the terminal to complete the rendering of the image. The image rendering process of the game is completed by the target GPU in the GPU resource pool, which reduces the terminal load and the requirements for the terminal hardware configuration. At the same time, since the image rendering is completed by the GPU in the GPU resource pool, the processing power of the GPU is higher than the image processing power of the terminal, which further improves the image rendering processing efficiency and reduces the processing time. The cloud game processing process is completely clouded, that is, the image rendering and processing are completed by the server, and the terminal is only used to issue touch information and display, which fundamentally solves the problem of truly completing the image rendering function on the server without relying on the terminal GPU in the current cloud game technical solution, and improves the user experience. On the other hand, compared with the traditional practice of the terminal completing the image rendering and image encoding to generate a video stream, the target GPU in this application encodes the rendered image, generates a video stream, and directly transmits the video stream information to the terminal. The terminal is only used to display the content of the video stream information, which reduces the terminal's processing process and resource consumption. Moreover, the process of completing the above-mentioned image encoding and generating video stream information by the GPU in the GPU resource pool can reduce the processing time, and the terminal is only used to display the above-mentioned video stream information, further improving the user experience.

[0174] The embodiment of the present invention also provides a Figure 3 The system architecture diagram of the cloud game is shown in FIG. The system includes a first server and a second server, wherein the first server is used to execute Figure 3 or Figure 4 The first server is the execution subject of the operation steps, and the second server is used to execute Figure 3 or Figure 4 The operation steps in which the second server is the execution entity will not be repeated here.

[0175] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0176] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application.

[0177] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0178] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] The above is only a specific embodiment of the present invention. Those skilled in the art may conceive of changes or substitutions based on the specific embodiments provided by the present invention, which should all be included in the protection scope of the present invention.

Claims

1. A method for data processing, characterized in that: The method comprises: The first server intercepts the instruction stream information of the game, the instruction stream information is used to indicate information of generating an image by executing operation content of the game, the game is run in a terminal, and the terminal communicates with the first server through a network; The first server determines a second server matching the instruction stream information in a graphics processor GPU resource pool; The first server sends the instruction stream information to the second server to instruct the GPU in the second server to complete the image rendering operation of the game; The first server runs one or more Android container emulators AVD, and the first server intercepts the instruction stream information of the game, including: The first server intercepts instruction flow information of a calling instruction flow in an AVD associated with the game.

2. The method according to claim 1, characterized in that: Before the first server intercepts the instruction stream information of the game, the method further includes: The first server receives control flow information sent by the terminal, wherein the control flow information includes the game identification ID and touch input information, and the touch input information is used to identify the operation content of the user on the game through the terminal.

3. The method according to claim 2, characterized in that The first server intercepts the game instruction stream information, including: The first server monitors the game process identification ID, and when a new game process ID is detected, obtains the instruction stream information.

4. The method according to any one of claims 1 to 3, characterized in that: When the GPU resource pool only includes multiple GPUs of the second server, the first server determines the second server matching the instruction stream information in the graphics processor GPU resource pool, including: The first server selects any one GPU to execute processing of the instruction stream information.

5. The method according to any one of claims 1 to 3, characterized in that: When the GPU resource pool includes multiple servers, and each of the multiple servers includes one or more GPUs, the first server determines a second server matching the instruction stream information in the graphics processor GPU resource pool, including: The management server receives the instruction stream information, the management server is any server in the GPU resource pool, and the management server is configured with one or more GPUs; The management server selects a GPU of a second server to execute the processing of the instruction stream information according to the load of the GPU in the GPU resource pool, where the second server is a server where the GPU in the GPU resource pool is located; The management server sends a notification to the first server, where the notification is used to indicate that the second server is the server that executes the instruction flow information processing.

6. The method according to any one of claims 1 to 3, characterized in that: When the GPU resource pool includes a management server and multiple servers, and each of the multiple servers includes one or more GPUs, the first server determines a second server matching the instruction stream information in the graphics processor GPU resource pool, including: The management server receives the instruction stream information; The management server selects a GPU of a second server to execute the processing of the instruction stream information according to the load of the GPU in the GPU resource pool, where the second server is a server where the GPU in the GPU resource pool is located; The management server sends a notification to the first server, where the notification is used to indicate that the second server is the server that executes the instruction flow information processing.

7. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The second server selects a target GPU from among the GPUs of the second server to parse the instruction stream information.

8. The method according to claim 7, characterized in that The method further comprises: The target GPU of the second server converts the openGLES data of the instruction stream information into openGL data according to a preset open graphics library openGL and an open graphics library openGLES for embedded devices conversion mapping table.

9. A data processing device, characterized in that: The device comprises an interception unit, a processing unit and a sending unit. The interception unit is used to intercept the instruction stream information of the game, the instruction stream information is used to indicate the information of generating an image by executing the operation content of the game, the game is run on the terminal, and the terminal communicates with the first server through the network; The processing unit is used to determine a second server matching the instruction stream information in a graphics processing unit (GPU) resource pool; The sending unit is used to send the instruction stream information to the second server to instruct the GPU in the second server to complete the image rendering operation of the game; The device runs one or more Android container emulators AVD; the interception unit is specifically used to intercept instruction flow information of the call instruction flow in the AVD associated with the game.

10. The device according to claim 9, characterized in that The device also includes a receiving unit, which is used to receive control flow information sent by the terminal, wherein the control flow information includes the game identification ID and touch input information, and the touch input information is used to identify the operation content of the user on the game through the terminal.

11. The device according to claim 10, characterized in that The interception unit is further used to monitor the game process identification ID, and when a new game process ID is detected, obtain the instruction stream information.

12. The device according to any one of claims 9 to 10, characterized in that The processing unit is further configured to select any one GPU to execute processing of the instruction stream information when the GPU resource pool only includes multiple GPUs of the second server.

13. A server, characterized in that: The server includes a processor and a memory, wherein the memory stores computer-executable instructions, and the processor executes the computer-executable instructions to respectively implement the functions of the operating steps of the first server, the second server or the management server execution method as described in any one of claims 1 to 8.

14. A cloud gaming system, characterized in that: The cloud gaming system includes a first server, a second server or a management server, the first server or the second server respectively including a processor and a memory, the memory storing computer execution instructions, the processor executing the computer execution instructions stored in the memory to implement the functions of the operating steps of the execution method of the first server, the second server or the management server as claimed in any one of claims 1 to 8.

Images