A display processing method and device for cloud games

Abstract

The embodiment of the present application provides a display processing method and device for cloud games, which are applied in the field of cloud computing technology, including performing vertex processing on the first marked object and the second marked object at the frame time respectively to obtain the positional relationship between the first marked object and the second marked object; according to the positional relationship, performing rendering pipeline processing on the first marked object to obtain the first layer, and sending the frame time information to the server; receiving the return picture sent by the server, and decoding the return picture to obtain the second layer; the return picture is obtained by the server according to the positional relationship obtained by the frame time information, and performing rendering pipeline processing on the second marked object; according to the first layer and the second layer, the frame time image is obtained. By rendering the interactive content and strong real-time content that users are more concerned about on the client, the screen delay of the content that users are more concerned about is effectively reduced, and the user's gaming experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing, and in particular to a display processing method and device for cloud games. Background Art

[0002] Cloud gaming is a gaming method based on cloud computing. In the cloud gaming mode, all games are run on the server side, and the rendered game screen is compressed and encoded and transmitted to the user client through the network. In the cloud gaming mode, the user client needs to send the operation instructions to the cloud server, and the cloud server transmits the corresponding game screen to the client according to the operation instructions.

[0003] However, the rendering of cloud games is all done on the server side, which will cause excessive rendering pressure on the server. In addition, the game logic operation time, audio and video rendering time, compression encoding time, and network transmission time on the server will cause delays in the game screen of the user client, reducing the game user experience. Summary of the invention

[0004] The embodiments of the present application provide a display processing method and device for cloud games, which are used to solve the delay problem of client game screens and improve the gaming user experience.

[0005] In a first aspect, an embodiment of the present application provides a display processing method for a cloud game, which is applied to a client and includes:

[0006] Performing vertex processing on the first marked object and the second marked object at the frame time respectively to obtain the positional relationship between the first marked object and the second marked object; the first marked object and the second marked object are objects to be displayed in the frame time image corresponding to the frame time;

[0007] According to the positional relationship, the first marked object is subjected to rendering pipeline processing to obtain a first layer, and frame time information is sent to a server;

[0008] receiving a returned picture sent by the server, and decoding the returned picture to obtain a second layer; the returned picture is obtained by the server performing rendering pipeline processing on the second marked object according to the position relationship obtained by the server according to the frame time information;

[0009] The frame moment image is obtained according to the first layer and the second layer.

[0010] By dividing the content of the game screen into two categories according to user needs, the interactive content and strong real-time content that users are more concerned about are rendered on the client, which greatly reduces the rendering pressure on the server, balances the load on the server, reduces the consumption on the server, and can support more clients at the same time. At the same time, it avoids the problem of screen delay of the content that users are concerned about due to slow server transmission speed and network delay, thereby improving the user's gaming experience.

[0011] Optionally, the first marking object is interactive content determined based on a game operation of a user on the client; and the second marking object is background content other than the interactive content determined by the game operation of the client.

[0012] The interactive content and real-time content that users are more concerned about are rendered on the client, which reduces the delay of these contents and improves the user's gaming experience.

[0013] Optionally, the first marking object and the second marking object are obtained by marking objects defined in the cloud game through a software development kit.

[0014] Objects defined in cloud games are marked through the software development kit to facilitate identification by the client and server.

[0015] Optionally, the frame time information includes camera information of the client at the frame time, operation event information corresponding to a game operation of a user on the client, and game event information obtained based on the operation event information.

[0016] Optionally, performing rendering pipeline processing on the first marked object according to the positional relationship to obtain a first layer includes:

[0017] Determining an unobstructed portion of the first marked object according to the positional relationship;

[0018] The unobstructed portion of the first marked object is subjected to rendering pipeline processing to obtain a first layer.

[0019] Through the positional relationship, the client clearly identifies the part that needs to be rendered, and performs a complete rendering pipeline process on the unobstructed part, while not performing a complete rendering pipeline process on the obstructed part.

[0020] Optionally, the client obtains the frame time image according to the first layer and the second layer, including:

[0021] Determining a superposition mode between the first layer and the second layer according to a preset superposition mode between each first marking object and each second marking object;

[0022] The frame moment image is obtained after superposition processing is performed according to the superposition mode between the first layer and the second layer.

[0023] By presetting the overlay mode between each first marking object and each second marking object, the client can perform overlay processing according to the overlay mode of each marking object when overlaying layers, which can improve the game screen quality of the client and enhance the user's visual experience.

[0024] In a second aspect, an embodiment of the present application provides a display processing method for a cloud game, which is applied to a server, including:

[0025] Receive frame time information under the frame time sent by the client;

[0026] According to the frame time information, vertex processing is performed on the first marked object and the second marked object at the frame time respectively to obtain the positional relationship between the first marked object and the second marked object; the first marked object and the second marked object are objects to be displayed in the frame time image corresponding to the frame time;

[0027] According to the positional relationship, the second marked object is subjected to rendering pipeline processing to obtain a returned image, and the returned image is sent to the client; the returned image is used by the client to generate the frame moment image.

[0028] By dividing the content of the game screen into two categories according to user needs, the interactive content and strong real-time content that users are more concerned about are rendered on the client, which greatly reduces the rendering pressure on the server, balances the load on the server, reduces the consumption on the server, and can support more clients at the same time.

[0029] Optionally, the frame time information includes the camera information of the client at the frame time, operation event information corresponding to the game operation of the user on the client, and game event information obtained based on the operation event information;

[0030] According to the frame time information, vertex processing is performed on the first marked object and the second marked object at the frame time respectively, including:

[0031] Determine the camera information of the server at the frame time according to the camera information of the client at the frame time;

[0032] Determine a first marked object and a second marked object at the frame time according to the operation event information and / or the game event information;

[0033] According to the camera information of the server at the frame time, vertex processing is performed on the first marked object and the second marked object respectively.

[0034] The camera information of the server and the client is kept consistent, and the server can obtain a more accurate positional relationship between the first marked object and the second marked object according to the frame time information.

[0035] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, so that the processor executes the display processing method of the cloud game described in the first aspect above.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the program runs on the computer device, the computer device executes the display processing method of the cloud game described in the first aspect above.

[0037] Rendering interactive content and highly real-time content that users are more concerned about on the client reduces the rendering pressure on the server, balances the load on the server, reduces the consumption on the server, and can support more clients at the same time. At the same time, it avoids the problem of screen delay of the content that users are concerned about due to slow server transmission speed and network delay, thereby improving the user's gaming experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 A schematic diagram of a cloud gaming system architecture provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a flow chart of a display processing method for a cloud game provided by an embodiment of the present invention;

[0041] Figure 3A A schematic diagram of a game viewing angle provided by an embodiment of the present invention;

[0042] Figure 3B A schematic diagram of a game viewing angle provided by an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a flow chart of a display processing method for a cloud game provided by an embodiment of the present invention;

[0044] Figure 5A A schematic diagram of a game viewing angle provided by an embodiment of the present invention;

[0045] Figure 5B A schematic diagram of a game viewing angle provided by an embodiment of the present invention;

[0046] Figure 6 A schematic diagram of a cloud gaming device provided by an embodiment of the present invention;

[0047] Figure 7 A schematic diagram of the structure of a computing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] refer to Figure 1 , which is a cloud gaming system architecture diagram applicable to the embodiment of the present application, and the system architecture may include a client 101 and a server 102. The client 101 and the server 102 are connected via a wired network or a wireless network. The client 101 may be a computer, a mobile phone, a tablet, etc., and the server 102 may be a high-performance ARM cluster server or a processor of a complex instruction set computer (CISC) architecture.

[0050] Based on the above description, Figure 2 The flow of a display processing method of a cloud game is exemplarily shown. The flow of the method is executed by the client 101 and includes the following steps:

[0051] Step S201 , performing vertex processing on the first marked object and the second marked object at the frame time respectively to obtain the positional relationship between the first marked object and the second marked object; the first marked object and the second marked object are objects to be displayed in the frame time image corresponding to the frame time.

[0052] Specifically, the first marking object can be game content with small rendering workload, strong correlation with user game operations, and users with high timeliness requirements; the second marking object can be game content with large rendering workload, weak correlation with user game operations, and users do not need high real-time requirements.

[0053] In some embodiments, the first marking object and the second marking object are obtained by marking objects defined in the cloud game through a software development kit. The first marking object is the interactive content determined based on the game operation of the user on the client; the second marking object is the background content other than the interactive content determined by the game operation of the client.

[0054] Specifically, in a cloud game, there are multiple first objects and multiple second objects, where the game developer defines the first object as the interactive content determined by the user in the game operation, or the game content that the user is more concerned about, including the operated character, the enemy close to the character, and the user interface (User Interface, referred to as UI) related to the character, etc., so the first object has strong real-time performance; the developer defines the second object as the background content other than the interactive content determined by the user in the game operation, including the buildings, ground, vegetation, trees, etc. in the game scene. The developer uses the Software Development Kit (SDK) to mark the first object and the second object, and obtains the first marked object and the second marked object respectively.

[0055] For example, in a multiplayer competitive game, the first marking object can be the character operated by the player, close enemies, and user interfaces related to the character, including the following: map UI, energy bar UI, team UI, skill UI, etc.; the second marking object is the house, mountain, land, etc. in the game screen.

[0056] Taking a first marked object and a second marked object in a frame as an example, the client will use a simple model to perform vertex processing on the first marked object and the second marked object to obtain the positional relationship between the first marked object and the second marked object, that is, to correctly handle the impact of the second marked object on the first marked object.

[0057] Possible positional relationships between the first marking object and the second marking object include: the second marking object completely covers the first marking object, the second marking object partially covers the first marking object, and the second marking object collides with the first marking object.

[0058] For example, if the first marking object is a character operated by the user, and the second marking object is a pillar, at the nth frame, after the character runs to the pillar, the pillar completely blocks the character, that is, the second marking object completely blocks the first marking object;

[0059] If the first marked object is a character operated by the user, and the second marked object is a stone platform, at the n+1th frame, after the character runs to the stone platform, the stone platform only blocks the character's legs, and the character's upper body and head are not blocked by the stone platform, that is, the second marked object partially blocks the first marked object;

[0060] If the first marked object is a machete held by a character operated by the user, and the second marked object is a wooden barrel, at the n+2th frame, the character chops down the barrel, the machete sinks into the barrel, and the blade of the machete collides with the barrel, that is, a collision event occurs between the second marked object and the first marked object.

[0061] The client uses a simple model to process vertices of the first and second marked objects, and can correctly handle the occlusion and collision events of the first and second marked objects in the game, and prepare for the subsequent complete rendering pipeline steps. The simple model can be a line drawing, a point line drawing, etc.

[0062] Step S202: Perform rendering pipeline processing on the first marked object according to the position relationship to obtain a first layer, and send frame time information to the server.

[0063] Specifically, the rendering pipeline processing is a process of converting a 3D scene into a 2D image on the screen. The complete rendering pipeline process is: vertex data input, vertex shader, tessellation process, geometry shader, primitive assembly, clipping and culling, rasterization, fragment shader, and mixed test. Therefore, based on the result of the vertex processing in step S201, the client performs rendering pipeline processing after vertex data input on all first marked objects, including vertex shader, tessellation process, geometry shader, primitive assembly, clipping and culling, rasterization, fragment shader, and mixed test, and finally obtains the first layer.

[0064] In the above step S202, it also includes: determining the unobstructed part of the first marked object according to the position relationship; and performing rendering pipeline processing on the unobstructed part of the first marked object to obtain a first layer.

[0065] For example, if the first marking object is a character operated by the user, and the second marking object is a pillar, if at the nth frame, the character runs to the pillar and the pillar completely blocks the character, that is, the second marking object completely blocks the first marking object, then the client does not need to perform rendering pipeline processing on the character after vertex data input;

[0066] If the first marked object is a character operated by the user, and the second marked object is a stone platform, if at the n+1th frame, the character runs to the stone platform, the stone platform only blocks the character's legs, and the character's upper body and head are not blocked by the stone platform, that is, the second marked object partially blocks the first marked object; at this time, the client needs to perform rendering pipeline processing after vertex data input for the part of the character that is not blocked by the stone platform;

[0067] If the first marked object is a machete held by a character operated by the user, and the second marked object is a wooden barrel, at the n+2th frame, the character chops down the barrel, the machete sinks into the barrel, the blade of the machete collides with the barrel, and the machete part of the first marked object is blocked by the second marked object, the barrel. At this time, the client performs rendering pipeline processing on the unblocked part of the machete after inputting vertex data.

[0068] After the client performs complete rendering pipeline processing on the first marked object, the frame time information is sent to the server.

[0069] By dividing the content of the game screen into two categories according to user needs, the client is responsible for rendering a part of the content in the cloud game screen, which greatly reduces the rendering pressure on the server; the interactive content and strong real-time content that users are more concerned about are rendered on the client, which effectively reduces the screen delay of the content that users are more concerned about and improves the user's gaming experience.

[0070] Step S203, receiving the returned picture sent by the server, and decoding the returned picture to obtain the second layer; the returned picture is obtained by the server performing rendering pipeline processing on the second marked object according to the position relationship obtained by the frame time information.

[0071] Specifically, the server receives the frame time information sent by the client, and uses a simple model to perform vertex processing on the first marked object and the second marked object respectively to obtain the positional relationship between the first marked object and the second marked object. The server performs rendering pipeline processing on the second marked object after vertex data input according to the positional relationship to obtain a returned image. The server encodes the returned image and sends it to the client. The client receives the encoded returned image and decodes it to obtain the second layer.

[0072] The frame time information includes the camera information of the client at the frame time, the operation event information corresponding to the game operation of the user on the client, and the game event information obtained based on the operation event information.

[0073] The camera information includes the angle information of the client's imaging camera, the camera coordinate system, etc.

[0074] The operation event information is the coordinate information of the user's touch point and the data information generated by the touch when the user clicks a certain place on the client interface (game screen).

[0075] Game event information is data information generated by users controlling game characters on the client, or data information generated after users operate characters to interact with surrounding scenes, non-player characters (NPCs), etc.

[0076] For example, when a user clicks on a dropped gem on the game screen, the player character will bend down to pick it up; the user's clicking action is a game operation, and the coordinate information of the click position obtained is the operation event information. Based on the game operation and the operation event information, the player character's bending action, the player character's picking up action, the number of gems and the total value of the gems are the game event information.

[0077] If the user continuously clicks the attack button on the game screen, the player character will draw a gun and shoot the enemy, and the enemy's health will decrease; wherein, the user continuously clicking the attack button is a game operation, the coordinate information of the attack button and the coordinate information of the attack object are operation event information, and based on the game operation and operation event information, the damage caused to the enemy by the player's attack and the enemy's remaining health are obtained as game event information.

[0078] The frame time information changes with different operations of the user, and also affects the positional relationship between the first marking object and the second marking object. Therefore, the positional relationship between the first marking object and the second marking object corresponding to different frame time information is also different.

[0079] like Figure 3A As shown, the first marked object is a character 301 operated by the user, and the second marked object is a pillar 302. The user uses a third-person perspective. At the nth frame, the camera is located behind the character 301, and the pillar 302 is located directly in front of the character 301. At this time, the second marked object pillar 302 is partially blocked by the first marked object character 301. The server needs to perform rendering pipeline processing on the part of the pillar 302 that is not blocked by the character 301 after vertex data input;

[0080] If at the n+1th frame time, Figure 3B As shown, the user slides the game perspective and rotates the camera perspective 90 degrees to the left. The camera is located on the right side of the character 301. That is, at this time, the second marked object column 302 is not blocked by the first marked object character 301. The server performs rendering pipeline processing on the column 302 after inputting vertex data.

[0081] Step S204, obtaining a frame time image according to the first layer and the second layer.

[0082] The client renders the first marked object to obtain the first layer, receives the returned picture sent by the server and decodes it to the second layer, and overlays the first layer and the second layer to obtain the game image at the frame time.

[0083] By dividing the content of the game screen into two categories according to user needs, the client is responsible for rendering a part of the content in the cloud game screen, which greatly reduces the rendering pressure on the server; the interactive content and strong real-time content that users are more concerned about are rendered on the client, avoiding the problem of screen delay of the content that users are concerned about due to slow transmission speed and network delay on the server, thereby improving the user's gaming experience.

[0084] In the above step S204, the client obtains a frame moment image based on the first layer and the second layer, including: determining the overlay mode between the first layer and the second layer according to the preset overlay mode between each first marked object and each second marked object; and obtaining the frame moment image after overlay processing according to the overlay mode between the first layer and the second layer.

[0085] In some embodiments, directly overlaying the first layer and the second layer does not produce an ideal effect, and therefore an optimal overlay mode is preset for each first marked object and each second marked object, and the client performs marking and determines a mode for overlaying the first layer and the second layer. The client overlays the first layer and the second layer according to the determined overlay mode to obtain a frame moment image.

[0086] For example, in a cloud game, the developer uses a software development kit to mark the shadow of the game character as the first marked object and the ground as the second marked object. The best overlay mode for the shadow of the game character and the ground is Alpha aliasing. The developer uses the software development kit to mark the overlay mode of the shadow of the game character and the ground. The client performs a complete rendering pipeline process on all the first marked objects to obtain the first layer. The client marks the pixel position of the shadow of the game character in the first layer. The server performs a complete rendering pipeline process on all the second marked objects to obtain the second layer. When the client superimposes the first layer where the first marked object is located and the second layer where the second marked object is located, the Alpha aliasing operation will be performed on the pixels where the shadow of the game character is located and the ground, and the effect is better than directly superimposing the first layer and the second layer. By performing an Alpha aliasing operation on the pixels where the shadow of the game character is located and the ground, the material of the ground can be clearly seen after superposition.

[0087] By marking the overlay mode between each first marking object and each second marking object through the software development kit, the client can perform overlay processing according to the overlay mode marked by each marking object when overlaying layers, which can improve the game screen quality of the client and enhance the user's visual experience.

[0088] Figure 4 The flow of a display processing method of a cloud game is exemplarily shown. The flow of the method is executed by the server and includes the following steps:

[0089] Step S401, receiving frame time information under the frame time sent by the client.

[0090] The server receives the frame time information sent by the client, which includes the camera information of the client at the frame time, the operation event information corresponding to the game operation of the user on the client, and the game event information obtained based on the operation event information.

[0091] The camera information includes the angle information of the client's imaging camera, the camera coordinate system, etc.

[0092] The operation event information is the coordinate information of the user's touch point and the data information generated by the touch when the user clicks a certain place on the client interface (game screen).

[0093] Game event information is data information generated by users controlling game characters on the client, or data information generated after users operate characters to interact with surrounding scenes, non-player characters (NPCs), etc.

[0094] Step S402, according to the frame time information, perform vertex processing on the first marked object and the second marked object at the frame time respectively to obtain the positional relationship between the first marked object and the second marked object; the first marked object and the second marked object are objects to be displayed in the frame time image corresponding to the frame time.

[0095] In the above step S402, vertex processing is performed on the first marked object and the second marked object at the frame time respectively according to the frame time information, including: determining the camera information of the server at the frame time according to the camera information of the client at the frame time; determining the first marked object and the second marked object at the frame time according to the operation event information and / or the game event information; and performing vertex processing on the first marked object and the second marked object respectively according to the camera information of the server at the frame time.

[0096] Specifically, the camera information of the server and the client must be kept consistent, that is, the camera angle information, camera coordinate system, etc., to ensure that when the second layer generated by the server and the first layer generated by the client are superimposed, there will be no problem with the generated frame time image. Therefore, based on the camera information of the client at frame n, the camera information of the server at frame n is determined.

[0097] The frame time information changes with different user operations, and also affects the positional relationship between the first marking object and the second marking object. Therefore, the positional relationship between the first marking object and the second marking object corresponding to different frame time information is also different. Figure 3A , 3B As shown, according to the camera information of the server at the frame time, simple models are used to perform vertex processing on the first marked object and the second marked object respectively.

[0098] Step S403, according to the positional relationship, the second marked object is subjected to rendering pipeline processing to obtain a returned image, and the returned image is sent to the client; the returned image is used by the client to generate a frame time image.

[0099] Specifically, the server performs complete rendering pipeline processing on the second marked object according to the positional relationship between the first marked object and the second marked object obtained by vertex processing in step S402 to obtain a returned image, and the server compresses and encodes the returned image and sends it to the client.

[0100] By dividing the content of the game screen into two categories according to user needs, the client is responsible for rendering a part of the content in the cloud game screen, effectively utilizing the performance redundancy of the client, reducing the rendering pressure on the server, balancing the load on the server, reducing the consumption on the server, and supporting more clients at the same time.

[0101] Various cloud games can present game screens through the technical solutions in the embodiments of the present application. The following takes cloud game C as an example. The client of cloud game C is a mobile phone and the service end is a server.

[0102] Cloud Game C is an adventure mobile game in which players need to control the characters to move around and level up by killing monsters and completing tasks.

[0103] During the development stage of cloud game C, the developer uses a software development kit (SDK) to perform a first mark on the characters, enemy characters, and user interface (UI) controlled by the client, and obtains several first mark objects; and performs a second mark on the houses, ground, trees, non-player characters (NPC) in the game scene, and obtains several second mark objects.

[0104] Taking a frame in cloud game C as an example, the mobile phone performs simple modeling and vertex processing on several first-marked objects and several second-marked objects respectively to correctly handle the problem of second-marked objects blocking first-marked objects. The mobile phone performs rendering pipeline processing on each first-marked object according to the positional relationship between each first-marked object and each second-marked object, generates layer 1 on the mobile phone, and sends the camera information, operation event information, and game event information of the mobile phone at the moment of the frame to the server.

[0105] For example, Figure 5A As shown, one of the first marked objects is a character 501 controlled by a player, and one of the second marked objects is a wall 502. At this time, the character 501 stands in front of the wall 502, and the character 501 blocks part of the wall 502. The mobile terminal will perform rendering pipeline processing on the complete character 501.

[0106] If at a certain frame time, the character 501 stands behind the wall 502, the wall 502 blocks part of the character 501, such as Figure 5B As shown, the mobile phone does not perform rendering pipeline processing on the part where the wall 502 blocks the character 501 , but performs rendering pipeline processing on the part where the wall 502 does not block the character 501 .

[0107] The server receives the camera information, operation event information and game event information sent by the mobile phone, and performs simple modeling and vertex processing on several first mark objects and several second mark objects respectively, so as to correctly handle the problem of the first mark object occluding the second mark object; the server performs rendering pipeline processing on each second mark object according to the positional relationship between each first mark object and each second mark object, encodes the rendered picture, obtains the returned picture, and sends the returned picture to the mobile phone.

[0108] For example, Figure 5AAs shown, the first marked object is the character 501 controlled by the player, and the second marked object is the wall 502. At this time, the character 501 stands in front of the wall 502, and the character 501 blocks part of the wall 502. The server will perform complete rendering pipeline processing on the wall 502 that is not blocked by the character 501, and will not perform complete rendering pipeline processing on the wall blocked by the character 501.

[0109] If at a certain frame time, the character 501 stands behind the wall 502, the wall 502 blocks part of the area of ​​the character 502, such as Figure 5B As shown, the server performs complete rendering pipeline processing on the entire wall 502.

[0110] The mobile phone receives the returned picture sent by the server and decodes it to obtain layer 2. The mobile phone superimposes layer 1 and layer 2 according to the superposition mode marked for each first marked object and the superposition mode marked for each second marked object, and finally obtains the actual game picture of the frame.

[0111] By dividing the content of the game screen into two categories according to user needs, the mobile phone is responsible for rendering a part of the content in the cloud game screen, reducing the rendering pressure on the server; the interactive content and strong real-time content that users are more concerned about in the game are rendered on the mobile phone, avoiding the screen delay problem caused by slow server transmission speed and network delay, and also avoiding the problem of blurred screen of the content that users are concerned about due to video encoding, thereby improving the picture quality of cloud games and enhancing the user's gaming experience.

[0112] Based on the same technical concept, the present application embodiment provides a schematic diagram of the structure of a cloud gaming device, such as Figure 6 As shown, the cloud gaming device 600 includes:

[0113] Processing module 601 performs vertex processing on a first marked object and a second marked object at a frame time respectively to obtain a positional relationship between the first marked object and the second marked object; the first marked object and the second marked object are objects to be displayed in a frame time image corresponding to the frame time;

[0114] The processing module 601 is further used to perform rendering pipeline processing on the first marked object according to the position relationship to obtain a first layer, and send frame time information to the server;

[0115] The acquisition module 602 is used to receive the returned picture sent by the server, and decode the returned picture to obtain the second layer; the returned picture is obtained by the server performing rendering pipeline processing on the second marked object according to the position relationship obtained by the frame time information;

[0116] The processing module 601 is used to obtain the frame moment image according to the first layer and the second layer.

[0117] Optionally, the processing module 601 is further used for:

[0118] Determining an unobstructed portion of the first marked object according to the positional relationship;

[0119] The unobstructed portion of the first marked object is subjected to rendering pipeline processing to obtain a first layer.

[0120] Optionally, the processing module 601 is further used for:

[0121] Determining a superposition mode between the first layer and the second layer according to a preset superposition mode between each first marking object and each second marking object;

[0122] The frame moment image is obtained after superposition processing is performed according to the superposition mode between the first layer and the second layer.

[0123] Optionally, the acquisition module 602 is further used to:

[0124] Receive frame time information under the frame time sent by the client;

[0125] According to the frame time information, vertex processing is performed on the first marked object and the second marked object at the frame time respectively to obtain the positional relationship between the first marked object and the second marked object; the first marked object and the second marked object are objects to be displayed in the frame time image corresponding to the frame time;

[0126] According to the positional relationship, the second marked object is subjected to rendering pipeline processing to obtain a returned image, and the returned image is sent to the client; the returned image is used by the client to generate the frame moment image.

[0127] Optionally, the processing module 601 is further used for:

[0128] Determine the camera information of the server at the frame time according to the camera information of the client at the frame time;

[0129] Determine a first marked object and a second marked object at the frame time according to the operation event information and / or the game event information;

[0130] According to the camera information of the server at the frame time, vertex processing is performed on the first marked object and the second marked object respectively.

[0131] By dividing the content of the game screen into two categories according to user needs, the client is responsible for rendering a part of the content in the cloud game screen, which greatly reduces the rendering pressure on the server; the interactive content and strong real-time content that users are more concerned about are rendered on the client, avoiding the screen delay problem caused by slow transmission speed and network delay on the server, and improving the user's gaming experience.

[0132] Based on the same technical concept, the embodiment of the present application provides a computer device, which can be Figure 1 The client 101 and / or server 102 shown in FIG. Figure 7 As shown, it includes at least one processor 701 and a memory 702 connected to the at least one processor. The specific connection medium between the processor 701 and the memory 702 is not limited in the embodiment of the present application. Figure 7 For example, the processor 701 and the memory 702 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0133] In an embodiment of the present application, the memory 702 stores instructions that can be executed by at least one processor 701, and the at least one processor 701 can execute the steps of the above-mentioned cloud game display processing method by executing the instructions stored in the memory 702.

[0134] Among them, the processor 701 is the control center of the computer device, and can use various interfaces and lines to connect various parts of the computer device, and realize the display of the frame moment image by running or executing the instructions stored in the memory 702 and calling the data stored in the memory 702. Optionally, the processor 701 may include one or more processing units, and the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 702 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0135] Processor 701 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.

[0136] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 702 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 702 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer device, but is not limited thereto. The memory 702 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0137] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned cloud game display processing method.

[0138] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0139] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0142] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A display processing method for cloud games, applied to a client, characterized in that: include: Performing vertex processing on the first marking object and the second marking object at the frame time respectively to obtain a positional relationship between the first marking object and the second marking object; The first marked object and the second marked object are objects to be displayed in the frame time image corresponding to the frame time; the position relationship is used to characterize the influence of the second marked object on the first marked object; According to the positional relationship, the first marked object is subjected to rendering pipeline processing to obtain a first layer, and frame time information is sent to a server; receiving a returned picture sent by the server, and decoding the returned picture to obtain a second layer; the returned picture is obtained by the server performing rendering pipeline processing on the second marked object according to the position relationship obtained by the frame time information; The frame moment image is obtained according to the first layer and the second layer.

2. The method according to claim 1, characterized in that The first marking object is interactive content determined based on a game operation performed by a user on the client; and the second marking object is background content other than the interactive content determined by the game operation performed by the client.

3. The method according to claim 1, characterized in that The first marking object and the second marking object are obtained by marking objects defined in the cloud game through a software development kit.

4. The method according to claim 1, characterized in that The frame time information includes the camera information of the client at the frame time, operation event information corresponding to the game operation of the user on the client, and game event information obtained based on the operation event information.

5. The method according to claim 1, characterized in that According to the positional relationship, rendering pipeline processing is performed on the first marked object to obtain a first layer, including: Determining an unobstructed portion of the first marked object according to the positional relationship; The unobstructed portion of the first marked object is subjected to rendering pipeline processing to obtain a first layer.

6. The method according to any one of claims 1 to 5, characterized in that: The client obtains the frame time image according to the first layer and the second layer, including: Determining a superposition mode between the first layer and the second layer according to a preset superposition mode between each first marking object and each second marking object; The frame moment image is obtained after superposition processing is performed according to the superposition mode between the first layer and the second layer.

7. A display processing method for cloud games, applied to a server, characterized in that: include: Receive frame time information under the frame time sent by the client; According to the frame time information, performing vertex processing on the first marked object and the second marked object at the frame time respectively to obtain a positional relationship between the first marked object and the second marked object; The first marked object and the second marked object are objects to be displayed in the frame time image corresponding to the frame time; the position relationship is used to characterize the influence of the second marked object on the first marked object; According to the positional relationship, the second marked object is subjected to rendering pipeline processing to obtain a returned image, and the returned image is sent to the client; the returned image is used by the client to generate the frame moment image.

8. The method according to claim 7, characterized in that The frame time information includes the camera information of the client at the frame time, the operation event information corresponding to the game operation of the user at the client, and the game event information obtained based on the operation event information; According to the frame time information, vertex processing is performed on the first marked object and the second marked object at the frame time respectively, including: Determine the camera information of the server at the frame time according to the camera information of the client at the frame time; Determine a first marked object and a second marked object at the frame time according to the operation event information and / or the game event information; According to the camera information of the server at the frame time, vertex processing is performed on the first marked object and the second marked object respectively.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: It stores a computer program executable by a computer device. When the program is run on the computer device, the computer device executes the steps of the method according to any one of claims 1 to 8.

Images