Cloud game processing method, server and system

Abstract

The invention provides a cloud game processing method, a cloud game server comprises a rendering module and a video stream processing module, and the method comprises the following steps: the rendering module renders a background in a scene of a cloud game to obtain a video stream of a background picture of the cloud game; the video stream processing module cuts the size of a picture in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, and the size of the game background picture in the target video stream is matched with the view angle range of a game role corresponding to the terminal equipment; the video stream processing module compresses and encodes the target video stream and then sends the processed target video stream to the terminal device, and the terminal device is used for locally generating a game role and / or a prop of the game role and superposing the game role and / or the prop of the game role on a background picture of the processed target video stream. The game scene of the cloud game under the view angle of the game role is presented to the user. According to the invention, the problem of unsmooth operation of the user terminal equipment caused by network delay is solved.

Description

Technical Field

[0001] The present application relates to the field of cloud technology, and in particular to a cloud game processing method, server and system. Background Art

[0002] With the continuous development of cloud technology, cloud gaming has become a hot topic in the information industry in recent years. Cloud gaming is a gaming method based on cloud computing. In the cloud gaming mode, all games are run on the cloud server, and the rendered game screen is compressed and transmitted to the user's terminal device through the network. The user's terminal device does not require any high-end processors and graphics cards, only basic video decompression capabilities. Therefore, cloud gaming has many characteristics such as "cloud operation, high image quality, and click-to-play". With the advent of the 5G era, breaking through the bandwidth problem of real-time cloud rendering has also given cloud gaming greater imagination. However, an inevitable problem with cloud gaming is latency. Therefore, how to improve the running stuttering problem of cloud gaming on user terminal devices caused by network latency and improve the user's gaming experience has become a problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a cloud gaming processing method, server, and system to solve the problem of running freezes on user terminal devices due to network delays and improve the user's gaming experience.

[0004] In a first aspect, the present application provides a cloud game processing method, which is applied to a cloud game server, the cloud game server including a rendering module and a video stream processing module, and a user can manipulate a game character in the cloud game on a terminal device, the method comprising: the rendering module renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game; the video stream processing module crops the picture size in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, and the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; the video stream processing module compresses and encodes the target video stream, and sends the processed target video stream to the terminal device, the terminal device is used to generate game characters and / or props of the game characters locally, and superimpose them on the background picture of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

[0005] Based on the above method, the cloud game server and the terminal device render the content in the cloud game scene respectively and then display it on the terminal for overlay. Through the end-cloud collaboration, the terminal device shares the rendering task in the cloud game scene, avoiding the problem of too large target video stream of the cloud game caused by insufficient network bandwidth due to the rendering task in the cloud game scene being completed by the cloud server. At the same time, the size of the background screen of the cloud game will be pre-cropped according to the screen display range of the terminal device, and the cropped game screen will be transmitted to the terminal, further reducing the amount of data transmitted, thereby alleviating the network delay problem caused by insufficient network bandwidth.

[0006] In a possible implementation of the first aspect, the size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range, and the size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

[0007] Based on the above method, the game background screen in the target video stream sent by the cloud game server to the terminal device includes a buffer range. The buffer range utilizes the visual temporary storage phenomenon of the game screen on the terminal device by the human eye, which can solve the lag problem caused to the user by the network delay in the target video stream after the terminal device receives and processes it.

[0008] In a possible implementation manner of the first aspect, the value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

[0009] Based on the above method, the buffer range is obtained based on two parameters: the maximum movement speed of the user manipulating the cloud game and the network delay of the target video stream. Based on the above parameters, the calculation of the buffer range can be made more accurate.

[0010] In a possible implementation of the first aspect, the cloud game server also includes a sending module, and the method also includes: the sending module sends the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device and the spatial depth information of the cloud game to the terminal device, and the terminal device is used to superimpose and display the background picture of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, and the terminal device is also used to manipulate the game character in the cloud game according to the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device.

[0011] Based on the above method, the terminal device overlays the processed background image of the target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, so that the background image and the game characters and props in the cloud game scene processed by the cloud game server and the terminal device respectively match more accurately. At the same time, the processing method of spatial depth information is used to make the picture in the cloud game scene more realistic.

[0012] In a possible implementation of the first aspect, the rendering module renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, including: identifying the perspective type in the cloud game scene, and when the perspective type is a first-person perspective, using VR virtual reality technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0013] Based on the above method, when the perspective type of the cloud game is identified as the first-person perspective, VR virtual reality technology is used to render the scene background in the cloud game, so that the rendered picture is more in line with the gaming requirements of the first-person perspective, and the rendering effect better meets the actual experience needs of users playing games as a first-person perspective.

[0014] In a possible implementation of the first aspect, the rendering module renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, including: identifying the perspective type in the cloud game scene, and when the perspective type is a third-person perspective, using plane graphics rendering technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0015] Based on the above method, when it is identified that the perspective type of the cloud game is a third-person perspective, the plane graphics rendering technology is used to render the scene background in the cloud game, so that the rendered picture is more in line with the gaming requirements of the third-person perspective, and the rendering effect better meets the actual experience needs of users playing games as a third-person perspective.

[0016] On the second aspect, the present application also provides a cloud game processing method, which is applied to a cloud game system, and a user can manipulate a game character in the cloud game on a terminal device. The method includes: the cloud game server renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game; the cloud game server crops the picture size in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, and the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; the cloud game server compresses and encodes the target video stream, and sends the processed target video stream to the terminal device; the terminal device locally generates game characters and / or props of the game characters, and superimposes them on the background picture of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

[0017] Based on the above method, the cloud game server and the terminal device render the content in the cloud game scene respectively and then overlay the content on the terminal for display. Through the end-cloud collaboration, the terminal device shares the rendering task in the cloud game scene, avoiding the problem of excessive target video stream of the cloud game caused by insufficient network bandwidth causing network latency.

[0018] In a possible implementation of the second aspect, the size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range, and the size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

[0019] Based on the above method, the game background screen in the target video stream sent by the cloud game server to the terminal device includes a buffer range. The buffer range utilizes the visual temporary storage phenomenon of the game screen on the terminal device by the human eye, which can solve the lag problem caused to the user by the network delay in the target video stream after the terminal device receives and processes it.

[0020] In a possible implementation of the second aspect, the value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

[0021] Based on the above method, the buffer range is obtained based on two parameters: the maximum movement speed of the user manipulating the cloud game and the network delay of the target video stream. Based on the above parameters, the calculation of the buffer range can be made more accurate.

[0022] In a possible implementation of the second aspect, the method also includes: the cloud game server sends the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device and the spatial depth information of the cloud game to the terminal device, the terminal device is used to superimpose and display the background picture of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, and the terminal device is also used to manipulate the game character in the cloud game according to the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device.

[0023] Based on the above method, the terminal device overlays the processed background image of the target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, so that the background image and the game characters and props in the cloud game scene processed by the cloud game server and the terminal device respectively match more accurately. At the same time, the processing method of spatial depth information is used to make the picture in the cloud game scene more realistic.

[0024] In a possible implementation of the second aspect, the cloud game server renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, including: identifying the perspective type in the cloud game scene, and when the perspective type is a first-person perspective, using VR virtual reality technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0025] Based on the above method, when the perspective type of the cloud game is identified as the first-person perspective, VR virtual reality technology is used to render the scene background in the cloud game, so that the rendered picture is more in line with the gaming requirements of the first-person perspective, and the rendering effect better meets the actual experience needs of users playing games as a first-person perspective.

[0026] In a possible implementation manner of the second aspect, the terminal device locally generates a game character and / or a prop of the game character, including: the terminal device locally constructs a VR virtual reality rendering sphere, and renders the game character and / or the prop of the game character inside the rendering sphere.

[0027] Based on the above method, the terminal device uses the same type of VR virtual reality rendering technology as the cloud game server, so that the background images in the scenes of the superimposed cloud games are more matched with the game characters and game props, giving users a better game picture experience.

[0028] In a possible implementation of the second aspect, the cloud game server renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, including: identifying the perspective type in the cloud game scene, and when the perspective type is a third-person perspective, using plane graphics rendering technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0029] Based on the above method, when it is identified that the perspective type of the cloud game is a third-person perspective, the plane graphics rendering technology is used to render the scene background in the cloud game, so that the rendered picture is more in line with the gaming requirements of the third-person perspective, and the rendering effect better meets the actual experience needs of users playing games as a third-person perspective.

[0030] In a possible implementation manner of the second aspect, the terminal device locally generates the game character and / or the props of the game character, including: the terminal device locally constructs a rendering plane, and renders the game character and / or the props of the game character within the rendering plane.

[0031] Based on the above method, the terminal device uses the same type of plane rendering technology as the cloud game server, so that the background images in the scenes of the superimposed cloud games are more matched with the game characters and game props, giving users a better game picture experience.

[0032] In a possible implementation of the second aspect, the method also includes: the terminal device receives an operation instruction from the user to move to the left, and moves the background picture of the game scene of the cloud game from the perspective of the game character to the left; the terminal device receives an operation instruction from the user to move to the right, and moves the background picture of the game scene of the cloud game from the perspective of the game character to the right.

[0033] Based on the above method, the terminal device responds to the user's operation instructions based on the locally generated game characters combined with the background pictures rendered by the cloud game server, so that the user has a smoother gaming experience.

[0034] In a possible implementation of the second aspect, the method also includes: the terminal device receives an operation instruction from the user to move forward, and zooms in on the background image of the game scene of the cloud game from the perspective of the game character; the terminal device receives an operation instruction from the user to move backward, and zooms out on the background image of the game scene of the cloud game from the perspective of the game character.

[0035] Based on the above method, the terminal device responds to the user's operation instructions based on the locally generated game characters combined with the background pictures rendered by the cloud game server. By zooming in or out of the picture, the user's gaming experience caused by network delays can be alleviated.

[0036] On the third aspect, the present application also provides a cloud gaming server, which includes: a rendering module, which is used to render the background in the cloud gaming scene to obtain a video stream of the background screen of the cloud gaming; a video stream processing module, which is used to crop the screen size in the video stream of the background screen of the cloud gaming to obtain a target video stream of the cloud gaming, wherein the size of the game background screen in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; the video stream processing module is also used to compress and encode the target video stream, and then send the processed target video stream to the terminal device, the terminal device is used to generate game characters and / or game character props locally, and superimpose them on the background screen of the processed target video stream to present the game scene of the cloud gaming from the perspective of the game character to the user.

[0037] The third aspect or any implementation method of the third aspect is a step implementation of the device corresponding to the first aspect or any implementation method of the first aspect. The description in the third aspect or any implementation method of the third aspect is applicable to the first aspect or any implementation method of the first aspect and will not be repeated here.

[0038] In a fourth aspect, the present application also provides a cloud gaming system, which includes: a cloud gaming server, which is used to render the background in the scene of the cloud gaming to obtain a video stream of the background picture of the cloud gaming; the cloud gaming server, which is also used to crop the picture size in the video stream of the background picture of the cloud gaming to obtain a target video stream of the cloud gaming, wherein the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; the cloud gaming server, which is also used to compress and encode the target video stream, and then send the processed target video stream to the terminal device; the terminal device, which is used to generate game characters and / or props of the game characters locally, and superimpose them on the background picture of the processed target video stream to present the game scene of the cloud gaming from the perspective of the game character to the user.

[0039] The fourth aspect or any implementation method of the fourth aspect is a step implementation of the device corresponding to the second aspect or any implementation method of the second aspect. The description in the fourth aspect or any implementation method of the fourth aspect is applicable to the second aspect or any implementation method of the second aspect and will not be repeated here.

[0040] In a fifth aspect, the present application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory. The processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method disclosed in the first aspect and any possible implementation of the first aspect.

[0041] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computer device cluster, enables the computer device cluster to implement the method disclosed in the first aspect and any possible implementation manner of the first aspect.

[0042] In a seventh aspect, the present application provides a computer-readable storage medium, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method disclosed in the first aspect and any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of an AR game interface in the prior art;

[0044] Figure 2 is a schematic diagram of a cloud gaming interface according to an embodiment of the present invention;

[0045] Figure 3 is a schematic diagram of a cloud gaming system architecture based on a cloud gaming processing method according to an embodiment of the present invention;

[0046] Figure 4 is a flow chart of a cloud game processing method according to an embodiment of the present invention;

[0047] Figure 5 is a schematic diagram of a cloud gaming scene from a first-person perspective according to an embodiment of the present invention;

[0048] Figure 6 is a schematic diagram of a cloud gaming scene from a third-person perspective according to an embodiment of the present invention;

[0049] Figure 7 is a schematic diagram of a buffer range of a cloud gaming interface according to an embodiment of the present invention;

[0050] Figure 8 is a schematic diagram of another buffer range of the cloud game interface according to an embodiment of the present invention;

[0051] Fig. 9 is a schematic diagram of the computing device structure of the cloud game processing method according to an embodiment of the present invention;

[0052] Fig.10 It is a structural diagram of a computing device cluster of the cloud game processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In order to facilitate understanding of the embodiments of the present application, first, some terms involved in the present invention are explained.

[0055] Cloud gaming: Also known as game on demand, it is an online gaming technology based on cloud computing technology. Cloud gaming technology enables lightweight devices with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud gaming scenario, the game is not played on the player's game terminal, but on a cloud server, which renders the game scene into a video and audio stream and transmits it to the player's game terminal over the network. The player's game terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input commands and send them to the cloud server.

[0056] Virtual Reality (VR) is the combination of virtual and reality. Theoretically, VR is a computer simulation system that can create and experience a virtual world. It uses computers to generate a simulated environment and immerse users in it. VR uses real-life data and electronic signals generated by computer technology, and combines them with various output devices to transform them into phenomena that people can feel. These phenomena can be real objects in reality or materials that are invisible to our naked eyes, expressed through three-dimensional models. Because these phenomena are not what we can see directly, but are simulated in the real world through computer technology, they are called virtual reality.

[0057] Augmented reality (AR) is a technology that cleverly integrates virtual information with the real environment. It simulates computer-generated text, images, three-dimensional models, music, videos and other virtual information and applies them to the real environment. Virtual information and information in the real environment complement each other, thus achieving "enhancement" of the real environment.

[0058] like Figure 1 As shown, Figure 1This is a schematic diagram of an AR game interface in the prior art. The existing AR game generates virtual game characters and / or game props required for the game on the terminal device, uses the camera device on the terminal device to use real-world objects as the background screen of the game, and presents the AR game interface in a way that combines the real world with the virtual game characters and / or game props. The game interface of this method uses a combination of virtual and real. The background screen of the game is only the real world. The clarity of the background screen depends on the camera device on the terminal device. The picture sense of the background screen is relatively simple, and the display effect is not exquisite enough.

[0059] With the continuous development of cloud technology, cloud games rely on the powerful data processing capabilities of cloud resources to solve the game's requirements for the hardware performance of user terminal devices. After the game is on the cloud, the game screen can be rendered on the cloud server and the video stream of the cloud game can be transmitted to the user's terminal device through the network. However, the network transmission method is affected by the quality of network bandwidth. If the game is completely processed by the cloud server, network latency will affect the gaming experience.

[0060] In view of the problems existing in the above technologies, this application can solve the problem that the video stream of cloud games is too large and easily affected by network delays after the background images and virtual game characters and / or game props in cloud games are processed by the cloud server side by rendering the background images in the cloud game scene on the cloud server side and generating them on the terminal device. At the same time, it also avoids the problem that the background images in existing AR games are single and not exquisite enough.

[0061] like Figure 2 As shown, Figure 2 It is a schematic diagram of a cloud game interface of an embodiment of the present invention. The present invention provides a method for processing cloud games, which is applied to a cloud game system. A user can manipulate a game character in a cloud game on a terminal device. The method comprises: a cloud game server renders the background in a scene of the cloud game to obtain a video stream of a background picture of the cloud game; the cloud game server crops the size of the picture in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, and the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; the cloud game server compresses and encodes the target video stream, and sends the processed target video stream to the terminal device; the terminal device locally generates a game character and / or props of the game character, and superimposes them on the background picture of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user. Figure 2The background of the cloud game interface is rendered by the cloud game server in the cloud, and the game characters and / or props of the cloud game are generated locally by the terminal device, finally presenting the superimposed game effect.

[0062] Based on the above method, the cloud game server and the terminal device render the content in the cloud game scene respectively and then overlay the content on the terminal for display. Through the end-cloud collaboration, the terminal device shares the rendering task in the cloud game scene, avoiding the problem of excessive target video stream of the cloud game caused by insufficient network bandwidth and network latency caused by the rendering task in the cloud game scene being completed by the cloud server.

[0063] The embodiment of the present invention further discloses a method for realizing Figure 2 The cloud game system architecture of the cloud game processing method is described below:

[0064] like Figure 3 As shown, Figure 3 A schematic diagram of a cloud gaming system architecture based on a cloud gaming processing method according to an embodiment of the present invention. Figure 3 In the illustrated embodiment, the cloud game system includes: a cloud game server 10 and a terminal device 20, wherein the cloud game server 10 and the terminal device 20 perform data transmission via the Internet 30, wherein the cloud game server 10 includes a rendering module 101, a video stream processing module 102 and a sending module 103. The rendering module 101 is used to render the background in the scene of the cloud game to obtain a video stream of the background screen of the cloud game; the video stream processing module 102 is used to crop the screen size in the video stream of the background screen of the cloud game to obtain a target video stream of the cloud game, wherein the size of the game background screen in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; the video stream processing module 102 is also used to compress and encode the target video stream, and then send the processed target video stream to the terminal device 20, wherein the terminal device 20 is used to generate a game character and / or a prop of the game character locally, and superimpose it on the background screen of the processed target video stream, so as to present the game scene of the cloud game from the perspective of the game character to the user.

[0065] It should be noted that the rendering module 101, the video stream processing module 102 and the sending module 103 can be implemented by software or by hardware. Exemplarily, the implementation of the rendering module 101 is described below by taking the rendering module 101 as an example. Similarly, the implementation of the video stream processing module 102 and the sending module 103 can refer to the implementation of the rendering module 101.

[0066] When implemented by software, the rendering module 101 may be an application or a code block running on a computer device. The computer device may be at least one of a physical host, a virtual machine, a container, and other computing devices. Furthermore, the above-mentioned computer device may be one or more. For example, the rendering module 101 may be an application running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application may be distributed in the same availability zone (AZ) or in different AZs. The multiple hosts / virtual machines / containers used to run the application may be distributed in the same region or in different regions. Typically, a region may include multiple AZs.

[0067] Similarly, multiple hosts / virtual machines / containers for running the application can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Typically, a region can include multiple VPCs, and a VPC can include multiple AZs.

[0068] When implemented by hardware, the rendering module 101 may include at least one computing device, such as a server, etc. Alternatively, the rendering module 101 may also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc. The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0069] The multiple computing devices included in the rendering module 101 can be distributed in the same AZ or in different AZs. The multiple computing devices included in the rendering module 101 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the rendering module 101 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0070] It should be noted that any module among the rendering module 101, the video stream processing module 102 and the sending module 103 can be used to execute part or all of the steps in the cloud game processing method.

[0071] The various modules of the cloud gaming server 10 disclosed in the embodiment of the present invention have clear division of labor and close cooperation. The various modules cooperate with each other to efficiently complete the performance tuning of the cloud business.

[0072] Based on the above cloud gaming system architecture, the embodiment of the present invention further discloses the detailed steps of the cloud gaming processing method, which are specifically described below:

[0073] like Figure 4 As shown, Figure 4 This is a flow chart of a cloud game processing method according to an embodiment of the present invention. The method includes:

[0074] S101: The rendering module renders the background in the cloud game scene to obtain a video stream of the background image of the cloud game.

[0075] After receiving the command to start the cloud game from the user's terminal device, the cloud game server finds the 3D scene required to start the cloud game that matches the user's target cloud game in the cloud game library. The rendering module in the cloud game server extracts the panoramic image of the background in the 3D scene, and after rendering the panoramic image, adds the corresponding audio to form a video stream of the background picture of the cloud game.

[0076] Specifically, the rendering module needs to identify the perspective type in the cloud game scene before rendering the background in the cloud game scene. When the perspective type is a first-person perspective, VR virtual reality technology is used to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game. When the perspective type is a third-person perspective, plane graphics rendering technology is used to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0077] like Figure 5 and 6 As shown, Figure 5 This is a schematic diagram of a cloud gaming scene from a first-person perspective according to an embodiment of the present invention. Figure 6 This is a schematic diagram of a cloud game scene from a third-person perspective of an embodiment of the present invention. Since different game perspectives have different focuses on rendering the game background screen, the rendering from the first-person perspective focuses more on the immersive picture sense, and the rendering from the third-person perspective focuses more on the overall picture sense of the game. Therefore, using different rendering methods for different cloud game perspective types will make the rendering effect of the background in the cloud game scene more meet the actual experience needs of users playing games.

[0078] S102: The video stream processing module crops the picture size in the video stream of the background picture of the cloud game to obtain the target video stream of the cloud game.

[0079] The video stream processing module in the cloud game server crops the picture size in the video stream of the background picture of the cloud game obtained after rendering, so that the picture size in the video stream of the background picture of the cloud game meets the requirements of the user terminal device.

[0080] It should be noted that the size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the viewing angle range of the game character corresponding to the terminal device matches the maximum rotation shooting range of the user's terminal device in the case of the first-person perspective, and the size of the viewing angle range of the game character corresponding to the terminal device matches the size of the display screen of the user's terminal device in the case of the third-person perspective; the size of the buffer range outside the viewing angle range is determined according to the network delay of the terminal device receiving the processed target video stream.

[0081] Specifically, the value of the buffer range is obtained by multiplying the maximum movement speed of the user's manipulation of the cloud game by the value of the network delay of the processed target video stream. In the case of the first-person perspective, the maximum movement speed of the user's manipulation of the cloud game is the angular velocity of the camera rotation of the terminal device when the user manipulates the terminal device, and in the case of the third-person perspective, the maximum movement speed of the user's manipulation of the cloud game is the speed at which the background image in the terminal device moves when the user manipulates the terminal device.

[0082] like Figure 7 As shown, Figure 7 This is a schematic diagram of a buffer range of a cloud gaming interface according to an embodiment of the present invention, taking VR virtual reality rendering as an example: the current average network latency is 0.15s, and the maximum angular velocity of rotation of a user manipulating a cloud game is 60° / s. In order to avoid the sense of delay in cloud gaming caused by network latency, it is necessary to retain the buffer range of the cloud gaming background screen. The angle value of the buffer range is 60° / s×0.15s=9°, that is, 9° needs to be cached to the left and right on both sides of the internal sector angle in the figure, and the shaded portion is the size of the cache angle, 9°.

[0083] like Figure 8 As shown, Figure 8This is a schematic diagram of another buffer range of the cloud gaming interface in an embodiment of the present invention, taking plane rendering as an example: the size of the display screen of the user's terminal device is 10cm×10cm, the current average network delay is 0.1s, and the maximum movement speed of the user manipulating the cloud game is 10cm / s. In order to avoid the sense of delay in cloud gaming caused by network delays, it is necessary to retain the buffer range of the cloud gaming background screen. The length and width of the buffer range are 10cm / s×0.1s=1cm, that is, the size of the inner square in the figure is the size of the display screen 10cm×10cm, and the size of the outer square is the size of the buffer range plus 11cm×11cm. The shaded part at the oblique line is the size of the buffer range 1cm.

[0084] S103: The video stream processing module compresses and encodes the target video stream, and then sends the processed target video stream to the terminal device.

[0085] The video stream processing module compresses and encodes the cropped video streams, and sends the processed target video streams to the terminal device through the real-time communication technology of WebRTC (Web Real-Time Communications). The use of WebRTC technology can make the transmission of video streams more convenient and faster.

[0086] S104: The terminal device generates a game character and / or props of the game character locally, and overlays them on the background screen of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

[0087] The terminal device generates game characters and / or game character props locally to match the technology used by the rendering module to render the background in the cloud game scene. Specifically, when the perspective type is the first-person perspective, the rendering module uses VR virtual reality technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, and the corresponding terminal device constructs a VR virtual reality rendering sphere locally, and renders the game characters and / or game character props inside the rendering sphere; when the perspective type is the third-person perspective, the rendering module uses plane graphics rendering technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, and the corresponding terminal device constructs a rendering plane locally, and renders the game characters and / or game character props inside the rendering plane.

[0088] The terminal device uses the same type of rendering technology as the cloud game server, so that the background images in the scenes of the superimposed cloud games are more matched with the game characters and game props, giving users a better gaming experience.

[0089] S105: The sending module sends the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device and the spatial depth information of the cloud game to the terminal device.

[0090] Since the terminal device needs to superimpose and display the cloud game background image rendered on the cloud game server and the game characters and / or props of the game characters generated locally on the terminal device on the terminal device, the sending module in the cloud game server is needed to send the coordinate information of the game characters in the cloud game that the user can manipulate on the terminal device to the terminal device. At the same time, in order to make the background and game characters and / or props in the cloud game more consistent with the relative position relationship between objects in real life when they are superimposed, the sending module in the cloud game server is also needed to send the spatial depth information of the cloud game to the terminal device.

[0091] S106: The terminal device overlays and displays the processed background image of the target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, and manipulates the game character in the cloud game according to the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device.

[0092] Based on the spatial projection relationship, the terminal device overlays the processed background image of the target video stream on the spatial depth information of the cloud game, making the image in the cloud game scene more realistic. At the same time, the user can manipulate the game character in the cloud game on the terminal device according to the coordinate information of the game character in the cloud game.

[0093] Specifically, the terminal device receives an operation instruction from the user to move left, and moves the background picture of the game scene of the cloud game from the perspective of the game character to the left; the terminal device receives an operation instruction from the user to move right, and moves the background picture of the game scene of the cloud game from the perspective of the game character to the right. The terminal device receives an operation instruction from the user to move forward, and zooms in the background picture of the game scene of the cloud game from the perspective of the game character; the terminal device receives an operation instruction from the user to move backward, and zooms out the background picture of the game scene of the cloud game from the perspective of the game character.

[0094] It should be noted that the way to present cloud games on the terminal device can be by downloading cloud game related applications on the terminal device, or by directly playing cloud games on demand through the web page, that is, a click-to-play method. The embodiment of the present invention does not limit the way cloud games are presented on the terminal device.

[0095] The present invention also provides a computing device, see below Fig. 9 , Fig. 91 is a schematic diagram of the computing device structure of the cloud game processing method according to an embodiment of the present invention. The computing device 100 includes: a bus 104, a processor 106, a memory 105, and a communication interface 107. The processor 106, the memory 105, and the communication interface 107 communicate with each other through the bus 104. The computing device 100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.

[0096] The bus 104 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The bus 104 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 104 may include a path for transmitting information between various components of the computing device 100 (eg, the memory 105, the processor 106, and the communication interface 107).

[0097] The processor 106 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0098] The memory 105 may include a volatile memory, such as a random access memory (RAM). The processor 106 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0099] The memory 105 stores executable program codes, and the processor 106 executes the executable program codes to respectively implement the functions of the aforementioned rendering module 101, the video stream processing module 102, and the sending module 103, thereby implementing the configuration method of the cloud connection service based on the public cloud. That is, the memory 105 stores instructions for the cloud management platform to execute the configuration method of the cloud connection service based on the public cloud.

[0100] The communication interface 107 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0101] An embodiment of the present invention further provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0102] See below Fig.10 , Fig.10 1 is a schematic diagram of a computing device cluster of a cloud gaming processing method according to an embodiment of the present invention. The computing device cluster includes at least one computing device 100. The memory 105 in one or more computing devices 100 in the computing device cluster may store the same cloud management platform for executing instructions of a configuration method for a cloud connection service based on a public cloud.

[0103] In some possible implementations, one or more computing devices 100 in the computing device cluster may also be used to execute some instructions of the cloud management platform for executing the configuration method of the cloud connection service based on the public cloud. In other words, the combination of one or more computing devices 100 may jointly execute the instructions of the cloud management platform for executing the configuration method of the cloud connection service based on the public cloud.

[0104] It should be noted that the memory 105 in different computing devices 100 in the computing device cluster can store different instructions for executing some functions of the cloud management platform. That is, the instructions stored in the memory 105 in different computing devices 100 can implement the functions of one or more modules in the rendering module 101, the video stream processing module 102 and the sending module 103.

[0105] The embodiment of the present invention also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computer device, the at least one computer device executes the configuration method for the cloud management platform to execute a cloud connection service based on a public cloud.

[0106] An embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned configuration method applied to a cloud management platform for executing a cloud connection service based on a public cloud.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing cloud games, characterized in that: The method is applied to a cloud game server, which includes a rendering module and a video stream processing module. A user can manipulate a game character in the cloud game on a terminal device. The method includes: The rendering module renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game; The video stream processing module crops the screen size in the video stream of the background screen of the cloud game to obtain the target video stream of the cloud game, wherein the size of the game background screen in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; After the video stream processing module compresses and encodes the target video stream, the processed target video stream is sent to the terminal device. The terminal device is used to locally generate the game character and / or the props of the game character, and superimpose them on the background screen of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

2. The method according to claim 1, characterized in that The size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

3. The method according to claim 2, characterized in that The value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

4. The method according to any one of claims 1 to 3, characterized in that: The cloud game server further includes a sending module, and the method further includes: The sending module sends the coordinate information of the game character in the cloud game that can be manipulated by the user on the terminal device and the spatial depth information of the cloud game to the terminal device. The terminal device is used to superimpose and display the background picture of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship. The terminal device is also used to manipulate the game character in the cloud game according to the coordinate information of the game character in the cloud game that can be manipulated by the user on the terminal device.

5. The method according to any one of claims 1 to 4, characterized in that: The rendering module renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game, including: Identify the perspective type in the scene of the cloud game. When the perspective type is a first-person perspective, use VR virtual reality technology to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

6. The method according to any one of claims 1 to 4, characterized in that: The rendering module renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game, including: Identify the perspective type in the scene of the cloud game. When the perspective type is a third-person perspective, use a plane graphics rendering technology to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

7. A method for processing cloud games, characterized in that: The method is applied to a cloud gaming system, and a user can manipulate a game character in the cloud gaming on the terminal device, and the method comprises: The cloud game server renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game; The cloud game server crops the size of the picture in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, wherein the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; The cloud game server compresses and encodes the target video stream, and sends the processed target video stream to the terminal device; The terminal device locally generates the game character and / or the props of the game character, and superimposes them on the background screen of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

8. The method according to claim 7, characterized in that: The size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

9. The method according to claim 8, characterized in that The value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

10. The method according to any one of claims 7 to 9, characterized in that: The method further comprises: The cloud game server sends coordinate information of a game character in the cloud game that can be manipulated by the user on the terminal device and spatial depth information of the cloud game to the terminal device; The terminal device overlays and displays the background picture of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship; The terminal device also manipulates the game character in the cloud game according to the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device.

11. The method according to any one of claims 7 to 10, characterized in that: The cloud game server renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game, including: Identify the perspective type in the scene of the cloud game. When the perspective type is a first-person perspective, use VR virtual reality technology to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

12. The method according to claim 11, characterized in that The terminal device locally generates the game character and / or the game character's props including: The terminal device constructs a VR virtual reality rendering sphere locally, and renders the game character and / or the props of the game character inside the rendering sphere.

13. The method according to any one of claims 7 to 10, characterized in that: The cloud game server renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game, including: Identify the perspective type in the scene of the cloud game. When the perspective type is a third-person perspective, use a plane graphics rendering technology to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

14. The method according to claim 13, characterized in that The terminal device locally generates the game character and / or the game character's props including: The terminal device constructs a rendering plane locally, and renders the game character and / or the props of the game character within the rendering plane.

15. The method according to any one of claims 7 to 14, characterized in that: The method further comprises: The terminal device receives the user's operation instruction to move left, and moves the background image of the game scene of the cloud game from the perspective of the game character to the left; The terminal device receives the user's operation instruction to move right, and moves the background image of the game scene of the cloud game from the perspective of the game character to the right.

16. The method according to any one of claims 7 to 15, characterized in that: The method further comprises: The terminal device receives the user's operation instruction to move forward, and enlarges the background image of the game scene of the cloud game from the perspective of the game character; The terminal device receives the user's operation instruction to move backward, and reduces the background image of the game scene of the cloud game from the perspective of the game character.

17. A cloud gaming server, characterized in that: The cloud gaming server comprises: A rendering module, used to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game; A video stream processing module, used to crop the size of the picture in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, wherein the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; The video stream processing module is also used to compress and encode the target video stream, and then send the processed target video stream to the terminal device. The terminal device is used to locally generate the game character and / or the props of the game character, and superimpose them on the background screen of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

18. The cloud gaming server according to claim 17, characterized in that: The size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

19. The cloud gaming server according to claim 18, characterized in that: The value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

20. The cloud gaming server according to any one of claims 17 to 19, characterized in that: The cloud gaming server further includes: A sending module is used to send coordinate information of a game character in the cloud game that can be manipulated by the user on the terminal device and spatial depth information of the cloud game to the terminal device, the terminal device is used to superimpose and display the background image of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, and the terminal device is also used to manipulate the game character in the cloud game according to the coordinate information of the game character in the cloud game that can be manipulated by the user on the terminal device.

21. The cloud gaming server according to any one of claims 17 to 20, characterized in that: The rendering module is specifically used to identify the perspective type in the scene of the cloud game. When the perspective type is a first-person perspective, VR virtual reality technology is used to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

22. The cloud gaming server according to any one of claims 17 to 20, characterized in that: The rendering module is specifically used to identify the perspective type in the scene of the cloud game. When the perspective type is a third-person perspective, the background in the scene of the cloud game is rendered using plane graphics rendering technology to obtain a video stream of the background picture of the cloud game.

23. A cloud gaming system, characterized in that: The cloud gaming system comprises: A cloud game server, configured to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game; The cloud game server is further used to crop the picture size in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, wherein the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; The cloud game server is further configured to compress and encode the target video stream and then send the processed target video stream to the terminal device; The terminal device is used to locally generate the game character and / or the props of the game character, and superimpose them on the background picture of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

24. The cloud gaming system according to claim 23, characterized in that: The size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

25. The cloud gaming system according to claim 24, characterized in that: The value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

26. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

27. A computer program product comprising instructions, characterized in that When the instructions are executed by a computer device cluster, the computer device cluster executes the method according to any one of claims 1 to 6.

28. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

Images