Cloud game processing method, server, and system

Abstract

A cloud game processing method. A cloud game server comprises a rendering module and a video stream processing module. The method comprises: a rendering module renders a background in a scene of a cloud game to obtain a video stream of a background picture of the cloud game; a video stream processing module crops the picture size in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, wherein the size of the game background picture in the target video stream is matched with the visual angle range of a game character corresponding to a terminal device; the video stream processing module performs compression and encoding processing on the target video stream, and then sends the processed target video stream to the terminal device; and the terminal device is used for locally generating a game character and / or a prop of the game character, and superposing the game character and / or the prop on the background picture of the processed target video stream, so as to present to a user a game scene of the cloud game under the visual angle of the game character.

Description

Cloud gaming processing method, server, and system Technical Field

[0001] The present application relates to the field of cloud technology, and in particular to a method, server, and system for processing cloud games. Background Art

[0002] With the continuous development of cloud technology, cloud gaming has become a hot topic in the information industry in recent years. Cloud gaming is a gaming format based on cloud computing. In this mode, all games are run on cloud servers, and the rendered game images are compressed and transmitted to the user's terminal device via the network. The user's terminal device does not require any high-end processors or graphics cards, only basic video decompression capabilities. Therefore, cloud gaming boasts numerous characteristics such as "running in the cloud, high-quality graphics, and instant play." With the advent of the 5G era, breaking through the bandwidth constraints of real-time cloud rendering has given cloud gaming greater potential. However, an unavoidable issue with cloud gaming is latency. Therefore, how to improve the performance of cloud games on user devices caused by network latency and enhance the user gaming experience has become a pressing issue.

[0003] Summary of the Invention

[0004] This application provides a cloud gaming processing method, server, and system to solve the problem of game lag caused by network delays on user terminal devices and improve the user's gaming experience.

[0005] In the first aspect, the present application provides a cloud game processing method, which is applied to a cloud game server. The cloud game server includes a rendering module and a video stream processing module. Users can manipulate game characters in the cloud game on a terminal device. The method includes: the rendering module renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game; the video stream processing module crops the picture size in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, and the size of the game background picture in the target video stream matches the viewing range of the game character corresponding to the terminal device; the video stream processing module compresses and encodes the target video stream, and sends the processed target video stream to the terminal device. The terminal device is used to generate game characters and / or game character props locally, and superimpose them on the background picture of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

[0006] Based on this method, the cloud gaming server and terminal device independently render the content of the cloud gaming scene, which is then overlaid on the terminal for display. Through end-cloud collaboration, the terminal device shares the rendering tasks of the cloud gaming scene, avoiding the problem of excessively large target video streams for the cloud gaming, which would otherwise result from insufficient network bandwidth and cause network latency issues, as all rendering tasks in the cloud gaming scene are completed by the cloud server. Furthermore, the background image of the cloud game is pre-cropped based on the display range of the terminal device, and the cropped game image is transmitted to the terminal, further reducing the amount of data transmitted and thus alleviating network latency issues caused by insufficient network bandwidth.

[0007] In a possible implementation of the first aspect, the size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

[0008] Based on the above method, the game background screen in the target video stream sent by the cloud game server to the terminal device includes a buffer range. The buffer range utilizes the visual temporary storage phenomenon of the game screen by the human eye on the terminal device, which can solve the lag problem caused to the user by the network delay in the target video stream after the terminal device receives and processes it.

[0009] In a possible implementation of the first aspect, the value of the buffer range is obtained by multiplying the maximum movement speed of the user manipulating the cloud game by the network delay of the processed target video stream.

[0010] Based on the above method, the buffer range is obtained based on two parameters: the maximum movement speed of the user manipulating the cloud game and the network delay of the target video stream. Based on the above parameters, the calculation of the buffer range can be made more accurate.

[0011] In a possible implementation of the first aspect, the cloud game server also includes a sending module, and the method also includes: the sending module sends the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device and the spatial depth information of the cloud game to the terminal device, and the terminal device is used to superimpose and display the background picture of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, and the terminal device is also used to manipulate the game character in the cloud game according to the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device.

[0012] Based on the above method, the terminal device superimposes the processed background image of the target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, so that the background image and game characters and props in the cloud game scene processed by the cloud game server and the terminal device respectively match more accurately. At the same time, the processing method of spatial depth information is used to make the picture in the cloud game scene more realistic.

[0013] In a possible implementation of the first aspect, the rendering module renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, including: identifying the perspective type in the cloud game scene, and when the perspective type is a first-person perspective, using VR virtual reality technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0014] Based on the above method, when it is identified that the perspective type of the cloud game is the first-person perspective, VR virtual reality technology is used to render the scene background in the cloud game, so that the rendered picture is more in line with the gaming requirements of the first-person perspective, and the rendering effect better meets the user's actual experience needs of playing the game as a first-person perspective.

[0015] In a possible implementation of the first aspect, the rendering module renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, including: identifying the perspective type in the cloud game scene, and when the perspective type is a third-person perspective, using plane graphics rendering technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0016] Based on the above method, when it is identified that the perspective type of the cloud game is a third-person perspective, the plane graphics rendering technology is used to render the scene background in the cloud game, so that the rendered picture is more in line with the gaming requirements of the third-person perspective, and the rendering effect better meets the user's actual experience needs of playing the game as a third-person perspective.

[0017] On the second aspect, the present application also provides a cloud game processing method, which is applied to a cloud game system, and users can manipulate game characters in the cloud game on a terminal device. The method includes: the cloud game server renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game; the cloud game server crops the picture size in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, and the size of the game background picture in the target video stream matches the viewing range of the game character corresponding to the terminal device; the cloud game server compresses and encodes the target video stream, and sends the processed target video stream to the terminal device; the terminal device generates game characters and / or game character props locally, and superimposes them on the background picture of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

[0018] Based on the above method, the cloud game server and the terminal device render the content in the cloud game scene respectively and then overlay it on the terminal for display. Through the end-cloud collaboration, the terminal device shares the rendering task in the cloud game scene, avoiding the problem of excessively large target video stream of the cloud game caused by all rendering tasks in the cloud game scene being completed by the cloud server, thereby leading to insufficient network bandwidth and causing network latency problems.

[0019] In a possible implementation of the second aspect, the size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

[0020] Based on the above method, the game background screen in the target video stream sent by the cloud game server to the terminal device includes a buffer range. The buffer range utilizes the visual temporary storage phenomenon of the game screen by the human eye on the terminal device, which can solve the lag problem caused to the user by the network delay in the target video stream after the terminal device receives and processes it.

[0021] In a possible implementation of the second aspect, the value of the buffer range is obtained by multiplying the maximum movement speed of the user manipulating the cloud game by the network delay of the processed target video stream.

[0022] Based on the above method, the buffer range is obtained based on two parameters: the maximum movement speed of the user manipulating the cloud game and the network delay of the target video stream. Based on the above parameters, the calculation of the buffer range can be made more accurate.

[0023] In a possible implementation of the second aspect, the method also includes: the cloud game server sends the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device and the spatial depth information of the cloud game to the terminal device, and the terminal device is used to superimpose and display the background picture of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, and the terminal device is also used to manipulate the game character in the cloud game according to the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device.

[0024] Based on the above method, the terminal device superimposes the processed background image of the target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, so that the background image and game characters and props in the cloud game scene processed by the cloud game server and the terminal device respectively match more accurately. At the same time, the processing method of spatial depth information is used to make the picture in the cloud game scene more realistic.

[0025] In a possible implementation of the second aspect, the cloud game server renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, including: identifying the perspective type in the cloud game scene, and when the perspective type is a first-person perspective, using VR virtual reality technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0026] Based on the above method, when it is identified that the perspective type of the cloud game is the first-person perspective, VR virtual reality technology is used to render the scene background in the cloud game, so that the rendered picture is more in line with the gaming requirements of the first-person perspective, and the rendering effect better meets the user's actual experience needs of playing the game as a first-person perspective.

[0027] In a possible implementation of the second aspect, the terminal device locally generates a game character and / or props for the game character, including: the terminal device locally constructs a VR virtual reality rendering sphere, and renders the game character and / or props for the game character inside the rendering sphere.

[0028] Based on the above method, the terminal device uses the same type of VR virtual reality rendering technology as the cloud game server, so that the background images in the superimposed cloud game scenes are more matched with the game characters and game props, giving users a better game picture experience.

[0029] In a possible implementation of the second aspect, the cloud game server renders the background in the cloud game scene to obtain a video stream of the background picture of the cloud game, including: identifying the perspective type in the cloud game scene, and when the perspective type is a third-person perspective, using plane graphics rendering technology to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0030] Based on the above method, when it is identified that the perspective type of the cloud game is a third-person perspective, the plane graphics rendering technology is used to render the scene background in the cloud game, so that the rendered picture is more in line with the gaming requirements of the third-person perspective, and the rendering effect better meets the user's actual experience needs of playing the game as a third-person perspective.

[0031] In a possible implementation of the second aspect, the terminal device locally generates the game character and / or the game character's props, including: the terminal device locally constructs a rendering plane, and renders the game character and / or the game character's props within the rendering plane.

[0032] Based on the above method, the terminal device uses the same type of plane rendering technology as the cloud game server, so that the background images in the scenes of the superimposed cloud games are more matched with the game characters and game props, giving users a better game picture experience.

[0033] In a possible implementation of the second aspect, the method also includes: the terminal device receives an operation instruction from the user to move to the left, and moves the background picture of the game scene of the cloud game from the perspective of the game character to the left; the terminal device receives an operation instruction from the user to move to the right, and moves the background picture of the game scene of the cloud game from the perspective of the game character to the right.

[0034] Based on the above method, the terminal device responds to the user's operation instructions based on the locally generated game character combined with the background picture rendered by the cloud game server, so that the user has a smoother gaming experience.

[0035] In a possible implementation of the second aspect, the method also includes: the terminal device receives an operation instruction from the user to move forward, and enlarges the background picture of the game scene of the cloud game from the perspective of the game character; the terminal device receives an operation instruction from the user to move backward, and reduces the background picture of the game scene of the cloud game from the perspective of the game character.

[0036] Based on the above method, the terminal device responds to the user's operation instructions based on the locally generated game character and the background picture rendered by the cloud game server. By zooming in or out of the picture, the user's gaming experience caused by network delay can be alleviated.

[0037] On the third aspect, the present application also provides a cloud gaming server, which includes: a rendering module for rendering the background in the cloud gaming scene to obtain a video stream of the background picture of the cloud gaming; a video stream processing module for cropping the picture size in the video stream of the background picture of the cloud gaming to obtain the target video stream of the cloud gaming, wherein the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; the video stream processing module is also used to compress and encode the target video stream, and then send the processed target video stream to the terminal device, and the terminal device is used to generate game characters and / or game character props locally, and superimpose them on the background picture of the processed target video stream to present the game scene of the cloud gaming to the user from the perspective of the game character.

[0038] The third aspect or any implementation of the third aspect is the step implementation of the device corresponding to the first aspect or any implementation of the first aspect. The description in the third aspect or any implementation of the third aspect is applicable to the first aspect or any implementation of the first aspect and will not be repeated here.

[0039] Fourthly, the present application also provides a cloud gaming system, which includes: a cloud gaming server, which is used to render the background in the cloud gaming scene to obtain a video stream of the background picture of the cloud gaming; the cloud gaming server is also used to crop the picture size in the video stream of the background picture of the cloud gaming to obtain the target video stream of the cloud gaming, and the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; the cloud gaming server is also used to compress and encode the target video stream, and then send the processed target video stream to the terminal device; the terminal device is used to generate game characters and / or game character props locally, and superimpose them on the background picture of the processed target video stream to present the game scene of the cloud gaming to the user from the perspective of the game character.

[0040] The fourth aspect or any implementation of the fourth aspect is the step implementation of the device corresponding to the second aspect or any implementation of the second aspect. The description in the fourth aspect or any implementation of the fourth aspect is applicable to the second aspect or any implementation of the second aspect and will not be repeated here.

[0041] In a fifth aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device including a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method disclosed in the first aspect and any possible implementation of the first aspect.

[0042] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computer device cluster, enables the computer device cluster to implement the method disclosed in the first aspect and any possible implementation manner of the first aspect.

[0043] In a seventh aspect, the present application provides a computer-readable storage medium comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method disclosed in the first aspect and any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] FIG1 is a schematic diagram of an AR game interface in the prior art;

[0045] FIG2 is a schematic diagram of a cloud gaming interface according to an embodiment of the present invention;

[0046] FIG3 is a schematic diagram of a cloud gaming system architecture based on a cloud gaming processing method according to an embodiment of the present invention;

[0047] FIG4 is a flow chart of a cloud gaming processing method according to an embodiment of the present invention;

[0048] FIG5 is a schematic diagram of a cloud gaming scene from a first-person perspective according to an embodiment of the present invention;

[0049] FIG6 is a schematic diagram of a cloud gaming scene from a third-person perspective according to an embodiment of the present invention;

[0050] FIG7 is a schematic diagram of a buffer range of a cloud gaming interface according to an embodiment of the present invention;

[0051] FIG8 is a schematic diagram of another buffer range of a cloud gaming interface according to an embodiment of the present invention;

[0052] FIG9 is a schematic diagram of the computing device structure of the cloud gaming processing method according to an embodiment of the present invention;

[0053] FIG10 is a structural diagram of a computing device cluster of a cloud gaming processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following describes the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] In order to facilitate understanding of the embodiments of the present application, first, some terms involved in the present invention are explained.

[0056] Cloud gaming, also known as game-on-demand, is an online gaming technology based on cloud computing. Cloud gaming enables lightweight devices with relatively limited graphics and data processing capabilities to run high-quality games. In cloud gaming scenarios, the game is not played on the player's gaming terminal, but rather on a cloud server. The cloud server renders the game scene into a video and audio stream, which is transmitted to the player's gaming terminal over the network. The player's gaming terminal does not need powerful graphics and data processing capabilities; it only needs basic streaming capabilities and the ability to receive player input and transmit it to the cloud server.

[0057] Virtual reality (VR) technology is the fusion of virtual and real life. Theoretically, VR is a computer simulation system that allows users to create and experience a virtual world. It uses computers to generate a simulated environment, immersing the user in it. VR technology utilizes real-life data, electronic signals generated by computer technology, and combines them with various output devices to transform them into perceptible phenomena. These phenomena can be real objects or invisible materials, represented through three-dimensional models. Because these phenomena are not directly visible to the human eye, but rather simulated through computer technology, they are called virtual reality.

[0058] Augmented reality (AR) is a technology that cleverly integrates virtual information with the real environment. It simulates computer-generated text, images, 3D models, music, and videos, and then applies them to the real environment. The virtual information and the real environment complement each other, thereby enhancing the real environment.

[0059] As shown in Figure 1, which is a schematic diagram of an AR game interface in the prior art, existing AR games generate virtual game characters and / or game props required for the game on a terminal device. Using the camera on the terminal device, real-world objects are used as the game background, presenting the AR game interface by combining the real world with the virtual game characters and / or game props. This game interface uses a combination of virtual and real, with the game background being solely the real world. The clarity of the background depends on the camera on the terminal device, resulting in a relatively monotonous visual experience and a less than polished presentation.

[0060] With the continuous development of cloud technology, cloud gaming relies on the powerful data processing capabilities of cloud resources to solve the game's requirements for the hardware performance of user terminal devices. After the game is on the cloud, the game screen can be rendered and processed on the cloud server, and the video stream of the cloud game can be transmitted to the user's terminal device through the network. However, the network transmission method is affected by the quality of network bandwidth. If the game is completely processed by the cloud server, network latency will affect the gaming experience.

[0061] In view of the problems existing in the above technologies, this application solves the problem that the video stream of cloud games is too large and easily affected by network latency after both the background images and virtual game characters and / or game props in cloud games are processed by the cloud server. It also avoids the problem of single and less exquisite background images in existing AR games.

[0062] As shown in FIG2 , FIG2 is a schematic diagram of a cloud gaming interface according to an embodiment of the present invention. The present invention provides a cloud gaming processing method, which is applied to a cloud gaming system, wherein a user can manipulate a game character in a cloud gaming on a terminal device. The method comprises: a cloud gaming server rendering the background of the cloud gaming scene to obtain a video stream of the cloud gaming background image; the cloud gaming server cropping the image size of the cloud gaming background image video stream to obtain a target video stream of the cloud gaming, wherein the size of the game background image in the target video stream matches the viewing angle of the game character corresponding to the terminal device; the cloud gaming server compressing and encoding the target video stream and transmitting the processed target video stream to the terminal device; the terminal device locally generating the game character and / or the game character's props and superimposing them on the background image of the processed target video stream to present the cloud gaming scene from the game character's perspective to the user. In FIG2 , the background of the cloud gaming interface is rendered in the cloud by the cloud gaming server, and the game character and / or the game character's props are generated locally by the terminal device, ultimately presenting the superimposed gaming effect.

[0063] Based on the above method, the cloud game server and the terminal device render the content in the cloud game scene respectively and then overlay it on the terminal for display. Through the end-cloud collaboration, the terminal device shares the rendering task in the cloud game scene, avoiding the problem of excessively large target video stream of the cloud game caused by all rendering tasks in the cloud game scene being completed by the cloud server, thereby leading to insufficient network bandwidth and causing network latency problems.

[0064] The embodiment of the present invention further discloses a cloud gaming system architecture for implementing the cloud gaming processing method of FIG2 , as detailed below:

[0065] As shown in Figure 3, Figure 3 is a schematic diagram of a cloud gaming system architecture based on a cloud gaming processing method according to an embodiment of the present invention. In the embodiment shown in Figure 3, the cloud gaming system includes: a cloud gaming server 10 and a terminal device 20. The cloud gaming server 10 and the terminal device 20 transmit data via the Internet 30. The cloud gaming server 10 includes a rendering module 101, a video stream processing module 102, and a sending module 103. The rendering module 101 is configured to render the background of the cloud gaming scene to obtain a video stream of the cloud gaming background image; the video stream processing module 102 is configured to crop the image size of the cloud gaming background image in the video stream to obtain a target video stream of the cloud gaming, where the size of the game background image in the target video stream matches the viewing angle of the corresponding game character on the terminal device; the video stream processing module 102 is further configured to compress and encode the target video stream and send the processed target video stream to the terminal device 20. The terminal device 20 is configured to locally generate a game character and / or game character props and overlay them on the background image of the processed target video stream to present the cloud gaming scene from the game character's perspective to the user.

[0066] It should be noted that the rendering module 101, video stream processing module 102, and sending module 103 can all be implemented via software or hardware. For example, the implementation of the rendering module 101 will be described below using the rendering module 101 as an example. Similarly, the implementation of the video stream processing module 102 and sending module 103 can refer to the implementation of the rendering module 101.

[0067] When implemented by software, the rendering module 101 can be an application or code block running on a computer device. The computer device can be at least one of a physical host, a virtual machine, a container, and other computing devices. Furthermore, the above-mentioned computer device can be one or more. For example, the rendering module 101 can be an application running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application can be distributed in the same availability zone (AZ) or in different AZs. The multiple hosts / virtual machines / containers used to run the application can be distributed in the same region or in different regions. Generally, a region can include multiple AZs.

[0068] Similarly, multiple hosts / virtual machines / containers used to run the application can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Typically, a region can include multiple VPCs, and a VPC can include multiple AZs.

[0069] When implemented by hardware, the rendering module 101 may include at least one computing device, such as a server. Alternatively, the rendering module 101 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0070] The multiple computing devices included in the rendering module 101 can be distributed in the same AZ or in different AZs. The multiple computing devices included in the rendering module 101 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the rendering module 101 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0071] It should be noted that any module among the rendering module 101, the video stream processing module 102 and the sending module 103 can be used to execute part or all of the steps in the cloud game processing method.

[0072] The various modules of the cloud gaming server 10 disclosed in the embodiment of the present invention have clear division of labor and close cooperation. The various modules cooperate with each other to efficiently complete the performance tuning of the cloud business.

[0073] Based on the above cloud gaming system architecture, the embodiments of the present invention further disclose detailed steps of a cloud gaming processing method, which are described below:

[0074] As shown in Figure 4, Figure 4 is a flow chart of a cloud game processing method according to an embodiment of the present invention. The method includes:

[0075] S101: The rendering module renders the background in the cloud game scene to obtain a video stream of the background image of the cloud game.

[0076] After receiving a command from a user's terminal device to launch a cloud game, the cloud gaming server finds a 3D scene in the cloud gaming library that matches the user's target cloud game. The rendering module in the cloud gaming server extracts a panoramic image of the background in the 3D scene, renders it, and adds the corresponding audio to form a video stream of the cloud game's background image.

[0077] Specifically, the rendering module needs to identify the perspective type in the cloud game scene before rendering the background in the cloud game scene. When the perspective type is a first-person perspective, VR virtual reality technology is used to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game. When the perspective type is a third-person perspective, plane graphics rendering technology is used to render the background in the cloud game scene to obtain a video stream of the background picture of the cloud game.

[0078] As shown in Figures 5 and 6, Figure 5 is a schematic diagram of a cloud gaming scene from a first-person perspective according to an embodiment of the present invention, and Figure 6 is a schematic diagram of a cloud gaming scene from a third-person perspective according to an embodiment of the present invention. Since different game perspectives place different emphasis on rendering the game background, rendering from a first-person perspective focuses more on an immersive visual experience, while rendering from a third-person perspective focuses more on the overall visual experience of the game. Therefore, adopting different rendering methods for different cloud gaming perspective types will make the background rendering effect in the cloud gaming scene more satisfying the actual experience requirements of users playing the game.

[0079] S102: The video stream processing module crops the image size in the video stream of the background image of the cloud game to obtain the target video stream of the cloud game.

[0080] The video stream processing module in the cloud game server crops the picture size in the video stream of the background picture of the cloud game obtained after rendering, so that the picture size in the video stream of the background picture of the cloud game meets the requirements of the user terminal device.

[0081] It should be noted that the size of the game background image in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. Specifically, the viewing angle range of the game character corresponding to the terminal device matches the maximum rotation range of the user's terminal device in the first-person perspective, and matches the size of the display screen of the user's terminal device in the third-person perspective. The size of the buffer range outside the viewing angle range is determined based on the network latency of the terminal device receiving the processed target video stream.

[0082] Specifically, the buffer range is calculated by multiplying the maximum movement speed of the user's cloud game operation by the network latency of the processed target video stream. From a first-person perspective, the maximum movement speed of the user's cloud game operation is the angular velocity of the terminal device's camera rotation. From a third-person perspective, the maximum movement speed of the user's cloud game operation is the speed at which the background image on the terminal device moves.

[0083] As shown in Figure 7, Figure 7 is a schematic diagram of a buffer range of the cloud game interface according to an embodiment of the present invention. Taking VR virtual reality rendering as an example: the current average network delay is 0.15s, and the maximum rotational angular velocity of the user manipulating the cloud game is 60° / s. In order to avoid the delay of the cloud game caused by the network delay to the user, the buffer range of the cloud game background screen needs to be retained. The angle value of the buffer range is 60° / s×0.15s=9°, that is, 9° needs to be cached to the left and right on both sides of the internal sector angle in the figure, and the shaded part is the size of the cache angle, 9°.

[0084] As shown in Figure 8, Figure 8 is a schematic diagram of another buffer range of the cloud game interface according to an embodiment of the present invention. Taking plane rendering as an example: the size of the display screen of the user's terminal device is 10cm×10cm, the current average network delay is 0.1s, and the maximum movement speed of the user in manipulating the cloud game is 10cm / s. In order to avoid the sense of delay in the cloud game caused by the network delay to the user, it is necessary to retain the buffer range of the cloud game background screen. The length and width of the buffer range are 10cm / s×0.1s=1cm, that is, the size of the inner square in the figure is the size of the display screen 10cm×10cm, and the size of the outer square is the size of the buffer range plus 11cm×11cm. The shaded part at the diagonal line is the size of the buffer range 1cm.

[0085] S103: The video stream processing module compresses and encodes the target video stream, and then sends the processed target video stream to the terminal device.

[0086] The video stream processing module compresses and encodes the cropped video stream and sends the processed target video stream to the terminal device via WebRTC (Web Real-Time Communications) real-time communication technology. Using WebRTC technology can make the transmission of video streams more convenient and faster.

[0087] S104: The terminal device generates a game character and / or props of the game character locally, and superimposes them on the background screen of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

[0088] The terminal device generates game characters and / or game character props locally, which match the technology used by the rendering module to render the background of the cloud game scene. Specifically, when the perspective type is the first-person perspective, the rendering module uses VR virtual reality technology to render the background of the cloud game scene to obtain a video stream of the background picture of the cloud game. The corresponding terminal device constructs a VR virtual reality rendering sphere locally and renders the game characters and / or game character props inside the rendering sphere. When the perspective type is the third-person perspective, the rendering module uses plane graphics rendering technology to render the background of the cloud game scene to obtain a video stream of the background picture of the cloud game. The corresponding terminal device constructs a rendering plane locally and renders the game characters and / or game character props inside the rendering plane.

[0089] The terminal device uses the same type of rendering technology as the cloud game server, so that the background images in the superimposed cloud game scenes are more closely matched with the game characters and game props, giving users a better gaming experience.

[0090] S105: The sending module sends the coordinate information of the game character that the user can manipulate in the cloud game on the terminal device and the spatial depth information of the cloud game to the terminal device.

[0091] Since the terminal device needs to superimpose and display the cloud game background screen rendered on the cloud game server and the game characters and / or game character props generated locally on the terminal device, the sending module in the cloud game server is required to send the coordinate information of the game characters in the cloud game that the user can manipulate on the terminal device to the terminal device. At the same time, in order to make the background and game characters and / or game character props in the cloud game more consistent with the relative position relationship between objects in real life when superimposed, the sending module in the cloud game server is also required to send the spatial depth information of the cloud game to the terminal device.

[0092] S106: The terminal device overlays and displays the processed background image of the target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, and manipulates the game character in the cloud game according to the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device.

[0093] Based on the spatial projection relationship, the terminal device overlays the processed target video stream's background image on the cloud game's spatial depth information, making the cloud game scene more realistic. At the same time, users can manipulate the cloud game character based on the coordinate information of the character on the terminal device.

[0094] Specifically, the terminal device receives an operation instruction from the user to move left, and moves the background image of the cloud game scene from the perspective of the game character to the left; the terminal device receives an operation instruction from the user to move right, and moves the background image of the cloud game scene from the perspective of the game character to the right. The terminal device receives an operation instruction from the user to move forward, and zooms in on the background image of the cloud game scene from the perspective of the game character; the terminal device receives an operation instruction from the user to move backward, and zooms out on the background image of the cloud game scene from the perspective of the game character.

[0095] It should be noted that the way to present cloud games on the terminal device can be by downloading cloud game related applications on the terminal device, or by directly playing cloud games on demand through the web page, that is, a click-to-play method. The embodiment of the present invention does not limit the way cloud games are presented on the terminal device.

[0096] The present invention also provides a computing device. See Figure 9 below, which is a schematic diagram of the computing device structure of the cloud gaming processing method according to an embodiment of the present invention. Computing device 100 includes a bus 104, a processor 106, a memory 105, and a communication interface 107. Processor 106, memory 105, and communication interface 107 communicate with each other via bus 104. Computing device 100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 100.

[0097] Bus 104 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG5 shows a single line, but this does not imply a single bus or type of bus. Bus 104 may include a path for transmitting information between various components of computing device 100 (e.g., memory 105, processor 106, and communication interface 107).

[0098] The processor 106 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0099] The memory 105 may include a volatile memory, such as a random access memory (RAM). The processor 106 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0100] Memory 105 stores executable program code, which processor 106 executes to implement the functions of rendering module 101, video stream processing module 102, and sending module 103, thereby implementing the method for configuring a cloud connection service based on a public cloud. Specifically, memory 105 stores instructions for the cloud management platform to execute the method for configuring a cloud connection service based on a public cloud.

[0101] The communication interface 107 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0102] An embodiment of the present invention further provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0103] Please refer to Figure 10 below, which is a schematic diagram of a computing device cluster structure for a cloud gaming processing method according to an embodiment of the present invention. The computing device cluster includes at least one computing device 100. The memory 105 of one or more computing devices 100 in the computing device cluster may store instructions for executing the configuration method for a cloud connection service based on a public cloud on the same cloud management platform.

[0104] In some possible implementations, one or more computing devices 100 in the computing device cluster may also be used to execute some of the instructions of the cloud management platform for executing the configuration method for a public cloud-based cloud connection service. In other words, the combination of one or more computing devices 100 may jointly execute the instructions of the cloud management platform for executing the configuration method for a public cloud-based cloud connection service.

[0105] It should be noted that the memory 105 in different computing devices 100 in the computing device cluster can store different instructions for executing some functions of the cloud management platform. In other words, the instructions stored in the memory 105 in different computing devices 100 can implement the functions of one or more of the rendering module 101, the video stream processing module 102, and the sending module 103.

[0106] An embodiment of the present invention further provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be executed on a computing device or stored on any available medium. When executed on at least one computing device, the computer program product causes the at least one computing device to execute the aforementioned configuration method for a cloud management platform for executing a public cloud-based cloud connection service.

[0107] An embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the configuration method described above for a cloud management platform for executing a public cloud-based cloud connection service.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for processing cloud games, characterized in that: The method is applied to a cloud game server, which includes a rendering module and a video stream processing module. A user can manipulate a game character in the cloud game on a terminal device. The method includes: The rendering module renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game; The video stream processing module crops the size of the picture in the video stream of the background picture of the cloud game to obtain the target video stream of the cloud game, and the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; After the video stream processing module compresses and encodes the target video stream, the processed target video stream is sent to the terminal device. The terminal device is used to locally generate the game character and / or the props of the game character, and superimpose them on the background screen of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

2. The method according to claim 1, characterized in that The size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

3. The method according to claim 2, characterized in that The value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

4. The method according to any one of claims 1 to 3, characterized in that: The cloud game server further includes a sending module, and the method further includes: The sending module sends the coordinate information of the game character in the cloud game that can be manipulated by the user on the terminal device and the spatial depth information of the cloud game to the terminal device. The terminal device is used to superimpose and display the background picture of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship. The terminal device is also used to manipulate the game character in the cloud game according to the coordinate information of the game character in the cloud game that can be manipulated by the user on the terminal device.

5. The method according to any one of claims 1 to 4, characterized in that: The rendering module renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game, including: Identify the perspective type in the scene of the cloud game. When the perspective type is a first-person perspective, use VR virtual reality technology to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

6. The method according to any one of claims 1 to 4, characterized in that: The rendering module renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game, including: Identify the perspective type in the scene of the cloud game. When the perspective type is a third-person perspective, use a plane graphics rendering technology to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

7. A method for processing cloud games, characterized in that: The method is applied to a cloud gaming system, and a user can manipulate a game character in the cloud gaming on the terminal device, and the method comprises: The cloud game server renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game; The cloud game server crops the size of the picture in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, wherein the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; The cloud game server compresses and encodes the target video stream, and sends the processed target video stream to the terminal device; The terminal device locally generates the game character and / or the props of the game character, and superimposes them on the background screen of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

8. The method according to claim 7, characterized in that: The size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

9. The method according to claim 8, characterized in that The value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

10. The method according to any one of claims 7 to 9, characterized in that: The method further comprises: The cloud game server sends coordinate information of a game character in the cloud game that can be manipulated by the user on the terminal device and spatial depth information of the cloud game to the terminal device; The terminal device overlays and displays the background picture of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship; The terminal device also manipulates the game character in the cloud game according to the coordinate information of the game character in the cloud game that the user can manipulate on the terminal device.

11. The method according to any one of claims 7 to 10, characterized in that: The cloud game server renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game, including: Identify the perspective type in the scene of the cloud game. When the perspective type is a first-person perspective, use VR virtual reality technology to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

12. The method according to claim 11, characterized in that The terminal device locally generates the game character and / or the game character's props including: The terminal device constructs a VR virtual reality rendering sphere locally, and renders the game character and / or the props of the game character inside the rendering sphere.

13. The method according to any one of claims 7 to 10, characterized in that: The cloud game server renders the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game, including: Identify the perspective type in the scene of the cloud game. When the perspective type is a third-person perspective, use a plane graphics rendering technology to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

14. The method according to claim 13, characterized in that The terminal device locally generates the game character and / or the game character's props including: The terminal device constructs a rendering plane locally, and renders the game character and / or the props of the game character within the rendering plane.

15. The method according to any one of claims 7 to 14, characterized in that: The method further comprises: The terminal device receives the user's operation instruction to move left, and moves the background image of the game scene of the cloud game from the perspective of the game character to the left; The terminal device receives the user's operation instruction to move right, and moves the background image of the game scene of the cloud game from the perspective of the game character to the right.

16. The method according to any one of claims 7 to 15, characterized in that: The method further comprises: The terminal device receives the user's operation instruction to move forward, and enlarges the background image of the game scene of the cloud game from the perspective of the game character; The terminal device receives the user's operation instruction to move backward, and reduces the background image of the game scene of the cloud game from the perspective of the game character.

17. A cloud gaming server, characterized in that: The cloud gaming server comprises: A rendering module, used to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game; A video stream processing module, used to crop the size of the picture in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, wherein the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; The video stream processing module is also used to compress and encode the target video stream, and then send the processed target video stream to the terminal device. The terminal device is used to locally generate the game character and / or the props of the game character, and superimpose them on the background screen of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

18. The cloud gaming server according to claim 17, characterized in that: The size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

19. The cloud gaming server according to claim 18, characterized in that: The value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

20. The cloud gaming server according to any one of claims 17 to 19, characterized in that: The cloud gaming server further includes: A sending module is used to send coordinate information of a game character in the cloud game that can be manipulated by the user on the terminal device and spatial depth information of the cloud game to the terminal device, the terminal device is used to superimpose and display the background image of the processed target video stream on the spatial depth information of the cloud game based on the spatial projection relationship, and the terminal device is also used to manipulate the game character in the cloud game according to the coordinate information of the game character in the cloud game that can be manipulated by the user on the terminal device.

21. The cloud gaming server according to any one of claims 17 to 20, characterized in that: The rendering module is specifically used to identify the perspective type in the scene of the cloud game. When the perspective type is a first-person perspective, VR virtual reality technology is used to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game.

22. The cloud gaming server according to any one of claims 17 to 20, characterized in that: The rendering module is specifically used to identify the perspective type in the scene of the cloud game. When the perspective type is a third-person perspective, the background in the scene of the cloud game is rendered using plane graphics rendering technology to obtain a video stream of the background picture of the cloud game.

23. A cloud gaming system, characterized in that: The cloud gaming system comprises: A cloud game server, configured to render the background in the scene of the cloud game to obtain a video stream of the background picture of the cloud game; The cloud game server is further used to crop the picture size in the video stream of the background picture of the cloud game to obtain a target video stream of the cloud game, wherein the size of the game background picture in the target video stream matches the viewing angle range of the game character corresponding to the terminal device; The cloud game server is further configured to compress and encode the target video stream and then send the processed target video stream to the terminal device; The terminal device is used to locally generate the game character and / or the props of the game character, and superimpose them on the background picture of the processed target video stream to present the game scene of the cloud game from the perspective of the game character to the user.

24. The cloud gaming system according to claim 23, characterized in that: The size of the game background screen in the target video stream includes the viewing angle range of the game character corresponding to the terminal device and the buffer range outside the viewing angle range. The size of the buffer range is determined based on the network delay of the terminal device receiving the processed target video stream.

25. The cloud gaming system according to claim 24, characterized in that: The value of the buffer range is obtained by multiplying the value of the maximum movement speed of the user manipulating the cloud game by the value of the network delay of the processed target video stream.

26. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

27. A computer program product comprising instructions, characterized in that When the instructions are executed by a computer device cluster, the computer device cluster executes the method according to any one of claims 1 to 6.

28. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

Images