Coding method, device, electronic device and storage medium for scene picture

Abstract

The present application discloses a method, apparatus, electronic device, and storage medium for encoding a scene picture. The method includes: obtaining position information of virtual objects manipulated by each target client in a target virtual scene; determining a target scene area corresponding to each target client in the target virtual scene according to the position information of the virtual objects manipulated by each target client; determining a target scene picture corresponding to each target client according to the scene data of each target scene area; performing video stream encoding on the target scene pictures corresponding to each target client to obtain video streams of each target client; and sending the video streams to the corresponding target clients. This solution realizes generating scene pictures and encoding by dividing scene areas, so that it is not necessary to generate a global scene picture, nor to encode the global scene picture, improving the picture encoding efficiency. This solution can be applied to cloud games.

Description

[0001] This application claims the priority of a Chinese application with the application number 202111348276.5 filed on November 15, 2021, the entire content of which is incorporated herein by reference. Technical Field

[0002] This application relates to the technical field of virtual scenarios, and more specifically, to a method, apparatus, electronic device, and storage medium for encoding a scene picture. Background Art

[0003] In the related art, after the game server executes the game control logic, it sends the global game data of the game scene to the devices where each player in the game session is located. Each device where a player is located performs screen rendering to generate the global scene picture of the game scene, and then intercepts the picture corresponding to the player and displays it on the display screen. In this process, each device where a player is located repeatedly renders the global scene picture of the game scene, wasting the processing resources of the device. Summary of the Invention

[0004] In view of the above problems, embodiments of the present application propose a method, apparatus, electronic device, and storage medium for encoding a scene picture to improve the above problems.

[0005] According to one aspect of the embodiments of the present application, a method for encoding a scene picture is provided, including: obtaining the position information of virtual objects controlled by each target client in a target virtual scene, where the target client refers to a client that joins a virtual session, and the target virtual scene refers to the virtual scene corresponding to the virtual session; determining, in the target virtual scene, a target scene area corresponding to each target client according to the position information of the virtual objects controlled by each target client; determining a target scene picture corresponding to each target client according to the scene data of each target scene area; performing video stream encoding on the target scene pictures corresponding to each target client to obtain a video stream for each target client; and sending the video stream to the corresponding target client.

[0006] According to one aspect of the embodiments of the present application, there is provided an encoding device for a scene picture, including: a position information acquisition module, configured to acquire the position information of virtual objects manipulated by each target client in a target virtual scene, where the target client refers to a client that joins a virtual game, and the target virtual scene refers to the virtual scene corresponding to the virtual game; a target scene area determination module, configured to determine the target scene area corresponding to each target client in the target virtual scene according to the position information of the virtual objects manipulated by each target client; a target scene picture determination module, configured to determine the target scene picture corresponding to each target client according to the scene data of each target scene area; a video stream encoding module, configured to perform video stream encoding on the target scene pictures corresponding to each target client to obtain the video streams of each target client; and a video stream sending module, configured to send the video streams to the corresponding target clients.

[0007] In some embodiments of the present application, the target virtual scene is a three-dimensional scene; in this embodiment, the target scene area determination module includes: a viewing angle information acquisition unit, configured to acquire the viewing angle information corresponding to each target client; and a target scene area determination unit, configured to determine the target scene area corresponding to each target client in the target virtual scene according to the viewing angle information corresponding to each target client and the position information of the virtual objects manipulated by each target client.

[0008] In some embodiments of the present application, the target virtual scene is a three-dimensional scene; in this embodiment, the target scene picture determination module includes: a scene data acquisition unit, configured to acquire the scene data of each target scene area; the scene data includes model data and rendering data; a geometry processing unit, configured to perform geometry processing on the model data of each target scene area to obtain an intermediate scene picture of each target scene area in screen space; and a rasterization processing unit, configured to perform rasterization processing on the intermediate scene picture of each target scene area in screen space and the rendering data corresponding to each target scene area to obtain the target scene picture corresponding to each target client.

[0009] In some embodiments of the present application, the model data includes the first coordinate information of each vertex in the corresponding target scene area; the target scene screen determination module includes: a first transformation unit, configured to transform each vertex from the model space to the camera space under the corresponding virtual camera according to the first coordinate information of each vertex in the model data of each target scene area, so as to obtain the second coordinate information of each vertex in the target scene area; a second transformation unit, configured to transform each vertex from the corresponding camera space to the homogeneous clipping space according to the second coordinate information of each vertex in the target scene area, so as to obtain the third coordinate information of each vertex in the target scene area; a third transformation unit, configured to transform each vertex from the homogeneous clipping space to the screen space based on the third coordinate information of each vertex in the target scene area, so as to obtain the fourth coordinate information of each vertex in the target scene area; an intermediate scene screen generation unit, configured to determine the intermediate scene screen of each target scene area in the screen space according to the fourth coordinate information of all vertices in each target scene area.

[0010] In some embodiments of the present application, the rasterization processing unit includes: a first processing unit, configured to perform rasterization processing on the intermediate scene screen corresponding to each target scene area according to the rendering data corresponding to the target scene area, so as to obtain the target scene screen corresponding to the target client.

[0011] In some embodiments of the present application, the intermediate scene screen indicates the fourth coordinate information of each vertex in the corresponding target scene area in the screen space; in this embodiment, the first processing unit includes: a fragment determination unit, configured to determine a plurality of fragments in the target scene area according to the fourth coordinate information of each vertex in the target scene area for each target scene area; a coloring and rendering unit, configured to perform coloring and rendering on the plurality of fragments in the target scene area according to the rendering data corresponding to the target scene area, so as to obtain the target scene screen corresponding to the target client.

[0012] In some embodiments of the present application, the rasterization processing unit includes: a splicing unit, configured to splice the intermediate scene screens corresponding to all target scene areas in the target virtual scene to obtain a spliced screen; a second processing unit, configured to perform rasterization processing on the spliced screen according to the rendering data corresponding to all target scene areas, so as to obtain a spliced scene screen; a determination unit, configured to determine the target scene screen corresponding to each target client in the spliced scene screen according to the position information of the intermediate scene screen corresponding to each target client in the spliced screen.

[0013] In some embodiments of the present application, the target virtual scene is a two-dimensional scene; the scene data includes background data and texture data of the virtual objects manipulated by the corresponding target clients; in this embodiment, the target scene image determination module includes: a background texture generation unit, configured to generate background textures for each of the target scene areas according to the background data of each of the target scene areas; a texture processing unit, configured to perform texture processing in the background textures of each of the target scene areas according to the texture data of the virtual objects manipulated by each of the target clients, and correspondingly obtain the target scene images corresponding to each of the target clients.

[0014] In some embodiments of the present application, the encoding device for the scene image further includes: a size information acquisition module, configured to acquire the size information of the target virtual scene; if the size indicated by the size information exceeds a set size threshold, then transfer to the target scene area determination module.

[0015] In some embodiments of the present application, the encoding device for the scene image further includes: a global background texture acquisition module, configured to acquire the global background texture of the target virtual scene if the size indicated by the size information does not exceed the set size threshold; a texture mapping module, configured to perform texture mapping of the object textures of all the virtual objects manipulated by each of the target clients in the global background texture according to the position information of the virtual objects manipulated by each of the target clients in the target virtual scene, to obtain a global scene image; a scene image area determination module, configured to determine the target scene images corresponding to each of the target clients in the global scene image according to the position information of the virtual objects manipulated by each of the target clients and the preset image size information; a video stream generation module, configured to perform video stream encoding according to the target scene images corresponding to each of the target clients in the global scene image, to obtain the video streams corresponding to each of the target clients.

[0016] In some embodiments of the present application, the video stream encoding module includes: an area determination unit, configured to determine the overlapping image areas and non-overlapping image areas of each target scene image relative to other target scene images according to the target scene images corresponding to each of the target clients; an encoding unit, configured to encode the overlapping image areas and the non-overlapping image areas respectively; a combination unit, configured to combine the encoding information of the overlapping image areas and the encoding information of the non-overlapping image areas in each target scene image, to obtain the encoding information of the target scene image corresponding to each of the target clients; a video stream generation unit, configured to generate the video streams corresponding to each of the target clients according to the encoding information of the target scene image corresponding to each of the target clients.

[0017] According to one aspect of the embodiments of the present application, an electronic device is provided, including: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the encoding method of the scenario picture as described above is implemented.

[0018] According to one aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the encoding method of the scenario picture as described above is implemented.

[0019] According to one aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, and when the computer instructions are executed by a processor, the encoding method of the scenario picture as described above is implemented.

[0020] In the solution of the present application, first, according to the position information of the virtual objects controlled by each target client in the virtual game, the target scenario area corresponding to each target client is determined in the virtual scenario. Then, according to the scenario data of each target scenario area, the target scenario picture corresponding to each target client is determined. Furthermore, video stream encoding is performed according to the target scenario pictures corresponding to each target client to obtain the video streams of each target client; in this process, it is not necessary for different target clients to repeatedly render the global scenario picture of the target virtual scenario, and the target scenario picture corresponding to each target client is generated according to the target scenario area that each target client needs to display. Thus, on the basis of ensuring the display requirements of the scenario pictures of each target client, processing resources are saved, and the problem of waste of processing resources caused by the need to render pictures in all clients in the virtual game in the related art is effectively solved.

[0021] In the related art, since the picture rendering is performed by the terminal devices where each target client is located, and the processing capabilities of different terminal devices are different, the scenario pictures presented by different target clients in the same virtual game are not synchronized. However, the solution of the present application can be executed by the same electronic device to obtain multiple video streams, and each video stream corresponds to a target client, and there is no difference in the progress of generating the scenario picture due to device differences. Thus, the consistency of the scenario pictures presented by different target clients in the same virtual game can be effectively ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a schematic diagram of rendering a game scene screen in the related art.

[0024] Figure 2 It is a schematic diagram of an implementation environment of a method for encoding a scene screen according to an embodiment of the present application.

[0025] Figure 3 It is a schematic diagram of an implementation environment of a method for encoding a scene screen according to another embodiment of the present application.

[0026] Figure 4 It is a flowchart of a method for encoding a scene screen according to an embodiment of the present application.

[0027] Figure 5 It is a schematic diagram of determining a target scene area corresponding to each target client according to an embodiment of the present application.

[0028] Figure 6 It is a schematic diagram of a target scene area corresponding to a client under different viewing perspectives according to an embodiment of the present application.

[0029] Figure 7A It is a flowchart of a method for encoding a scene screen according to an embodiment of the present application.

[0030] Figure 7B It is an architecture diagram of a cloud game system according to an embodiment of the present application.

[0031] Figure 7C It is an interaction sequence diagram in a cloud game scenario according to an embodiment of the present application.

[0032] Figure 8 It is a flowchart of screen rendering according to an embodiment of the present application.

[0033] Figure 9 is Figure 2 A flowchart of step 430 in an embodiment in the embodiment.

[0034] Figure 10A It is a flowchart of geometric processing according to an embodiment of the present application.

[0035] Figure 10B Shows a schematic diagram of a projection transformation frustum.

[0036] Figure 11 is Figure 9 A flowchart of step 920 in an embodiment in the embodiment.

[0037] Figure 12 It is a flowchart of a method for encoding a scene screen according to another embodiment of the present application.

[0038] Figure 13 It is a flowchart of a method for encoding a scene picture shown in another embodiment of the present application.

[0039] Figure 14 It is a flowchart of encoding a scene picture shown in another embodiment of the present application.

[0040] Figure 15 It is a flowchart of encoding a scene picture shown in another embodiment of the present application.

[0041] Figure 16 It is a block diagram of an encoding device for a scene picture shown in an embodiment of the present application.

[0042] Figure 17 It shows a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application. Detailed implementation manners

[0043] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0044] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0045] The block diagrams shown in the accompanying drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0046] The flowcharts shown in the accompanying drawings are only illustrative and do not necessarily include all the content and operations / steps, nor do they necessarily have to be executed in the described order. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0047] It should be noted that: "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0048] Before making a specific description, the following explanations are given for the terms involved in this application:

[0049] Virtual scene: It can be a simulation environment scene of the real world, or a semi-simulated and semi-fictional three-dimensional environment scene, or a purely fictional three-dimensional environment scene. The virtual scene can be a two-dimensional virtual scene or a three-dimensional virtual scene. For example, the scene constructed in a game, and various

[0050] Virtual object: It refers to an object in a virtual scene, which can be a virtual character, a virtual animal, an anime character, a virtual prop, etc. For example: the characters, animals, plants, oil drums, walls, stones, etc. displayed in the virtual scene. The controlled virtual object can be a virtual image in the virtual scene that represents the user.

[0051] A virtual scene can include multiple virtual objects. Each virtual object has its own shape and volume in the virtual scene and occupies a part of the space in the virtual scene. Optionally, when the virtual scene is a three-dimensional virtual scene, optionally, the virtual object can be a three-dimensional solid model, and the three-dimensional solid model can be a three-dimensional character constructed based on three-dimensional human bone technology. The same virtual object can display different external images by wearing different skins.

[0052] Optionally, the controlled virtual object can be a player character controlled by operations on the client, or an artificial intelligence (AI) set in the virtual scene battle through training, or a non-player character (NPC) set in the virtual scene interaction. Optionally, the controlled virtual object can be a virtual character competing in the virtual scene. Optionally, the number of virtual objects participating in the interaction in the virtual scene can be preset or dynamically determined according to the number of clients joining the interaction.

[0053] ROI (Region of Interest) encoding: Encoding a part of the region in the complete image and ignoring the other parts. The part of the region encoded in the complete image is the region of interest in the complete image.

[0054] Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In the cloud gaming scenario, the game does not run on the player's game terminal but on the cloud server. The cloud server renders the game scene into a video and audio stream and transmits it to the player's game terminal through the network. The player's game terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain the player's input instructions and send them to the cloud server.

[0055] Figure 1 It is a flowchart for rendering the game scene screen in the related technology. As Figure 1 shown, after executing the game control logic in the game server 120, the global scene data of the game scene is sent to the terminal devices where each game client in the game session is located, such as Figure 1 the first terminal device 111, the second terminal device 112, and the third terminal device 113 in . In this way, the first terminal device 111, the second terminal device 112, and the third terminal device 113 all need to render the global game scene screen, and then intercept the corresponding scene screen from the global game scene screen, such as the first scene screen corresponding to the first terminal device 111, the second scene screen corresponding to the second terminal device 112, and the third scene screen corresponding to the third device 113. It can be seen that in this way, the global game scene screen of the game scene is repeatedly rendered on different terminal devices in the game session, wasting the rendering processing resources of the terminal devices and resulting in a relatively high requirement for the processing power of the terminal. Therefore, how to reduce or avoid repeated rendering of the screen in the virtual scene is an urgent problem to be solved in the existing technology. To solve this problem, the solution of this application is proposed.

[0056] Figure 2 It is a schematic diagram of the implementation environment of a method for encoding a scene screen provided by an embodiment of this application. Refer to Figure 2 , this implementation environment includes: terminals ( Figure 2 the first terminal 211 and the second terminal 212 are exemplarily shown in ) and a server 220.

[0057] The server 220 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server 220 is used to provide background services for application programs that support virtual scenarios. The terminal and the server 220 can be directly or indirectly connected through wired or wireless communication methods, and this application does not limit this here.

[0058] The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, a smart TV, a game terminal, etc., but is not limited thereto. The terminal installs and runs an application program that supports virtual scenarios. At this time, the terminal correspondingly serves as a client that supports virtual scenarios. The application program can be any one of MOBA (Multiplayer Online Battle Arena) games, Massively Multiplayer Online Role Playing Games (MMORPG), First-Person Shooting games (FPS), Third-Person Shooting games, virtual reality application programs, three-dimensional map programs, or multiplayer gunfight survival games.

[0059] The user can interact with the server 220 based on the application program that supports virtual scenarios running on the terminal. For example, the user can trigger a virtual object to perform activities in the virtual scenario in the scene picture of the virtual scenario displayed on the terminal. The activities include but are not limited to: releasing skills, adjusting body postures, crawling, walking, running, cycling, jumping, driving, picking up, shooting, attacking, throwing, etc. The terminal generates corresponding control instructions according to the control operations triggered by the user in the display interface of the terminal, and sends the control instructions to the server 220. The control instructions are, for example, instructions for controlling the virtual object to crawl, release skills, attack, etc., and are not specifically limited here. For example Figure 2 As shown, the first control instruction generated by the first terminal 211 and the second control instruction generated by the second terminal 212 are correspondingly sent to the server 220.

[0060] In other embodiments, the user can also use other control terminals to trigger a virtual object to perform activities in the virtual scenario. The control terminal is, for example Figure 1 the gamepad 230 in. In this case, the terminal can display the picture of the virtual scenario, receive the control instructions from the gamepad 230, and send the control instructions to the server 220.

[0061] After receiving the control instruction sent by the terminal, the server 220 executes the corresponding control logic, such as the control logic for releasing skills. At the same time, in order to ensure that the virtual scene after executing the control logic is displayed in real time on the display screen of the terminal, the server 220 further performs screen rendering and video stream encoding according to the scene data of the virtual scene after executing the control logic, so as to display the virtual scene picture on the display screen of the terminal. Specifically, the server 220 can encode the scene picture corresponding to the virtual scene into a video stream according to the method of the present application and send the video stream to the terminal.

[0062] In a virtual game session (such as a game round), there are at least two clients participating in the virtual game session. The virtual objects controlled by different clients in the same virtual game session are in the same virtual scene. At this time, the virtual object controlled by one client can perform adversarial interactions with other virtual objects in the virtual scene. For example, the virtual objects can be divided into different camps or teams, and the virtual objects in a hostile or adversarial relationship can fight by releasing skills to each other. The virtual objects belonging to the same camp or the same team can have temporary communication permissions. In this case, one virtual object can release a healing skill to other virtual objects belonging to the same camp or team.

[0063] In a virtual game session, although the virtual objects controlled by each client are in the same virtual scene, due to the different virtual objects or perspectives controlled by each client (such as players), the pictures presented on different clients are different, and the pictures presented on different user interfaces may be completely different pictures.

[0064] Based on this, in order to improve the efficiency of screen rendering and encoding, the server 220 can, according to the method of the present application, determine the target scene area corresponding to each target client in the target virtual scene according to the position information of the virtual objects controlled by each target client participating in the virtual game session in the target virtual scene; then, determine the target scene picture corresponding to each target client according to the scene data of each target scene area; then, perform video stream encoding according to the target scene picture corresponding to each target client to obtain the video stream of each target client; finally, send the video stream to the corresponding target client. Thus, according to the solution of the present application, the video stream corresponding to each target client can be obtained respectively.

[0065] Suppose the client for joining a virtual game includes Client I in the first terminal 211 and Client II in the second terminal 212. The server 220 can, according to the method of this application, determine the first target scene area 221 corresponding to Client I and the second target scene area 222 corresponding to Client II in the target virtual scene. Furthermore, based on the determined first target scene area 221 and second target scene area 222, perform screen rendering and video stream encoding to obtain Video Stream 1 corresponding to the first terminal 211 and Video Stream 2 corresponding to the second terminal 212 respectively.

[0066] It is worth mentioning that the application programs installed on the terminals can be the same, or the application programs installed on the two terminals are of the same type but on different operating system platforms. Those skilled in the art can know that the number of the above terminals can be more or less.

[0067] Figure 3 It is a schematic diagram of an implementation environment shown according to another embodiment of this application. Compared with the Figure 1 shown implementation environment, in this implementation, an intermediate device 240 is also provided between the server 220 and the terminals. The intermediate device 240 can be a deployed game console, a desktop computer, a laptop computer, or other electronic devices that can provide screen rendering capabilities.

[0068] In this embodiment, each terminal sends control instructions in the virtual scene (such as the first control instruction generated by the first terminal 211 and the second control instruction generated by the second terminal 212) to the server 220. The server 220 executes the control logic in the virtual scene. Since after executing the control logic, the states of some virtual objects in the virtual scene will change (such as the position change of a virtual object in the virtual scene), therefore, after the server 220 executes the control logic, the corresponding scene data of the virtual scene is updated, and then the scene data is sent to the intermediate device 240, and the intermediate device 240 executes the method of this application to obtain Video Stream 1 corresponding to Client I in the first terminal 211 and Video Stream 2 corresponding to Client II in the second terminal 212.

[0069] Figure 4 It is a flowchart of a scene screen encoding method shown according to an embodiment of this application. This method can be executed by an electronic device with processing capabilities, such as a server, a terminal, an edge computing device, etc., which is not specifically limited here. Refer to Figure 4 shown, this method at least includes steps 410 to 450, which are introduced in detail as follows:

[0070] Step 410: Obtain the position information of the virtual objects controlled by each target client in the target virtual scene. The target client refers to the client that joins the virtual game, and the target virtual scene refers to the virtual scene corresponding to the virtual game.

[0071] In a competitive scenario involving multiple users, the competition can be carried out in units of games (or rounds). One game of competition is called a virtual game. It can be understood that the clients participating in the virtual game include at least two clients, which can be game clients, virtual reality application clients, etc., and no specific limitations are made here. In this solution, the clients participating in a virtual game are called target clients. The virtual scene where a virtual game is located is called the target virtual scene.

[0072] In a virtual game, each target client is assigned a virtual object representing the user logged in by the target client. The assigned virtual object representing the user logged in by the target client can be controlled by the user, for example, to release skills, jump, move forward, move backward, lie down, etc.

[0073] The position information of the virtual object controlled by the target client in the target virtual scene is used to indicate the position of the virtual object controlled by the target client in the target virtual scene. It can be understood that since the virtual object can be controlled by the target client to move, therefore, at different moments during the progress of the virtual game, the positions of the virtual objects controlled by each target client in the target virtual scene may also change correspondingly.

[0074] In some embodiments, after the server receives the control instructions from each target client, the server executes the corresponding control logic. Thus, if the control instruction is to indicate controlling the virtual object to move in the virtual scene, the corresponding moving distance and moving direction can be calculated according to the control instruction, and combined with the historical position information of the virtual object in the target virtual scene, to determine the current position information of the virtual object in the target virtual scene after executing the control instruction. Thus, the position information of the virtual object in the target virtual scene is updated and stored correspondingly, and the position information of the corresponding virtual object can be read when the position information of the virtual object is required.

[0075] Step 420: Determine the target scene areas corresponding to each target client in the target virtual scene according to the position information of the virtual objects controlled by each target client.

[0076] For a target client, the observation range of the target client in the target virtual scene is limited, and its observation range is only a partial scene area in the target virtual scene. Correspondingly, in order to facilitate the user to know the current situation of the virtual object controlled in the target virtual environment, the picture displayed by the target client is also the picture of a partial scene area in the target virtual scene.

[0077] Therefore, in the solution of the present application, before performing screen rendering, first determine the target scene area corresponding to the target client based on the position information of the virtual objects controlled by each target client. Herein, the target scene area refers to the scene area corresponding to the scene screen to be displayed by the target client. It can be understood that the target scene area is a partial area of the target virtual scene area. It can be understood that in step 420, it is necessary to determine the position and size of the target scene area corresponding to the target client in the target virtual scene.

[0078] In some embodiments, the size information of the viewing range of the virtual objects controlled by each target client can be preset. Herein, the size information of the viewing range is used to indicate the size of the target scene area corresponding to the corresponding target client. Furthermore, taking the position of the virtual object in the target virtual scene as the position reference, and then combining the reference position and the size information of the viewing range to determine the position of the target scene area corresponding to the target client in the target virtual scene.

[0079] Figure 5 is a schematic diagram showing the determination of the target scene areas corresponding to each target client according to an embodiment of the present application. Assume that the target clients participating in the virtual game include Client I, Client II, and Client III, and the viewing ranges of the virtual objects are set with the position where the virtual object is located as the center point, and the area defined by the size of a*b. As Figure 5 shown, if it is set that the virtual object controlled by Client I is located at Figure 5 the position point S1 in, the virtual object controlled by Client II is located at Figure 5 the position point S2 in, and the virtual object controlled by Client III is located at Figure 5 the position point S3 in, then according to the position where the virtual object is located as the center point and the size of a*b, the target scene area corresponding to Client I is determined as the first scene area 510, the target scene area corresponding to Client II is determined as the second scene area 520, and the target scene area corresponding to Client III is determined as the third scene area 530.

[0080] In some embodiments, since the viewing angles of the virtual objects are different, their viewing ranges also correspondingly vary. Therefore, in this case, the viewing angles of the virtual objects can be further combined to determine the target scene areas corresponding to each target client.

[0081] In some embodiments, the target virtual scene is a three-dimensional scene; step 420 includes: obtaining the viewing angle information corresponding to each target client; and determining the target scene areas corresponding to each target client in the target virtual scene according to the viewing angle information corresponding to each target client and the position information of the virtual objects controlled by each target client.

[0082] The perspective information corresponding to the target client is used to indicate the viewing perspective of the virtual object controlled by the target client. The viewing perspective can be a first-person perspective, a third-person perspective, or a specified viewing perspective, etc. Among them, in the first-person perspective, the virtual camera is placed inside the virtual object controlled by the client. Its advantage is that it can accurately aim and bring a sense of immersion, and its visual range is limited to the front area of the virtual object. In the third-person perspective, the virtual camera is generally placed behind the virtual object controlled by the client and moves with the controlled virtual object. Its advantage is that the field of view is relatively wide and it is conducive to interacting with the surrounding virtual objects.

[0083] In some embodiments, the viewing perspectives corresponding to the virtual objects in the virtual scene can be preset in advance. Thus, after determining the virtual object controlled by the target virtual scene, the perspective information corresponding to the target client can be obtained accordingly.

[0084] In some embodiments, if the user can change the viewing perspective of the virtual object controlled by the target client in the target client, after the target client detects the trigger operation for changing the viewing perspective, it generates a perspective change instruction and sends the perspective change instruction to the server so that the server can determine the current viewing perspective of the virtual object controlled by the target client.

[0085] In some embodiments, the corresponding relationship between each viewing perspective and the relative position information of the reference point can be set, where the relative position information of the reference point is used to indicate the position of the reference point relative to the target scene area. Thus, the relative position information of the reference point is determined according to the viewing perspective, and then the position of the virtual object controlled by the client in the target virtual scene is used as the reference point, and combined with the size information of the target scene area to determine the position of the target scene area corresponding to each client in the target virtual scene.

[0086] Figure 6 It is a schematic diagram of the target scene area corresponding to the client under different viewing perspectives shown in an embodiment of the present application. If it is set that in the first-person perspective, the position of the reference point relative to the target scene area is at the center of the lower edge of the target scene area; in the third-person perspective, the intersection of the left edge and the lower edge of the reference point relative to the target scene area; if the size of the target scene area is set as a*b, then if the virtual object controlled by a client is located at Figure 6 the position point S4 in, then in the first-person perspective, the position of the target scene area corresponding to the client in the target virtual scene is as shown by the first area 610 surrounded by the dotted line in Figure 6 ; in the third-person perspective, the position of the target scene area corresponding to the client in the target virtual scene is as shown by Figure 6The second region 620 enclosed by the center dash-dot line.

[0087] In some embodiments, since different virtual objects are applicable to different viewing perspectives, a first correspondence between the viewing perspective of a virtual object and the object type to which the virtual object belongs can be set. Furthermore, based on this first correspondence, the viewing perspective corresponding to the virtual object can be determined first according to the object type to which the virtual object belongs, and then the target scene area corresponding to the virtual object can be determined in combination with the viewing perspective corresponding to the virtual object.

[0088] In some embodiments, when the viewing perspectives applicable to a virtual object at different positions are different. For example, if the virtual object is in an indoor environment, the viewing perspective corresponding to the virtual object is set as the first-person perspective, and if the virtual object is in an outdoor environment, the viewing perspective corresponding to the virtual object is set as the third-person perspective. Therefore, a second correspondence between the position and the viewing perspective can be set. Thus, based on this second correspondence, the viewing perspective corresponding to the virtual object can be determined according to the position information of the virtual object in the target virtual scene, and then the target scene area corresponding to the virtual object can be determined in combination with the viewing perspective corresponding to the virtual object and the position information of the virtual object.

[0089] In some embodiments, in order to create an immersive experience, different-sized viewing ranges (i.e., target scene areas) are provided corresponding to the same virtual object at different positions. Thus, based on the third correspondence between the position of the virtual object and the size of the scene area, the size of the target scene area corresponding to the client that controls the virtual object can be determined according to the current position information of the virtual object. Furthermore, the position of the target scene area corresponding to the client that controls the virtual object can be determined in combination with the size of the target scene area and the reference point determined based on the position information of the virtual object.

[0090] Step 430, determine the target scene images corresponding to the respective target clients according to the scene data of the respective target scene areas.

[0091] The target scene area corresponding to the target client is a partial area in the target virtual scene. One or more virtual objects are included in this target scene area. The virtual object can be an active virtual object controlled by the target client or a virtual object that remains stationary in the target virtual scene, such as virtual trees, virtual rocks, virtual buildings, virtual lawns, etc.

[0092] The scene data of the target scene area includes the model data of each virtual object included in the target virtual scene, as well as the rendering data of each virtual object. Among them, the model data of the virtual object includes the coordinate information of each vertex in the virtual object, and the vertex of the virtual object refers to the key point in the model for constructing the virtual object, such as bone key points, etc. The rendering data of the virtual object is used to indicate information such as the color and texture of the virtual object.

[0093] The scene data of the target scene area represents the situation of the virtual objects included in the target virtual scene in the model space. In the target client, it is necessary to present the situation in the target scene area under a two-dimensional display plane (i.e., the display screen in the terminal device where the target client is located). Therefore, it is necessary to convert the virtual objects included in the target scene area into the screen space to obtain a two-dimensional scene image in the screen space, and the obtained two-dimensional scene image in the screen space is the target scene image.

[0094] It can be understood that since there are differences in the target scene areas corresponding to different target clients, the target scene images determined for different target clients are also correspondingly different.

[0095] Step 440: Perform video stream encoding according to the target scene images corresponding to each target client to obtain the video streams of each target client.

[0096] Through the process of the above steps 410 - 430, the target scene images corresponding to each target client at different times can be determined. On this basis, at a set frame rate, the multiple target scene images of each target client at multiple times are encoded into a video stream, so as to obtain the video stream of each target client.

[0097] Step 450: Send the video stream to the corresponding target client.

[0098] Through the above process, each target client in a virtual game can present the corresponding target scene image based on the received video stream.

[0099] In some embodiments, if background music or sound effect stunts are also set for the target virtual scene, the background music and sound effect stunts can also be encoded into an audio stream, and the audio stream and the video streams corresponding to each target client are combined into an audio - video stream, and the audio - video streams corresponding to each target client are sent to the corresponding target client.

[0100] In the solution of the present application, first, according to the position information of the virtual objects controlled by each target client in the virtual game, the target scene areas corresponding to each target client are determined in the virtual scene. Then, according to the scene data of each target scene area, the target scene images corresponding to each target client are determined. Furthermore, video stream encoding is performed according to the target scene images corresponding to each target client to obtain the video streams of each target client. In this process, it is not necessary for all the clients in the virtual game to render the global scene images based on the global scene data of the target virtual scene and then respectively intercept partial images from the global scene images, and there is no need to repeatedly render the target virtual scene. Thus, on the basis of ensuring the display requirements of the scene images on each target client, processing resources are saved, and the problem of waste of processing resources caused by the need to perform image rendering on all the clients in the virtual game in the related art is effectively solved.

[0101] In the related art, since the image rendering is performed by the terminal devices where each target client is located, the processing capabilities of different terminal devices are different, and there are differences in the speed of image rendering, resulting in the out-of-synchronization of the scene images presented by different target clients in the same virtual game. However, the solution of the present application can be executed by the same electronic device to obtain multiple video streams, with each video stream corresponding to a target client, and there is no difference in the rendering progress caused by device differences. Thus, the synchronization of the scene images presented by different target clients in the same virtual game can be effectively ensured.

[0102] Figure 7A It is a flowchart of a method for encoding a scene image according to an embodiment of the present application. As Figure 7A shown, it is assumed that the target clients participating in the virtual game include Client I, Client II, and Client III. The target scene area determined for Client I in the target virtual scene is Scene Area C1, the target scene area determined for Client II is Scene Area C2, and the target scene area determined for Client III is Scene Area C3.

[0103] After that, image rendering is performed based on Scene Area C1 to determine the target scene image D1, image rendering is performed based on Scene Area C2 to determine the target scene image D2, and image rendering is performed based on Scene Area C2 to determine the target scene image D2. Then, video stream encoding is performed according to the target scene image D1 corresponding to Client I to obtain the video stream E1; video stream encoding is performed according to the target scene image D2 corresponding to Client II to obtain the video stream E2; video stream encoding is performed according to the target scene image D3 corresponding to Client III to obtain the video stream E3. Finally, the video stream E1 is sent to Client I, the video stream E2 is sent to Client II, and the video stream E3 is sent to Client III.

[0104] In this way, for Client I, Client II, and Client III, they do not need to perform screen rendering but only need to play the video stream. For the server, since it encodes the target scene areas corresponding to each target client and does not need to perform global screen rendering on the target virtual scene, but only performs screen rendering on the target scene areas to be displayed in the target client, thus saving the server's processing resources and improving the screen rendering efficiency.

[0105] The solution of this application can be applied to cloud games. Thus, the cloud game server encodes the video stream of each game client in the game session according to the method of this application, ensuring the synchronization of the game screens presented by the game clients in the same game session. In the cloud game scenario, adopting this solution can reduce the number of hardware devices used for screen rendering and also reduce the amount of repeated calculations. Taking a 3D game as an example, originally multiple player devices needed to run the rendering pipeline separately. Using this solution, it can be achieved that the rendering pipeline only runs once on one device, reducing the rendering hardware requirements for terminal devices and avoiding repeated screen rendering.

[0106] Figure 7B is an architecture diagram of a cloud game system shown according to an embodiment of this application, as Figure 7B shown, the cloud game system includes a cloud game server 711 and a game terminal 712 that are communicatively connected.

[0107] The game terminal 712 can be an electronic device that supports running an application for cloud games. The electronic device can be, for example, a smart TV, a smartphone, a tablet computer, a laptop computer, a desktop computer, a vehicle-mounted terminal, etc. During the process of running the application for cloud games on the game terminal 712, the game terminal 712 can be regarded as a game client correspondingly. In the cloud game scenario, the game terminal 712 does not need to perform complex screen rendering. After receiving the video stream sent by the cloud game server 711, it only needs to decode the video stream and display the game screen.

[0108] Based on the game screen displayed on the game terminal 712, the user can trigger game control operations in the game interface or trigger game control operations through an input device (such as a game controller) communicatively connected to the game terminal. The game terminal 712 generates game control instructions according to the detected game control operations. The game control instructions can be, for example, control instructions for controlling a virtual object to move forward, lie down, release skills, etc., and send the game control instructions to the cloud game server 711.

[0109] The cloud game server 711 is a cloud server that provides cloud game services. The cloud game services include, for example, game account management services, cloud game deployment services, screen rendering services, game logic services, video stream encoding services, etc.

[0110] After the cloud game server 711 receives the game control instruction sent by the game terminal 712, it executes the game control logic according to the game control instruction, and correspondingly updates the game scene data. Then, it executes rendering calculation and video stream encoding correspondingly to obtain a video stream, and sends the video stream to the game terminal 712. In each game session of the cloud game that requires at least two game terminals to participate, the cloud game server 711 can generate the video streams of each game client in the game session according to the encoding method of the scene picture provided in this application.

[0111] Figure 7C It is an interaction timing diagram in the cloud game scenario shown according to an embodiment of this application. In Figure 7C In the corresponding embodiment, it is assumed that the clients joining the game session include a first game client and a second game client. Of course, in other embodiments, there can be more clients joining the game session. During the progress of the game session, the game client generates a game control instruction according to the detected game control operation, and sends the game control instruction to the cloud game server.

[0112] In this embodiment, for the convenience of description, the game control instruction generated by the first game client is called the first game control instruction, and the game control instruction generated by the second game client is called the second game control instruction. After generating the game control instruction, the first game client executes step 721: sending the first game control instruction, and the second game client executes step 722, sending the second game control instruction.

[0113] After the cloud game server receives the game control instruction (the first game control instruction and / or the second game control instruction), it encodes the game scene picture according to the following steps 723-727.

[0114] Step 723, according to the game control instruction, execute the game control logic. Among them, after executing the game control logic, correspondingly update the target game scene corresponding to the game session joined by the first game client and the second game client. For example, if the first control instruction is an instruction to indicate that the virtual object Q lies down, the cloud game server correspondingly controls the virtual object Q to lie down in the game scene of the game session.

[0115] Step 724, obtain the position information of the virtual objects respectively controlled by each game client in the target game scene. The target cloud game scene refers to the game scene corresponding to the game session currently joined by the first game client and the second game client.

[0116] Step 725: Determine the target game scene areas corresponding to each game client in the target game scene. Specifically, through step 725, determine the target game scene area corresponding to the first game client and the target game scene corresponding to the second game client. Herein, the target game scene area corresponding to each game client is a partial scene area in the target game scene.

[0117] Step 726: Render the scene data of the target scene areas corresponding to each game client to determine the game scene images corresponding to each game client.

[0118] Step 727: Perform video stream encoding based on the game scene images corresponding to each game client to obtain the video streams of each game client. In this embodiment, for the convenience of distinction, the video stream obtained for the first game client is referred to as the first video stream, and the video stream obtained for the second game client is referred to as the second video stream.

[0119] Step 728: Send the first video stream. After that, after the first game client receives the first video stream sent by the cloud game server, perform steps 730 and 731. Herein, step 730: Decode the first video stream; step 731: Display the game scene image corresponding to the first game client.

[0120] Step 729: Send the second video stream. After that, after the second game client receives the second video stream sent by the cloud game server, perform steps 732 and 733. Herein, step 732: Decode the second video stream; step 733: Display the game scene image corresponding to the second game client.

[0121] In Figure 7C the embodiments, for the specific implementation processes of the cloud game server to determine the target game scene areas, game scene images, and video stream encoding corresponding to each game client, refer to the descriptions above and below, and will not be elaborated herein.

[0122] In the cloud game scene of this embodiment, in a game session participated by multiple game clients, the cloud game server first determines the target game scene areas of each game client in the game session, and then renders the scene data of each target game scene area to obtain the game scene images of each game client. In this process, it is not necessary for the cloud game server to perform global scene rendering based on the global scene data of the target game scene, but to perform local game scene rendering based on the target game scene areas corresponding to each game client. Thus, the rendering calculation amount of the cloud game server is reduced, the efficiency of scene rendering is improved, and correspondingly, the encoding efficiency of the game scene images is improved.

[0123] In the case where the cloud game is a 3D game, since the viewing perspectives corresponding to different game clients are different, if the cloud game server performs global screen rendering based on the global scene data, it is necessary to perform global screen rendering multiple times according to the viewing perspectives corresponding to different game clients. By adopting the method of the present application, the screen rendering of the local game scene is performed for the viewing perspective of each game client, greatly reducing the rendering calculation amount of the cloud game server.

[0124] In some embodiments, if the target virtual scene is a three-dimensional scene, the screen rendering can be performed according to the Figure 8 process shown, as Figure 8 shown, which specifically includes three stages, namely: the application stage 810, the geometry processing stage 820, and the rasterization processing stage 830.

[0125] In the application stage 810, the scene data corresponding to each target scene area is loaded into the video memory, and the rendering state (such as the state of materials, textures, shaders, etc.) is set to define how the meshes in the virtual objects in the corresponding target scene area should be rendered, and then the GPU pipeline is called to prepare for the subsequent geometry processing stage 820 and rasterization processing stage 830.

[0126] In the geometry processing stage 820, the virtual objects are transformed from the model space to the screen space.

[0127] Rasterization is a process of converting geometric primitives into two-dimensional images. In the rasterization processing stage 830, the coordinates of the vertices obtained in the geometry processing stage in the screen space are calculated for the covered pixels, and pixel shading is performed correspondingly.

[0128] Combined with this solution, if the target virtual scene is a three-dimensional scene; as Figure 9 shown, step 430 includes: step 910, obtaining the scene data of each target scene area; the scene data includes model data and drawing data.

[0129] As described above, the model data is used to indicate the coordinate information of each vertex in the virtual object. The vertex of the virtual object refers to the key points in the model for constructing the virtual object, such as bone key points, etc. In the solution of the present application, for the sake of distinction, the coordinate information of each vertex in the model space indicated by the model data is referred to as the first coordinate information. The drawing data of the virtual object is used to indicate information such as the color and texture of the virtual object.

[0130] Step 920, performing geometry processing according to the model data of each target scene area to obtain the intermediate scene images of each target scene area in the screen space.

[0131] As described above, geometric processing transforms virtual objects from model space to screen space. In the solution of this application, the scene picture obtained by transforming each virtual object included in the target scene area from model space to screen space before shading processing is called an intermediate scene picture.

[0132] In the geometric processing stage, the following processes are involved: 1) Model transformation from model space to world space;

[0133] 2) View transformation from world space to camera space; 3) Projection transformation from camera space to homogeneous clip space; 4) Screen mapping from homogeneous clip space to screen space. Among them, model space refers to the space under the model coordinate system set when constructing the model of the virtual object. Camera space refers to the coordinate system space constructed with the position of the virtual camera as the origin.

[0134] Figure 10A is a flowchart of geometric processing shown according to an embodiment of the present application. As Figure 10A shown, it includes: after transforming the model of the virtual object from model space to world space, according to the virtual cameras corresponding to the virtual objects controlled by each target client, transforming the corresponding target scene area into the camera space (such as Figure 10A camera space 1, camera space 2... camera space n) where the corresponding virtual camera is located, and then performing projection, clipping, and screen mapping in sequence.

[0135] The scene picture presented by the client can be understood as the picture obtained by shooting the virtual scene through the virtual camera corresponding to the client. The set positions of the virtual cameras corresponding to different clients may be different. Correspondingly, the camera spaces corresponding to each client are also different.

[0136] In camera space, looking from the camera anchor point of the virtual camera at the virtual object, a projection transformation frustum can be determined correspondingly. The projection transformation frustum is the area that determines what the virtual camera can see and is composed of 6 planes. These 6 planes are called clipping planes, and the clipping planes include a near clipping plane and a far clipping plane, which determine the depth range that the virtual camera can see. Figure 10B is a schematic diagram showing a projection transformation frustum. As Figure 10B shown, the coordinate origin in it is the anchor point of the virtual camera. Figure 10B The plane T1 perpendicular to the z-axis in it is the near clipping plane, and the plane T2 perpendicular to the z-axis is the far clipping plane.

[0137] After that, for the convenience of calculation, the virtual objects in the projection transformation frustum are transformed into the homogeneous clip space. Transforming the virtual objects from the camera space to the homogeneous clip space is the projection process shown in Figure 10. In the homogeneous clip space, based on Figure 10B the projection transformation frustum in it, clipping is performed to clip off the vertices that are not within the projection transformation frustum; after that, the vertices retained after clipping are screen-mapped to the screen space to obtain the corresponding pixel positions.

[0138] In some embodiments, the model data includes the first coordinate information of each vertex in the corresponding target scene area; as Figure 11 shown, step 920 includes:

[0139] Step 1110, according to the first coordinate information of each vertex in the model data of each target scene area, transform each vertex from the model space to the camera space under the corresponding virtual camera to obtain the second coordinate information of each vertex in the target scene area.

[0140] Specifically, in step 1110, first transform each vertex in the model data from the model space to the world space to obtain the coordinates of each vertex in the world space. In this process, it can be transformed to the world space through the way of scaling, rotating and then translating; after that, then transform each vertex from the world space to the camera space under the corresponding virtual camera to obtain the second coordinate information of each vertex in the target scene area under the corresponding camera space. In this process, the coordinate information of each vertex in the camera space (i.e., the second coordinate information) can be obtained successively through translation, rotation, scaling and negation.

[0141] Step 1120, according to the second coordinate information of each vertex in the target scene area, transform each vertex from the corresponding camera space to the homogeneous clip space to obtain the third coordinate information of each vertex in the target scene area.

[0142] In a specific embodiment, each vertex can be transformed from the corresponding camera space to the homogeneous clip space through a perspective projection matrix or an orthographic projection matrix to obtain the coordinate information of each vertex in the homogeneous clip space (i.e., the third coordinate information).

[0143] Step 1130, based on the third coordinate information of each vertex in the target scene area, transform each vertex from the homogeneous clip space to the screen space to obtain the fourth coordinate information of each vertex in the target scene area.

[0144] Step 1140, according to the fourth coordinate information of all vertices in each target scene area, determine the intermediate scene picture of each target scene area in the screen space.

[0145] The coordinates indicated by the fourth coordinate information are the pixel positions corresponding to the vertices in the screen space. Therefore, by combining pixels based on the fourth coordinate information of each vertex, the intermediate scene image of the target scene area in the screen space is obtained.

[0146] Please continue to refer to Figure 9 , step 930, perform rasterization processing based on the intermediate scene images of each target scene area in the screen space and the rendering data corresponding to each target scene area to obtain the target scene images corresponding to each target client.

[0147] In some embodiments, step 930 includes: for each target scene area, perform rasterization processing on the intermediate scene image corresponding to the target scene area according to the rendering data corresponding to the target scene area to obtain the target scene image corresponding to the target client.

[0148] In some embodiments, the intermediate scene image indicates the fourth coordinate information of each vertex in the screen space of the corresponding target scene area; in this embodiment, for each target scene area, the step of performing rasterization processing on the intermediate scene image corresponding to the target scene area according to the rendering data corresponding to the target scene area to obtain the target scene image corresponding to the target client further includes: for each target scene area, determine a plurality of fragments in the target scene area according to the fourth coordinate information of each vertex in the target scene area; color and render the plurality of fragments in the target scene area according to the rendering data corresponding to the target scene area to obtain the target scene image corresponding to the target client.

[0149] Specifically, according to the fourth coordinate information of each vertex in the target scene area, based on the coordinate range of the set triangular mesh, check whether each vertex is covered by the triangular mesh. If it is covered, generate a fragment, and use the vertex coordinates of the three vertices in the triangular mesh to interpolate the entire covered area, and then output a sequence of fragments in the target scene area.

[0150] In some embodiments, the fragments can be colored and rendered through a fragment shader. Specifically, since the rendering data indicates information such as color and texture, the state of the fragment shader (such as setting color, texture, etc.) can be set first based on the rendering data, and then the fragments can be colored one by one through the fragment shader.

[0151] In this embodiment, the intermediate scene image corresponding to each target scene area is rasterized once respectively. Thus, the intermediate scene images corresponding to all clients under the same virtual game can be rasterized synchronously, improving the rasterization processing efficiency.

[0152] In some other embodiments of the present application, step 930 includes: splicing the intermediate scene images corresponding to all target scene regions in the target virtual scene to obtain a spliced image; rasterizing the spliced image according to the rendering data corresponding to all target scene regions to obtain a spliced scene image; and determining the target scene images corresponding to each target client in the spliced scene image according to the position information of the intermediate scene images corresponding to each target client in the spliced image.

[0153] In this embodiment, before rasterization, first splice the intermediate scene images corresponding to all target scene regions respectively to obtain a spliced image, and then rasterize the spliced image. Thus, the rasterization of the spliced image can be performed through one rasterization process, rather than through multiple rasterization processes.

[0154] Figure 12 It is a flowchart showing a method for encoding a scene image according to an embodiment of the present application, as Figure 12 shown, including: step 1210, determining the target scene regions corresponding to each target client;

[0155] Step 1220, performing geometric processing on each target scene region; intermediate scene images corresponding to the respective target scene regions can be obtained accordingly; for the specific process of geometric processing, refer to the above description and will not be elaborated here.

[0156] Step 1230, splicing the intermediate scene images; a spliced image can be obtained accordingly.

[0157] Step 1240, performing coloring rendering on the spliced image; this coloring rendering process is the rasterization process described above. The spliced scene image is obtained through step 1240.

[0158] Step 1250, performing ROI encoding on the spliced scene image; in this encoding process, the region of interest in the spliced scene image is the region where the target scene image corresponding to the target client in the spliced scene image is located. It can be understood that the region where the intermediate scene image is located in the spliced scene image is the region where the target scene image corresponding to the target client is located in the spliced scene image. Therefore, the target scene images corresponding to each target client can be determined in the spliced scene image according to the position information of the intermediate scene images corresponding to each target client in the spliced image. Thus, through step 1250, the target scene images corresponding to each target client in the same virtual game session can be obtained respectively, such as Figure 12 the target scene image D1, the target scene image D2... the target scene image Dn, etc.

[0159] Step 1260, video stream encoding. In this process, video stream encoding is respectively performed based on the target scene images corresponding to each target client, and the video streams corresponding to each target client are obtained accordingly. For example, Figure 12 the video streams E1, E2,..., En in

[0160] In some other embodiments of the present application, the target virtual scene is a two-dimensional scene; the scene data includes background data and texture data of the virtual objects controlled by the corresponding target clients. In this embodiment, step 430 includes: generating background texture maps for each target scene area according to the background data of each target scene area; performing texture mapping processing in the background texture maps of each target scene area according to the texture data of the virtual objects controlled by each target client, and correspondingly obtaining the target scene images corresponding to each target client.

[0161] For a two-dimensional target virtual scene, it does not involve the transformation from a three-dimensional model space to a two-dimensional screen space. Therefore, the corresponding two-dimensional images can be directly generated based on the corresponding scene data. In this embodiment, the scene data of the two-dimensional target scene includes background data and texture data of the virtual objects controlled by the corresponding target clients. The background data of each target scene area is used to generate the background texture map corresponding to the background scene area, and the texture data of the virtual objects controlled by each target client is used to generate the virtual object texture maps of the virtual objects controlled by the corresponding target clients. Thus, according to the position information of the virtual objects controlled by each target client in the target virtual scene, the virtual object texture maps of the virtual objects controlled by the target client are correspondingly combined into the background texture map of the target scene area corresponding to the target client, that is, the target scene images corresponding to each target client are obtained.

[0162] It can be understood that in the virtual scene, virtual objects serving as the background, such as virtual buildings, virtual trees, virtual rocks, etc., basically remain unchanged in position during the progress of the virtual game, while the virtual objects controlled by the client are movable. Therefore, in this embodiment, the data of the virtual objects serving as the background and the movable virtual objects in the target virtual scene are distinguished, so as to facilitate the subsequent combination of the background texture map and the texture map of the movable virtual objects, that is, the texture mapping processing.

[0163] In some embodiments, when the target virtual scene is a two-dimensional scene, the method further includes: obtaining the size information of the target virtual scene; if the size indicated by the size information exceeds the set size threshold, then execute step 420. Then, correspondingly, according to the process in the above embodiment, the corresponding target scene images are generated for the scene data of each target scene area.

[0164] When the size of the two-dimensional target virtual scene is large, for example, when the length of the two-dimensional target virtual scene exceeds the set length threshold and the width of the target virtual scene exceeds the set width threshold, there are many scene areas in the target virtual scene that do not need to be presented on the target client. Therefore, in this case, in order to reduce the processing volume, the corresponding target scene area is determined for each target client, and then the target scene pictures of each target client are generated accordingly.

[0165] In some embodiments, in the case where the target virtual scene is a two-dimensional scene, after the step of obtaining the size information of the target virtual scene, as Figure 13 shown, the method further includes:

[0166] Step 1310, if the size indicated by the size information does not exceed the set size threshold, obtain the global background map of the target virtual scene.

[0167] Step 1320, according to the position information of the virtual objects controlled by each target client in the target virtual scene, perform mapping processing on the object maps of the virtual objects controlled by all target clients in the global background map to obtain the global scene picture.

[0168] Step 1330, according to the position information of the virtual objects controlled by each target client and the preset picture size information, determine the target scene pictures corresponding to each target client in the global scene picture. The preset picture size information is used to indicate the size of the scene area presented on the picture displayed by the client.

[0169] Step 1340, perform video stream encoding according to the target scene pictures corresponding to each target client in the global scene picture to obtain the video streams corresponding to each target client.

[0170] In step 1340, ROI encoding can be performed on the global scene picture. During the ROI encoding process, the area of interest is the area where the target scene picture corresponding to the target client is located, so as to obtain the video stream corresponding to each target client accordingly.

[0171] In this embodiment, since the size of the two-dimensional target virtual scene is small, for example, the length of the two-dimensional target virtual scene does not exceed a set length threshold, and the width of the target virtual scene does not exceed a set width threshold. Even though the scene areas displayed on each target client are only partial areas in the target virtual scene, overall, the scene areas in the target virtual scene that are not displayed on the target client are few. In this case, compared with the processing amount of separately determining the target scene area for each target client and then determining the corresponding target scene picture, directly generating a global picture based on the panoramic scene data consumes less computing power in determining the target scene picture corresponding to each target client from the global picture. Therefore, the target scene picture corresponding to each target client can be determined in the global scene picture in the manner of this embodiment.

[0172] Figure 14 It is a flowchart of scene picture encoding shown according to another embodiment of the present application. Figure 14 The embodiment can be applied to a two-dimensional game scene. As Figure 14 shown, the method of the present application can be executed by a cloud game server. Specifically, after the cloud game server executes the game logic, it first generates a global background texture map of the target game scene based on the background scene data of the target game scene; then, it combines the game character texture maps (such as Figure 14 the game character 1 texture map, game character 2 texture map... game character n texture map) controlled by the game characters on each player's client into the global background texture map to obtain a global scene picture, and then performs ROI encoding based on the global background texture map to obtain the target scene picture corresponding to each target client (such as Figure 14 the target scene picture P1, target scene picture P2... target scene picture Pn in Figure 14 ); then, video stream encoding is performed according to the target scene picture corresponding to each target client to obtain the video stream of each target client (such as in

[0173] the video stream 1 corresponding to the client where player 1 is located, the video stream 2 corresponding to the client where player 2 is located...), and then the video stream is sent to the corresponding player.

[0173] Figure 15 It is a flowchart of scene picture encoding shown according to another embodiment of the present application. Compared with Figure 14 the embodiment shown in Figure 15 In the embodiment of Figure 15Among them, the video stream 1 corresponding to the client where player 1 is located, the video stream 2 corresponding to the client where player 2 is located...). The rendering device in the local area network can be a desktop computer, a game console, etc., which will not be specifically limited here.

[0174] In some embodiments, step 1340 includes: determining, according to the target scene pictures corresponding to each target client, the overlapping picture area and the non-overlapping picture area of each target scene picture relative to other target scene pictures; respectively encoding the overlapping picture area and the non-overlapping picture area; combining the encoded information of the overlapping picture area and the encoded information of the non-overlapping picture area in each target scene picture to obtain the encoded information of the target scene picture corresponding to each target client; generating a video stream corresponding to each target client according to the encoded information of the target scene picture corresponding to each target client.

[0175] The overlapping picture area refers to the picture area in a target scene picture that overlaps with other target scene pictures, where the overlapping picture area can be the picture area where two or more target scene pictures overlap. The non-overlapping picture area is the picture area in a target picture area that does not overlap with other target picture areas.

[0176] In this embodiment, for each target scene picture, the overlapping picture area and the non-overlapping picture area of the target scene picture relative to other target scene areas are determined. Thus, in the process of encoding the target scene picture, the encoded information of the overlapping picture area can be shared, and there is no need to repeatedly encode the overlapping picture area, thereby improving the encoding efficiency of the picture.

[0177] In some embodiments, after determining the overlapping picture area, the size information of the overlapping picture area can be further determined. If the size information of the overlapping picture area is greater than the set second size threshold, the overlapping picture area and the non-overlapping picture area are respectively encoded for pictures to share the encoded information of the overlapping picture area; otherwise, if the size information of the overlapping picture area is not greater than the set second size threshold, the picture is still encoded according to the target scene picture, and the overlapping picture area and the non-overlapping picture area are not respectively encoded for pictures. When the size information of the overlapping picture area is not greater than the set second size threshold, it indicates that the overlapping picture area is small. In this case, the improvement in encoding efficiency brought by sharing the encoded information of the overlapping picture area may not be significant. Therefore, in this case, the overlapping picture area and the non-overlapping picture area can be not respectively encoded for pictures.

[0178] The following introduces the device embodiments of the present application, which can be used to execute the methods in the above embodiments of the present application. For the details not disclosed in the device embodiments of the present application, please refer to the above method embodiments of the present application.

[0179] Figure 16 is a block diagram of an encoding device for a scene picture shown according to an embodiment, as Figure 16 shown. The encoding device for the scene picture includes: a position information acquisition module 1610, configured to acquire the position information of virtual objects manipulated by each target client in a target virtual scene, where the target client refers to a client that joins a virtual game, and the target virtual scene refers to the virtual scene corresponding to the virtual game; a target scene area determination module 1620, configured to determine the target scene area corresponding to each target client in the target virtual scene according to the position information of the virtual objects manipulated by each target client; a target scene picture determination module 1630, configured to determine the target scene picture corresponding to each target client according to the scene data of each target scene area; a video stream encoding module 1640, configured to perform video stream encoding on the target scene pictures corresponding to each target client to obtain the video streams of each target client; and a video stream sending module 1650, configured to send the video streams to the corresponding target clients.

[0180] In some embodiments of the present application, the target virtual scene is a three-dimensional scene; in this embodiment, the target scene area determination module 1620 includes: a viewing angle information acquisition unit, configured to acquire the viewing angle information corresponding to each target client; and a target scene area determination unit, configured to determine the target scene area corresponding to each target client in the target virtual scene according to the viewing angle information corresponding to each target client and the position information of the virtual objects manipulated by each target client.

[0181] In some embodiments of the present application, the target virtual scene is a three-dimensional scene; in this embodiment, the target scene picture determination module 1630 includes: a scene data acquisition unit, configured to acquire the scene data of each target scene area; the scene data includes model data and rendering data; a geometry processing unit, configured to perform geometry processing on the model data of each target scene area to obtain an intermediate scene picture of each target scene area in screen space; and a rasterization processing unit, configured to perform rasterization processing on the intermediate scene picture of each target scene area in screen space and the rendering data corresponding to each target scene area to obtain the target scene picture corresponding to each target client.

[0182] In some embodiments of the present application, the model data includes the first coordinate information of each vertex in the corresponding target scene area; the target scene image determination module 1630 includes: a first transformation unit, configured to transform each vertex from the model space to the camera space under the corresponding virtual camera according to the first coordinate information of each vertex in the model data of each target scene area, to obtain the second coordinate information of each vertex in the target scene area; a second transformation unit, configured to transform each vertex from the corresponding camera space to the homogeneous clipping space according to the second coordinate information of each vertex in the target scene area, to obtain the third coordinate information of each vertex in the target scene area; a third transformation unit, configured to transform each vertex from the homogeneous clipping space to the screen space based on the third coordinate information of each vertex in the target scene area, to obtain the fourth coordinate information of each vertex in the target scene area; an intermediate scene image generation unit, configured to determine the intermediate scene image of each target scene area in the screen space according to the fourth coordinate information of all vertices in each target scene area.

[0183] In some embodiments of the present application, the rasterization processing unit includes: a first processing unit, configured to perform rasterization processing on the intermediate scene image corresponding to each target scene area according to the rendering data corresponding to the target scene area, to obtain the target scene image corresponding to the target client.

[0184] In some embodiments of the present application, the intermediate scene image indicates the fourth coordinate information of each vertex in the corresponding target scene area in the screen space; in this embodiment, the first processing unit includes: a fragment determination unit, configured to determine a plurality of fragments in the target scene area according to the fourth coordinate information of each vertex in the target scene area for each target scene area; a shading and rendering unit, configured to perform shading and rendering on the plurality of fragments in the target scene area according to the rendering data corresponding to the target scene area, to obtain the target scene image corresponding to the target client.

[0185] In some embodiments of the present application, the rasterization processing unit includes: a splicing unit, configured to splice the intermediate scene images corresponding to all target scene areas in the target virtual scene to obtain a spliced image; a second processing unit, configured to perform rasterization processing on the spliced image according to the rendering data corresponding to all target scene areas, to obtain a spliced scene image; a determination unit, configured to determine the target scene image corresponding to each target client in the spliced scene image according to the position information of the intermediate scene image corresponding to each target client in the spliced image.

[0186] In some embodiments of the present application, the target virtual scene is a two-dimensional scene; the scene data includes background data and texture data of virtual objects controlled by the corresponding target clients; in this embodiment, the target scene image determination module 1630 includes: a background texture generation unit, configured to generate background textures for each target scene area according to the background data of each target scene area; a texture processing unit, configured to perform texture processing in the background textures of each target scene area according to the texture data of the virtual objects controlled by each target client, and correspondingly obtain the target scene images corresponding to each target client.

[0187] In some embodiments of the present application, the encoding device for the scene image further includes: a size information acquisition module, configured to acquire the size information of the target virtual scene; if the size indicated by the size information exceeds a set size threshold, then transfer to the target scene area determination module 1620.

[0188] In some embodiments of the present application, the encoding device for the scene image further includes: a global background texture acquisition module, configured to acquire the global background texture of the target virtual scene if the size indicated by the size information does not exceed the set size threshold; a texture module, configured to perform texture processing on the object textures of all the virtual objects controlled by each target client in the global background texture according to the position information of the virtual objects controlled by each target client in the target virtual scene, to obtain a global scene image; a scene image area determination module, configured to determine the target scene images corresponding to each target client in the global scene image according to the position information of the virtual objects controlled by each target client and the preset image size information; a video stream generation module, configured to perform video stream encoding according to the target scene images corresponding to each target client in the global scene image, to obtain the video streams corresponding to each target client.

[0189] In some embodiments of the present application, the video stream encoding module includes: an area determination unit, configured to determine the overlapping image area and non-overlapping image area of each target scene image relative to other target scene images according to the target scene images corresponding to each target client; an encoding unit, configured to encode the overlapping image area and non-overlapping image area respectively; a combination unit, configured to combine the encoding information of the overlapping image area and the encoding information of the non-overlapping image area in each target scene image, to obtain the encoding information of the target scene image corresponding to each target client; a video stream generation unit, configured to generate the video stream corresponding to each target client according to the encoding information of the target scene image corresponding to each target client.

[0190] Figure 17 FIG. shows a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application. It should be noted that Figure 17The computer system 1700 of the illustrated electronic device is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0191] As Figure 17 shown, the computer system 1700 includes a processor, such as the Central Processing Unit (CPU) 1701 in Figure 17 it, which can perform various appropriate actions and processes according to the program stored in the Read-Only Memory (ROM) 1702 or the program loaded from the storage section 1708 into the Random Access Memory (RAM) 1703, such as executing the methods in the above embodiments. In the RAM 1703, various programs and data required for system operation are also stored. The CPU 1701, ROM 1702, and RAM 1703 are connected to each other via a bus 1704. The Input / Output (I / O) interface 1705 is also connected to the bus 1704.

[0192] The following components are connected to the I / O interface 1705: an input section 1706 including a keyboard, a mouse, etc.; an output section 1707 including, for example, a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc. and a speaker, etc.; a storage section 1708 including a hard disk, etc.; and a communication section 1709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1709 performs communication processing via a network such as the Internet. A drive 1710 is also connected to the I / O interface 1705 as needed. A removable medium 1711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1710 as needed so that a computer program read from it can be installed into the storage section 1708 as needed.

[0193] Particularly, according to the embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments of the present application include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 1709, and / or installed from the removable medium 1711. When the computer program is executed by the Central Processing Unit (CPU) 1701, various functions defined in the system of the present application are executed.

[0194] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0195] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0196] The units involved in the embodiments of the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the unit itself in some cases.

[0197] On the other hand, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist alone without being assembled into the electronic device. The above computer-readable storage medium carries computer-readable instructions, and when the computer-readable storage instructions are executed by a processor, the method in any of the above embodiments is implemented.

[0198] According to one aspect of the present application, an electronic device is also provided, which includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method in any of the above embodiments is implemented.

[0199] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method in any of the above embodiments.

[0200] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0201] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0202] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include well-known knowledge or conventional technical means in the technical field not disclosed in the present application.

[0203] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A method for encoding a scene picture, characterized in that, it includes: Obtain the position information of the virtual objects controlled by each target client in the target virtual scene, where the target client refers to the client that joins the virtual game, and the target virtual scene refers to the virtual scene corresponding to the virtual game; there are at least two target clients; According to the position information of the virtual objects controlled by each target client, determine the target scene area corresponding to each target client in the target virtual scene; According to the scene data of each target scene area, determine the target scene picture corresponding to each target client; Perform video stream encoding according to the target scene pictures corresponding to each target client to obtain the video streams of each target client; Send the video stream to the corresponding target client; If the target virtual scene is a three-dimensional scene, the step of determining the target scene picture corresponding to each target client according to the scene data of each target scene area includes: Obtain the scene data of each target scene area; the scene data includes model data and rendering data; Perform geometric processing according to the model data of each target scene area to obtain the intermediate scene picture of each target scene area in screen space; Perform rasterization processing according to the intermediate scene picture of each target scene area in screen space and the rendering data corresponding to each target scene area to obtain the target scene picture corresponding to each target client, including: splicing the intermediate scene pictures corresponding to all target scene areas in the target virtual scene to obtain a spliced picture; through a rasterization process, rasterize the spliced picture according to the rendering data corresponding to all target scene areas to obtain a spliced scene picture; according to the position information of the intermediate scene picture corresponding to each target client in the spliced picture, determine the target scene picture corresponding to each target client in the spliced scene picture.

2. The method according to claim 1, characterized in that, the target virtual scene is a three-dimensional scene; the step of determining the target scene area corresponding to each target client in the target virtual scene according to the position information of the virtual objects controlled by each target client includes: Obtain the perspective information corresponding to each target client; According to the perspective information corresponding to each target client and the position information of the virtual objects controlled by each target client, determine the target scene area corresponding to each target client in the target virtual scene.

3. The method according to claim 1, characterized in that, the model data includes the first coordinate information of each vertex in the corresponding target scene area; the step of performing geometric processing according to the model data of each target scene area to obtain the intermediate scene picture of each target scene area in screen space includes: According to the first coordinate information of each vertex in the model data of each target scene area, transform each vertex from model space to camera space under the corresponding virtual camera to obtain the second coordinate information of each vertex in the target scene area; According to the second coordinate information of each vertex in the target scene area, transform each vertex from the corresponding camera space to the homogeneous clipping space to obtain the third coordinate information of each vertex in the target scene area; Based on the third coordinate information of each vertex in the target scene area, transform each vertex from the homogeneous clipping space to the screen space to obtain the fourth coordinate information of each vertex in the target scene area; According to the fourth coordinate information of all vertices in each of the target scene areas, determine the intermediate scene picture of each of the target scene areas in the screen space.

4. A coding device for a scene picture, characterized in that, it includes: A position information acquisition module, configured to acquire the position information of the virtual objects manipulated by each target client in the target virtual scene, where the target client refers to the client that joins the virtual game, and the target virtual scene refers to the virtual scene corresponding to the virtual game; there are at least two target clients; A target scene area determination module, configured to determine the target scene area corresponding to each target client in the target virtual scene according to the position information of the virtual objects manipulated by each target client; A target scene picture determination module, configured to determine the target scene picture corresponding to each target client according to the scene data of each target scene area; A video stream encoding module, configured to perform video stream encoding according to the target scene pictures corresponding to each target client to obtain the video streams of each target client; A video stream sending module, configured to send the video stream to the corresponding target client; wherein, if the target virtual scene is a three-dimensional scene, the target scene picture determination module includes: A scene data acquisition unit, configured to acquire the scene data of each target scene area; the scene data includes model data and rendering data; A geometry processing unit, configured to perform geometry processing according to the model data of each target scene area to obtain the intermediate scene picture of each target scene area in the screen space; A rasterization processing unit, configured to perform rasterization processing according to the intermediate scene picture of each target scene area in the screen space and the rendering data corresponding to each target scene area to obtain the target scene picture corresponding to each target client; the rasterization processing unit includes: a splicing unit, configured to splice the intermediate scene pictures corresponding to all target scene areas in the target virtual scene to obtain a spliced picture; a second processing unit, configured to perform rasterization processing on the spliced picture according to the rendering data corresponding to all target scene areas through one rasterization process to obtain a spliced scene picture; a determination unit, configured to determine the target scene picture corresponding to each target client in the spliced scene picture according to the position information of the intermediate scene picture corresponding to each target client in the spliced picture.

5. The device according to claim 4, characterized in that, the target scene area determination module includes: A viewing angle information acquisition unit, configured to acquire the viewing angle information corresponding to each target client; A target scene area determination unit, configured to determine, in the target virtual scene, a target scene area corresponding to each target client according to the perspective information corresponding to each target client and the position information of the virtual object manipulated by each target client.

6. The apparatus according to claim 4, wherein, the model data includes first coordinate information of each vertex in the corresponding target scene area; the geometric processing unit includes: a first transformation unit, configured to transform each vertex from the model space to the camera space under the corresponding virtual camera according to the first coordinate information of each vertex in the model data of each target scene area, so as to obtain second coordinate information of each vertex in the target scene area; a second transformation unit, configured to transform each vertex from the corresponding camera space to the homogeneous clipping space according to the second coordinate information of each vertex in the target scene area, so as to obtain third coordinate information of each vertex in the target scene area; a third transformation unit, configured to transform each vertex from the homogeneous clipping space to the screen space based on the third coordinate information of each vertex in the target scene area, so as to obtain fourth coordinate information of each vertex in the target scene area; an intermediate scene image generation unit, configured to determine an intermediate scene image of each target scene area in the screen space according to the fourth coordinate information of all vertices in each target scene area.

7. An electronic device, wherein, it includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method according to any one of claims 1-3 is implemented.

8. A computer-readable storage medium, on which computer-readable instructions are stored, wherein, when the computer-readable instructions are executed by a processor, the method according to any one of claims 1-3 is implemented.

9. A computer program product, including computer instructions, wherein, when the computer instructions are executed by a processor, the method according to any one of claims 1-3 is implemented.

Images