Picture generation method, apparatus, device, and medium

Abstract

The application discloses a picture generation method and device, equipment and medium, and relates to the field of image processing. The method comprises the following steps: rendering a three-dimensional virtual scene to obtain a scene background rendering picture; rendering a first foreground target reported by a first client to obtain a first foreground target rendering picture; mixing the scene background rendering picture and the first foreground target rendering picture to obtain a first picture, wherein the first picture comprises the first foreground target displayed in the three-dimensional virtual scene; and providing the first picture to the first client. The application realizes the technology of thousands of people with thousands of faces, and can improve the rendering efficiency and save the cost.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, apparatus, device and medium for generating a picture. Background Art

[0002] Cloud gaming refers to gaming based on cloud computing. A server runs the game and sends the rendered game screen to the client, which is primarily responsible for displaying the game screen.

[0003] In somatosensory cloud games, the relevant technology requires the camera to capture user images, and the client sends the user images to the server. After receiving the user images, the server will process the user images and generate a corresponding user model. The server then places the user model in a three-dimensional virtual environment, renders the user model and the three-dimensional virtual environment as a whole, obtains the game screen, and sends the game screen to the client, which displays the game screen.

[0004] However, when there are multiple clients playing cloud games in a virtual room, the relevant technology will repeatedly render the three-dimensional virtual environment, resulting in low rendering efficiency. Summary of the Invention

[0005] The embodiments of the present application provide a method, apparatus, device, and storage medium for generating a picture, which only requires rendering a three-dimensional virtual environment once, has high rendering efficiency, and saves costs. The technical solution is as follows:

[0006] According to one aspect of the present application, a screen generation method is provided, which is applied to a server and includes:

[0007] Rendering the three-dimensional virtual scene to obtain a scene background rendering;

[0008] Rendering the first foreground object reported by the first client to obtain a rendering image of the first foreground object;

[0009] Mixing the scene background rendering image with the first foreground object rendering image to obtain a first picture, wherein the first picture includes the first foreground object displayed in the three-dimensional virtual scene;

[0010] The first picture is provided to the first client.

[0011] According to one aspect of the present application, a screen generation device is provided, the device comprising:

[0012] A rendering module is used to render a three-dimensional virtual scene and obtain a scene background rendering;

[0013] The rendering module is configured to render the first foreground target reported by the first client to obtain a first foreground target rendering image.

[0014] a mixing module, configured to mix the scene background rendering image with the first foreground object rendering image to obtain a first picture, wherein the first picture includes the first foreground object displayed in the three-dimensional virtual scene;

[0015] A transceiver module is configured to provide the first picture to the first client.

[0016] According to another aspect of the present application, an interactive method is provided, which is applied to at least two clients and includes:

[0017] The first client captures a first foreground object through a first camera; the second client captures a second foreground object through a second camera;

[0018] The first client displays a first image of the three-dimensional virtual scene, the first image including the first foreground object displayed in the three-dimensional virtual scene; the second client displays a second image of the three-dimensional virtual scene, the second image including the second foreground object displayed in the three-dimensional virtual scene;

[0019] The foreground objects in the first picture and the second picture are different, the scene backgrounds in the first picture and the second picture are the same, and the first foreground object and the second foreground object move in the same three-dimensional virtual scene.

[0020] According to another aspect of the present application, an interactive system is provided, the system comprising at least two interactive devices, the system comprising:

[0021] The first interactive device captures a first foreground object through a first camera; the second interactive device captures a second foreground object through a second camera;

[0022] The first interactive device displays a first image of the three-dimensional virtual scene, the first image including the first foreground object displayed in the three-dimensional virtual scene; the second interactive device displays a second image of the three-dimensional virtual scene, the second image including the second foreground object displayed in the three-dimensional virtual scene;

[0023] The foreground objects in the first picture and the second picture are different, the scene backgrounds in the first picture and the second picture are the same, and the first foreground object and the second foreground object move in the same three-dimensional virtual scene.

[0024] According to another aspect of the present application, a computer device is provided, comprising: a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, a code set, or an instruction set is loaded and executed by the processor to implement the screen generation method described above, or the interactive method described above.

[0025] According to another aspect of the present application, a computer storage medium is provided, in which at least one program code is stored. The program code is loaded and executed by a processor to implement the screen generation method described above, or the interactive method described above.

[0026] According to another aspect of the present application, a computer program product or computer program is provided. The computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the screen generation method or the interactive method described above.

[0027] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0028] To generate an image, the 3D virtual scene only needs to be rendered once to obtain a background rendering. This rendering is then blended with the first foreground rendering to create the final image. Because the 3D virtual scene is rendered only once throughout the entire process, this achieves the goal of achieving personalized experiences while reducing the number of rendering cycles. This results in lower performance overhead, a higher throughput, higher rendering efficiency, and cost savings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] Figure 1 is a structural block diagram of a computer system provided by an exemplary embodiment of the present application;

[0031] Figure 2 This is a flowchart of a screen generation method provided by an exemplary embodiment of the present application;

[0032] Figure 3 is a schematic diagram of a three-dimensional virtual environment provided by an exemplary embodiment of the present application;

[0033] Figure 4 is a schematic diagram of a first foreground target provided by an exemplary embodiment of the present application;

[0034] Figure 5 is a schematic diagram of a second foreground target provided by an exemplary embodiment of the present application;

[0035] Figure 6 is a schematic diagram of a first screen provided by an exemplary embodiment of the present application;

[0036] Figure 7 is a schematic diagram of a second screen provided by an exemplary embodiment of the present application;

[0037] Figure 8 This is a flowchart of a method for generating a cloud game screen provided by an exemplary embodiment of the present application;

[0038] Figure 9 This is a flowchart of a screen generation method provided by an exemplary embodiment of the present application;

[0039] Figure 10 is a schematic diagram of a scene background depth rendering provided by an exemplary embodiment of the present application;

[0040] Figure 11 is a schematic diagram of a first depth information rendering provided by an exemplary embodiment of the present application;

[0041] Figure 12 is a schematic diagram of a second depth information rendering provided by an exemplary embodiment of the present application;

[0042] Figure 13 This is a flowchart of a screen generation method provided by an exemplary embodiment of the present application;

[0043] Figure 14 is a schematic diagram of a method for calculating pixel color provided by an exemplary embodiment of the present application;

[0044] Figure 15 is a flowchart of an interactive method provided by an exemplary embodiment of the present application;

[0045] Figure 16 This is a flowchart of a live interactive method provided by an exemplary embodiment of the present application;

[0046] Figure 17 is a schematic diagram of a first screen provided by an exemplary embodiment of the present application;

[0047] Figure 18 is a schematic diagram of a second screen provided by an exemplary embodiment of the present application;

[0048] Figure 19 is a schematic diagram of a third screen provided by an exemplary embodiment of the present application;

[0049] Figure 20 This is a flowchart of a game battle method provided by an exemplary embodiment of the present application;

[0050] Figure 21 This is a flowchart of an online conference method provided by an exemplary embodiment of the present application;

[0051] Figure 22 This is a structural block diagram of a screen generating device provided by an exemplary embodiment of the present application;

[0052] Figure 23 is a structural block diagram of an interactive system provided by an exemplary embodiment of the present application;

[0053] Figure 24 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0055] First, the nouns involved in the embodiments of this application are introduced:

[0056] Three-dimensional virtual scene: a virtual world composed of models, textures and other resources used in program development.

[0057] Depth: used to describe the distance between an object and the camera. The farther the distance, the greater the depth.

[0058] Rendering: The process of converting a three-dimensional virtual scene into a computer screen using program code logic, combined with a game engine, CPU (Central Processing Unit), and graphics card.

[0059] Cloud gaming: The game logic runs on the server side. Users only need to have the most basic cloud gaming client, which only needs to receive video streams and send user input data. Users can experience high-quality games on low-configuration devices.

[0060] Thousands of personalized technology: This means that multiple clients connected to the server display the same background scene but different foreground objects. For example, a first client and a second client are connected to the same server. The first client displays the first image, which includes a forest background and the foreground object of User A's avatar. The second client displays the second image, which also includes the forest background and the foreground object of User B's avatar.

[0061] Game Engine: A game engine is a pre-programmed, editable computer game system or the core component of some interactive real-time graphics applications. These systems provide technicians with the tools necessary to create games, allowing them to easily and quickly create game programs.

[0062] UV (U stands for horizontal direction, V stands for vertical direction): Also known as UV coordinates, UV coordinates refer to the fact that all images are two-dimensional. Through this flat, two-dimensional UV coordinate system, any pixel in the image can be located.

[0063] Texture: In this application, texture refers to a two-dimensional image in the game engine.

[0064] Figure 1 FIG. 1 is a block diagram of a computer system according to an exemplary embodiment of the present application. The computer system 100 includes a terminal 120 and a server 140 .

[0065] The terminal 120 includes at least one terminal. A client related to a three-dimensional virtual scene is running on the terminal 120. The client can be a small program in an app (application), a dedicated application, or a web client. Exemplarily, a cloud game related to a three-dimensional virtual scene is running on the terminal 120. After the terminal 120 receives the game screen sent by the server, the terminal 120 displays the game screen. The terminal 120 is at least one of a smart phone, a tablet computer, an e-book reader, an MP3 player, an MP4 player, a laptop computer, and a desktop computer. A camera is provided on the terminal 120, or the terminal 120 is connected to a camera.

[0066] The terminal 120 is connected to the server 140 via a wireless network or a wired network.

[0067] Server 140 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 140 is used to provide backend services for the client of the three-dimensional virtual scene and send the image to terminal 120. Optionally, server 140 performs the primary computing work, and terminal 120 performs the secondary computing work.

[0068] Figure 2 The flow chart of the screen generation method provided by one embodiment of the present application is shown. The method is applied in the server. The method can be applied to Figure 1 In the server 140 shown, the method includes the following steps:

[0069] Step 202: Render the three-dimensional virtual scene to obtain a scene background rendering.

[0070] The three-dimensional virtual scene can be set up by technicians or users.

[0071] The scene background rendering is a two-dimensional image obtained by rendering a three-dimensional virtual scene. For example, Figure 3 As shown, the three-dimensional virtual scene 301 is rendered to obtain a scene background rendering.

[0072] Step 204: Render the first foreground object reported by the first client to obtain a first foreground object rendering image.

[0073] The first foreground object rendering image is a two-dimensional image obtained by rendering the first foreground object. For example, Figure 4 As shown, the first foreground object 401 is rendered to obtain a first foreground object rendering image.

[0074] Optionally, the first foreground target is obtained through a first camera, and the first camera is a camera on a terminal where the first client is located, or the first camera is a camera connected to the terminal where the first client is located.

[0075] In an optional embodiment of the present application, the first foreground target is a user image. Optionally, when the first camera captures multiple user portraits, all of the multiple user portraits are used as the first foreground target, or the user portrait closest to the first camera is used as the first foreground target, or the user portrait within a specified area is used as the first foreground target, or the first target user portrait among the multiple user portraits is used as the first foreground target, where the first target user portrait is a user portrait that has been recognized through facial recognition.

[0076] Step 206: Mix the scene background rendering image and the first foreground object rendering image to obtain a first picture, where the first picture includes the first foreground object displayed in the three-dimensional virtual scene.

[0077] In an embodiment of the present application, the first image is a front-to-back hierarchical relationship between a three-dimensional virtual scene and a foreground object determined based on the scene depth information of the three-dimensional virtual scene and the depth information of the foreground object. Optionally, based on the scene depth information of the three-dimensional virtual scene and the first depth information, the scene background rendering and the first foreground object rendering are blended to obtain the first image; and based on the scene depth information and the second depth information, the scene background rendering and the second foreground object rendering are blended to obtain the second image.

[0078] It is worth mentioning that, in the case where the first foreground object has first depth information relative to the first camera, the first picture includes the first foreground object displayed in the three-dimensional virtual scene based on the first depth information. In this case, the embodiment can also ensure that the front-back hierarchical relationship between the scene background and the foreground object is correct. For example, in Figure 3 In the figure, the aircraft 302 is located at the top layer. Figure 6 In the figure, the aircraft 302 blocks the first foreground object 401, which means that the aircraft 302 and the first foreground object 401 are located in different layers, and the layer where the aircraft 302 is located is in the front layer, indicating that the front-back hierarchical relationship between the aircraft 302 and the first foreground object 401 is correct.

[0079] Step 208: Provide the first picture to the first client.

[0080] After receiving the first picture, the first client displays the first picture.

[0081] In an actual implementation, the server encodes the first picture to obtain a first code; the server provides the first code to the first client.

[0082] In summary, this embodiment only needs to render the 3D virtual scene once to generate a scene image, obtaining a scene background rendering. This rendering is then blended with the first foreground object rendering to produce the resulting image. Because the 3D virtual scene is rendered only once throughout the entire process, this achieves personalized user experience while minimizing performance overhead, increasing throughput, improving rendering efficiency, and saving costs.

[0083] In one embodiment of the present application, Figure 2 The embodiment shown is used in a cloud gaming scenario to realize the "thousand faces for thousands of people" technology, and when generating different pictures, the three-dimensional virtual scene only needs to be rendered once, which can improve rendering efficiency and save costs.

[0084] Figure 8 The following is a flow chart of a method for generating a cloud game screen according to an embodiment of the present application. The method is applied to a server connected to at least two clients. The method can be applied to Figure 1 In the server 140 shown, the method includes the following steps:

[0085] Step 801: Render a three-dimensional virtual scene to obtain a scene background rendering.

[0086] The three-dimensional virtual scene can be set up by technicians or users.

[0087] The scene background rendering is a two-dimensional image obtained by rendering a three-dimensional virtual scene. For example, Figure 3 As shown, the three-dimensional virtual scene 301 is rendered to obtain a scene background rendering.

[0088] Step 802: Render the first foreground object reported by the first client to obtain a first foreground object rendering image; render the second foreground object reported by the second client to obtain a second foreground object rendering image.

[0089] In a case where the first foreground object has first depth information relative to the first camera, the first picture includes the first foreground object displayed in the three-dimensional virtual scene based on the first depth information.

[0090] The first foreground object rendering image is a two-dimensional image obtained by rendering the first foreground object. For example, Figure 4 As shown, the first foreground object 401 is rendered to obtain a first foreground object rendering image.

[0091] In a case where the second foreground object has second depth information relative to the second camera, the second picture includes the second foreground object displayed in the three-dimensional virtual scene based on the second depth information.

[0092] The second foreground object rendering is a two-dimensional image obtained by rendering the second foreground object. Figure 5 As shown, the second foreground object 501 is rendered to obtain a second foreground object rendering image.

[0093] The first camera is a camera on the terminal where the first client is located, or the first camera is a camera connected to the terminal where the first client is located. Similarly, the second camera is a camera on the terminal where the second client is located, or the second camera is a camera connected to the terminal where the first client is located.

[0094] In an optional embodiment of the present application, the first foreground target is a user image. Optionally, when the first camera captures multiple user portraits, all of the multiple user portraits are used as the first foreground target, or the user portrait closest to the first camera is used as the first foreground target, or the user portrait within a specified area is used as the first foreground target, or the first target user portrait among the multiple user portraits is used as the first foreground target, where the first target user portrait is a user portrait that has been recognized through facial recognition.

[0095] In an optional embodiment of the present application, the second foreground target is a user image. Optionally, when the second camera captures multiple user portraits, all of the multiple user portraits are used as the second foreground target, or the user portrait closest to the second camera is used as the second foreground target, or the user portrait within a specified area is used as the second foreground target, or the second target user portrait among the multiple user portraits is used as the second foreground target, where the second target user portrait is a user portrait that has been recognized through facial recognition.

[0096] Step 803: Mix the scene background rendering image with the first foreground target rendering image and the second foreground target rendering image respectively to obtain the first cloud game screen and the second cloud game screen.

[0097] It should be noted that the foreground targets in the first cloud game screen and the second cloud game screen are different, and the scene backgrounds in the first cloud game screen and the second cloud game screen are the same scene background.

[0098] For example, Figure 6 and Figure 7 As shown, in Figure 6 In the first cloud game screen, the foreground target is the first foreground target 401, and the scene background is the three-dimensional virtual scene 301. Figure 7 In the example, the foreground target in the second cloud game screen is the second foreground target 501, and the scene background is the three-dimensional virtual scene 301. Therefore, the foreground targets in the first cloud game screen and the second cloud game screen are different, and the scene backgrounds in the first cloud game screen and the second cloud game screen are the same scene background.

[0099] In an embodiment of the present application, the first cloud game screen and the second cloud game screen are based on the scene depth information of the three-dimensional virtual scene and the depth information of the foreground target to determine the front-back hierarchical relationship between the three-dimensional virtual scene and the foreground target. Optionally, based on the scene depth information of the three-dimensional virtual scene and the first depth information, the scene background rendering and the first foreground target rendering are mixed to obtain the first cloud game screen; based on the scene depth information and the second depth information, the scene background rendering and the second foreground target rendering are mixed to obtain the second cloud game screen.

[0100] It is worth mentioning that this embodiment can also ensure that the front-back hierarchical relationship between the scene background and the foreground object is correct. Figure 3 In the figure, the aircraft 302 is located at the top layer. Figure 6 In the figure, the aircraft 302 blocks the first foreground object 401, which means that the aircraft 302 and the first foreground object 401 are located in different layers, and the layer where the aircraft 302 is located is in the front layer, indicating that the front-back hierarchical relationship between the aircraft 302 and the first foreground object 401 is correct. Figure 7 In the figure, the aircraft 302 blocks the second foreground target 501, which means that the aircraft 302 and the second foreground target 501 are located in different layers, and the layer where the aircraft 302 is located is in the front layer, indicating that the front-back hierarchical relationship between the aircraft 302 and the second foreground target 501 is correct.

[0101] Optionally, when generating the first and second cloud game screens, the first and second cloud game screens also include a third foreground target, which is captured by a third client through a third camera. The server is connected to the third client, and the type of the third client is different from other clients connected to the server. In this case, the first cloud game screen is obtained by mixing the scene background color image, the third foreground target rendering image, and the first foreground target color image. The second cloud game screen is obtained by mixing the scene background color image, the third foreground target rendering image, and the second foreground target color image. The third foreground target rendering image is obtained by rendering the third foreground target sent by the third client.

[0102] Step 804: Provide a first cloud game screen to the first client, and provide a second cloud game screen to the second client.

[0103] After receiving the first cloud game screen, the first client displays the first cloud game screen. After receiving the second cloud game screen, the second client displays the second cloud game screen.

[0104] In one practical implementation, the server encodes the first cloud game screen to obtain a first code, and provides the first code to the first client. The server encodes the second cloud game screen to obtain a second code, and provides the second code to the second client.

[0105] In summary, this embodiment only needs to render the 3D virtual scene once to generate multiple different cloud game screens, obtaining a scene background rendering. This rendering is then blended with the first and second foreground target renderings to produce two different cloud game screens. Because the entire process only renders the 3D virtual scene once, this achieves the "Thousand Faces" technology while also reducing performance overhead, increasing throughput, improving rendering efficiency, and saving costs.

[0106] Figure 9 The flow chart of the screen generation method provided by one embodiment of the present application is shown. The method is applied to a server, which is connected to at least two clients. The method can be applied to Figure 1 In the server 140 shown, the method includes the following steps:

[0107] Step 901: Render a three-dimensional virtual scene to obtain a scene background rendering.

[0108] The three-dimensional virtual scene can be set up by technicians or users.

[0109] The scene background rendering is a two-dimensional image obtained by rendering a three-dimensional virtual scene. For example, Figure 3 As shown, the three-dimensional virtual scene is rendered to obtain a scene background rendering image 301.

[0110] Step 902: Render the first foreground object reported by the first client to obtain a first foreground object rendering image; render the second foreground object reported by the second client to obtain a second foreground object rendering image.

[0111] The first foreground object has first depth information relative to a first camera, where the first depth information refers to the distance from the first foreground object to the first camera. The first camera is used to capture the first foreground object. The second foreground object has second depth information relative to a second camera, where the second depth information refers to the distance from the second foreground object to the second camera. The second camera is used to capture the second foreground object.

[0112] The first camera is a camera on the terminal where the first client is located, or the first camera is a camera connected to the terminal where the first client is located. Similarly, the second camera is a camera on the terminal where the second client is located, or the second camera is a camera connected to the terminal where the first client is located.

[0113] Step 903: Obtain the first scene background color of each pixel in the first picture according to the scene depth information, the first depth information and the scene background rendering image.

[0114] Optionally, render the scene depth information to obtain a scene depth information rendering image; render the first depth information to obtain a first depth information rendering image; and obtain the first scene background color of each pixel in the first picture based on the scene depth information rendering image, the first depth information rendering image, and the scene background rendering image. Figure 10 As shown, the scene depth information is rendered to obtain a scene depth information rendering image. Figure 11 As shown, the first depth information is rendered to obtain a first depth information rendering image.

[0115] The scene depth information refers to the distance between an object in a three-dimensional virtual scene and a scene camera. The scene camera is a virtual camera used to capture the three-dimensional virtual scene.

[0116] The first scene background color is used to indicate the degree of influence of the color of the scene background rendering image on the color of the first picture.

[0117] Taking the i-th pixel in the first picture as an example, i is a positive integer and is related to the number of pixels in the first picture, this step may include the following sub-steps:

[0118] 1. For the i-th pixel in the first picture, determine a first depth weight parameter of the i-th pixel according to the i-th scene depth information corresponding to the i-th pixel in the scene depth information and the i-th first depth information corresponding to the i-th pixel in the first depth information.

[0119] Assume that the i-th scene depth information corresponding to the i-th pixel in the scene depth information is D0, and the i-th first depth information corresponding to the i-th pixel in the first depth information is D1. When D0>D1, the first depth weight parameter P of the i-th pixel is recorded as 0, and when D0≤D1, the first depth weight parameter P of the i-th pixel is recorded as 1.

[0120] 2. Calculate the sum of the first depth weight parameter of the i-th pixel and the transparency of the i-th pixel in the first foreground target rendering image to obtain the first color mixing weight of the i-th pixel.

[0121] Assume that the transparency of the i-th pixel in the first foreground target rendering is A1, then the first color mixing weight of the i-th pixel is P0 = P + (1-A1), and the clamp function (interval clamping function) is used to clamp P0 to the interval [0, 1]. If P0 is in the interval [0, 1], the return value of the clamp function is P0; if P0 is less than 0, the return value of the clamp function is 0; if P0 is greater than 0, the return value of the clamp function is 1.

[0122] 3. Calculate the product of the first color mixing weight of the i-th pixel and the color of the i-th pixel in the scene background rendering to obtain the first scene background color of the i-th pixel in the first picture.

[0123] Assume that the color of the i-th pixel in the scene background rendering is C0, then the first scene background color of the i-th pixel in the first picture is R0=C0*P0 (P0 here is P0 limited by the clamp function).

[0124] 4. Repeat the above three steps until the first scene background color of each pixel in the first picture is obtained.

[0125] Since the above sub-steps 1 to 3 only obtain the first scene background color of the i-th pixel in the first picture, it is necessary to repeat the above sub-steps 1 to 3 to obtain the first scene background color of each pixel in the first picture.

[0126] Step 904: Obtain the first foreground object color of each pixel in the first picture according to the scene depth information, the first depth information and the first foreground object rendering image.

[0127] The first foreground target color is used to indicate the degree of influence of the color of the first foreground target rendering image on the color of the first picture.

[0128] Taking the j-th pixel in the first picture as an example, where j is a positive integer and is related to the number of pixels in the first picture, this step may include the following sub-steps:

[0129] 1. For the jth pixel in the first picture, determine a first depth weight parameter of the jth pixel according to the jth scene depth information corresponding to the jth pixel in the scene depth information and the jth first depth information corresponding to the jth pixel in the first depth information.

[0130] Assume that the jth scene depth information corresponding to the jth pixel in the scene depth information is D0, and the jth first depth information corresponding to the jth pixel in the first depth information is D1. When D0>D1, the first depth weight parameter P of the jth pixel is recorded as 0, and when D0≤D1, the first depth weight parameter P of the jth pixel is recorded as 1.

[0131] 2. Calculate the product of the first depth weight parameter of the j-th pixel, the transparency of the j-th pixel in the first foreground object rendering, and the color of the j-th pixel in the first foreground object rendering to obtain the first foreground object color of the j-th pixel in the first picture.

[0132] Assume that the transparency of the i-th pixel in the first foreground object rendering is A1, and the color of the j-th pixel in the first foreground object rendering is C1, then the first foreground object color of the j-th pixel in the first picture is R1=(1-P)*C1*A1.

[0133] 3. Repeat the above two steps until the first foreground target color of each pixel in the first picture is obtained.

[0134] Since the above sub-steps 1 and 2 only obtain the first scene background color of the j-th pixel in the first picture, it is necessary to repeat the above sub-steps 1 to 3 to obtain the first scene background color of each pixel in the first picture.

[0135] Step 905: Calculate the sum of the first scene background color and the first foreground object color of each pixel point in the first picture to obtain the first picture.

[0136] For any pixel in the first picture, assuming that the first scene background color is R0 and the first foreground object color is R1, the color of any pixel in the first picture is R=R0+R1.

[0137] Step 906: Obtain the second scene background color of each pixel in the second picture according to the scene depth information, the second depth information, and the scene background rendering.

[0138] Optionally, render the scene depth information to obtain a scene depth information rendering image; render the first and second depth information to obtain a second depth information rendering image; and obtain the second scene background color of each pixel in the second picture based on the scene depth information rendering image, the second depth information rendering image, and the scene background rendering image. Figure 9 As shown, the scene depth information is rendered to obtain a scene depth information rendering image. Figure 11 As shown, the second depth information is rendered to obtain a second depth information rendering image.

[0139] The second scene background color is used to indicate the degree of influence of the color of the scene background rendering image on the color of the second picture.

[0140] Taking the mth pixel in the second picture as an example, where m is a positive integer and is related to the number of pixels in the first picture, this step may include the following sub-steps:

[0141] 1. For the mth pixel in the second picture, determine the second depth weight parameter of the mth pixel according to the mth scene depth information corresponding to the mth pixel in the scene depth information and the mth second depth information corresponding to the mth pixel in the second depth information.

[0142] Assume that the mth scene depth information corresponding to the mth pixel in the scene depth information is D0, and the mth second depth information corresponding to the mth pixel in the second depth information is D1. When D0>D1, the second depth weight parameter P of the mth pixel is recorded as 0; when D0≤D1, the second depth weight parameter P of the mth pixel is recorded as 1.

[0143] 2. Calculate the sum of the second depth weight parameter of the m-th pixel and the transparency of the m-th pixel in the second foreground target rendering image to obtain the second color mixing weight of the m-th pixel.

[0144] Assume that the transparency of the mth pixel in the second foreground target rendering is A1, then the second color mixing weight of the mth pixel is P0 = P + (1-A1), and the clamp function (interval clamping function) is used to clamp P0 to the interval [0, 1]. If P0 is in the interval [0, 1], the return value of the clamp function is P0; if P0 is less than 0, the return value of the clamp function is 0; if P0 is greater than 0, the return value of the clamp function is 1.

[0145] 3. Calculate the product of the second color mixing weight of the m-th pixel and the color of the m-th pixel in the scene background rendering to obtain the second scene background color of the m-th pixel in the second picture.

[0146] Assume that the color of the m-th pixel in the scene background rendering image is C0, then the second scene background color R0 of the m-th pixel in the second picture is R0=C0*P0.

[0147] 4. Repeat the above three steps until the second scene background color of each pixel in the second picture is obtained.

[0148] Since the above sub-steps 1 to 3 only obtain the second scene background color of the mth pixel in the second picture, it is necessary to repeat the above sub-steps 1 to 3 to obtain the second scene background color of each pixel in the second picture.

[0149] Step 907: Obtain the second foreground object color of each pixel in the first image based on the scene depth information, the first depth information, and the second foreground object rendering.

[0150] The second foreground target color is used to indicate the degree of influence of the color of the second foreground target rendering image on the color of the second picture.

[0151] Taking the nth pixel in the second picture as an example, where n is a positive integer and is related to the number of pixels in the first picture, this step may include the following sub-steps:

[0152] 1. For the nth pixel in the second picture, determine a second depth weight parameter of the nth pixel according to the nth scene depth information corresponding to the nth pixel in the scene depth information and the nth second depth information corresponding to the nth pixel in the second depth information.

[0153] Assume that the nth scene depth information corresponding to the nth pixel in the scene depth information is D0, and the nth second depth information corresponding to the nth pixel in the second depth information is D1. When D0>D1, the second depth weight parameter P of the nth pixel is recorded as 0, and when D0≤D1, the second depth weight parameter P of the nth pixel is recorded as 1.

[0154] 2. Calculate the product of the second depth weight parameter of the nth pixel, the transparency of the nth pixel in the second foreground object rendering image, and the color of the nth pixel in the second foreground object rendering image to obtain the second foreground object color of the nth pixel in the second picture.

[0155] Assume that the transparency of the i-th pixel in the second foreground object rendering is A1, and the color of the n-th pixel in the second foreground object rendering is C1, then the second foreground object color of the n-th pixel in the second picture is R1 = (1-P)*C1*A1.

[0156] 3. Repeat the above two steps until the second foreground target color of each pixel in the second picture is obtained.

[0157] Since the above sub-steps 1 and 2 only obtain the second scene background color of the n-th pixel in the second picture, it is necessary to repeat the above sub-steps 1 to 3 to obtain the second scene background color of each pixel in the second picture.

[0158] Step 908: Calculate the sum of the scene background color and the second foreground object color of each pixel point in the second picture to obtain the second picture.

[0159] For any pixel point in the second picture, assuming that the second scene background color is R0 and the second foreground object color is R1, the color of any pixel point in the second picture is R=R0+R1.

[0160] Step 909: Provide the first picture to the first client, and provide the second picture to the second client.

[0161] The foreground objects in the first picture and the second picture are different, and the scene backgrounds in the first picture and the second picture are the same scene background.

[0162] After receiving the first picture, the first client displays the first picture. After receiving the second picture, the second client displays the second picture.

[0163] In an actual implementation, the server encodes the first picture to obtain a first code, and provides the first code to the first client. The server encodes the second picture to obtain a second code, and provides the second code to the first and second clients.

[0164] In summary, this embodiment only needs to render the 3D virtual scene once to generate multiple different images, obtaining a scene background rendering. This rendering is then blended with the first and second foreground object renderings to produce two different images. Because the 3D virtual scene is rendered only once throughout the entire process, this achieves the goal of enabling personalized experiences for each person while also reducing performance overhead, increasing throughput, improving rendering efficiency, and saving costs.

[0165] Moreover, this embodiment can mix the front and back hierarchical relationship effect, highlight the spatial hierarchical relationship between the foreground object and the scene background, and obtain a more reasonable picture.

[0166] Figure 13 The flow chart of the screen generation method provided by one embodiment of the present application is shown. The method is applied to a server, which is connected to at least two clients. The method can be applied to Figure 1 In the server 140 shown, the method includes the following steps:

[0167] Step 1301: Create a first virtual camera for acquiring scene background.

[0168] The first virtual camera is used to obtain data corresponding to the scene background. Optionally, when the server obtains data corresponding to the scene background through other means, it is not necessary to create the first virtual camera.

[0169] Step 1302: Create a second virtual camera for acquiring scene depth information.

[0170] The second virtual camera is used to obtain data corresponding to the scene depth information. Optionally, when the server obtains data corresponding to the scene depth information through other means, there is no need to create the second virtual camera.

[0171] Step 1303: Create a third virtual camera for capturing the target foreground.

[0172] The third virtual camera is used to obtain data corresponding to the target foreground. Optionally, when the server obtains data corresponding to the target foreground through other means, there is no need to create the third virtual camera.

[0173] Step 1304: Create a fourth virtual camera for acquiring foreground depth information.

[0174] The fourth virtual camera is used to obtain data corresponding to the foreground depth information. Optionally, when the server obtains data corresponding to the foreground depth information through other means, there is no need to create the fourth virtual camera.

[0175] It should be noted that the above steps 1301 to 1304 are not in any particular order and can be arranged in any combination.

[0176] Step 1305: Render the scene background through the first virtual camera to obtain a scene background rendering image; render the scene depth information through the second virtual camera to obtain a scene background depth map; render the target foreground through the third virtual camera to obtain a target foreground rendering image; render the foreground depth information through the fourth virtual camera to obtain a target foreground depth map.

[0177] Optionally, after the server renders the scene background through the first virtual camera to obtain a scene background rendering image, the server saves the scene background in a texture corresponding to the scene background.

[0178] Optionally, after the server renders the scene depth information through the second virtual camera to obtain the scene background depth map, the server saves the scene depth information in a texture corresponding to the scene depth information.

[0179] Optionally, after the server renders the target foreground through the third virtual camera to obtain a target foreground rendering image, the server saves the target foreground in a texture corresponding to the target foreground.

[0180] Optionally, after the server renders the foreground depth information through the fourth virtual camera to obtain a target foreground depth map, the server saves the foreground depth information in a texture corresponding to the foreground depth information.

[0181] Step 1306: Mix the scene background rendering image, the scene background depth map, the target foreground rendering image, and the target foreground depth map to obtain a mixed image.

[0182] The mixed picture includes a foreground object displayed in a scene background based on the scene depth information and the foreground depth information.

[0183] Step 1307: Encode the mixed picture to obtain video encoding.

[0184] The server will provide the video codec to the client. After receiving the video codec, the client will decode the video codec to obtain the mixed image.

[0185] In summary, this embodiment uses a virtual camera to create a rendering and depth map. The scene background rendering and the target foreground rendering are then blended using scene depth information and foreground depth information to produce a mixed image. Because the entire process only renders the 3D virtual scene once, this reduces performance overhead, increases the load limit, improves rendering efficiency, and saves costs.

[0186] Figure 14 A schematic diagram of a method for calculating pixel color provided by an embodiment of the present application is shown. The method is applied to a server connected to at least two clients. The method can be applied to Figure 1 The method is used to calculate the server 140. Figure 2 The colors of the pixels of the first picture or the second picture in the embodiment shown.

[0187] This diagram includes scene background depth 1401, foreground object depth 1402, foreground object color 1404, and background scene color 1407. The R terminal of scene background depth 1401 outputs scene depth information for the pixel. The R terminal of foreground object depth 1402 outputs foreground depth information for the pixel. The RGB terminal of foreground object color 1404 outputs the color of the pixel in the foreground object rendering, and the A terminal of foreground object color 1404 outputs the transparency of the pixel in the foreground object rendering. The RGB terminal of background scene color 1407 outputs the color of the pixel in the background scene rendering.

[0188] In this diagram, the scene depth information for a pixel outputted by scene background depth 1401 and the foreground depth information for a pixel outputted by foreground object depth 1402 are inputted into conditional determiner 1403. The output of conditional determiner 1403 is the depth weight parameter P for that pixel. Conditional determiner 1403 is used to determine whether the scene background should be rendered. When the scene depth information is greater than the foreground depth information, the depth weight parameter P outputted by conditional determiner 1403 is 0; when the scene depth information is less than or equal to the foreground depth information, the depth weight parameter P outputted by conditional determiner 1403 is 1.

[0189] After obtaining the depth weight parameter P, the output of conditional judge 1403 is divided into two paths. One path is input to adder 1405, and the other path is input to subtractor 1409. First, the path of the output of conditional judge 1403 that is input to adder 1405 is described. The input of adder 1405 is the depth weight parameter P of the pixel and the output of the A terminal of the foreground target color 1404, which outputs the transparency (1-A1) of the pixel in the foreground target rendering image. (It should be noted that the output of the A terminal of the foreground target color 1404 here is the inverse of the transparency of the pixel in the foreground target rendering image. In other optional embodiments, the output of the A terminal of the foreground target color 1404 can also be the transparency A1 of the pixel in the foreground target rendering image.) The output of adder 1405 is the color mixing weight P0 = P + (1-A1). Adder 1405 inputs the color mixing weight into the interval limiter 1406, which is used to limit the input value to the interval [0, 1]. If the input color blending weight P0 falls within the interval [0, 1], the output of interval qualifier 1406 is P0. If the input color blending weight P0 is less than 0, the output of interval qualifier 1406 is 0. If the input color blending weight P0 is greater than 1, the output of interval qualifier 1406 is 1. Next, the output P3 of interval qualifier 1406 and the color C0 of the pixel in the background scene rendering image outputted by the RGB terminal of background scene color 1407 are combined. Finally, the scene background color R0 of the pixel is inputted into adder 1413.

[0190] Next, we will introduce the path from the output of conditional judge 1403 to subtractor 1409. The output of subtractor 1409 is (1-P), which is input to multiplier 1410. Multiplier 1410 multiplies the input (1-P) by the color C1 of the pixel in the foreground target rendering image, which is output from the RGB end of foreground target color 1404, to obtain (1-P)*C1. Then, multiplier 1410 inputs (1-P)*C1 to multiplier 1412. Another input to multiplier 1412 is the transparency A1 of the pixel in the foreground target rendering image, which is output by subtractor 1411. Here, the input to subtractor 1411 is the transparency (1-A1) of the pixel in the foreground target rendering image. Subtractor 1411 calculates 1-(1-A1)=A1 to obtain the output of subtractor 1411. Multiplier 1412 multiplies the input (1-P)*C1 and A1 to obtain the foreground target color R1 of the pixel. Finally, the foreground target color R1 of the pixel is input to adder 1413. Adder 1413 adds the scene background color R0 and the foreground target color R1 to obtain the color R of the pixel in the first or second image.

[0191] Figure 14The flow chart of the interactive method provided by an embodiment of the present application is shown. The method is applied to at least two clients. Figure 1 The terminal 120 shown is implemented, wherein the method includes the following steps:

[0192] Step 1501: A first client captures a first foreground object through a first camera.

[0193] The first client is at least one of the at least two clients.

[0194] The first camera is a camera on the terminal where the first client is located, or the first camera is a camera connected to the terminal where the first client is located.

[0195] In an optional embodiment of the present application, the first foreground target is a user image. Optionally, when the first camera captures multiple user portraits, all of the multiple user portraits are used as the first foreground target, or the user portrait closest to the first camera is used as the first foreground target, or the user portrait within a specified area is used as the first foreground target, or the first target user portrait among the multiple user portraits is used as the first foreground target, where the first target user portrait is a user portrait that has been recognized through facial recognition.

[0196] For example, Figure 4 As shown, the first client collects the first foreground target 401.

[0197] Step 1502: The second client captures a second foreground object through a second camera.

[0198] The second client is at least one of the at least two clients.

[0199] The second camera is a camera on the terminal where the second client is located, or the second camera is a camera connected to the terminal where the second client is located.

[0200] In an optional embodiment of the present application, the second foreground target is a user portrait. Optionally, when the second camera captures multiple user portraits, all of the multiple user portraits are used as the second foreground target, or the user portrait closest to the second camera is used as the second foreground target, or the user portrait within a specified area is used as the second foreground target, or the second target user portrait among the multiple user portraits is used as the second foreground target, where the second target user portrait is a user portrait that has been recognized through facial recognition.

[0201] For example, Figure 5 As shown, the second client collects the second foreground target 501.

[0202] Step 1503: The first client displays a first screen of the three-dimensional virtual scene, the first screen including a first foreground target displayed in the three-dimensional virtual scene; the second client displays a second screen of the three-dimensional virtual scene, the second screen including a second foreground target displayed in the three-dimensional virtual scene; wherein the foreground targets in the first screen and the second screen are different, and the scene backgrounds in the first screen and the second screen are the same scene background, and the first foreground target and the second foreground target are active in the same three-dimensional virtual scene.

[0203] It should be noted that the first image is obtained by the server mixing the first foreground target rendering image and the scene background rendering image, and the first foreground target rendering image is obtained by the server rendering the first foreground target; the second image is obtained by the server mixing the second foreground target rendering image and the scene background rendering image, and the second foreground target rendering image is obtained by the server rendering the second foreground target. Among them, the scene background rendering image is obtained by rendering a three-dimensional virtual scene. For example, Figure 3 As shown, the first picture displays a first foreground in the three-dimensional virtual scene, and the second picture displays a second foreground target in the three-dimensional virtual scene.

[0204] Optionally, the first foreground target and the second foreground target participate in the same battle in the same three-dimensional virtual scene. For example, in a cloud gaming scenario, if the first client and the second client participate in the same battle, the first foreground target and the second foreground target will be active in the same three-dimensional virtual scene.

[0205] In other optional embodiments of the present application, the displayed first foreground target and the second foreground target can be replaced with other images. Optionally, the first screen includes a first virtual image displayed in a three-dimensional virtual scene based on the first depth information, and a second virtual image displayed in a three-dimensional virtual scene based on the second depth information, wherein the first virtual image corresponds to the first foreground target, and the second virtual image corresponds to the second foreground target. Exemplarily, when the target foreground is a real person, the first foreground target corresponds to a virtual image with a monkey appearance, and the second foreground target corresponds to a virtual image with a panda appearance. Then, when the first screen is displayed, the first screen includes a virtual image with a monkey appearance in the three-dimensional virtual scene, and when the second screen is displayed, the second screen includes a virtual image with a panda appearance in the three-dimensional virtual scene.

[0206] In summary, this embodiment provides an interactive method that can display images including different foreground objects on different clients, thereby realizing personalized display solutions for users and facilitating interaction between users.

[0207] For example, Figure 15 The embodiment shown can be applied in multiple real-life scenarios. In the following embodiments, live broadcast is used as an example for description. Figure 16 The flowchart of the live interactive method provided by one embodiment of the present application is shown. The method is applied to at least three clients, and the at least three clients are connected to a server for providing a three-dimensional virtual scene, wherein the type of the third client is different from the other clients connected to the server. Figure 1 The method is implemented on the computer system 100 shown. The method includes the following steps:

[0208] Step 1601: A first client captures a first foreground object through a first camera.

[0209] The first camera is a camera on the terminal where the first client is located, or the first camera is a camera connected to the terminal where the first client is located.

[0210] The first foreground object has first depth information relative to the first camera.

[0211] In another optional implementation manner of the present application, the first client obtains the first foreground target through a memory, or the first client obtains the first foreground target by accessing a network.

[0212] Step 1602: The first client provides a first foreground target to the server.

[0213] The first client sends a first target foreground to the server.

[0214] Step 1603: The second client captures a second foreground target through a second camera.

[0215] The second camera is a camera on the terminal where the second client is located, or the second camera is a camera connected to the terminal where the second client is located.

[0216] In another optional implementation manner of the present application, the second client obtains the second foreground target through a memory, or the second client obtains the second foreground target by accessing a network.

[0217] The second foreground object has second depth information relative to the second camera.

[0218] Step 1604: The second client provides the second foreground target to the server.

[0219] The second client sends the second foreground target to the server.

[0220] Step 1605: The third client captures a third foreground target through a third camera.

[0221] In a specific implementation of the present application, the third client is a host client, and the other clients are user clients.

[0222] The third camera is a camera on the terminal where the third client is located, or the third camera is a camera connected to the terminal where the third client is located.

[0223] In another optional implementation manner of the present application, the third client obtains the third foreground target through a memory, or the third client obtains the third foreground target by accessing a network.

[0224] The third foreground object has third depth information relative to the third camera.

[0225] Step 1606: The third client provides the third foreground target to the server.

[0226] The third client sends a third foreground target to the server.

[0227] Step 1607: the server renders the three-dimensional virtual scene to obtain a scene background rendering image; the server renders the first foreground object to obtain a first foreground object rendering image; the server renders the second foreground object to obtain a second foreground object rendering image; and the server renders the third foreground object to obtain a third foreground object rendering image.

[0228] The scene background rendering is a two-dimensional image obtained by rendering the three-dimensional virtual scene.

[0229] The first foreground object rendering image is a two-dimensional image obtained by rendering the first foreground object.

[0230] The second foreground object rendering image is a two-dimensional image obtained by rendering the second foreground object.

[0231] The third foreground object rendering image is a two-dimensional image obtained by rendering the third foreground object.

[0232] Step 1608: The server mixes the first foreground target rendering image, the third foreground target rendering image and the scene background rendering image to obtain a first picture; the server mixes the second foreground target rendering image, the third foreground target rendering image and the scene background rendering image to obtain a second picture; the server mixes the candidate foreground target rendering image and the scene background rendering image to obtain a third picture.

[0233] The first picture includes a first foreground object displayed in the three-dimensional virtual scene based on the first depth information and a third foreground object displayed in the three-dimensional virtual scene based on the third depth information.

[0234] The second picture includes a second foreground object displayed in the three-dimensional virtual scene based on the second depth information and a third foreground object displayed in the three-dimensional virtual scene based on the third depth information.

[0235] The second picture includes candidate foreground targets displayed in the three-dimensional virtual scene based on the candidate depth information, where the candidate foreground targets are foreground targets corresponding to other clients, for example, the ambiguous foreground targets mentioned above.

[0236] Step 1609: The server provides the first picture to the first client.

[0237] The server sends the encoded first picture to the first client.

[0238] Step 1610: The first client displays a first picture.

[0239] For example, Figure 17 As shown, the first picture includes a first foreground object 1701 and a third foreground object 1702 .

[0240] Step 1611: The server provides a second picture to the second client.

[0241] The server sends the encoded second picture to the second client.

[0242] Step 1612: The second client displays the second picture.

[0243] For example, Figure 18 As shown, the second picture includes a second foreground object 1801 and a third foreground object 1702 .

[0244] Step 1613: The server provides the third picture to the third client.

[0245] The server sends the encoded third picture to the third client.

[0246] Step 1614: The third client displays the third picture.

[0247] If the third client is a live streamer client, the user corresponding to the third client can select candidate foreground targets to be displayed on the second client. Alternatively, the third client displays user information of other clients. In response to a selection of target user information within the user information on the third client, the third client displays the foreground targets corresponding to the target user information. For example, as shown in the figure, after the user corresponding to the third client clicks on the target user information, the corresponding foreground targets are displayed on the third screen.

[0248] Optionally, in response to a hiding operation on the third client, the third client cancels displaying the user information.

[0249] For example, Figure 19As shown, a third foreground target 1702 is displayed in the third screen 191, and user information is displayed on the right side of the third screen. The user clicks on the target user information 1901 in the user information, and an interface as shown in the third screen 192 is displayed. The foreground target 1902 corresponding to the target user information 1901 is displayed in the third screen 192.

[0250] During the actual live broadcast, the host can select the content of the live broadcast and thus modify the 3D virtual scene used. Optionally, in response to a scene switching operation on the third client, the 3D virtual scene is switched. For example, a 3D virtual scene of a city night scene can be switched to a 3D virtual scene of a starry sky.

[0251] In summary, this embodiment, when used in live broadcasts, renders the 3D virtual scene only once to produce a background rendering. This rendering is then mixed with different foreground object renderings to produce different images. Because the 3D virtual scene is rendered only once throughout the entire process, performance consumption is reduced, the load limit is increased, rendering efficiency is improved, and costs are saved.

[0252] In the following embodiments, game battles are taken as an example for description. Figure 20 The flowchart of the game battle method provided by one embodiment of the present application is shown. The method is applied to at least two clients. Figure 1 The method is implemented on the computer system 100 shown. The method includes the following steps:

[0253] Step 2001: A first client captures a first foreground target through a first camera.

[0254] The first camera is a camera on the terminal where the first client is located, or the first camera is a camera connected to the terminal where the first client is located.

[0255] The first foreground object has first depth information relative to the first camera.

[0256] In another optional implementation manner of the present application, the first client obtains the first foreground target through a memory, or the first client obtains the first foreground target by accessing a network.

[0257] Step 2002: The first client provides a first foreground target to the server.

[0258] The first client sends a first target foreground to the server.

[0259] Step 2003: The second client captures a second foreground target through a second camera.

[0260] The second camera is a camera on the terminal where the second client is located, or the second camera is a camera connected to the terminal where the second client is located.

[0261] In another optional implementation manner of the present application, the second client obtains the second foreground target through a memory, or the second client obtains the second foreground target by accessing a network.

[0262] The second foreground object has second depth information relative to the second camera.

[0263] Step 2004: The second client provides the second foreground target to the server.

[0264] The second client sends the second foreground target to the server.

[0265] Step 2005: the server renders the three-dimensional virtual scene to obtain a scene background rendering image; the server renders the first foreground object to obtain a first foreground object rendering image; and the server renders the second foreground object to obtain a second foreground object rendering image.

[0266] The scene background rendering is a two-dimensional image obtained by rendering the three-dimensional virtual scene.

[0267] The first foreground object rendering image is a two-dimensional image obtained by rendering the first foreground object.

[0268] The second foreground object rendering image is a two-dimensional image obtained by rendering the second foreground object.

[0269] Step 2006: The server mixes the first foreground target rendering image set and the scene background rendering image to obtain a first game screen; the server mixes the second foreground target rendering image set and the scene background rendering image to obtain a second game screen.

[0270] The first foreground object rendering image set includes at least one foreground object rendering image corresponding to a client connected to a server for providing a three-dimensional virtual scene. The first foreground object rendering image set may be determined by a user corresponding to the first client.

[0271] The foreground targets included in the first game screen correspond to the first foreground target rendering image set. For example, there are four clients connected to a server for providing a three-dimensional virtual scene. During a game, user A wishes to display users B and C, who are also participating in the game, on the first game screen. When the server generates the first game screen, the server mixes the foreground target rendering images corresponding to users B and C with the scene background rendering image to generate the first screen. The resulting first game screen includes users B and C.

[0272] The second foreground object rendering image set includes at least one foreground object rendering image corresponding to a client connected to a server for providing a three-dimensional virtual scene. The second foreground object rendering image set may be determined by a user corresponding to the second client.

[0273] The foreground object included in the second game screen corresponds to the second foreground object rendering image set.

[0274] Step 2007: The server provides the first game screen to the first client.

[0275] The server sends the encoded first game screen to the first client.

[0276] Step 2008: The first client displays the first game screen.

[0277] It should be noted that users participating in a game match will be eliminated as the game progresses, so the foreground target displayed on the first game screen will change. For example, before a game match begins, the first game screen includes user A and user B as foreground targets. During the game match, if user B is eliminated, user B will no longer be displayed on the first game screen. For example, before a game match begins, the first game screen includes user A and user B as foreground targets. During the game match, if user C joins the game midway, user C will be displayed on the first game screen.

[0278] Step 2009: The server provides a second game screen to the second client.

[0279] The server sends the encoded second game screen to the second client.

[0280] Step 2010: The second client displays the second game screen.

[0281] It should be noted that users participating in the game battle will be eliminated as the game battle progresses, so the foreground target displayed on the second game screen will change.

[0282] In summary, this embodiment, when applied to gaming, renders the 3D virtual scene only once to produce a background rendering. This background rendering is then blended with different foreground object renderings to create different images. Because the 3D virtual scene is rendered only once throughout the entire process, performance consumption is reduced, the load limit is increased, rendering efficiency is improved, and costs are saved.

[0283] In the following embodiments, an online meeting is taken as an example for description. Figure 21The flowchart of the method for realizing an online conference provided by an embodiment of the present application is shown. The method is applied to at least two clients, and at least two clients are connected to a server for providing a three-dimensional virtual scene. Figure 1 The method is implemented on the computer system 100 shown. The method includes the following steps:

[0284] Step 2101: The first client captures a first foreground target through a first camera.

[0285] The first camera is a camera on the terminal where the first client is located, or the first camera is a camera connected to the terminal where the first client is located.

[0286] The first foreground object has first depth information relative to the first camera.

[0287] In another optional implementation manner of the present application, the first client obtains the first foreground target through a memory, or the first client obtains the first foreground target by accessing a network.

[0288] Step 2102: The first client provides a first foreground target to the server.

[0289] The first client sends a first target foreground to the server.

[0290] Step 2103: The second client captures a second foreground target through a second camera.

[0291] The second camera is a camera on the terminal where the second client is located, or the second camera is a camera connected to the terminal where the second client is located.

[0292] In another optional implementation manner of the present application, the second client obtains the second foreground target through a memory, or the second client obtains the second foreground target by accessing a network.

[0293] The second foreground object has second depth information relative to the second camera.

[0294] Step 2104: The second client provides the second foreground target to the server.

[0295] The second client sends the second foreground target to the server.

[0296] Step 2105: the server renders the three-dimensional virtual scene to obtain a scene background rendering image; the server renders the first foreground object to obtain a first foreground object rendering image; and the server renders the second foreground object to obtain a second foreground object rendering image.

[0297] The scene background rendering is a two-dimensional image obtained by rendering the three-dimensional virtual scene.

[0298] The first foreground object rendering image is a two-dimensional image obtained by rendering the first foreground object.

[0299] The second foreground object rendering image is a two-dimensional image obtained by rendering the second foreground object.

[0300] Step 2106: The server mixes the first foreground target rendering image set and the scene background rendering image to obtain a first game screen; the server mixes the second foreground target rendering image set and the scene background rendering image to obtain a second game screen.

[0301] The first foreground object rendering image set includes at least one foreground object rendering image corresponding to a client connected to a server for providing a three-dimensional virtual scene. The first foreground object rendering image set may be determined by a user corresponding to the first client.

[0302] The foreground targets included in the first conference screen correspond to the first foreground target rendering image set. For example, there are six clients connected to a server for providing a three-dimensional virtual scene. During a conference, user A wishes to display user B and user C, who are also participating in the conference, on the first conference screen. When the server generates the first conference screen, the server mixes the foreground target rendering images corresponding to user B and user C with the scene background rendering image to obtain the first screen. The first conference screen thus obtained includes user B and user C.

[0303] The second foreground object rendering image set includes at least one foreground object rendering image corresponding to a client connected to a server for providing a three-dimensional virtual scene. The second foreground object rendering image set may be determined by a user corresponding to the second client.

[0304] The foreground objects included in the second conference screen correspond to the second foreground object rendering image set.

[0305] Step 2107: The server provides the first conference screen to the first client.

[0306] The server sends the encoded first conference picture to the first client.

[0307] Step 2108: The first client displays the first conference screen.

[0308] It should be noted that during a meeting, users may leave or join the current meeting, so the foreground target displayed on the first meeting screen may change. For example, before the meeting starts, the first meeting screen includes user A and user B as foreground targets. During the meeting, user B leaves the meeting, and user B is no longer displayed on the first meeting screen. For example, before the meeting starts, the first meeting screen includes user A and user B as foreground targets. During the meeting, user C joins the meeting midway, and user C is displayed on the first meeting screen.

[0309] Step 2109: The server provides the second conference screen to the second client.

[0310] The server sends the encoded second conference picture to the second client.

[0311] Step 2110: The second client displays the second conference screen.

[0312] It should be noted that during the conference, some users may leave the current conference or some users may enter the current conference, so the foreground object displayed on the second conference screen may change.

[0313] In summary, this embodiment, when applied to conference games, renders the 3D virtual scene only once to obtain a scene background rendering. This rendering is then blended with different foreground object renderings to produce different images. Because the 3D virtual scene is rendered only once throughout the entire process, performance consumption is reduced, the load limit is increased, rendering efficiency is improved, and costs are saved.

[0314] Figure 22 The following is a schematic diagram of the structure of a screen generation device provided by an exemplary embodiment of the present application. The device 2200 can be implemented as all or part of a computer device through software, hardware, or a combination of both, and specifically includes:

[0315] Rendering module 2201, used to render the three-dimensional virtual scene to obtain a scene background rendering;

[0316] The rendering module 2201 is further configured to render the first foreground object reported by the first client to obtain a rendering image of the first foreground object;

[0317] a mixing module 2202 configured to mix the scene background rendering image with the first foreground object rendering image to obtain a first image, wherein the first image includes the first foreground object displayed in the three-dimensional virtual scene;

[0318] The transceiver module 2203 is configured to provide the first picture to the first client.

[0319] In an optional design of the present application, the mixing module 2202 is also used to mix the scene background rendering image and the first foreground target rendering image according to the scene depth information and the first depth information of the three-dimensional virtual scene to obtain the first picture, and the first depth information is the depth information of the first foreground target relative to the first camera.

[0320] In an optional design of the present application, the mixing module 2202 is also used to obtain the first scene background color of each pixel point in the first picture according to the scene depth information, the first depth information and the scene background rendering map, and the first scene background color is used to indicate the degree of influence of the color of the scene background rendering map on the color of the first picture; obtain the first foreground target color of each pixel point in the first picture according to the scene depth information, the first depth information and the first foreground target rendering map, and the first foreground target color is used to indicate the degree of influence of the color of the first foreground target rendering map on the color of the first picture; calculate the sum of the first scene background color and the first foreground target color of each pixel point in the first picture to obtain the first picture.

[0321] In an optional design of the present application, the mixing module 2202 is further used to determine, for the i-th pixel in the first picture, a first depth weight parameter of the i-th pixel according to the i-th scene depth information corresponding to the i-th pixel in the scene depth information and the i-th first depth information corresponding to the i-th pixel in the first depth information, where i is a positive integer; calculate the sum of the first depth weight parameter of the i-th pixel and the transparency of the i-th pixel in the first foreground target rendering image to obtain the first color mixing weight of the i-th pixel; calculate the product of the first color mixing weight of the i-th pixel and the color of the i-th pixel in the scene background rendering image to obtain the first scene background color of the i-th pixel in the first picture; repeat the above three steps until the first scene background color of each pixel in the first picture is obtained.

[0322] In an optional design of the present application, the mixing module 2202 is further used to determine, for the jth pixel in the first picture, a first depth weight parameter of the jth pixel according to the jth scene depth information corresponding to the jth pixel in the scene depth information and the jth first depth information corresponding to the jth pixel in the first depth information, where j is a positive integer; calculate the product of the first depth weight parameter of the jth pixel, the transparency of the jth pixel in the first foreground target rendering image, and the color of the jth pixel in the first foreground target rendering image to obtain the first foreground target color of the jth pixel in the first picture; repeat the above two steps until the first foreground target color of each pixel in the first picture is obtained.

[0323] In an optional design of the present application, the server is connected to at least two clients. The rendering module 2201 is further used to render the second foreground target reported by the second client to obtain a second foreground target rendering image, wherein the second foreground target has second depth information relative to the second camera; the mixing module 2202 is further used to mix the scene background rendering image with the second foreground target rendering image to obtain a second picture, wherein the second picture includes the second foreground target displayed in the three-dimensional virtual scene; the transceiver module 2203 is further used to provide the second picture to the second client; wherein the foreground targets in the first picture and the second picture are different, and the scene backgrounds in the first picture and the second picture are the same scene background.

[0324] In an optional design of the present application, when the first foreground target has first depth information relative to the first camera, the first picture includes the first foreground target displayed in the three-dimensional virtual scene based on the first depth information; when the second foreground target has second depth information relative to the second camera, the second picture includes the second foreground target displayed in the three-dimensional virtual scene based on the second depth information.

[0325] In an optional design of the present application, the server is also connected to a third client, and the type of the third client is different from other clients connected to the server. The first picture and the second picture also include a third foreground target, and the third foreground target is acquired by the third client through a third camera; the rendering module 2201 is also used to render the third foreground target sent by the third client to obtain a third foreground target rendering image; the scene background color image, the third foreground target rendering image and the first foreground target color image are mixed to obtain the first picture; the mixing module is also used to mix the scene background color image, the third foreground target rendering image and the second foreground target color image to obtain the second picture.

[0326] In an optional design of the present application, the rendering module 2201 is also used to add a scene mark on the three-dimensional virtual scene; add a foreground mark on the first foreground target and the second foreground target, and the scene mark and the foreground mark are used to distinguish the three-dimensional virtual scene and the foreground target.

[0327] In summary, when generating multiple different images, the 3D virtual scene only needs to be rendered once to obtain a scene background rendering. This rendering is then blended with the first and second foreground object renderings to produce two different images. Because the 3D virtual scene is rendered only once throughout the entire process, this achieves the "Thousand Faces" technology while reducing the number of renderings. This results in lower performance overhead, a higher throughput, higher rendering efficiency, and cost savings.

[0328] Figure 23 The following is a schematic diagram showing the structure of an interactive system provided by an exemplary embodiment of the present application. The system can be implemented as all or part of a computer device through software, hardware, or a combination of both, and specifically includes:

[0329] The first interactive device 2301 captures a first foreground object through a first camera; the second interactive device 2302 captures a second foreground object through a second camera;

[0330] The first interactive device 2301 displays a first image of the three-dimensional virtual scene, the first image including the first foreground object displayed in the three-dimensional virtual scene; the second interactive device 2302 displays a second image of the three-dimensional virtual scene, the second image including the second foreground object displayed in the three-dimensional virtual scene;

[0331] The foreground objects in the first picture and the second picture are different, and the scene backgrounds in the first picture and the second picture are the same.

[0332] In an optional design of the present application, the first picture is obtained by the server mixing a first foreground target rendering image and a scene background rendering image, and the first foreground target rendering image is obtained by the server rendering the first foreground target; the second picture is obtained by the server mixing a second foreground target rendering image and the scene background rendering image, and the second foreground target rendering image is obtained by the server rendering the second foreground target; wherein, the scene background rendering image is obtained by rendering the three-dimensional virtual scene.

[0333] In an optional design of the present application, the server is further connected to a third interactive device 2303, which is of a different type than the other interactive devices connected to the server. The first interactive device 2301 provides the first foreground object to the server; in response to the first interactive device 2301 receiving the first image, the first interactive device 2301 displays the first image, and the first image and the second image also include a third foreground object. The third foreground object is captured by the third interactive device 2303 using a third camera, and the third foreground object has third depth information relative to the third camera.

[0334] In an optional design of the present application, the server is further connected to a third interactive device 2303, which is of a different type than the other interactive devices connected to the server. The second interactive device 2302 provides the second foreground object to the server; in response to the second interactive device 2302 receiving the second image, the second interactive device 2302 displays the second image, wherein the first image and the second image also include a third foreground object, which is captured by the third interactive device 2303 using a third camera and has third depth information relative to the third camera.

[0335] In an optional design of the present application, the third interactive device 2303 displays a third screen, which includes candidate foreground targets displayed in the three-dimensional virtual scene based on candidate depth information, and the candidate foreground targets are foreground targets corresponding to the other interactive devices.

[0336] In an optional design of the present application, when the first foreground target has first depth information relative to the first camera, the first picture includes the first foreground target displayed in the three-dimensional virtual scene based on the first depth information; when the second foreground target has second depth information relative to the second camera, the second picture includes the second foreground target displayed in the three-dimensional virtual scene based on the second depth information.

[0337] In an optional design of the present application, the third interactive device 2303 displays the user information of the other interactive devices; in response to the selection operation of the target user information in the user information on the third interactive device 2303, the third interactive device 2303 displays the foreground target corresponding to the target user information.

[0338] In an optional design of the present application, the three-dimensional virtual scene is switched in response to a scene switching operation on the third interactive device 2303 .

[0339] To sum up, this embodiment can display different pictures on different interactive devices, providing personalized picture display for each user. At the same time, it not only realizes the technology of thousands of faces for thousands of people, but also reduces the number of rendering times, resulting in lower performance consumption, higher load limit, higher rendering efficiency, and cost savings.

[0340] Figure 24 2 is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment. The computer device 2400 includes a central processing unit (CPU) 2401, a system memory 2404 including a random access memory (RAM) 2402 and a read-only memory (ROM) 2403, and a system bus 2405 connecting the system memory 2404 and the CPU 2401. The computer device 2400 also includes a basic input / output system (I / O system) 2406 for facilitating information transmission between various components within the computer device, and a mass storage device 2407 for storing an operating system 2413, application programs 2414, and other program modules 2415.

[0341] The basic input / output system 2406 includes a display 2408 for displaying information and an input device 2409 such as a mouse and keyboard for user input. The display 2408 and the input device 2409 are both connected to the central processing unit 2401 via an input / output controller 2410 connected to the system bus 2405. The basic input / output system 2406 may also include an input / output controller 2410 for receiving and processing input from a variety of other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 2410 also provides output to a display screen, printer, or other types of output devices.

[0342] The mass storage device 2407 is connected to the central processing unit 2401 via a mass storage controller (not shown) connected to the system bus 2405. The mass storage device 2407 and its associated computer-readable medium provide non-volatile storage for the computer device 2400. In other words, the mass storage device 2407 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM drive.

[0343] Without loss of generality, the computer device readable medium may include computer device storage media and communication media. Computer device storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer device readable instructions, data structures, program modules or other data. Computer device storage media include RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), CD-ROM, Digital Video Disc (DVD) or other optical storage, tape cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer device storage media is not limited to the above-mentioned ones. The above-mentioned system memory 2404 and mass storage device 2407 can be collectively referred to as memory.

[0344] According to various embodiments of the present disclosure, the computer device 2400 may also be connected to a remote computer device on a network such as the Internet for operation. That is, the computer device 2400 may be connected to the network 2411 via the network interface unit 2412 connected to the system bus 2405, or the network interface unit 2412 may be used to connect to other types of networks or remote computer device systems (not shown).

[0345] The memory also includes one or more programs, which are stored in the memory. The central processing unit 2401 implements the above-mentioned image generation method, or all or part of the steps of the interactive method described above by executing the one or more programs.

[0346] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the image generation method provided by the above-mentioned various method embodiments, or the interactive method as described above.

[0347] The present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the image generation method provided in the above method embodiment, or the interactive method as described above.

[0348] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the image generation method provided in the above embodiments, or the interactive method described above.

[0349] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0350] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0351] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0352] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A screen generation method, characterized in that: Applied to a server connected to at least two clients, the method includes: Rendering the three-dimensional virtual scene to obtain a scene background rendering; Rendering a first foreground target captured and reported by the first client through the first camera to obtain a first foreground target rendering image; the first foreground target is a user avatar; For an i-th pixel in the first picture, determine a first depth weight parameter of the i-th pixel according to the i-th scene depth information corresponding to the i-th pixel in the scene depth information and the i-th first depth information corresponding to the i-th pixel in the first depth information, where i is a positive integer; calculate the sum of the first depth weight parameter of the i-th pixel and the transparency of the i-th pixel in the first foreground target rendering image to obtain a first color mixing weight of the i-th pixel; calculate the product of the first color mixing weight of the i-th pixel and the color of the i-th pixel in the scene background rendering image to obtain a first scene background color of the i-th pixel in the first picture; repeat the above three steps until the first scene background color of each pixel in the first picture is obtained; the scene depth information indicates the distance from an object in the three-dimensional virtual scene to the scene camera, the first depth information is the depth information of the first foreground target relative to the first camera, and the first scene background color is used to indicate the degree of influence of the color of the scene background rendering image on the color of the first picture; For the j-th pixel in the first picture, determine a first depth weight parameter of the j-th pixel according to the j-th scene depth information corresponding to the j-th pixel in the scene depth information and the j-th first depth information corresponding to the j-th pixel in the first depth information, where j is a positive integer; calculate the product of the first depth weight parameter of the j-th pixel, the transparency of the j-th pixel in the first foreground target rendering image, and the color of the j-th pixel in the first foreground target rendering image to obtain a first foreground target color of the j-th pixel in the first picture; repeat the above two steps until the first foreground target color of each pixel in the first picture is obtained; the first foreground target color is used to indicate the degree of influence of the color of the first foreground target rendering image on the color of the first picture; calculating the sum of the first scene background color and the first foreground object color of each pixel point in the first picture to obtain the first picture, where the first picture includes the first foreground object displayed in the three-dimensional virtual scene; providing the first picture to the first client; Rendering a second foreground object reported by the second client to obtain a second foreground object rendering image, where the second foreground object has second depth information relative to the second camera; Mixing the scene background rendering image with the second foreground object rendering image to obtain a second picture, wherein the second picture includes the second foreground object displayed in the three-dimensional virtual scene; providing the second picture to the second client; The foreground objects in the first picture and the second picture are different, and the scene backgrounds in the first picture and the second picture are the same.

2. The method according to claim 1, characterized in that In a case where the first foreground object has first depth information relative to the first camera, the first picture includes the first foreground object displayed in the three-dimensional virtual scene based on the first depth information; In a case where the second foreground object has second depth information relative to the second camera, the second picture includes the second foreground object displayed in the three-dimensional virtual scene based on the second depth information.

3. The method according to claim 1, characterized in that The server is further connected to a third client, the type of the third client being different from other clients connected to the server, the first picture and the second picture further including a third foreground object, the third foreground object being captured by the third client using a third camera, and the method further including: Rendering the third foreground object sent by the third client to obtain a rendering image of the third foreground object; Mixing the scene background color image, the third foreground object rendering image, and the first foreground object color image to obtain the first picture; The scene background color image, the third foreground object rendering image and the second foreground object color image are mixed to obtain the second image.

4. An interactive method, characterized in that The method is applied to at least two clients, and includes: The first client captures a first foreground object through a first camera; the first foreground object is a user's head portrait; the second client captures a second foreground object through a second camera; The first client displays a first picture of a three-dimensional virtual scene, wherein the first picture includes the first foreground target displayed in the three-dimensional virtual scene; the first picture is rendered by the server to obtain a scene background rendering image of the three-dimensional virtual scene, and the first foreground target is rendered to obtain a first foreground target rendering image. For an i-th pixel in the first picture, a first depth weight parameter of the i-th pixel is determined according to the i-th scene depth information corresponding to the i-th pixel in the scene depth information and the i-th first depth information corresponding to the i-th pixel in the first depth information, where i is a positive integer; the sum of the first depth weight parameter of the i-th pixel and the transparency of the i-th pixel in the first foreground target rendering image is calculated to obtain a first color mixing weight of the i-th pixel; the product of the first color mixing weight of the i-th pixel and the color of the i-th pixel in the scene background rendering image is calculated to obtain a first scene background color of the i-th pixel in the first picture, and the above three steps are repeated until the first scene background color of each pixel in the first picture is obtained. For a j-th pixel in the first picture, a first depth weight parameter of the i-th pixel is determined according to the j-th scene depth information corresponding to the j-th pixel in the scene depth information. determining a first depth weight parameter of the jth pixel based on the jth first depth information corresponding to the jth pixel in the first depth information and the first depth information, where j is a positive integer; calculating the product of the first depth weight parameter of the jth pixel, the transparency of the jth pixel in the first foreground target rendering image, and the color of the jth pixel in the first foreground target rendering image to obtain the first foreground target color of the jth pixel in the first picture, repeating the above two steps until the first foreground target color of each pixel in the first picture is obtained, and calculating the sum of the first scene background color and the first foreground target color of each pixel in the first picture; the scene depth information indicates the distance from the object in the three-dimensional virtual scene to the scene camera, the first depth information is the depth information of the first foreground target relative to the first camera, the first scene background color is used to indicate the degree of influence of the color of the scene background rendering image on the color of the first picture, and the first foreground target color is used to indicate the degree of influence of the color of the first foreground target rendering image on the color of the first picture; the second client displays a second picture of the three-dimensional virtual scene, where the second picture includes the second foreground target displayed in the three-dimensional virtual scene; The foreground objects in the first picture and the second picture are different, the scene backgrounds in the first picture and the second picture are the same, and the first foreground object and the second foreground object move in the same three-dimensional virtual scene.

5. The method according to claim 4, characterized in that The second picture is obtained by the server mixing a second foreground object rendering image and the scene background rendering image, and the second foreground object rendering image is obtained by the server rendering the second foreground object.

6. The method according to claim 4, characterized in that The server is further connected to a third client, and the type of the third client is different from other clients connected to the server; The first client displays a first picture of the three-dimensional virtual scene, including: The first client provides the first foreground target to the server; In response to the first client receiving the first picture, the first client displays the first picture, and the first picture and the second picture also include a third foreground target, which is captured by the third client through a third camera, and the third foreground target has third depth information relative to the third camera.

7. The method according to claim 4, characterized in that The server is further connected to a third client, and the type of the third client is different from other clients connected to the server; The second client displays a second picture of the three-dimensional virtual scene, including: The second client provides the second foreground target to the server; In response to the second client receiving the second picture, the second client displays the second picture, and the first picture and the second picture also include a third foreground target, which is captured by the third client through a third camera, and the third foreground target has third depth information relative to the third camera.

8. The method according to claim 6 or 7, characterized in that The method further comprises: The third client displays a third picture, which includes candidate foreground targets displayed in the three-dimensional virtual scene based on candidate depth information, and the candidate foreground targets are foreground targets corresponding to the other clients.

9. The method according to claim 8, characterized in that The method further comprises: The third client displays user information of the other clients; In response to a selection operation on the third client on target user information in the user information, the third client displays a foreground target corresponding to the target user information.

10. The method according to any one of claims 4 to 9, characterized in that In a case where the second foreground object has second depth information relative to the second camera, the second picture includes the second foreground object displayed in the three-dimensional virtual scene based on the second depth information.

11. A screen generating device, characterized in that: The device is connected to at least two clients, and includes: A rendering module is used to render a three-dimensional virtual scene and obtain a scene background rendering; The rendering module is further configured to render a first foreground target captured and reported by the first client via the first camera to obtain a first foreground target rendering image; the first foreground target is a user avatar; A mixing module is used to determine, for an i-th pixel in a first picture, a first depth weight parameter of the i-th pixel according to the i-th scene depth information corresponding to the i-th pixel in the scene depth information and the i-th first depth information corresponding to the i-th pixel in the first depth information, where i is a positive integer; calculate the sum of the first depth weight parameter of the i-th pixel and the transparency of the i-th pixel in the first foreground target rendering image to obtain a first color mixing weight of the i-th pixel; calculate the product of the first color mixing weight of the i-th pixel and the color of the i-th pixel in the scene background rendering image to obtain a first scene background color of the i-th pixel in the first picture; repeat the above three steps until the first scene background color of each pixel in the first picture is obtained, the scene depth information indicates the distance from the object in the three-dimensional virtual scene to the scene camera, the first depth information is the depth information of the first foreground target relative to the first camera, and the first scene background color is used to indicate the color of the scene background rendering image relative to the the degree of influence of the color of the first picture; for the jth pixel in the first picture, determining the first depth weight parameter of the jth pixel according to the jth scene depth information corresponding to the jth pixel in the scene depth information and the jth first depth information corresponding to the jth pixel in the first depth information, where j is a positive integer; calculating the product of the first depth weight parameter of the jth pixel, the transparency of the jth pixel in the first foreground target rendering image, and the color of the jth pixel in the first foreground target rendering image to obtain the first foreground target color of the jth pixel in the first picture; repeating the above two steps until the first foreground target color of each pixel in the first picture is obtained, the first foreground target color being used to indicate the degree of influence of the color of the first foreground target rendering image on the color of the first picture; calculating the sum of the first scene background color and the first foreground target color of each pixel in the first picture to obtain the first picture, where the first picture includes the first foreground target displayed in the three-dimensional virtual scene; a transceiver module, configured to provide the first picture to the first client; The rendering module is further configured to render a second foreground object reported by the second client to obtain a second foreground object rendering image, wherein the second foreground object has second depth information relative to the second camera; The mixing module is further configured to mix the scene background rendering image with the second foreground object rendering image to obtain a second picture, wherein the second picture includes the second foreground object displayed in the three-dimensional virtual scene; The transceiver module is further configured to provide the second picture to the second client; The foreground objects in the first picture and the second picture are different, and the scene backgrounds in the first picture and the second picture are the same.

12. An interactive system, characterized in that: The system comprises at least two interactive devices, the system comprising: The first interactive device captures a first foreground object through a first camera; the second interactive device captures a second foreground object through a second camera; The first interactive device displays a first screen of a three-dimensional virtual scene, wherein the first screen includes the first foreground target displayed in the three-dimensional virtual scene; the first screen is rendered by a server to obtain a scene background rendering image of the three-dimensional virtual scene, and the first foreground target is rendered to obtain a first foreground target rendering image. For an i-th pixel in the first screen, a first depth weight parameter of the i-th pixel is determined according to the i-th scene depth information corresponding to the i-th pixel in the scene depth information and the i-th first depth information corresponding to the i-th pixel in the first depth information, where i is a positive integer; the sum of the first depth weight parameter of the i-th pixel and the transparency of the i-th pixel in the first foreground target rendering image is calculated to obtain a first color mixing weight of the i-th pixel; the product of the first color mixing weight of the i-th pixel and the color of the i-th pixel in the scene background rendering image is calculated to obtain a first scene background color of the i-th pixel in the first screen, and the above three steps are repeated until the first scene background color of each pixel in the first screen is obtained. For a j-th pixel in the first screen, a first depth weight parameter of the i-th pixel is determined according to the j-th scene depth information corresponding to the j-th pixel in the scene depth information. determining a first depth weight parameter of the jth pixel based on the jth first depth information corresponding to the jth pixel in the first depth information and the first depth information, where j is a positive integer; calculating the product of the first depth weight parameter of the jth pixel, the transparency of the jth pixel in the first foreground target rendering image, and the color of the jth pixel in the first foreground target rendering image to obtain the first foreground target color of the jth pixel in the first picture, repeating the above two steps until the first foreground target color of each pixel in the first picture is obtained, and calculating the sum of the first scene background color and the first foreground target color of each pixel in the first picture; the scene depth information indicates the distance from the object in the three-dimensional virtual scene to the scene camera, the first depth information is the depth information of the first foreground target relative to the first camera, the first scene background color is used to indicate the degree of influence of the color of the scene background rendering image on the color of the first picture, and the first foreground target color is used to indicate the degree of influence of the color of the first foreground target rendering image on the color of the first picture; the second interactive device displays a second picture of the three-dimensional virtual scene, the second picture including the second foreground target displayed in the three-dimensional virtual scene; The foreground objects in the first picture and the second picture are different, the scene backgrounds in the first picture and the second picture are the same, and the first foreground object and the second foreground object move in the same three-dimensional virtual scene.

13. A computer device, characterized in that: The computer device includes: a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the screen generation method according to any one of claims 1 to 3, or the interactive method according to any one of claims 4 to 10.

14. A computer-readable storage medium, characterized in that At least one program code is stored in the computer-readable storage medium, and the program code is loaded and executed by the processor to implement the screen generation method according to any one of claims 1 to 3, or the interactive method according to any one of claims 4 to 10.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the screen generation method according to any one of claims 1 to 3 or the interactive method according to any one of claims 4 to 10 is implemented.

Images