Simultaneous screen rendering methods, devices and electronic equipment

Abstract

This invention provides a method, apparatus, and electronic device for rendering simultaneous screen images. The method involves capturing each frame of a first virtual object and a second virtual object using a first scene capture component and a second scene capture component, respectively. Each frame of the first and second virtual objects is then masked using a first black-and-white mask and a second black-and-white mask, respectively, resulting in corresponding masked frames. These masked frames are then combined to obtain simultaneous screen images. In this method, different scene capture components can capture each frame of their respective virtual objects, perform masking using their respective black-and-white masks, and combine the masked frames to obtain the simultaneous screen images. This eliminates the need for pre-creating simultaneous screen images using additional manpower and resources, thus shortening the production cycle, reducing costs, and allowing for user modification and improved scalability.

Description

Technical Field

[0001] This invention relates to the field of screen rendering technology, and in particular to a method, apparatus and electronic device for rendering screens on the same screen. Background Technology

[0002] In some games, the plot serves as the driving force behind the entire game's development. Describing a complete story is the most fundamental theme of a game, and there are various ways to depict the plot. This can be done by inserting pre-made videos and then playing them, or by completing main quests to lead into the next part of the story. Some games feature cutscenes with multiple camera angles rendered on screen simultaneously. The technology for this usually involves pre-creating the shots, rendering them into videos through post-production, and then playing the videos directly at specific times in the game. This method requires additional manpower and resources to create the videos, and each time there are content changes, the videos need to be re-output, resulting in a long production cycle, high costs, limited modifiability, and poor scalability. Summary of the Invention

[0003] The purpose of this invention is to provide a method, apparatus, and electronic device for rendering multiple screens on the same screen, so as to shorten the production cycle of rendering multiple screens on the same screen, reduce costs, facilitate modification, and improve scalability.

[0004] This invention provides a method for rendering simultaneous screen images. The virtual scene of a game includes at least one virtual object. The method includes: capturing each frame of the first virtual object using a first scene capture component; capturing each frame of the second virtual object using a second scene capture component; obtaining a preset first black-and-white mask and a preset black-and-white mask; wherein the second black-and-white mask is obtained by inverting the first black-and-white mask; each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask have the same size; applying the first black-and-white mask to each frame of the first virtual object to obtain a masked image of each frame of the first virtual object; applying the second black-and-white mask to each frame of the second virtual object to obtain a masked image of each frame of the second virtual object; and combining the masked images of each frame of the first virtual object and the masked images of each frame of the second virtual object to render simultaneous screen images.

[0005] This invention provides a simultaneous screen rendering device. The virtual scene of a game includes at least one virtual object. The device includes: a capture module for capturing each frame of the first virtual object using a first scene capture component; and capturing each frame of the second virtual object using a second scene capture component; an acquisition module for acquiring a preset first black-and-white mask and a preset black-and-white mask; wherein the second black-and-white mask is obtained by inverting the first black-and-white mask; each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask have the same size; a processing module for applying the first black-and-white mask to each frame of the first virtual object to obtain a masked frame of the first virtual object; applying the second black-and-white mask to each frame of the second virtual object to obtain a masked frame of the second virtual object; and a combination module for combining the masked frames of the first and second virtual objects to render each frame of the simultaneous screen scene.

[0006] The present invention provides an electronic device, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement any of the above-mentioned methods for rendering screen images simultaneously.

[0007] The present invention provides a machine-readable storage medium storing machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement any of the above-mentioned methods for rendering screen images simultaneously.

[0008] The present invention provides a method, apparatus, and electronic device for rendering simultaneous screen images. The method involves capturing each frame of a first virtual object in a game using a first scene capture component; capturing each frame of a second virtual object in the game using a second scene capture component; obtaining a preset first black-and-white mask and a second black-and-white mask; wherein the second black-and-white mask is obtained by inverting the first black-and-white mask; each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask have the same size; applying the first black-and-white mask to each frame of the first virtual object to obtain a masked image of each frame of the first virtual object; applying the second black-and-white mask to each frame of the second virtual object to obtain a masked image of each frame of the second virtual object; and combining the masked images of each frame of the first virtual object and the masked images of each frame of the second virtual object to render simultaneous screen images. In this method, different scene capture components can capture each frame of their respective virtual objects, perform masking processing using their respective black and white masks, and combine the masked images to obtain the same-screen image. This eliminates the need for additional manpower and resources to pre-produce the same-screen image, thereby shortening the production cycle, reducing costs, and making it easy for users to modify, thus improving scalability. Attached Figure Description

[0009] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0010] Figure 1 A flowchart of a method for rendering images on the same screen provided in an embodiment of the present invention;

[0011] Figure 2 A schematic diagram of a grayscale region provided in an embodiment of the present invention;

[0012] Figure 3 A schematic diagram of a grayscale region provided in an embodiment of the present invention;

[0013] Figure 4 A schematic diagram of a grayscale region provided in an embodiment of the present invention;

[0014] Figure 5 A schematic diagram of a first black-and-white mask provided in an embodiment of the present invention;

[0015] Figure 6 A schematic diagram of a graphical user interface provided in an embodiment of the present invention;

[0016] Figure 7 A schematic diagram of a viewport image provided in an embodiment of the present invention;

[0017] Figure 8 A schematic diagram of a graphical user interface provided in an embodiment of the present invention;

[0018] Figure 9 A schematic diagram of a viewport image provided in an embodiment of the present invention;

[0019] Figure 10 A schematic diagram of node connection relationship provided in an embodiment of the present invention;

[0020] Figure 11 A schematic diagram of a graphical user interface provided in an embodiment of the present invention;

[0021] Figure 12 A schematic diagram of a graphical user interface provided in an embodiment of the present invention;

[0022] Figure 13 A schematic diagram of a graphical user interface provided in an embodiment of the present invention;

[0023] Figure 14 A schematic diagram of a graphical user interface provided in an embodiment of the present invention;

[0024] Figure 15 This is a schematic diagram of the structure of a screen-on-screen rendering device provided in an embodiment of the present invention;

[0025] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] In the development of some Japanese RPGs (Role-Playing Games), the plot is the driving force behind the entire game's progression. For example, the Tales series often features cutscenes with multiple camera shots rendered simultaneously. This typically requires pre-creating in-game character shots, then post-producing a coherent story sequence, which is then played at specific moments within the game to advance the plot. This method requires additional manpower and resources to produce videos, necessitating re-output for each content change, resulting in a long production cycle, high costs, limited flexibility, and poor scalability. Therefore, this invention provides a method, apparatus, and electronic device for rendering multiple screens simultaneously. This technology can be applied to scenarios requiring the simultaneous display of multiple images.

[0028] In one embodiment of this disclosure, the screen-sharing rendering method can run on a local terminal device or a server. When the screen-sharing rendering method runs on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0029] In an optional implementation, various cloud applications, such as cloud gaming, can run under the cloud interaction system. Taking cloud gaming as an example, cloud gaming refers to a gaming method based on cloud computing. In the cloud gaming operating mode, the game program and the game screen presentation are separate. The storage and execution of the on-screen rendering method are completed on the cloud gaming server. The client device is used for data reception, transmission, and game screen presentation. For example, the client device can be a display device with data transmission capabilities located close to the user, such as a mobile terminal, television, computer, or PDA; however, the information processing is performed by the cloud gaming server in the cloud. When playing the game, the player operates the client device to send operation commands to the cloud gaming server. The cloud gaming server runs the game according to the operation commands, encodes and compresses the game screen and other data, returns it to the client device via the network, and finally, the client device decodes and outputs the game screen.

[0030] In an optional implementation, taking a game as an example, the local terminal device stores the game program and is used to display the game screen. The local terminal device is used to interact with the player through a graphical user interface (GUI), i.e., conventionally by downloading, installing, and running the game program via an electronic device. The local terminal device can provide the GUI to the player in various ways, such as rendering it on the terminal's display screen or providing it to the player via holographic projection. For example, the local terminal device can include a display screen for displaying the GUI, which includes game screens, and a processor for running the game, generating the GUI, and controlling the display of the GUI on the display screen.

[0031] In one possible implementation, this disclosure provides a method for rendering screen-on-screen, which provides a graphical user interface through a terminal device. The terminal device can be either the aforementioned local terminal device or a client device in the aforementioned cloud interaction system.

[0032] To facilitate understanding of this embodiment, we will first describe a screen-sharing rendering method disclosed in this invention. The virtual scene of the game includes at least one virtual object. Specifically, the virtual scene can be provided through a graphical user interface of an electronic device; this electronic device can be a computer, iPad, or other terminal device. The virtual scene of the game can include at least one virtual object such as a virtual character or virtual item, and typically also includes a scene background. Users can operate and control the relevant virtual objects in the virtual scene through the graphical user interface. Figure 1 As shown, the method includes the following steps:

[0033] Step S102: Capture each frame of the first virtual object in the game using the first scene capture component; capture each frame of the second virtual object in the game using the second scene capture component.

[0034] The aforementioned scene capture component, also known as SceneCaptureComponent2D, captures images rendered by the engine for each frame. Its working principle can be understood as capturing a scene view from the camera's view frustum and storing the captured view as an image, which can then be used in materials. In games, this scene capture component is often used as a monitor-like function. The first and second virtual objects can both be virtual characters or virtual items in the game scene. They can be the same virtual object or different virtual objects. If they are the same virtual object, the first scene capture component can capture each frame of the virtual object from the first capture perspective, and the second scene capture component can capture each frame of the virtual object from the second capture perspective. In actual implementation, as the game's plot progresses, the first and second virtual objects are usually dynamically changing, such as changing expressions or actions. The first scene capture component can capture each frame of the first virtual object, and the second scene capture component can capture each frame of the second virtual object.

[0035] Step S104: Obtain a preset first black and white mask and a preset black and white mask; wherein, the second black and white mask is obtained by inverting the first black and white mask; each frame of the first virtual object, each frame of the second virtual object, the first black and white mask and the second black and white mask have the same size.

[0036] The positions of the black and white areas in the first black and white mask can be set according to actual needs. For example, if the first black and white mask is a rectangle, one diagonal of the rectangle can be drawn. In this first black and white mask, one side of the diagonal is a black area and the other side is a white area. The second black and white mask can be obtained by inverting the first black and white mask. That is to say, the position and size of the black area of ​​the first black and white mask and the white area of ​​the second black and white mask should match. The position and size of the white area of ​​the first black and white mask and the black area of ​​the second black and white mask should match. In order to make the masking effect meet the requirements of the same screen, the size of each frame of the first virtual object, each frame of the second virtual object, the first black and white mask and the second black and white mask should be the same.

[0037] Step S106: Apply a first black and white mask to each frame of the first virtual object to obtain each frame of the masked image of the first virtual object; apply a second black and white mask to each frame of the second virtual object to obtain each frame of the masked image of the second virtual object.

[0038] A first black and white mask is used to mask each frame of the first virtual object. Since the first black and white mask has the same size as each frame of the first virtual object, the entire frame of each frame of the first virtual object can be masked. For each frame of the first virtual object, the area corresponding to the black area of ​​the first black and white mask is masked, and the area corresponding to the white area is displayed normally, thus obtaining each frame of the masked image of the first virtual object.

[0039] A second black-and-white mask is used to mask each frame of the second virtual object. Since the second black-and-white mask has the same size as each frame of the second virtual object, the entire frame of each frame of the second virtual object can be masked. For each frame of the second virtual object, the area corresponding to the black area of ​​the second black-and-white mask is masked, while the area corresponding to the white area is displayed normally, thus obtaining the masked frame of the second virtual object.

[0040] Step S108: Combine each frame of the mask image of the first virtual object and each frame of the mask image of the second virtual object to render each frame of the same screen.

[0041] The first virtual object's mask frame and the second virtual object's mask frame are displayed in corresponding positions and combined to create a single-frame display. For example, to combine the first virtual object's mask frame with the second virtual object's mask frame, assuming the first black and white mask is a rectangle, a diagonal line is drawn within the rectangle, with the lower left corner corresponding to the white area and the upper right corner corresponding to the black area. Since the second black and white mask is obtained by inverting the first black and white mask, the lower left corner of the second black and white mask corresponds to the black area and the upper right corner corresponds to the white area. After combining, the first virtual object's mask frame is displayed in the lower left corner of the first single-frame display, and the second virtual object's mask frame is displayed in the upper right corner of the first single-frame display.

[0042] The above-described method for rendering simultaneous screen images involves: capturing each frame of a first virtual object in the game using a first scene capture component; capturing each frame of a second virtual object in the game using a second scene capture component; obtaining a preset first black-and-white mask and a second black-and-white mask; wherein the second black-and-white mask is obtained by inverting the first black-and-white mask; each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask have the same size; applying the first black-and-white mask to each frame of the first virtual object to obtain each frame of the masked image of the first virtual object; applying the second black-and-white mask to each frame of the second virtual object to obtain each frame of the masked image of the second virtual object; and combining each frame of the masked image of the first virtual object and the second frame of the masked image of the second virtual object to render each frame of the simultaneous screen image. In this method, different scene capture components can capture each frame of their respective virtual objects, perform masking processing using their respective black and white masks, and combine the masked images to obtain the same-screen image. This eliminates the need for additional manpower and resources to pre-produce the same-screen image, thereby shortening the production cycle, reducing costs, and making it easy for users to modify, thus improving scalability.

[0043] In an optional implementation, the method is based on the method of the above embodiments. In this method, the first area of ​​each frame of the same-screen display is used to play the mask image of the first virtual object, and the second area of ​​each frame of the same-screen display is used to play the mask image of the second virtual object. The first area actually corresponds to the white area of ​​the first black and white mask, so it displays the mask image of the first virtual object. The second area corresponds to the white area of ​​the second black and white mask, so it displays the mask image of the second virtual object. As the game plot progresses, the first and second virtual objects usually change dynamically, that is, the states of the first and second virtual objects can be dynamically played in real time through each frame of the same-screen display.

[0044] The number of first virtual objects and / or second virtual objects can be one or more, that is, the number of first virtual objects can be one or more, the number of second virtual objects can be one or more, and the first virtual object and the second virtual object can also be the same virtual object; it can be understood that this method can realize the same virtual object displayed on the same screen from different angles, can realize the screen display of one first virtual object and one second virtual object, and can also realize the screen display of multiple first virtual objects and multiple second virtual objects, thereby meeting a variety of different screen display needs.

[0045] To optimize the display effect of each frame of the final on-screen image and avoid distortion issues, the dimensions of each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask are typically matched to the screen size of the display screen showing each frame of the on-screen image. Specifically, the aspect ratios of each frame of the first and second virtual objects, as well as the aspect ratios of the first and second black-and-white masks, can be the same as the aspect ratio of the display screen. This method includes the following steps:

[0046] Step 1: Capture the initial frame of each virtual object in the game using the first scene capture component.

[0047] The initial frame of each frame of the first virtual object can be understood as the frame containing the first virtual object that is actually captured by the first scene capture component. In each initial frame, the first virtual object may be located in the left, right or middle area of ​​the frame.

[0048] Step 2: Process each initial frame of the first virtual object according to the preset processing method to obtain each frame of the first virtual object; wherein, the preset processing method includes at least one of the following: scaling processing, offset processing, and color adjustment processing.

[0049] The scaling process described above can specifically be to enlarge or shrink the initial image by a preset factor; the offset process described above can be to offset the initial image so that the first virtual object in the initial image is located in a specified area of ​​the image; the color adjustment process described above can be to adjust the color of the initial image so that the color-adjusted initial image meets the display requirements. It is understood that the above-mentioned preset processing method can also include other processing methods, which can be set according to actual needs and are not limited here. In actual implementation, considering that the initial frame of each frame of the first virtual object captured by the first scene capture component may not meet the requirements of subsequent screen display and cannot be used directly, it is usually necessary to process the initial frame of each frame of the captured first virtual object. For example, continuing to take the first black and white mask as a rectangle, with the lower left corner corresponding to the white area and the upper right corner corresponding to the black area, in the initial frame of the first image, the first virtual object is located in the middle area. Since the first frame of the first image is masked by the first black and white mask, the first virtual object may only be partially displayed in the first frame of the mask image, while the other part is hidden. Therefore, it is necessary to offset the initial frame of the first image so that the first virtual object is displayed in the lower left area. In this way, the first virtual object can be fully displayed in the lower left area in the first frame of the mask image after the masking process.

[0050] Step 3: Capture the initial frame of each frame of the game's second virtual object using the second scene capture component.

[0051] The initial frame of each frame of the aforementioned second virtual object can be understood as the frame containing the second virtual object actually captured by the second scene capture component. In each initial frame, the second virtual object may be located in the left, right, or middle area of ​​the frame.

[0052] Step 4: Process each initial frame of the second virtual object according to the preset processing method to obtain each frame of the second virtual object.

[0053] In actual implementation, considering that the initial frame of each captured second virtual object may not meet the requirements for subsequent simultaneous display and cannot be used directly, it is usually necessary to process the initial frame of each captured second virtual object. For example, continuing to take the second black and white mask as a rectangle, with the lower left corner corresponding to the black area and the upper right corner corresponding to the white area, in the first initial frame, the second virtual object is located in the middle area. Since the first initial frame is masked by the second black and white mask, the second virtual object may only be partially displayed in the first masked frame, while the other part is hidden. Therefore, it is necessary to offset the first initial frame so that the second virtual object is displayed in the upper right area. In this way, the second virtual object can be fully displayed in the upper right area in the first masked frame after the masking process.

[0054] Step 5: Obtain the preset first black and white mask and second black and white mask; wherein, the second black and white mask is obtained by inverting the first black and white mask; each frame of the first virtual object, each frame of the second virtual object, the first black and white mask and the second black and white mask have the same size.

[0055] The process of obtaining the first black and white mask is described below. This first black and white mask can be obtained through the following steps A to E:

[0056] Step A: Obtain the first texture image in the initial screen UV coordinate system.

[0057] A first black-and-white mask based on the screen can be calculated using material shader algorithms. This can be achieved using a High Level Shader Language (HLSL), a language owned and developed by Microsoft. HLSL operates independently on the Windows platform and is only compatible with Microsoft's Direct3D. HLSL is Microsoft's product to compete with GLSL (Graphics Library Shader Language), but it is not compatible with the OpenGL (Open Graphics Library) standard. It is very similar to Cg (C for Graphics, a high-level shader language). The main purpose of HLSL is to perform complex image processing quickly and efficiently on the graphics card, reducing the chance of programming errors when writing complex special effects compared to composite or low-level shader languages.

[0058] Step B involves converting the first texture image from the initial screen UV coordinate system to the screen UV Cartesian coordinate system, resulting in the converted second texture image.

[0059] The initial screen UV coordinate system is offset to obtain the screen UV Cartesian coordinate system, which is the standard coordinate system used for trigonometric function calculations. By subtracting 0.5 from ViewPortUV, the origin of the initial screen UV coordinate system is the top left corner. By offsetting, the origin of the screen UV Cartesian coordinate system can be shifted to the center of the screen. The offset screen UV Cartesian coordinate system has four quadrants, which facilitates the subsequent calculation to obtain the first black and white mask.

[0060] The corresponding code for this step is as follows:

[0061] float2 origin=float2(0.5f,0.5f);

[0062] float2 location=ViewportUV.xy-origin;

[0063] return location;

[0064] Here, float2 is a binary floating-point data type; ViewportUV.xy represents the coordinate value of each pixel in the initial screen UV coordinate system; and location represents the coordinate value of each pixel in the offset screen UV Cartesian coordinate system.

[0065] Step C: Calculate the slope of the pixel coordinates of each pixel in the second texture image relative to the positive direction of the horizontal axis of the screen UV Cartesian coordinate system.

[0066] The atan2 arctangent function is used to obtain the slope of the pixel coordinates of each pixel in the screen's UV Cartesian coordinate system with respect to the positive direction of the horizontal axis (the rightward direction of the horizontal axis is the positive direction). The slope range is -1 to 1, where -1 to 0 is black, 0-1 is gradient gray, and 1 is white.

[0067] The corresponding code for this step is as follows:

[0068] float2 origin=float2(0.5f,0.5f);

[0069] float2 location=ViewportUV.xy-origin;

[0070] float angle=atan2(location.y,location.x);

[0071] return angle;

[0072] Where location.x and location.y represent the x-coordinate and y-coordinate of each pixel in the screen's UV Cartesian coordinate system, respectively; atan2() represents the angle between the ray pointing from the origin to (location.x, location.y) and the positive x-axis in the coordinate plane.

[0073] The angle variable declared here can be understood as the slope of a branch line connecting the coordinates of any point on the current screen to the origin.

[0074] Step D: Determine the first mask corresponding to the second texture image based on the slope; wherein the first mask includes: black area, white area and grayscale area.

[0075] Based on the slope mentioned above, the first mask corresponding to the second texture image can be obtained. When the slope is -1 to 0, it corresponds to the black area of ​​the first mask; when the slope is between 0 and 1, it corresponds to the gray area of ​​the first mask; and when the slope is 1, it corresponds to the white area of ​​the first mask.

[0076] Step E: Based on the first mask, obtain the first black and white mask.

[0077] This step E can be obtained through the following steps a and b:

[0078] Step a: Adjust each slope based on preset variable values ​​to adjust the area range of the grayscale region according to each adjusted slope.

[0079] A variable value can be provided to adjust each slope, and the range of the grayscale area can be adjusted based on each adjusted slope. This variable value can be a pre-set value based on practical experience, such as... Figure 2 The diagram shown illustrates a grayscale region. Figure 3 A schematic diagram of a grayscale region is shown. Figure 4 The diagram shown illustrates a grayscale region, wherein... Figure 2 The location of the grayscale region when radian = 0. Figure 3 The location of the grayscale region when radian = 0.3. Figure 4 The image shows the position of the grayscale region when `radian` = 0.6. It demonstrates that the range of grayscale region positions can be adjusted by providing different `radian` values. The corresponding code is as follows:

[0080] float2 origin=float2(0.5f,0.5f);

[0081] float2 location=ViewportUV.xy-origin;

[0082] float angle=radian+atan2(location.y,location.x);

[0083] return angle;

[0084] Here, redian represents the variable value.

[0085] Step b: Based on each adjusted slope and the distance of each pixel in the first mask from the center of the screen, a first black and white mask is obtained.

[0086] Next, perform the radian calculation, obtain the above results, and calculate the cosine value. The corresponding code is as follows:

[0087] float2 origin=float2(0.5f,0.5f);

[0088] float2 location=ViewportUV.xy-origin;

[0089] float angle=radian+atan2(location.y,location.x);

[0090] location = cos(angle);

[0091] return location;

[0092] Here, cos() represents calculating the cosine value.

[0093] After obtaining the above results, performing one more step of calculation yields a sector-shaped mask. So far, only the mask has been calculated; we also need the actual distance of each pixel on the screen from the screen center. Therefore, we need to multiply by the distance of the current pixel from the center point. The corresponding code is as follows:

[0094] float2 origin=float2(0.5f,0.5f);

[0095] float2 location=ViewportUV.xy-origin;

[0096] float angle=radian+atan2(location.y,location.x);

[0097] float dist=length(location);

[0098] location = cos(angle) * dist;

[0099] return location;

[0100] Here, dist represents the actual distance of each pixel from the center of the screen.

[0101] By performing a step calculation on the final result, the first black and white mask based on the screen UV can be calculated, such as... Figure 5 The diagram shown illustrates a first type of black-and-white mask. The corresponding code for this step is as follows:

[0102] float2 origin=float2(0.5f,0.5f);

[0103] float2 location=ViewportUV.xy-origin;

[0104] float angle=radian+atan2(location.y,location.x);

[0105] float dist=length(location);

[0106] location = cos(angle) * dist;

[0107] return step(location.x,cutValue);

[0108] Here, cutValue represents the value set in the step node; step() means performing step calculation on the x-coordinate based on cutValue.

[0109] Step 6: Apply a first black and white mask to each frame of the first virtual object to obtain each frame of the masked image of the first virtual object; apply a second black and white mask to each frame of the second virtual object to obtain each frame of the masked image of the second virtual object.

[0110] Step 7: Combine each frame of the mask image of the first virtual object and each frame of the mask image of the second virtual object to render each frame of the same screen.

[0111] The above-described on-screen rendering method involves capturing each initial frame of the first virtual object in the game using a first scene capture component; processing each initial frame of the first virtual object according to a preset processing method to obtain each frame of the first virtual object; capturing each initial frame of the second virtual object in the game using a second scene capture component; processing each initial frame of the second virtual object according to a preset processing method to obtain each frame of the second virtual object; obtaining preset first and second black-and-white masks; wherein the second black-and-white mask is obtained by inverting the first black-and-white mask; each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask have the same size; applying the first black-and-white mask to each frame of the first virtual object to obtain each frame of the first virtual object mask; applying the second black-and-white mask to each frame of the second virtual object to obtain each frame of the second virtual object mask; and combining each frame of the first virtual object mask and each frame of the second virtual object mask to render each frame of the on-screen scene. This method allows for the creation of multi-camera scene rendering on the same screen directly within the game engine during game development. Furthermore, content modifications take effect in real time, and the method is highly scalable. This breaks free from the constraints of traditional video post-production compositing methods, enabling rapid production, rapid iteration, and low modification costs.

[0112] In an alternative implementation, the game is configured with a first interpolation component, which may be an interpolation node in the game engine, such as the Lerp node in the UE4 (Unreal Engine 4) engine. The Lerp node returns a mixed value between inputs A and B based on the alpha input. When the alpha value is 0, it returns 100% of the value of A, and when the alpha value is 1, it returns 100% of the value of B.

[0113] In response to the interpolation coefficient of the first interpolation component switching from a first value to a second value, a simultaneous multi-screen display event is determined to have occurred. The interpolation coefficient is based on a pre-set first material ball parameter switching from a first value to a second value. For example, taking a Lerp node as the first interpolation component, the aforementioned interpolation coefficient can be understood as the Alpha value of the Lerp node. To achieve screen switching, this Alpha value can be 0 or 1. For instance, when the Alpha value is 0, the current virtual scene of the game can be displayed in the graphical user interface, which can be understood as the currently displayed actual game scene. When the Alpha value switches from 0 to 1, the current virtual scene in the graphical user interface can be switched to displaying simultaneous screen images. In each frame of simultaneous screen images, the first area displays the mask image of each frame of the first virtual object, and the second area displays the mask image of each frame of the second virtual object.

[0114] For ease of explanation, let's take the UE4 engine as an example. To trigger the switching of interpolation coefficients, we can create a new "Material Parameter Set." This "Material Parameter Set" is a built-in component of the UE4 engine and can be dynamically controlled by writing program scripts. The specific steps are as follows: In the Content Browser, right-click to create a "Material Parameter Set." You can temporarily name the created "Material Parameter Set" "MPC_Test1." Double-click to open it, click "+" to create a scalar parameter and name it Active. In the Material Sphere, right-click again, enter collectionParameter, and specify the created MPC_Test1. Then, change the parameter name to Active and connect it to the Alpha of the Lerp node as a switch. Next, right-click in the Content Browser to create a Blueprint. Select Actor as the parent class. An Actor is an object that can be placed or dynamically generated in the world. Temporarily name the created Blueprint "BP_Test1." Blueprints are an important module of the UE4 engine and are a type of visual programming script. Double-click to open BP_Test1, and under the Event Graph, right-click and enter set. In Scala, select "Set Scalar Parameter Values," specify the created MPC_Test1 and its corresponding parameters. Then, click the plus sign to create a new Boolean variable. In the details pane, check "Editable Instance" and "Expose to Cutscenes." Next, hold down Ctrl and click "Active" to drag out the parameter value. The engine will automatically convert the Boolean value to a float type: a true Boolean value is converted to a float of 1.0, and a false Boolean value is converted to 0.0. This value is then directly passed to the Alpha input in the material, controlling whether the material enables in-screen rendering. Finally, drag the Blueprint into the sequencer, select the "Active" switch, and then you can use keyframes to determine when to enable the split-screen rendering function.

[0115] In response to the game's multi-screen display event, each frame of the multi-screen display is called. The aforementioned multi-screen display event can be understood as follows: when the game's plot progresses to a certain point and requires the display of multi-screen images, the material parameters are modified through a program script to switch the relevant on / off parameters, thereby triggering the multi-screen display event. In actual implementation, when the game's plot progresses to a certain point and the multi-screen display event occurs, it indicates that multi-screen images need to be displayed. In this case, the corresponding frame of the multi-screen display in the game can be called first.

[0116] In practical implementation, when it is necessary to display the same screen, the current virtual scene displayed in the graphical user interface can be switched to display each frame of the same screen. This can also be understood as using each frame of the same screen to occlude the current virtual scene. The user sees each frame of the same screen through the graphical user interface. In the first area of ​​each frame of the same screen, the mask of the first virtual object is played, and in the second area of ​​each frame of the same screen, the mask of the second virtual object is played. As long as the event of ending the display of multiple screens is not received, the states of the first and second virtual objects can be dynamically played in real time through each frame of the same screen.

[0117] In actual implementation, the game is configured with a second interpolation component. The first input of the second interpolation component is connected to the screen data node of the first virtual object, the second input of the second interpolation component is connected to the screen data node of the second virtual object, and the third input of the second interpolation component is connected to the region node of the first black and white mask.

[0118] The second interpolation component mentioned above can also be a hybrid interpolation node in a game engine, such as the Lerp function node in the UE4 engine; the above-mentioned screen data node can be implemented in the form of a rendering target. The rendering target is a rendering texture of the game engine, which is readable and writable, and the game engine can dynamically write color information through the program.

[0119] In practice, you can create a new blueprint named BP_ScreenSlicing_Capture, add a SceneCaptureComponent2D component (corresponding to the scene capture component), and rename it to Capture. Next, create a new RenderTarget2D (corresponding to the rendering target) named RT_LAYER and assign it to Capture for rendering. This allows the SceneCaptureComponent2D component to continuously render the current camera viewport image onto the RenderTarget2D. Double-clicking the RenderTarget2D and placing the BP_ScreenSlicing_Capture component in the scene and moving it freely will show the RenderTarget2D's image updating in real time.

[0120] For example, such as Figure 6 The diagram shown illustrates a graphical user interface, displaying the scene location of BP_ScreenSlicing_Capture. Figure 7 The diagram shown is a viewport view rendered from the scene captured by the scene capture component, such as... Figure 8The diagram shows a graphical user interface. If BP_ScreenSlicing_Capture is moved a certain distance to the left, the box in the RT rendering screen will move to the right, as shown below. Figure 9 The diagram shows a viewport view, which allows the current camera image to be captured in real time and written to a single RT.

[0121] For ease of explanation, we will still use the Lerp node as the second interpolation component as an example, such as... Figure 10 The diagram shown illustrates a node connection relationship. Figure 10 The Lerp node located at the top corresponds to the second interpolation component. To implement this embodiment, the first input end of the Lerp node is connected to the first rendering target corresponding to the first virtual object. The first rendering target renders each frame of the first virtual object captured by the first scene capture component. The second input end of the Lerp node is connected to the second rendering target corresponding to the second virtual object. The second rendering target renders each frame of the second virtual object captured by the second scene capture component. The third input end of the Lerp node is connected to the region node of the first black and white mask.

[0122] The first scene capture component captures each frame of the first virtual object in the game; each frame of the first virtual object is sent to the first input terminal; and the second interpolation component performs blending processing on each frame of the first virtual object and the first black and white mask to obtain each frame of the mask image of the first virtual object. In actual implementation, the first scene capture component captures each frame of the first virtual object and renders it to the corresponding first rendering target. The first rendering target is connected to the first input terminal of the Lerp node. The Lerp node performs blending interpolation processing on the first rendering target and the first black and white mask to obtain each frame of the mask image of the first virtual object.

[0123] Within the second interpolation component, the first black-and-white mask can be inverted to obtain a second black-and-white mask. The second scene capture component captures each frame of the game's second virtual object. Each frame of the second virtual object is sent to the second input terminal. The second interpolation component then blends each frame of the second virtual object with the second black-and-white mask to obtain each frame of the second virtual object's mask. In actual implementation, the second scene capture component captures each frame of the second virtual object and renders it to the corresponding second rendering target. The second rendering target is connected to the second input terminal of the Lerp node. The Lerp node then performs blending interpolation on the second rendering target and the second black-and-white mask to obtain each frame of the second virtual object's mask.

[0124] To facilitate understanding of the above process, further explanation is provided below, using the UE4 engine as an example. Post-processing materials are a way for the UE4 engine to achieve special effects, operating across the entire screen space. Different filter effects can be achieved by writing corresponding shader logic. In post-processing materials, typing `SceneTexture` will bring up the `SceneTexture:SceneColor` node. The `SceneTexture` node contains a lot of data, all of which are various data information needed by the UE4 engine to render the current scene. The default output is `SceneColor`, and other data includes `DiffuseColor` (the color of objects receiving light), `SpecularColor` (spectral information of all objects in the scene), and `SubSurfaceColor` (subsurface color information of objects), etc.

[0125] However, this embodiment requires the aggregation of all color information in the current scene, so it selects PostProcessInput0, then creates a new material ball, switches MaterialDomain to PostProcess, converts the material ball into a post-processing material, and applies a screen filter to affect the display effect of the scene without affecting the model or character.

[0126] Connect SceneTexture:PostProcessInput0 directly to Emissive (this is the only output for post-processing materials); then create a new scene for the UE4 engine (VR_Basic is sufficient, no special requirements). Find the post-processing box (the post-processing box enables post-processing materials by default when the camera is within its bounds), manually add the post-processing material, and then change the blending weight to 1 (meaning post-processing materials are enabled by default). At this point, the scene remains unchanged because no post-processing material has been applied. Then, you can further process the color information of the current scene, such as blending a color. To do this, multiply SceneTexture:PostProcessInput0 and Vector3. Returning to the scene, you will find that the entire image has been blended with a red filter.

[0127] The above describes how developers can perform blending calculations on the entire screen using the data from SceneTexture:PostProcessInput0 of post-processing materials. Similarly, blending can also be done using masks. SceneTexture:PostProcessInput0 will then be referred to as PPV for short, and then directly compared with... Figure 5Multiplying the black and white masks as shown yields the result. Since the left side of the mask is pure white, corresponding to the data 1, multiplying any data by 1 results in the same result as the data itself. Therefore, the left triangle remains unchanged. The right triangle's data is 0, so multiplying any data by 0 results in 0, which is pure black. Figure 11 The diagram shows a graphical user interface. When the position of the current scene capture component is moved, the left side of the screen updates, but the right side remains pure black. Therefore, the pure black part on the right can be used as a mask for developers to create various effects.

[0128] Instead of multiplication, a Lerp node is used for blending. The Lerp node has three inputs: A, B, and Alpha. The function of the Lerp node is to determine the output result based on the Alpha input. The closer the Alpha value is to 0, the closer the output result is to A; conversely, the closer the Alpha value is to 1, the closer the output result is to B. If the PPV data is given to B and A is given to 0, then the same result as multiplication will occur. It can be understood as first obtaining a black and white mask based on the screen UV, and then using the Lerp node for blending to divide the entire screen into two parts.

[0129] You can continue by multiplying the left part of the black and white mask by a color, which will then blend a green filter onto the left part. If you give different colors to the two inputs of Lerp, the result will be two solid colors. At this step, if you replace these two colors with images captured by two scene capture components, you can achieve the function of this embodiment.

[0130] like Figure 12 The diagram illustrates a graphical user interface, using an example where there is one first virtual object and one second virtual object. Two scene capture components capture the images of the two virtual objects respectively. When switching to a simultaneous screen view, the result is as follows: Figure 13 The diagram illustrates a graphical user interface that displays two virtual objects on the same screen. To enhance the display effect, a color can be overlaid on the material to achieve the desired result. Figure 14The diagram illustrates a graphical user interface, showing the segmented display effect of two virtual objects in a simultaneous screen. In actual implementation, the number of first virtual objects can be one or more, and the number of second virtual objects can also be one or more, depending on actual needs. For example, taking two first and two second virtual objects as an example, it is necessary to display each frame of four virtual objects simultaneously. This can be handled in pairs. For ease of explanation, the two first virtual objects are represented as object 1 and object 2, and the two second virtual objects are represented as object 3 and object 4. The black and white mask 1 corresponding to object 1 and object 3, and the black and white mask 2 of object 2 and object 4 can be obtained respectively. Following the aforementioned steps, object 1 and object 3 are used as the two inputs of the first Lerp node 1, and the black and white mask 1 is used as the Alpha input of Lerp node 1. Through interpolation, each frame of the mask of object 1 is captured and displayed in the first area of ​​the simultaneous screen 1, and each frame of the mask of object 3 is captured and displayed in the second area of ​​the simultaneous screen 1.

[0131] Similarly, using Object 2 and Object 4 as the two inputs to the second Lerp node 2, and the black and white mask 2 as the Alpha input to Lerp node 2, interpolation is used to capture each frame of the mask image of Object 2 and display it in the first area of ​​the screen-sharing image 2. The same method captures each frame of the mask image of Object 4 and displays it in the second area of ​​the screen-sharing image 2. Using each frame of the screen-sharing image 1 and each frame of the screen-sharing image 2 as the two inputs to the third Lerp node 3, and the black and white mask 3 as the Alpha input to Lerp node 3, interpolation is used to capture each frame of the screen-sharing image containing Object 1 and Object 3 and display it in the first area of ​​the screen-sharing image 3. The same method captures each frame of the screen-sharing image containing Object 2 and Object 4 and displays it in the second area of ​​the screen-sharing image 3. The black and white masks 1, 2, and 3 can be the same or different. Through this nested design of multiple Lerp nodes, four virtual objects can be displayed on the same screen. This method can be used to implement screen-sharing methods for multiple first virtual objects and multiple second virtual objects, and will not be elaborated further here.

[0132] In response to the interpolation coefficient of the first interpolation component switching from a second value to a first value, a simultaneous multi-screen display end event is determined; wherein, the interpolation coefficient is based on the pre-set second material ball parameters switching from a second value to a first value. In response to the game's simultaneous multi-screen display end event, the simultaneous screen displayed in the graphical user interface is switched back to the game's virtual scene.

[0133] Continuing with the first interpolation component as the Lerp node and the interpolation coefficient as the Alpha value of the Lerp node, with an Alpha value of 0 or 1; for example, when a multi-screen display event occurs, the Alpha value remains at 1. When the Alpha value switches from 1 to 0, the display of the multi-screen display in the graphical user interface can be switched to the display of the current virtual scene. Refer to the relevant descriptions in the previous steps to modify the material ball parameters through the program script to switch the relevant switch parameters and trigger the end event of the multi-screen display event.

[0134] The above-mentioned method of rendering multiple scenes on the same screen can be used to render multiple scenes on the same screen. For example, two different models can be used to represent two different characters in the game. When the game plot progresses to a certain point, an event occurs that greatly shocks both characters in the game. In order to show that both characters are shocked, it is necessary to display the current expression state of the two characters on the same screen. At this time, the solution in this application can be used to render the scenes of the two characters on the same screen at the same time.

[0135] The aforementioned method for rendering simultaneous scenes on screen uses a material shader algorithm to calculate a black and white mask based on the screen's UVs. Then, SceneCaptureComponent2D captures the scene's RGB (R: Red; G: Green; B: Blue) information in real time and renders it to a RenderTarget. By reading the RenderTarget's image information from the material, the RGB information of the current RenderTarget can be rendered in real time. Finally, the game engine's PostProcessMaterial is used to directly present it on the screen. In game development, this method can directly implement the function of rendering simultaneous multi-camera scenes within the game engine, and content modifications take effect in real time. It is highly scalable, thus breaking away from the constraints of traditional video post-production compositing methods, achieving the advantages of rapid production, rapid iteration, and low modification costs.

[0136] This invention provides a screen-on-screen rendering device, wherein the virtual scene of the game includes at least one virtual object, such as... Figure 15As shown, the device includes: a capture module 150, used to capture each frame of a first virtual object in the game through a first scene capture component; and to capture each frame of a second virtual object in the game through a second scene capture component; an acquisition module 151, used to acquire a preset first black and white mask and a second black and white mask; wherein the second black and white mask is obtained by inverting the first black and white mask; each frame of the first virtual object, each frame of the second virtual object, the first black and white mask, and the second black and white mask have the same size; a processing module 152, used to perform masking processing on each frame of the first virtual object using the first black and white mask to obtain each frame mask of the first virtual object; and to perform masking processing on each frame of the second virtual object using the second black and white mask to obtain each frame mask of the second virtual object; and a combination module 153, used to combine each frame mask of the first virtual object and each frame mask of the second virtual object to render each frame of the same-screen image.

[0137] The aforementioned screen-on-screen rendering device allows different scene capture components to capture each frame of their respective virtual objects, perform masking processing using their respective black and white masks, and combine the masked images to obtain the screen-on-screen image. This eliminates the need for additional manpower and resources to pre-produce the screen-on-screen image, thereby shortening the production cycle, reducing costs, and making it easy for users to modify, thus improving scalability.

[0138] In an optional implementation, the first area of ​​each frame of the same screen is used to play the mask image of the first virtual object, and the second area of ​​each frame of the same screen is used to play the mask image of the second virtual object.

[0139] In an alternative implementation, the number of the first virtual object and / or the second virtual object is one or more.

[0140] In an optional implementation, the capture module 150 is configured to: capture each frame of the initial image of the first virtual object in the game through the first scene capture component; process each frame of the initial image of the first virtual object according to a preset processing method to obtain each frame of the first virtual object; wherein the preset processing method includes at least one of the following: scaling processing, offset processing, and color adjustment processing.

[0141] In an optional implementation, the capture module 150 is configured to: capture each frame of the initial image of the second virtual object in the game through the second scene capture component; and process each frame of the initial image of the second virtual object according to a preset processing method to obtain each frame of the second virtual object.

[0142] In an optional implementation, the dimensions of each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask are all matched with the screen size of the display screen that displays each frame of the same image.

[0143] In an optional embodiment, the device further includes a first black-and-white mask acquisition module, which obtains the first black-and-white mask. This module is used to: acquire a first texture image in an initial screen UV coordinate system; convert the first texture image from the initial screen UV coordinate system to a screen UV Cartesian coordinate system to obtain a converted second texture image; calculate the slope of the pixel coordinates of each pixel in the second texture image relative to the positive horizontal axis of the screen UV Cartesian coordinate system; determine the first mask corresponding to the second texture image based on the slope; wherein the first mask includes: a black area, a white area, and a grayscale area; and obtain a first black-and-white mask based on the first mask.

[0144] In an optional implementation, the first black-and-white mask acquisition module is further configured to: adjust each slope based on a preset variable value to adjust the area range of the grayscale region according to each adjusted slope; and obtain the first black-and-white mask based on each adjusted slope and the distance of each pixel in the first mask from the center of the screen.

[0145] The screen-sharing rendering device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned screen-sharing rendering method embodiment. For the sake of brevity, any parts not mentioned in the screen-sharing rendering device embodiment can be referred to the corresponding content in the aforementioned screen-sharing rendering method embodiment.

[0146] This invention also provides an electronic device, see [link to relevant documentation]. Figure 16 As shown, the electronic device includes a processor 130 and a memory 131. The memory 131 stores machine-executable instructions that can be executed by the processor 130. The processor 130 executes the machine-executable instructions to implement the above-mentioned screen rendering method.

[0147] Furthermore, Figure 16 The electronic device shown also includes a bus 132 and a communication interface 133, with the processor 130, the communication interface 133 and the memory 131 connected via the bus 132.

[0148] The memory 131 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 133 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network. The bus 132 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 16 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0149] Processor 130 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 130 or by instructions in software form. Processor 130 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 131. The processor 130 reads information from memory 131 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiment. The virtual scene of the game includes at least one virtual object; specifically, the following steps are executed:

[0150] The game captures each frame of a first virtual object using a first scene capture component; captures each frame of a second virtual object using a second scene capture component; obtains a preset first black-and-white mask and a second black-and-white mask; wherein the second black-and-white mask is obtained by inverting the first black-and-white mask; each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask have the same size; the first black-and-white mask is used to mask each frame of the first virtual object to obtain each frame of the masked image of the first virtual object; the second black-and-white mask is used to mask each frame of the second virtual object to obtain each frame of the masked image of the second virtual object; the first and second frame masked images of the masked images of the first and second virtual objects are combined to render each frame of the same-screen image.

[0151] The specific implementation methods and working processes can be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0152] In an optional implementation, the first area of ​​each frame of the same screen is used to play the mask image of the first virtual object, and the second area of ​​each frame of the same screen is used to play the mask image of the second virtual object.

[0153] In an alternative implementation, the number of the first virtual object and / or the second virtual object is one or more.

[0154] In an optional implementation, when the processor captures each frame of the first virtual object in the game through the first scene capture component, it is specifically used to: capture each initial frame of the first virtual object in the game through the first scene capture component; process each initial frame of the first virtual object according to a preset processing method to obtain each frame of the first virtual object; wherein, the preset processing method includes at least one of the following: scaling processing, offset processing, and color adjustment processing.

[0155] In an optional implementation, when the processor captures each frame of the second virtual object in the game through the second scene capture component, it is specifically used to: capture each initial frame of the second virtual object in the game through the second scene capture component; process each initial frame of the second virtual object according to a preset processing method to obtain each frame of the second virtual object.

[0156] In an optional implementation, the dimensions of each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask are all matched with the screen size of the display screen that displays each frame of the same image.

[0157] In an optional implementation, when the processor acquires the first black-and-white mask, it specifically performs the following steps: acquires a first texture image in the initial screen UV coordinate system; converts the first texture image from the initial screen UV coordinate system to the screen UV Cartesian coordinate system to obtain a converted second texture image; calculates the slope of the pixel coordinates of each pixel in the second texture image relative to the positive direction of the horizontal axis of the screen UV Cartesian coordinate system; determines the first mask corresponding to the second texture image based on the slope; wherein the first mask includes: a black area, a white area, and a grayscale area; and obtains a first black-and-white mask based on the first mask.

[0158] In an optional implementation, when the processor acquires the first black and white mask, it is further configured to: adjust each slope based on a preset variable value to adjust the area range of the grayscale region according to each adjusted slope; and obtain the first black and white mask based on each adjusted slope and the distance of each pixel in the first mask from the center of the screen.

[0159] The above method involves capturing each frame of a first virtual object in the game using a first scene capture component; capturing each frame of a second virtual object in the game using a second scene capture component; obtaining preset first and second black-and-white masks; wherein the second black-and-white mask is obtained by inverting the first black-and-white mask; the dimensions of each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask are the same; masking is performed on each frame of the first virtual object using the first black-and-white mask to obtain each frame of the masked image of the first virtual object; masking is performed on each frame of the second virtual object using the second black-and-white mask to obtain each frame of the masked image of the second virtual object; and the first and second frame masked images of the masked images of the first and second virtual objects are combined to render each frame of the same-screen image. In this method, different scene capture components can capture each frame of their respective virtual objects, perform masking using their respective black-and-white masks, and combine the masked images to obtain the same-screen image. This eliminates the need for additional manpower and resources to pre-create the same-screen image, thus shortening the production cycle, reducing costs, and allowing users to easily modify it, improving scalability.

[0160] This invention also provides a machine-readable storage medium storing machine-executable instructions. When these machine-executable instructions are invoked and executed by a processor, they cause the processor to perform the following steps:

[0161] The game captures each frame of a first virtual object using a first scene capture component; captures each frame of a second virtual object using a second scene capture component; obtains a preset first black-and-white mask and a second black-and-white mask; wherein the second black-and-white mask is obtained by inverting the first black-and-white mask; each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask have the same size; the first black-and-white mask is used to mask each frame of the first virtual object to obtain each frame of the masked image of the first virtual object; the second black-and-white mask is used to mask each frame of the second virtual object to obtain each frame of the masked image of the second virtual object; the first and second frame masked images of the masked images of the first and second virtual objects are combined to render each frame of the same-screen image.

[0162] The specific implementation methods and working processes can be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0163] In an optional implementation, the first area of ​​each frame of the same screen is used to play the mask image of the first virtual object, and the second area of ​​each frame of the same screen is used to play the mask image of the second virtual object.

[0164] In an alternative implementation, the number of the first virtual object and / or the second virtual object is one or more.

[0165] In an optional implementation, when the processor captures each frame of the first virtual object in the game through the first scene capture component, it is specifically used to: capture each initial frame of the first virtual object in the game through the first scene capture component; process each initial frame of the first virtual object according to a preset processing method to obtain each frame of the first virtual object; wherein, the preset processing method includes at least one of the following: scaling processing, offset processing, and color adjustment processing.

[0166] In an optional implementation, when the processor captures each frame of the second virtual object in the game through the second scene capture component, it is specifically used to: capture each initial frame of the second virtual object in the game through the second scene capture component; process each initial frame of the second virtual object according to a preset processing method to obtain each frame of the second virtual object.

[0167] In an optional implementation, the dimensions of each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask are all matched with the screen size of the display screen that displays each frame of the same image.

[0168] In an optional implementation, when the processor acquires the first black-and-white mask, it specifically performs the following steps: acquires a first texture image in the initial screen UV coordinate system; converts the first texture image from the initial screen UV coordinate system to the screen UV Cartesian coordinate system to obtain a converted second texture image; calculates the slope of the pixel coordinates of each pixel in the second texture image relative to the positive direction of the horizontal axis of the screen UV Cartesian coordinate system; determines the first mask corresponding to the second texture image based on the slope; wherein the first mask includes: a black area, a white area, and a grayscale area; and obtains a first black-and-white mask based on the first mask.

[0169] In an optional implementation, when the processor acquires the first black and white mask, it is further configured to: adjust each slope based on a preset variable value to adjust the area range of the grayscale region according to each adjusted slope; and obtain the first black and white mask based on each adjusted slope and the distance of each pixel in the first mask from the center of the screen.

[0170] The above method involves capturing each frame of a first virtual object in the game using a first scene capture component; capturing each frame of a second virtual object in the game using a second scene capture component; obtaining preset first and second black-and-white masks; wherein the second black-and-white mask is obtained by inverting the first black-and-white mask; the dimensions of each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask are the same; masking is performed on each frame of the first virtual object using the first black-and-white mask to obtain each frame of the masked image of the first virtual object; masking is performed on each frame of the second virtual object using the second black-and-white mask to obtain each frame of the masked image of the second virtual object; and the first and second frame masked images of the masked images of the first and second virtual objects are combined to render each frame of the same-screen image. In this method, different scene capture components can capture each frame of their respective virtual objects, perform masking using their respective black-and-white masks, and combine the masked images to obtain the same-screen image. This eliminates the need for additional manpower and resources to pre-create the same-screen image, thus shortening the production cycle, reducing costs, and allowing users to easily modify it, improving scalability.

[0171] The computer program products of the screen-on-screen rendering method, apparatus and electronic device provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0172] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for rendering images on the same screen, characterized in that, The game's virtual scene includes at least one virtual object, and the method includes: The first scene capture component captures each frame of the first virtual object in the game; the second scene capture component captures each frame of the second virtual object in the game. Obtain a preset first black and white mask and a preset second black and white mask; wherein, the second black and white mask is obtained by inverting the first black and white mask; each frame of the first virtual object, each frame of the second virtual object, the first black and white mask, and the second black and white mask have the same size; The first black and white mask is used to mask each frame of the first virtual object to obtain each frame of the masked image of the first virtual object; the second black and white mask is used to mask each frame of the second virtual object to obtain each frame of the masked image of the second virtual object. The mask images of each frame of the first virtual object and the mask images of each frame of the second virtual object are combined to render each frame of the same screen. The steps of capturing each frame of the first virtual object in the game using the first scene capture component include: The initial frame of each virtual object in the game is captured by the first scene capture component; According to a preset processing method, each frame of the initial image of the first virtual object is processed to obtain each frame of the first virtual object; wherein, the preset processing method includes at least one of the following: scaling processing, offset processing, and color adjustment processing; The steps of capturing each frame of the second virtual object in the game using the second scene capture component include: The initial frame of each virtual object in the game is captured by the second scene capture component; According to the preset processing method, each frame of the initial image of the second virtual object is processed to obtain each frame of the second virtual object.

2. The method according to claim 1, characterized in that, The first area of ​​each frame of the same-screen image is used to play the mask image of each frame of the first virtual object, and the second area of ​​each frame of the same-screen image is used to play the mask image of each frame of the second virtual object.

3. The method according to claim 1, characterized in that, The number of the first virtual objects is one or more; the number of the second virtual objects is one or more.

4. The method according to claim 1, characterized in that, The dimensions of each frame of the first virtual object, each frame of the second virtual object, the first black-and-white mask, and the second black-and-white mask are all matched with the screen size of the display screen that displays each frame of the same-screen image.

5. The method according to claim 1, characterized in that, The first black-and-white mask is obtained in the following way: Obtain the first texture image in the initial screen UV coordinate system; The first texture image is converted from the initial screen UV coordinate system to the screen UV Cartesian coordinate system to obtain the converted second texture image; Calculate the slope of the pixel coordinates of each pixel in the second texture image relative to the positive horizontal axis of the screen UV Cartesian coordinate system; Based on the slope, a first mask corresponding to the second texture image is determined; wherein, the first mask includes: a black area, a white area, and a grayscale area; Based on the first mask, the first black and white mask is obtained.

6. The method according to claim 5, characterized in that, The steps to obtain the first black-and-white mask based on the first mask include: Each slope is adjusted based on a preset variable value, so as to adjust the area range of the grayscale region according to each adjusted slope; The first black and white mask is obtained based on each adjusted slope and the distance of each pixel in the first mask from the center of the screen.

7. A device for rendering images on the same screen, characterized in that, The game's virtual scene includes at least one virtual object, and the device includes: The capture module is used to capture each frame of a first virtual object in the game through a first scene capture component; and to capture each frame of a second virtual object in the game through a second scene capture component. The acquisition module is used to acquire a preset first black and white mask and a second black and white mask; wherein, the second black and white mask is obtained by inverting the first black and white mask; each frame of the first virtual object, each frame of the second virtual object, the first black and white mask, and the second black and white mask have the same size; The processing module is used to perform masking processing on each frame of the first virtual object using the first black and white mask to obtain each frame of the first virtual object mask; and to perform masking processing on each frame of the second virtual object using the second black and white mask to obtain each frame of the second virtual object mask. The combination module is used to combine each frame of the mask image of the first virtual object and each frame of the mask image of the second virtual object to render each frame of the same screen image. The capture module is also used for: The initial frame of each virtual object in the game is captured by the first scene capture component; According to a preset processing method, each frame of the initial image of the first virtual object is processed to obtain each frame of the first virtual object; wherein, the preset processing method includes at least one of the following: scaling processing, offset processing, and color adjustment processing; The initial frame of each virtual object in the game is captured by the second scene capture component; According to the preset processing method, each frame of the initial image of the second virtual object is processed to obtain each frame of the second virtual object.

8. An electronic device, characterized in that, The method includes a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the screen rendering method according to any one of claims 1-6.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the screen rendering method according to any one of claims 1-6.

Images