Rendering system and method for virtual scene

Abstract

The invention relates to a rendering system and method for a virtual scene. A first rendering server is used for performing rendering task division on the virtual scene needing to be rendered currently to determine image rendering tasks and brightness rendering tasks; executing a picture rendering task by using each second rendering server, and sending the generated target rendering picture to the first rendering server; the third rendering server is used for executing the brightness rendering task, a generated brightness rendering picture is sent to the first rendering server, and the brightness rendering picture is used for indicating brightness information corresponding to all positions in the whole picture of the virtual scene; and finally, adjusting the brightness of the plurality of target rendering pictures according to the brightness rendering picture by using the first rendering server, and splicing to obtain a spliced picture corresponding to the virtual scene. The transition in the spliced picture is more natural and smooth, and the application scene of multi-machine parallel rendering is expanded.

Description

Technical Field

[0001] The present disclosure relates to the field of virtual shooting technology, and in particular to a rendering system and method for a virtual scene. Background Art

[0002] Virtual photography is a photography method that uses computer-generated images and real-time rendering technology. In the virtual photography solution, it is necessary to project a real-time rendered virtual scene on the screen. In order to make the displayed virtual scene have higher clarity and frame rate, the related technology uses multi-machine parallel rendering to render the virtual scene. However, in this method, the rendering results of different machines need to be spliced ​​and fused into a complete virtual scene before it can be displayed. However, since the images rendered by different machines have certain brightness differences, there are problems such as excessive abruptness and lack of naturalness in the spliced ​​images. Summary of the invention

[0003] In view of this, the present disclosure proposes a rendering system and method for a virtual scene.

[0004] According to one aspect of the present disclosure, a rendering system for a virtual scene is provided, the system comprising a first rendering server, a plurality of second rendering servers and a third rendering server.

[0005] The first rendering server is used to divide the rendering tasks for the virtual scene that currently needs to be rendered, and determine the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering server;

[0006] Each of the second rendering servers is configured to execute the received picture rendering task and send the generated target rendering picture to the first rendering server;

[0007] The third rendering server is used to execute the received brightness rendering task and send the generated brightness rendering picture to the first rendering server, where the brightness rendering picture is used to indicate the brightness information corresponding to each position in the overall picture of the virtual scene;

[0008] The first rendering server is further configured to adjust the brightness of a plurality of target rendering pictures according to the brightness rendering picture and then stitch them together to obtain a stitched picture corresponding to the virtual scene.

[0009] In a possible implementation, dividing the rendering tasks for the virtual scene currently to be rendered, and determining the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering servers, includes:

[0010] Determining, according to the scene resolution of the virtual scene, a second resolution for the third rendering server to render brightness information, and generating a brightness rendering task for the third rendering server;

[0011] The brightness rendering task is used to indicate a second resolution of a brightness picture to be rendered and the position information of the camera, and the second resolution is smaller than the scene resolution.

[0012] In a possible implementation, the second resolution is related to the first resolution of the target picture rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering server match.

[0013] In a possible implementation, executing the received brightness rendering task and sending the generated brightness rendering picture to the first rendering server includes:

[0014] Rendering the virtual scene based on the posture information carried in the brightness rendering task and the second resolution, to obtain a virtual scene rendering result in RGB format and with the second resolution;

[0015] After converting the virtual scene rendering result from RGB format to YUV format, Y channel data is extracted to form a brightness rendering picture.

[0016] In a possible implementation, adjusting the brightness of a plurality of target rendering pictures according to the brightness rendering picture and then splicing them to obtain a spliced ​​picture corresponding to the virtual scene includes:

[0017] After adjusting the brightness of a plurality of target rendering pictures respectively according to the brightness rendering picture, a plurality of adjusted pictures are obtained;

[0018] The plurality of adjusted images are spliced ​​and merged according to the area range of each of the adjusted images in the virtual scene to obtain a spliced ​​image corresponding to the virtual scene.

[0019] In a possible implementation, after brightness adjustment is performed on a plurality of target rendering pictures respectively according to the brightness rendering picture, a plurality of adjusted pictures are obtained, including:

[0020] According to the scene resolution corresponding to the virtual scene, super-resolution processing is performed on the brightness rendering picture to obtain a processed brightness picture with the same resolution as the scene resolution;

[0021] According to the brightness information of each position in the processed brightness picture, the color value of the pixel at the corresponding position in each of the target rendering pictures is adjusted to obtain a plurality of adjusted pictures.

[0022] In a possible implementation, the picture rendering task is further used to indicate a rendering algorithm required to be used to render the target picture, and the rendering algorithm includes an algorithm related to lighting and / or shadow.

[0023] According to another aspect of the present disclosure, a rendering method for a virtual scene is provided, which is applied to a rendering system, wherein the rendering system includes a first rendering server, a plurality of second rendering servers, and a third rendering server, and the method includes:

[0024] Controlling the first rendering server to divide the rendering tasks for the virtual scene currently to be rendered, and determining the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering server;

[0025] Controlling each of the second rendering servers to execute the received picture rendering tasks to generate a target rendering picture;

[0026] Controlling the third rendering server to execute the received brightness rendering task and generate a brightness rendering picture, where the brightness rendering picture is used to indicate brightness information corresponding to each position in the overall picture of the virtual scene;

[0027] The first rendering server is controlled to adjust the brightness of multiple target rendering pictures according to the brightness rendering picture and then splice them to obtain a spliced ​​picture corresponding to the virtual scene.

[0028] In a possible implementation, controlling the first rendering server to divide the rendering tasks for the virtual scene currently to be rendered, and determining the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering server include:

[0029] Determining, according to the scene resolution of the virtual scene, a second resolution for the third rendering server to render brightness information, and generating a brightness rendering task for the third rendering server;

[0030] The brightness rendering task is used to indicate a second resolution of a brightness picture to be rendered and the position information of the camera, and the second resolution is smaller than the scene resolution.

[0031] In a possible implementation, the second resolution is related to the first resolution of the target picture rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering server match.

[0032] In a possible implementation manner, controlling the third rendering server to execute the received brightness rendering task to generate a brightness rendering picture includes:

[0033] Rendering the virtual scene based on the posture information carried in the brightness rendering task and the second resolution, to obtain a virtual scene rendering result in RGB format and with the second resolution;

[0034] After converting the virtual scene rendering result from RGB format to YUV format, Y channel data is extracted to form a brightness rendering picture.

[0035] In a possible implementation, controlling the first rendering server to adjust the brightness of multiple target rendering pictures according to the brightness rendering picture and then stitching them together to obtain a stitched picture corresponding to the virtual scene includes:

[0036] After adjusting the brightness of a plurality of target rendering pictures respectively according to the brightness rendering picture, a plurality of adjusted pictures are obtained;

[0037] The plurality of adjusted images are spliced ​​and merged according to the area range of each of the adjusted images in the virtual scene to obtain a spliced ​​image corresponding to the virtual scene.

[0038] In a possible implementation, after brightness adjustment is performed on a plurality of target rendering pictures respectively according to the brightness rendering picture, a plurality of adjusted pictures are obtained, including:

[0039] According to the scene resolution corresponding to the virtual scene, super-resolution processing is performed on the brightness rendering picture to obtain a processed brightness picture with the same resolution as the scene resolution;

[0040] According to the brightness information of each position in the processed brightness picture, the color value of the pixel at the corresponding position in each of the target rendering pictures is adjusted to obtain a plurality of adjusted pictures.

[0041] In a possible implementation, the picture rendering task is further used to indicate a rendering algorithm required to be used to render the target picture, and the rendering algorithm includes an algorithm related to lighting and / or shadow.

[0042] According to another aspect of the present disclosure, a rendering device for a virtual scene is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0043] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0044] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0045] The rendering system and method for virtual scenes provided by the embodiment of the present disclosure use a first rendering server to divide the rendering tasks for the virtual scene currently to be rendered, determine the picture rendering tasks of each second rendering server and the brightness rendering tasks of the third rendering server; use each second rendering server to execute the received picture rendering tasks, and send the generated target rendering picture to the first rendering server; use the third rendering server to execute the received brightness rendering tasks, and send the generated brightness rendering picture to the first rendering server, the brightness rendering picture is used to indicate the brightness information corresponding to each position in the overall picture of the virtual scene; finally, the first rendering server is used to adjust the brightness of multiple target rendering pictures according to the brightness rendering picture and then splice them to obtain a spliced ​​picture corresponding to the virtual scene. After adjusting the brightness of multiple target rendering pictures based on the brightness rendering picture, the brightness difference of different target rendering pictures is reduced, making the transition in the spliced ​​picture more natural and smooth, and because the effect of the spliced ​​picture is consistent with the effect of a single server performing virtual scene picture rendering, the scene that can only be rendered by a single machine can be changed to multi-machine parallel rendering, expanding the application scenario of multi-machine parallel rendering.

[0046] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0048] Figure 1 A structural block diagram of a rendering system for a virtual scene according to an embodiment of the present disclosure is shown.

[0049] Figure 2 A schematic diagram of a rendering process of a rendering system for a virtual scene according to an embodiment of the present disclosure is shown.

[0050] Figure 3 It is a block diagram of a device 1900 for rendering a virtual scene according to an exemplary embodiment. DETAILED DESCRIPTION

[0051] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0052] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0053] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0054] Virtual Production is a film and entertainment production method that uses computer graphics technology and real-time rendering technology to combine the digital world with the physical world in real time during the production process. With the continuous development of virtual shooting, the content of virtual scenes displayed on LED screens is becoming more and more complex, and the amount of data processing for picture rendering is also increasing. To solve this problem, related technologies use multi-machine parallel rendering. Each machine renders a part of the picture content separately. After obtaining the rendered pictures returned by each machine, multiple rendered pictures are spliced ​​and merged to obtain the final overall picture. However, in the process of rendering the picture on each machine, each machine will calculate the lighting and shadows in the picture based on the screen space algorithm (Screen space algorithms refer to the technology of performing calculations within a two-dimensional image or screen space. These algorithms usually involve post-processing operations on the rasterized images to enhance the rendering effect. The core of the screen space algorithm is to use the information of the screen space, such as depth, normals, color, etc., to simulate complex lighting and reflection effects, thereby improving the visual quality of the image). Therefore, there will be certain brightness differences in the rendered pictures of different machines, resulting in the joints in the overall stitched picture being too abrupt and unnatural.

[0055] In order to solve the above technical problems, the embodiment of the present disclosure provides a rendering system and method for a virtual scene, wherein a first rendering server is used to divide the rendering tasks for the virtual scene currently to be rendered, and the picture rendering tasks of each second rendering server and the brightness rendering tasks of the third rendering server are determined; each second rendering server is used to execute the received picture rendering tasks, and the generated target rendering picture is sent to the first rendering server; the third rendering server is used to execute the received brightness rendering tasks, and the generated brightness rendering picture is sent to the first rendering server, and the brightness rendering picture is used to indicate the brightness information corresponding to each position in the overall picture of the virtual scene; finally, the first rendering server is used to adjust the brightness of multiple target rendering pictures according to the brightness rendering picture and then splice them to obtain a spliced ​​picture corresponding to the virtual scene. After adjusting the brightness of multiple target rendering pictures based on the brightness rendering picture, the brightness difference of different target rendering pictures is reduced, and the transition in the spliced ​​picture is more natural and smooth. Moreover, since the effect of the spliced ​​picture is consistent with the effect of rendering the virtual scene picture by a single server, the scene that can only be rendered by a single machine can be changed to multi-machine parallel rendering, which expands the application scenario of multi-machine parallel rendering.

[0056] like Figure 1 , Figure 2 As shown, the rendering system for a virtual scene provided by an embodiment of the present disclosure includes multiple rendering servers, including a first rendering server, multiple second rendering servers and a third rendering server. In some embodiments, the rendering system can be set as a rendering cluster in virtual shooting.

[0057] The first rendering server is used to divide the rendering tasks for the virtual scene that currently needs to be rendered, and determine the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering server. The first rendering server, the second rendering server, and the third rendering server can be physically independent devices, or can be integrated on one or more physical devices, which is not limited in the present disclosure.

[0058] In this embodiment, in virtual shooting, the first rendering server can determine the viewing angle of the virtual scene that needs to be displayed on the screen based on the current posture of the camera (the posture includes the position and posture), and then determine the overall picture that needs to be displayed for the virtual scene, and then divide the rendering tasks for each overall picture of each virtual scene.

[0059] The image rendering task may refer to a task of rendering a target image in a partial area of ​​the overall image of the virtual scene, and the brightness rendering task may refer to a task of rendering brightness information of the overall image of the virtual scene. The area range size of the target image to be rendered by each second rendering server may be set according to the data processing capabilities of different second rendering servers and / or the rendering frame rate required for the overall image to meet the shooting requirements.

[0060] In a possible implementation, dividing the rendering tasks for the virtual scene that currently needs to be rendered, determining the image rendering tasks of each second rendering server and the brightness rendering tasks of the third rendering server, may include: determining the area range of the target image that each second rendering server needs to render and the corresponding first resolution according to the scene resolution of the virtual scene and the number of the second rendering servers, and generating the image rendering tasks of each second rendering server. The image rendering task is used to indicate the area range of the target image that needs to be rendered in the virtual scene and the corresponding first resolution, the camera's position information, etc., and the first resolution is smaller than the scene resolution.

[0061] The scene resolution may be the resolution corresponding to the overall image of the virtual scene, and the scene resolution may be determined based on the pixel pitch of the screen, the size of the screen, the distance between the camera and the screen in virtual shooting, the shooting angle of the camera, and the like. The smaller the pixel pitch of the screen and the larger the size of the screen, the higher the scene resolution. The distance between the camera and the screen affects the proportion of the screen in the camera's field of view. The closer the distance, the larger the proportion of the screen in the field of view, and the higher the required scene resolution. The shooting angle of the camera affects the way the screen image is presented in the camera's field of view. Therefore, the required scene resolution can be determined based on the requirements for maintaining the consistency and accuracy of the virtual scene display corresponding to different shooting angles. The higher the requirements, the higher the scene resolution.

[0062] Among them, for the overall picture of the same virtual scene, the posture of the camera determines the content of the overall picture that needs to be rendered in the end, so the posture information of the camera can be carried in the picture rendering task, so that the posture information can be used as the basis for picture rendering when the second rendering server executes the picture rendering task. The correspondence between the posture of the camera and the overall picture, as well as the correspondence between the overall picture and the picture rendering task of each second rendering server, can be set based on the actual situation and needs of the shooting. Among them, the posture information of the camera may include the camera position and posture. The posture information of the camera can be determined based on the tracking detection of the tracking device bound to the camera.

[0063] Among them, the area range of the target picture can be represented by the coordinate range of the target picture in the overall picture of the virtual scene, etc., and the present disclosure does not limit this. The first resolution can be the resolution of the target rendering picture that needs to be rendered, and the proportional relationship between the size of the first resolution and the scene resolution corresponds to the proportional relationship between the size of the target picture and the overall picture of the virtual scene. For example, assuming that the scene resolution of the overall picture of the virtual scene is 4K (3840×2160), if the target picture is 1 / 4 of the overall picture, the first resolution corresponding to the target picture can be 1920×1080.

[0064] In some embodiments, when the configuration and performance of the second rendering servers are the same, the rendering tasks can be divided by equally dividing the screen according to the number of second rendering servers. Then, according to the scene resolution of the virtual scene and the number of the second rendering servers, the area range of the target screen that each second rendering server needs to render and the corresponding first resolution are determined, and the screen rendering tasks of each second rendering server are generated, which may include: according to the scene resolution of the virtual scene and the number of the second rendering servers, the overall screen of the virtual scene is divided into multiple target screens according to the principle of equal division, and the number of the target screens is the same as the number of the second rendering servers; the area range and the corresponding first resolution corresponding to each of the target screens are determined, and the screen rendering tasks of each second rendering server are generated. For example, assuming that the scene resolution of the overall picture of the virtual scene is 4K (3840×2160) and there are four second rendering servers, the target pictures corresponding to the picture rendering tasks of the four second rendering servers are: target picture 1 in the upper left corner of the overall picture of the virtual scene, target picture 2 in the upper right corner of the overall picture of the virtual scene, target picture 3 in the lower left corner of the overall picture of the virtual scene, and target picture 4 in the upper right corner of the overall picture of the virtual scene; the first resolution corresponding to each target picture is 1920×1080.

[0065] In a possible implementation, the rendering tasks are divided for the virtual scene that needs to be rendered currently, and the picture rendering tasks of each second rendering server and the brightness rendering tasks of the third rendering server are determined, which may also include: determining the second resolution for the third rendering server to render the brightness information according to the scene resolution of the virtual scene, and generating the brightness rendering task of the third rendering server. The brightness rendering task is used to indicate the second resolution of the brightness picture that needs to be rendered and the position information of the camera, and the second resolution is smaller than the scene resolution.

[0066] In this implementation, there may be a proportional relationship between the scene resolution and the second resolution. In some embodiments, when the configuration and performance of each rendering server are the same, in order to meet the quality of the final spliced ​​picture, the proportional relationship may match the number of second rendering servers. For example, assuming that the scene resolution of the overall picture of the virtual scene is 4K (3840×2160) and there are 4 second rendering servers, the second resolution may be 1920×1080. In this way, while reducing the amount of data processing for rendering, it is ensured that the brightness rendering picture finally generated can carry the brightness information corresponding to each position in the overall picture of the virtual scene.

[0067] In this embodiment, the second resolution is related to the first resolution of the target picture rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering server match, ensuring that the entire rendering system can render the picture at a stable and efficient frame rate. In some embodiments, when the configuration and performance of the rendering servers (including the second rendering server and the third rendering server) are not completely the same, and / or the content complexity of different areas of the overall picture of the virtual scene varies greatly, the first rendering server can further set the area range and the first resolution and the second resolution of the target picture to be rendered by each second rendering server in real time based on the content complexity of different areas of the overall picture, the performance of each second rendering server, and the performance of the third rendering server in the process of dividing the rendering tasks, so as to make the processing speed of the second rendering server performing the picture rendering task and the third rendering server performing the brightness rendering task close as much as possible, thereby making the rendering frame rate of each target rendering picture and the rendering frame rate of the brightness rendering picture basically consistent, avoiding the overall frame rate drop due to the excessive time consumption of a certain rendering server (the second rendering server or the third rendering server), and ensuring that the entire rendering system can render the picture at a stable and efficient frame rate. The larger the area, the higher the first resolution, the slower the processing speed, and the lower the rendering frame rate of the target rendering picture. The higher the second resolution, the slower the processing speed and the lower the rendering frame rate of the brightness rendering picture. When the first resolution changes, the second resolution can be adaptively adjusted.

[0068] Each second rendering server is configured to execute the received screen rendering task and send the generated target rendered screen to the first rendering server. Each screen rendering task may indicate the area range, first resolution, camera position information, etc. of the target screen that the corresponding second rendering server needs to render, so that the second rendering server can render the virtual scene after receiving the screen rendering task, and finally form the target rendered screen.

[0069] The third rendering server is used to execute the received brightness rendering task and send the generated brightness rendering picture to the first rendering server, where the brightness rendering picture is used to indicate the brightness information corresponding to each position in the overall picture of the virtual scene.

[0070] In some embodiments, after receiving the brightness rendering task, the third rendering server can render the virtual scene based on the posture information carried in the brightness rendering task and the second resolution, and obtain a virtual scene rendering result in RGB format and with a second resolution; and then convert the virtual scene rendering result from RGB format to YUV format, and extract the Y channel data to form a brightness rendering picture. In other words, the third server can determine the overall picture of the virtual scene to be rendered based on the posture information of the camera, and then render a virtual scene rendering result in a two-dimensional RGB format and a size of the second resolution. After converting the virtual scene rendering result from RGB format to YUV format, the Y channel in the YUV format image represents brightness information, and the U and V channels represent chromaticity information, so by extracting the Y channel data in the virtual scene rendering result in YUV format, a brightness rendering picture representing the brightness information at each pixel position of the overall picture can be obtained. Those skilled in the art should understand that the third rendering server can also perform brightness rendering tasks in other ways, such as directly rendering a virtual scene rendering result in YUV format and with a size of the second resolution after determining the overall picture of the virtual scene to be rendered based on the position information of the camera, and extracting the data of the Y channel to obtain a brightness rendering picture. Alternatively, a brightness rendering picture with a size of the scene resolution can be rendered, and then downsampled to obtain a brightness rendering picture with the second resolution.

[0071] The first rendering server is further configured to adjust the brightness of multiple target rendering images according to the brightness rendering image and then splice them to obtain a spliced ​​image corresponding to the virtual scene. After obtaining the brightness rendering image, the first rendering server can adjust each target rendering image based on the brightness rendering image to make the brightness of different target rendering images harmonious and consistent, and ensure that the brightness transition between different target rendering images in the spliced ​​spliced ​​image is smooth and natural.

[0072] In some embodiments, adjusting the brightness of multiple target rendering images according to the brightness rendering image and then splicing them to obtain a spliced ​​image corresponding to the virtual scene may include: adjusting the brightness of multiple target rendering images according to the brightness rendering image to obtain multiple adjusted images; splicing and fusing the multiple adjusted images according to the area range of each adjusted image in the virtual scene to obtain a spliced ​​image corresponding to the virtual scene. Among them, since the brightness rendering image includes the brightness information of each position in the overall image, the color value of the pixel at the position can be adjusted according to the corresponding position of each position in the target rendering image in the overall image and the brightness information of the position in the brightness rendering image, so that the brightness of the position in the adjusted image is consistent with the brightness indicated by the brightness information of the position in the brightness rendering image, and it can ensure that the image effect is more natural when there are dynamic light and shadow changes or camera shaking in the virtual scene.

[0073] In some embodiments, in order to further improve the consistency between the brightness in the adjusted picture and the brightness rendering picture, multiple target rendering pictures are brightness adjusted respectively according to the brightness rendering picture to obtain multiple adjusted pictures, which may include: super-resolution processing of the brightness rendering picture according to the scene resolution corresponding to the virtual scene to obtain a processed brightness picture with the same resolution as the scene resolution; and then adjusting the color values ​​of pixels at corresponding positions in each of the target rendering pictures according to the brightness information of each position in the processed brightness picture to obtain multiple adjusted pictures.

[0074] For example, if the brightness rendering image after super-resolution processing has the same scene resolution as the complete virtual scene, then the sum of the number of pixels of the brightness rendering image after super-resolution processing is the same as that of all target rendering images, and the positions of the pixels have a one-to-one correspondence. The brightness information of any pixel position of the target rendering image can be replaced with the brightness information of the corresponding pixel position in the brightness rendering image after super-resolution processing to obtain the adjusted image.

[0075] Those skilled in the art should understand that the method of adjusting the brightness of multiple target rendering pictures respectively according to the brightness rendering picture is not limited to the above method. For example, super-resolution processing may be omitted, and the brightness information of a pixel in the brightness rendering picture may be directly used to replace the brightness information of multiple corresponding pixels in the target rendering picture. Taking the example of the number of second rendering servers as 4 described above, a pixel in the brightness rendering picture may correspond to four pixels of a target rendering picture, and the positions of the corresponding pixels in the complete virtual scene have a corresponding relationship. Alternatively, it is also possible to adjust the brightness of multiple target rendering pictures after making the required adjustments to the brightness information carried by the brightness rendering picture as needed.

[0076] In the related art, in order to ensure the natural transition between different areas in the spliced ​​picture, the rendering server will be controlled not to use algorithms related to lighting and shadows during the rendering process, and no rendering calculations will be performed on lighting and shadows during the picture rendering process. After the spliced ​​picture is formed, the lighting and shadow related calculations will be performed through post-processing, but this method will greatly affect the realism of the scene, the attractiveness of the visual effects of the picture, and the immersiveness. In the embodiment of the present disclosure, since a separate third rendering server is used in the system to render the brightness rendering picture, the accuracy of the brightness of each position in the spliced ​​picture can be guaranteed, so in the process of each second rendering server generating the target rendering picture, algorithms related to lighting and shadows can be used normally, such as the calculation of shadow maps (ShadowMap), the calculation of virtual shadow maps (Virtual ShadowMap), the calculation of shadow quality (Shadow Quality), the calculation of blooming, halo, Tyndall effect, and so on. Therefore, the picture rendering task is also used to indicate the rendering algorithm needed to render the target picture, and the rendering algorithm includes an algorithm related to lighting and / or shadow, so that each second rendering server can use an algorithm related to lighting and / or shadow during the picture rendering process, which can ensure the realism of the scene in the spliced ​​picture and enhance the attractiveness and immersion of the visual effects of the picture.

[0077] The embodiment of the present disclosure further provides a rendering method for a virtual scene, which is applied to a rendering system, wherein the rendering system includes a first rendering server, a plurality of second rendering servers, and a third rendering server, and the method includes:

[0078] Controlling the first rendering server to divide the rendering tasks for the virtual scene currently to be rendered, and determining the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering server;

[0079] Controlling each of the second rendering servers to execute the received picture rendering tasks to generate a target rendering picture;

[0080] Controlling the third rendering server to execute the received brightness rendering task and generate a brightness rendering picture, where the brightness rendering picture is used to indicate brightness information corresponding to each position in the overall picture of the virtual scene;

[0081] The first rendering server is controlled to adjust the brightness of multiple target rendering pictures according to the brightness rendering picture and then splice them to obtain a spliced ​​picture corresponding to the virtual scene.

[0082] In a possible implementation, controlling the first rendering server to divide the rendering tasks for the virtual scene currently to be rendered, and determining the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering server include:

[0083] Determining, according to the scene resolution of the virtual scene, a second resolution for the third rendering server to render brightness information, and generating a brightness rendering task for the third rendering server;

[0084] The brightness rendering task is used to indicate a second resolution of a brightness picture to be rendered and the position information of the camera, and the second resolution is smaller than the scene resolution.

[0085] In a possible implementation, the second resolution is related to the first resolution of the target picture rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering server match.

[0086] In a possible implementation manner, controlling the third rendering server to execute the received brightness rendering task to generate a brightness rendering picture includes:

[0087] Rendering the virtual scene based on the posture information carried in the brightness rendering task and the second resolution, to obtain a virtual scene rendering result in RGB format and with the second resolution;

[0088] After converting the virtual scene rendering result from RGB format to YUV format, Y channel data is extracted to form a brightness rendering picture.

[0089] In a possible implementation, controlling the first rendering server to adjust the brightness of multiple target rendering pictures according to the brightness rendering picture and then stitching them together to obtain a stitched picture corresponding to the virtual scene includes:

[0090] After adjusting the brightness of a plurality of target rendering pictures respectively according to the brightness rendering picture, a plurality of adjusted pictures are obtained;

[0091] The plurality of adjusted images are spliced ​​and merged according to the area range of each of the adjusted images in the virtual scene to obtain a spliced ​​image corresponding to the virtual scene.

[0092] In a possible implementation, after brightness adjustment is performed on a plurality of target rendering pictures respectively according to the brightness rendering picture, a plurality of adjusted pictures are obtained, including:

[0093] According to the scene resolution corresponding to the virtual scene, super-resolution processing is performed on the brightness rendering picture to obtain a processed brightness picture with the same resolution as the scene resolution;

[0094] According to the brightness information of each position in the processed brightness picture, the color value of the pixel at the corresponding position in each of the target rendering pictures is adjusted to obtain a plurality of adjusted pictures.

[0095] In a possible implementation, the picture rendering task is further used to indicate a rendering algorithm required to be used to render the target picture, and the rendering algorithm includes an algorithm related to lighting and / or shadow.

[0096] It should be noted that although the above embodiments are used as examples to introduce the rendering system and method for virtual scenes, those skilled in the art will appreciate that the present disclosure should not be limited thereto. In fact, the user can flexibly set each part and each step according to personal preferences and / or actual application scenarios, as long as it complies with the technical solution of the present disclosure.

[0097] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0098] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0099] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0100] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0101] Figure 3 1 is a block diagram of a device 1900 for rendering a virtual scene according to an exemplary embodiment. For example, the device 1900 may be provided as a server. Figure 3, the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the method performed by any of the rendering servers described above.

[0102] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0103] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0104] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0105] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0106] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0107] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0108] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0109] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0110] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0111] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0112] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A rendering system for a virtual scene, characterized in that: The system includes a first rendering server, a plurality of second rendering servers and a third rendering server, The first rendering server is used to divide the rendering tasks for the virtual scene that currently needs to be rendered, and determine the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering server; Each of the second rendering servers is configured to execute the received picture rendering task and send the generated target rendering picture to the first rendering server; The third rendering server is used to execute the received brightness rendering task and send the generated brightness rendering picture to the first rendering server, where the brightness rendering picture is used to indicate the brightness information corresponding to each position in the overall picture of the virtual scene; The first rendering server is further configured to adjust the brightness of a plurality of target rendering pictures according to the brightness rendering picture and then stitch them together to obtain a stitched picture corresponding to the virtual scene.

2. The system according to claim 1, characterized in that Dividing the rendering tasks for the virtual scene currently to be rendered, and determining the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering servers, includes: Determining, according to the scene resolution of the virtual scene, a second resolution for the third rendering server to render brightness information, and generating a brightness rendering task for the third rendering server; The brightness rendering task is used to indicate a second resolution of a brightness picture to be rendered and the position information of the camera, and the second resolution is smaller than the scene resolution.

3. The system according to claim 2, characterized in that The second resolution is related to the first resolution of the target picture rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering server match.

4. The system according to claim 2, characterized in that Executing the received brightness rendering task and sending the generated brightness rendering picture to the first rendering server includes: Rendering the virtual scene based on the posture information carried in the brightness rendering task and the second resolution, to obtain a virtual scene rendering result in RGB format and with the second resolution; After converting the virtual scene rendering result from RGB format to YUV format, Y channel data is extracted to form a brightness rendering picture.

5. The system according to claim 1, characterized in that Adjusting the brightness of a plurality of target rendering pictures according to the brightness rendering picture and then splicing them to obtain a spliced ​​picture corresponding to the virtual scene includes: After adjusting the brightness of a plurality of target rendering pictures respectively according to the brightness rendering picture, a plurality of adjusted pictures are obtained; The plurality of adjusted images are spliced ​​and merged according to the area range of each of the adjusted images in the virtual scene to obtain a spliced ​​image corresponding to the virtual scene.

6. The system according to claim 5, characterized in that After adjusting the brightness of a plurality of target rendering pictures respectively according to the brightness rendering picture, a plurality of adjusted pictures are obtained, including: According to the scene resolution corresponding to the virtual scene, super-resolution processing is performed on the brightness rendering picture to obtain a processed brightness picture with the same resolution as the scene resolution; According to the brightness information of each position in the processed brightness picture, the color value of the pixel at the corresponding position in each of the target rendering pictures is adjusted to obtain a plurality of adjusted pictures.

7. The system according to claim 5, characterized in that The picture rendering task is also used to indicate a rendering algorithm to be used for rendering a target picture, wherein the rendering algorithm includes an algorithm related to lighting and / or shadow.

8. A rendering method for a virtual scene, characterized in that: Applied to a rendering system, the rendering system includes a first rendering server, a plurality of second rendering servers and a third rendering server, the method includes: Controlling the first rendering server to divide the rendering tasks for the virtual scene currently to be rendered, and determining the picture rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering server; Controlling each of the second rendering servers to execute the received picture rendering tasks to generate a target rendering picture; Controlling the third rendering server to execute the received brightness rendering task and generate a brightness rendering picture, where the brightness rendering picture is used to indicate brightness information corresponding to each position in the overall picture of the virtual scene; The first rendering server is controlled to adjust the brightness of multiple target rendering pictures according to the brightness rendering picture and then splice them to obtain a spliced ​​picture corresponding to the virtual scene.

9. A rendering device for a virtual scene, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method of claim 8 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method of claim 8 is implemented.

11. A computer program product comprising computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, characterized in that: When the computer readable code is executed in a processor of an electronic device, the processor in the electronic device executes the method of claim 8 .

Images