Rendering System and Method for Virtual Scenes

Abstract

The present disclosure relates to a rendering system and method for a virtual scene. The first rendering server is used to divide the rendering tasks for the virtual scene that needs to be rendered currently to determine each frame rendering task and brightness rendering task; each second rendering server is used to execute the frame rendering task and send the generated target rendering frame to the first rendering server; the third rendering server is used to execute the brightness rendering task and send the generated brightness rendering frame to the first rendering server, and the brightness rendering frame is used to indicate the brightness information corresponding to each position in the overall frame of the virtual scene; finally, the first rendering server adjusts the brightness of multiple target rendering frames according to the brightness rendering frame and then splices them to obtain the spliced frame corresponding to the virtual scene. This makes the transition in the spliced frame more natural and smooth, and expands the application scenarios of multi-machine parallel rendering.

Description

Technical Field

[0001] The present disclosure relates to the field of virtual shooting technology, and in particular, to a rendering system and method for a virtual scene. Background Art

[0002] Virtual shooting is a shooting method that uses computer-generated images and real-time rendering technology. In a virtual shooting solution, it is necessary to project a real-time rendered virtual scene on a screen. To make the displayed virtual scene have higher clarity and frame rate, in related technologies, a multi-machine parallel rendering method is used to render the virtual scene. However, in this method, it is necessary to splice and fuse the rendering results of different machines into a complete virtual scene picture before it can be displayed. However, due to certain brightness differences in the pictures rendered by different machines, there are problems such as excessive abruptness and unnaturalness in the spliced picture. Summary of the Invention

[0003] In view of this, the present disclosure proposes a rendering system and method for a virtual scene.

[0004] According to one aspect of the present disclosure, there is provided a rendering system for a virtual scene, the system includes a first rendering server, a plurality of second rendering servers, and a third rendering server,

[0005] The first rendering server is configured to divide a rendering task for a currently required virtual scene to be rendered, and determine a picture rendering task for each of the second rendering servers and a brightness rendering task for the third rendering server;

[0006] Each of the second rendering servers is configured to execute the received picture rendering task and send the generated target rendering picture to the first rendering server;

[0007] The third rendering server is configured to execute the received brightness rendering task and send the generated brightness rendering picture to the first rendering server, and the brightness rendering picture is used to indicate the brightness information corresponding to each position in the overall picture of the virtual scene;

[0008] The first rendering server is further configured to adjust the brightness of a plurality of target rendering pictures according to the brightness rendering picture and then splice them to obtain a spliced picture corresponding to the virtual scene.

[0009] In a possible implementation manner, dividing a rendering task for a currently required virtual scene to be rendered, and determining a picture rendering task for each of the second rendering servers and a brightness rendering task for the third rendering server includes:

[0010] Determine the second resolution for the third rendering server to render the brightness information according to the scene resolution of the virtual scene, and generate a brightness rendering task for the third rendering server;

[0011] Wherein, the brightness rendering task is used to indicate the second resolution of the brightness screen to be rendered and the pose information of the camera, and the second resolution is smaller than the scene resolution.

[0012] In a possible implementation, the second resolution is related to the first resolution of the target screen rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering server match.

[0013] In a possible implementation, execute the received brightness rendering task, and send the generated brightness rendering screen to the first rendering server, including:

[0014] Render the virtual scene based on the pose information and the second resolution carried in the brightness rendering task to obtain a virtual scene rendering result in RGB format and with the second resolution;

[0015] After converting the virtual scene rendering result from RGB format to YUV format, extract the Y-channel data to form a brightness rendering screen.

[0016] In a possible implementation, adjust the brightness of multiple target rendering screens according to the brightness rendering screen and then splice them to obtain a spliced screen corresponding to the virtual scene, including:

[0017] Adjust the brightness of multiple target rendering screens respectively according to the brightness rendering screen to obtain multiple adjusted screens;

[0018] Splice and fuse the multiple adjusted screens according to the area ranges of the adjusted screens in the virtual scene to obtain a spliced screen corresponding to the virtual scene.

[0019] In a possible implementation, adjusting the brightness of multiple target rendering screens respectively according to the brightness rendering screen to obtain multiple adjusted screens, including:

[0020] Perform super-resolution processing on the brightness rendering screen according to the scene resolution corresponding to the virtual scene to obtain a processed brightness screen with the same resolution as the scene resolution;

[0021] Adjust the color values of the pixels at the corresponding positions in each target rendering screen according to the brightness information at each position in the processed brightness screen to obtain multiple adjusted screens.

[0022] In a possible implementation, the screen rendering task is further used to indicate the rendering algorithm required for rendering the target screen, and the rendering algorithm includes algorithms related to lighting and / or shadows.

[0023] According to another aspect of the present disclosure, a rendering method for a virtual scene is provided, which is applied to a rendering system. The rendering system includes a first rendering server, a plurality of second rendering servers, and a third rendering server. The method includes:

[0024] Controlling the first rendering server to divide the rendering task for the currently required virtual scene to be rendered, and determining the screen rendering tasks of the second rendering servers and the brightness rendering task of the third rendering server;

[0025] Controlling each of the second rendering servers to execute the received screen rendering task to generate a target rendering screen;

[0026] Controlling the third rendering server to execute the received brightness rendering task to generate a brightness rendering screen, where the brightness rendering screen is used to indicate the brightness information corresponding to each position in the overall screen of the virtual scene;

[0027] Controlling the first rendering server to adjust and splice the brightness of multiple target rendering screens according to the brightness rendering screen to obtain the spliced screen corresponding to the virtual scene.

[0028] In a possible implementation, controlling the first rendering server to divide the rendering task for the currently required virtual scene to be rendered, and determining the screen rendering tasks of the second rendering servers and the brightness rendering task of the third rendering server includes:

[0029] According to the scene resolution of the virtual scene, determining the second resolution for the third rendering server to render brightness information, and generating the brightness rendering task of the third rendering server;

[0030] Wherein, the brightness rendering task is used to indicate the second resolution of the brightness screen to be rendered and the pose information of the camera, and the second resolution is less than the scene resolution.

[0031] In a possible implementation, the second resolution is related to the first resolution of the target screen rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering servers are matched.

[0032] In a possible implementation, controlling the third rendering server to execute the received brightness rendering task to generate a brightness rendering screen includes:

[0033] Render the virtual scene based on the pose information carried in the brightness rendering task and the second resolution, to obtain a virtual scene rendering result in RGB format and with the second resolution;

[0034] After converting the virtual scene rendering result from RGB format to YUV format, extract the Y-channel data to form a brightness rendering image.

[0035] In a possible implementation, controlling the first rendering server to adjust and splice the brightness of multiple target rendering images according to the brightness rendering image to obtain a spliced image corresponding to the virtual scene includes:

[0036] After respectively adjusting the brightness of multiple target rendering images according to the brightness rendering image, obtain multiple adjusted images;

[0037] Splice and fuse the multiple adjusted images according to the regional scope of each adjusted image in the virtual scene to obtain a spliced image corresponding to the virtual scene.

[0038] In a possible implementation, after respectively adjusting the brightness of multiple target rendering images according to the brightness rendering image, obtaining multiple adjusted images includes:

[0039] Perform super-resolution processing on the brightness rendering image according to the scene resolution corresponding to the virtual scene to obtain a processed brightness image with the same resolution as the scene resolution;

[0040] According to the brightness information of each position in the processed brightness image, adjust the color values of the pixels at the corresponding positions in each target rendering image to obtain multiple adjusted images.

[0041] In a possible implementation, the image rendering task is further used to indicate the rendering algorithm required for rendering the target image, and the rendering algorithm includes algorithms related to lighting and / or shadows.

[0042] According to another aspect of the present disclosure, there is provided a rendering device for a virtual scene, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to implement the above method when executing the instructions stored in the memory.

[0043] According to another aspect of the present disclosure, there is provided a non-volatile computer-readable storage medium, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0044] According to another aspect of the present disclosure, there is provided a computer program product including computer-readable code or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0045] The rendering system and method for a virtual scene provided by the embodiments of the present disclosure divide the rendering tasks for the currently required virtual scene by using a first rendering server, and determine the screen rendering tasks of each second rendering server and the brightness rendering tasks of a third rendering server; each second rendering server is used to execute the received screen rendering tasks and send the generated target rendering screens to the first rendering server; the third rendering server is used to execute the received brightness rendering tasks and send the generated brightness rendering screens to the first rendering server, and the brightness rendering screens are used to indicate the brightness information corresponding to each position in the overall screen of the virtual scene; finally, the first rendering server adjusts the brightness of multiple target rendering screens according to the brightness rendering screens and then splices them to obtain the spliced screen corresponding to the virtual scene. After adjusting the brightness of multiple target rendering screens based on the brightness rendering screens, the brightness difference between different target rendering screens is reduced, making the transition in the spliced screen more natural and smooth. And since the effect of the spliced screen is the same as the effect of rendering the virtual scene screen by a single server, the scenarios that could originally only be rendered using a single machine can be changed to multi-machine parallel rendering, expanding the application scenarios of multi-machine parallel rendering.

[0046] Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings included in and constituting a part of this specification, together with the specification, illustrate the exemplary embodiments, features, and aspects of the present disclosure and are used to explain the principles of the present disclosure.

[0048] Figure 1 FIG. shows a block diagram of a rendering system for a virtual scene according to an embodiment of the present disclosure.

[0049] Figure 2 FIG. shows a schematic diagram of the rendering process of a rendering system for a virtual scene according to an embodiment of the present disclosure.

[0050] Figure 3 FIG. is a block diagram of an apparatus 1900 for rendering a virtual scene according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. Like reference numerals in the drawings denote functionally identical or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0052] As used herein, the term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

[0053] In addition, for a better description of the present disclosure, numerous specific details are given in the following detailed description. Those skilled in the art should understand that the present disclosure can be implemented without some of these specific details. In some instances, methods, means, elements, and circuits well-known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0054] Virtual Production is a film and entertainment production method that uses computer graphics technology and real-time rendering technology to combine the digital world with the physical world in real time during the production process. With the continuous development of virtual production, the content of the virtual scenes displayed on screens such as LED has become increasingly complex, and the amount of data processing for image rendering has also become larger and larger. To solve this problem, in related technologies, a multi-machine parallel rendering method is adopted, where each machine separately renders a part of the image content. After obtaining the rendered images returned by each machine, multiple rendered images are then stitched and fused to obtain the final overall image. However, during the process of each machine rendering the image, each machine will calculate the lighting and shadows in the image based on the screen space algorithm (Screen Space Algorithms refer to technologies that perform calculations within a two-dimensional image or screen space. These algorithms usually involve post-processing operations on the already rasterized image to enhance the rendering effect. The core of the screen space algorithm lies in using information in the screen space, such as depth, normal, color, etc., to simulate complex lighting and reflection effects, thereby improving the visual quality of the image), which will result in a certain brightness difference in the rendered images of different machines, causing the transition at the stitching point in the stitched overall image to be too abrupt and not natural enough.

[0055] To solve the above technical problems, embodiments of the present disclosure provide a rendering system and method for virtual scenes. The first rendering server is used to divide the rendering tasks for the virtual scene that needs to be rendered currently, and determine the frame rendering tasks of each second rendering server and the brightness rendering tasks of the third rendering server; each second rendering server is used to execute the received frame rendering tasks and send the generated target rendering frames to the first rendering server; the third rendering server is used to execute the received brightness rendering tasks and send the generated brightness rendering frames to the first rendering server, and the brightness rendering frames are used to indicate the brightness information corresponding to each position in the overall frame of the virtual scene; finally, the first rendering server adjusts and splices the brightness of multiple target rendering frames according to the brightness rendering frames to obtain the spliced frame corresponding to the virtual scene. After adjusting the brightness of multiple target rendering frames based on the brightness rendering frames, the brightness difference between different target rendering frames is reduced, making the transition in the spliced frame more natural and smooth. Moreover, since the effect of the spliced frame is the same as that of rendering the virtual scene frame by a single server, the scenario that could only be rendered by a single machine originally can be changed to multi-machine parallel rendering, expanding the application scenarios of multi-machine parallel rendering.

[0056] As Figure 1 、 Figure 2 shown, the rendering system for virtual scenes provided by embodiments of the present disclosure includes multiple rendering servers, and among the multiple rendering servers, there is a first rendering server, multiple second rendering servers, and a third rendering server. In some embodiments, the rendering system can be set as a rendering cluster in virtual shooting.

[0057] The first rendering server is used to divide the rendering tasks for the virtual scene that needs to be rendered currently, and determine the frame rendering tasks of each second rendering server and the brightness rendering tasks of the third rendering server. Among them, the first rendering server, the second rendering server, and the third rendering server can be physically independent devices, or integrated on one or more physical devices, and the present disclosure does not limit this.

[0058] In this embodiment, in virtual shooting, the first rendering server can determine the perspective of the virtual scene that needs to be displayed on the screen based on the current pose of the camera (the pose includes position and attitude), and then determine the overall frame of the virtual scene that needs to be displayed currently, and then divide the rendering tasks for each overall frame of each virtual scene.

[0059] Among them, the screen rendering task may refer to the task of rendering the target screen of a partial area in the overall screen of the virtual scene, and the brightness rendering task may refer to the task of rendering the brightness information of the overall screen of the virtual scene. Among them, the area range of the target screen to be rendered by each second rendering server can be set according to the data processing capabilities of different second rendering servers and / or the rendering frame rate required for the overall screen to meet the shooting needs.

[0060] In a possible implementation manner, for the virtual scene that needs to be rendered currently, the rendering tasks are divided to determine the screen rendering tasks of each second rendering server and the brightness rendering tasks of the third rendering server, which may include: according to the scene resolution of the virtual scene and the number of second rendering servers, determining the area range of the target screen that each second rendering server needs to render and the corresponding first resolution, and generating the screen rendering tasks of each second rendering server. Among them, the screen rendering task is used to indicate the area range of the target screen to be rendered in the virtual scene, the corresponding first resolution, the pose information of the camera, etc., and the first resolution is smaller than the scene resolution.

[0061] Among them, the scene resolution may be the resolution corresponding to the overall screen of the virtual scene, and this scene resolution can be determined according to the pixel pitch of the screen, the size of the screen, the distance between the camera and the screen in virtual shooting, the shooting angle of the camera, etc. Among them, the smaller the pixel pitch of the screen and the larger the size of the screen, the higher the scene resolution. The distance between the camera and the screen affects the proportion of the screen in the camera's field of view. The closer the distance, the larger the proportion of the screen in the field of view, and the higher the required scene resolution. The shooting angle of the camera affects the presentation mode of the screen image in the camera's field of view. Therefore, according to the requirements for maintaining the consistency and accuracy of the virtual scene display corresponding to different shooting angles, the required scene resolution can be determined, and the higher the requirement, the higher the scene resolution.

[0062] Among them, for the overall screen of the same virtual scene, the pose of the camera determines the content of the overall screen that finally needs to be rendered. Therefore, the pose information of the camera can be carried in the screen rendering task so that the pose information can be used as the basis for screen rendering when the second rendering server executes the screen rendering task. The correspondence between the pose of the camera and the overall screen, as well as the correspondence between the overall screen and the screen rendering task of each second rendering server, can be set based on the actual situation and needs of shooting. Among them, the pose information of the camera may include the camera position and attitude. The pose information of the camera can be determined based on the tracking detection of the tracking device bound to the camera.

[0063] Among them, the area range of the target screen can be represented by the coordinate range of the target screen in the overall screen of the virtual scene, etc., and the present disclosure does not limit this. The first resolution can be the resolution of the target rendering screen that needs to be rendered, and the proportional relationship between the size of the first resolution and the scene resolution corresponds to the proportional relationship between the size of the target screen and the overall screen of the virtual scene. For example, assuming that the scene resolution of the overall screen of the virtual scene is 4K (3840×2160), if the target screen is 1 / 4 of the overall screen, the first resolution corresponding to the target screen can be 1920×1080.

[0064] In some embodiments, when the configurations and performances of the second rendering servers are the same, the rendering tasks can be divided by equally dividing the screen according to the number of the second rendering servers. Then, according to the scene resolution of the virtual scene and the number of the second rendering servers, determining the area range of the target screen that each of the second rendering servers needs to render and the corresponding first resolution, and generating the screen rendering tasks for each of the second rendering servers may include: dividing the overall screen of the virtual scene into multiple target screens according to the principle of equal division according to the scene resolution of the virtual scene and the number of the second rendering servers, where the number of the target screens is the same as the number of the second rendering servers; determining the area range and the corresponding first resolution for each of the target screens, and generating the screen rendering tasks for each of the second rendering servers. For example, assuming that the scene resolution of the overall screen of the virtual scene is 4K (3840×2160) and the number of the second rendering servers is 4, the target screens corresponding to the screen rendering tasks of the 4 second rendering servers are, in sequence: target screen 1 in the upper left corner of the overall screen of the virtual scene, target screen 2 in the upper right corner of the overall screen of the virtual scene, target screen 3 in the lower left corner of the overall screen of the virtual scene, and target screen 4 in the upper right corner of the overall screen of the virtual scene; the first resolution corresponding to each target screen is 1920×1080.

[0065] In a possible implementation manner, for the virtual scene that needs to be rendered currently, when dividing the rendering tasks and determining the screen rendering tasks of each of the second rendering servers and the brightness rendering tasks of the third rendering server, it may further include: determining the second resolution for the third rendering server to render the brightness information according to the scene resolution of the virtual scene, and generating the brightness rendering tasks of the third rendering server. Among them, the brightness rendering task is used to indicate the second resolution of the brightness screen that needs to be rendered and the pose information of the camera, and the second resolution is less than the scene resolution.

[0066] In this implementation manner, there may be a proportional relationship between the scene resolution and the second resolution. In some embodiments, when the configurations and performances of the rendering servers are the same, to meet the quality of the final stitched image, this proportional relationship can be matched with the number of the second rendering servers. For example, assuming that the scene resolution of the overall image of the virtual scene is 4K (3840×2160) and the number of the second rendering servers is 4, the second resolution can be 1920×1080. In this way, while reducing the data processing volume of rendering, it can be ensured that the finally generated luminance rendering image can carry the luminance information corresponding to each position in the overall image of the virtual scene.

[0067] In this embodiment, the second resolution is related to the first resolution of the target image rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering server are matched, ensuring that the entire rendering system can render images at a stable and efficient frame rate. In some embodiments, when the configurations and performances of the rendering servers (including the second rendering servers and the third rendering servers) are not completely the same, and / or the content complexity of different regions of the overall image of the virtual scene varies greatly, during the process of the first rendering server dividing the rendering tasks, it can further set the region range, the first resolution, and the second resolution of the target images to be rendered by each second rendering server in real time based on the content complexity of different regions of the overall image, the performances of the second rendering servers, and the performance of the third rendering server, so as to make the processing speeds of the second rendering server executing the image rendering tasks and the third rendering server executing the luminance rendering tasks as close as possible, and then make the rendering frame rates of the target rendering images and the luminance rendering image basically the same, avoiding a significant decrease in the overall frame rate due to the excessive time consumption of a certain rendering server (the second rendering server or the third rendering server), and ensuring that the entire rendering system can render images at a stable and efficient frame rate. Among them, the larger the region range and the higher the first resolution, the slower the processing speed and the lower the rendering frame rate of the target rendering image. The higher the second resolution, the slower the processing speed and the lower the rendering frame rate of the luminance rendering image. When the first resolution changes, the second resolution can be adaptively adjusted.

[0068] Each of the second rendering servers is configured to execute the received image rendering tasks and send the generated target rendering images to the first rendering server. Each of the image rendering tasks can indicate the region range, the first resolution, the pose information of the camera, etc. of the target image that the corresponding second rendering server needs to render, so that the second rendering server can perform rendering on the virtual scene after receiving the image rendering tasks and finally form the target rendering images.

[0069] The third rendering server is configured to execute the received brightness rendering task and send the generated brightness rendering image to the first rendering server. The brightness rendering image is used to indicate the brightness information corresponding to each position in the overall image of the virtual scene.

[0070] In some embodiments, after receiving the brightness rendering task, the third rendering server may render the virtual scene based on the pose information carried in the brightness rendering task and the second resolution, to obtain a virtual scene rendering result in RGB format and with the second resolution. Then, after converting the virtual scene rendering result from RGB format to YUV format, the Y-channel data is extracted to form a brightness rendering image. That is to say, the third server can determine the overall image of the virtual scene to be rendered based on the pose information of the camera, and then render a virtual scene rendering result in two-dimensional RGB format and with a size of the second resolution. After converting the virtual scene rendering result from RGB format to YUV format, in the YUV format image, the Y channel represents brightness information, and the U and V channels represent chrominance information. Therefore, by extracting the data of the Y channel in the virtual scene rendering result in YUV format, a brightness rendering image representing the brightness information at each pixel position in the overall image can be obtained. Those skilled in the art should understand that the third rendering server can also execute the brightness rendering task in other ways. For example, after determining the overall image of the virtual scene to be rendered based on the pose information of the camera, it directly renders a virtual scene rendering result in YUV format and with a size of the second resolution, and extracts the data of the Y channel to obtain a brightness rendering image. Or it can also render a brightness rendering image with a size of the scene resolution, and then perform downsampling to obtain a brightness rendering image with the second resolution.

[0071] The first rendering server is further configured to adjust and splice the brightness of a plurality of target rendering images according to the brightness rendering image, to obtain a spliced image corresponding to the virtual scene. After obtaining the brightness rendering image, the first rendering server can adjust each target rendering image based on the brightness rendering image, so that the brightness between different target rendering images is harmonious and consistent, ensuring that the brightness transition between different target rendering images in the spliced image obtained by splicing is smooth and natural.

[0072] In some embodiments, after adjusting the brightness of multiple target rendering images according to the brightness rendering image and then stitching them, the stitched image corresponding to the virtual scene can be obtained, which may include: after respectively adjusting the brightness of multiple target rendering images according to the brightness rendering image, multiple adjusted images are obtained; according to the regional ranges of the adjusted images in the virtual scene, the multiple adjusted images are stitched and fused to obtain the stitched image corresponding to the virtual scene. Among them, since the brightness rendering image includes the brightness information of each position in the overall image, the color value of the pixel at each position in the target rendering image can be adjusted according to the corresponding position of each position in the target rendering image in the overall image and the brightness information of this position in the brightness rendering image, so that the brightness of this position in the adjusted image is consistent with the brightness indicated by the brightness information of this position in the brightness rendering image, and it can ensure that the image effect is more natural when there are dynamic light and shadow changes or camera shakes in the virtual scene.

[0073] In some embodiments, to further improve the consistency between the brightness in the adjusted image and the brightness rendering image, after respectively adjusting the brightness of multiple target rendering images according to the brightness rendering image and obtaining multiple adjusted images, it may include: performing super-resolution processing on the brightness rendering image according to the scene resolution corresponding to the virtual scene to obtain a processed brightness image with the same resolution as the scene resolution; then, according to the brightness information of each position in the processed brightness image, the color value of the corresponding pixel at each position in each target rendering image is adjusted to obtain multiple adjusted images.

[0074] For example, if the scene resolution of the super-resolution processed brightness rendering image is the same as that of the complete virtual scene, then it is the same as the sum of the pixel numbers of all target rendering images, and there is a one-to-one correspondence between the pixels. The brightness information of any pixel position in the target rendering image can be replaced with the brightness information of the corresponding pixel position in the super-resolution processed brightness rendering image to obtain the adjusted image.

[0075] Those skilled in the art should understand that the method of respectively adjusting the brightness of multiple target rendering images according to the brightness rendering image is not limited to the above method. For example, super-resolution processing may not be performed, and directly the brightness information of one pixel in the brightness rendering image is used to replace the brightness information of multiple corresponding pixels in the target rendering image. Taking the number of the second rendering servers mentioned above as 4 as an example, one pixel in the brightness rendering image can correspond to four pixels of one target rendering image, and the positions of the corresponding pixels in the complete virtual scene have a corresponding relationship. Or, according to needs, after adjusting the brightness information carried by the brightness rendering image as required, the brightness of multiple target rendering images can be adjusted.

[0076] In the related art, in order to ensure the natural transition between different regions in the stitched image, the rendering server is controlled not to use algorithms related to lighting and shadows during the rendering process. During the rendering of the image, lighting and shadows are no longer rendered and calculated. After the stitched image is formed, lighting and shadow related calculations are performed through post-processing. However, the realism of the scene, the attractiveness and immersion of the visual effect of the image will be greatly affected in this way. In the embodiments of the present disclosure, since a separate third rendering server is used in the system to render the brightness rendering image, the accuracy of the brightness at each position in the stitched image can be ensured. Therefore, during the process of each second rendering server generating the target rendering image, algorithms related to lighting and shadows can be normally used, such as the calculation of ShadowMap, the calculation of Virtual ShadowMap, the calculation of Shadow Quality, the calculation of bloom, halo, and Tyndall effect, and so on. Therefore, the image rendering task is also used to indicate the rendering algorithm required to render the target image, and the rendering algorithm includes algorithms related to lighting and / or shadows, so that each second rendering server can use algorithms related to lighting and / or shadows during the process of image rendering, which can ensure the realism of the scene in the stitched image and also enhance the attractiveness and immersion of the visual effect of the image.

[0077] Embodiments of the present disclosure also provide a rendering method for a virtual scene, which is applied to a rendering system. The rendering system includes a first rendering server, a plurality of second rendering servers, and a third rendering server. The method includes:

[0078] Controlling the first rendering server to divide the rendering task for the currently required virtual scene to be rendered, and determining the image rendering tasks of the second rendering servers and the brightness rendering task of the third rendering server;

[0079] Controlling each of the second rendering servers to execute the received image rendering task to generate a target rendering image;

[0080] Controlling the third rendering server to execute the received brightness rendering task to generate a brightness rendering image, where the brightness rendering image is used to indicate the brightness information corresponding to each position in the overall image of the virtual scene;

[0081] Controlling the first rendering server to adjust and stitch the brightness of the multiple target rendering images according to the brightness rendering image to obtain the stitched image corresponding to the virtual scene.

[0082] In a possible implementation, controlling the first rendering server to divide the rendering task for the virtual scene that needs to be rendered currently, and determining the frame rendering tasks of each second rendering server and the brightness rendering task of the third rendering server, includes:

[0083] According to the scene resolution of the virtual scene, determining a second resolution for the third rendering server to render brightness information, and generating the brightness rendering task of the third rendering server;

[0084] Wherein, the brightness rendering task is used to indicate the second resolution of the brightness frame to be rendered and the pose information of the camera, and the second resolution is smaller than the scene resolution.

[0085] In a possible implementation, the second resolution is related to the first resolution of the target frame rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering servers match.

[0086] In a possible implementation, controlling the third rendering server to execute the received brightness rendering task and generate a brightness rendering frame, includes:

[0087] Rendering the virtual scene based on the pose information carried in the brightness rendering task and the second resolution, to obtain a virtual scene rendering result in RGB format and with the second resolution;

[0088] After converting the virtual scene rendering result from RGB format to YUV format, extracting the Y-channel data to form a brightness rendering frame.

[0089] In a possible implementation, controlling the first rendering server to adjust and splice the brightness of multiple target rendering frames according to the brightness rendering frame, to obtain a spliced frame corresponding to the virtual scene, includes:

[0090] After respectively adjusting the brightness of multiple target rendering frames according to the brightness rendering frame, obtaining multiple adjusted frames;

[0091] According to the regional ranges of the adjusted frames in the virtual scene, splicing and fusing the multiple adjusted frames to obtain a spliced frame corresponding to the virtual scene.

[0092] In a possible implementation, after respectively adjusting the brightness of multiple target rendering frames according to the brightness rendering frame, obtaining multiple adjusted frames, includes:

[0093] Perform super-resolution processing on the brightness rendering image according to the scene resolution corresponding to the virtual scene to obtain a processed brightness image with the same resolution as the scene resolution;

[0094] Adjust the color values of the pixels at the corresponding positions in each of the target rendering images according to the brightness information at each position in the processed brightness image to obtain a plurality of adjusted images.

[0095] In a possible implementation manner, the image rendering task is further used to indicate the rendering algorithm to be used for rendering the target image, and the rendering algorithm includes algorithms related to lighting and / or shadows.

[0096] It should be noted that although the above embodiments are used as examples to introduce the rendering system and method for virtual scenes as above, those skilled in the art can understand that the present disclosure should not be limited thereto. In fact, users can flexibly set each part and each step according to personal preferences and / or actual application scenarios, as long as it conforms to the technical solution of the present disclosure.

[0097] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0098] The embodiments of the present disclosure also propose a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above methods are implemented. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0099] The embodiments of the present disclosure also propose an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to implement the above methods when executing the instructions stored in the memory.

[0100] The embodiments of the present disclosure also provide a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in the processor of an electronic device, the processor in the electronic device executes the above methods.

[0101] Figure 3 It is a block diagram of a device 1900 for rendering a virtual scene shown according to an exemplary embodiment. For example, the device 1900 can be provided as a server. Refer to Figure 3, Device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the methods executed by any of the above rendering servers.

[0102] Device 1900 may also include a power component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or the like.

[0103] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as the memory 1932 including computer program instructions, and the above computer program instructions can be executed by the processing component 1922 of the device 1900 to complete the above method.

[0104] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0105] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example—but not limited to—an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device, such as a punched card or raised structures in grooves storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as an instantaneous signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0106] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0107] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0108] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer - readable program instructions.

[0109] These computer - readable program instructions can be provided to a processor of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data - processing apparatus, create a means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions includes a manufacture, which includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0110] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0111] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified function or act, or by a combination of dedicated hardware and computer instructions.

[0112] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the market, or to enable other ordinary skilled artisans in the art to understand the embodiments disclosed herein.

Claims

1. A rendering system for a virtual scene, characterized in that, The system includes a first rendering server, multiple second rendering servers, and a third rendering server. The first rendering server is configured to divide a rendering task for a virtual scene to be rendered currently, and determine a frame rendering task for each of the second rendering servers and a brightness rendering task for the third rendering server. Each of the second rendering servers is configured to execute the received frame rendering task, and send the generated target rendering frame to the first rendering server. The third rendering server is configured to execute the received brightness rendering task, and send the generated brightness rendering frame to the first rendering server, where the brightness rendering frame is used to indicate brightness information corresponding to each position in the overall frame of the virtual scene. The first rendering server is further configured to adjust the brightness of multiple target rendering frames according to the brightness rendering frame and then splice them to obtain a spliced frame corresponding to the virtual scene.

2. The system according to claim 1, wherein Dividing a rendering task for a virtual scene to be rendered currently, and determining a frame rendering task for each of the second rendering servers and a brightness rendering task for the third rendering server includes: Determining a second resolution for the third rendering server to render brightness information according to the scene resolution of the virtual scene, and generating a brightness rendering task for the third rendering server; wherein the brightness rendering task is used to indicate the second resolution of the brightness frame to be rendered and the pose information of the camera, and the second resolution is smaller than the scene resolution.

3. The system according to claim 2, wherein The second resolution is related to the first resolution of the target frame rendered by each second rendering server, so that the rendering frame rates of the third rendering server and the second rendering servers match.

4. The system according to claim 2, wherein Executing the received brightness rendering task, and sending the generated brightness rendering frame to the first rendering server includes: Rendering the virtual scene based on the pose information and the second resolution carried in the brightness rendering task to obtain a rendering result of the virtual scene in RGB format and with the second resolution; After converting the rendering result of the virtual scene from RGB format to YUV format, extracting the Y channel data to form a brightness rendering frame.

5. The system according to claim 1, wherein Adjusting the brightness of multiple target rendering frames according to the brightness rendering frame and then splicing them to obtain a spliced frame corresponding to the virtual scene includes: Adjusting the brightness of multiple target rendering frames respectively according to the brightness rendering frame to obtain multiple adjusted frames; Splicing and fusing the multiple adjusted frames according to the regional ranges of the adjusted frames in the virtual scene to obtain a spliced frame corresponding to the virtual scene.

6. The system according to claim 5, wherein Adjusting the brightness of multiple target rendering frames respectively according to the brightness rendering frame to obtain multiple adjusted frames includes: Performing super-resolution processing on the brightness rendering frame according to the scene resolution corresponding to the virtual scene to obtain a processed brightness frame with the same resolution as the scene resolution; Adjusting the color values of the pixels at the corresponding positions in each target rendering frame according to the brightness information at each position in the processed brightness frame to obtain multiple adjusted frames.

7. The system according to claim 5, wherein The screen rendering task is further used to indicate the rendering algorithm to be used for rendering the target screen, and the rendering algorithm includes algorithms related to lighting and / or shadows.

8. A rendering method for a virtual scene, characterized in that Applied to a rendering system, the rendering system includes a first rendering server, a plurality of second rendering servers, and a third rendering server, and the method includes: Controlling the first rendering server to divide the rendering task for the virtual scene that needs to be rendered currently, and determining the screen rendering tasks of the second rendering servers and the brightness rendering task of the third rendering server; Controlling each of the second rendering servers to execute the received screen rendering task to generate a target rendering screen; Controlling the third rendering server to execute the received brightness rendering task to generate a brightness rendering screen, where the brightness rendering screen is used to indicate the brightness information corresponding to each position in the overall screen of the virtual scene; Controlling the first rendering server to adjust and splice the brightness of a plurality of target rendering screens according to the brightness rendering screen to obtain a spliced screen corresponding to the virtual scene.

9. A rendering device for a virtual scene, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to implement the method according to claim 8 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that, The computer program instructions, when executed by the processor, implement the method according to claim 8.

11. A computer program product, comprising computer-readable code or a non-volatile computer-readable storage medium carrying the computer-readable code, characterized in that When the computer-readable code runs in the processor of the electronic device, the processor in the electronic device executes the method according to claim 8.

Images