Three-dimensional scene rendering processing method and electronic device

Abstract

Embodiments of the present application disclose a three-dimensional scene rendering processing method and an electronic device. The method comprises: determining the number of rendering devices required for digital background shooting based on a three-dimensional scene; obtaining pose information of a camera in units of frames during shooting, so as to determine total rendering task information of the same frame according to the pose information; distributing the total rendering task of the same frame to each rendering device according to the number of rendering devices, so that the rendering devices render different sub-regional ranges within the total regional range respectively; after each rendering device returns a rendering result in the corresponding sub-regional range, the rendering results are spliced and combined to generate a target image frame, and the target image frame is displayed on a target screen associated with the digital background shooting. Through the embodiments of the present application, the hardware cost can be reduced, the rendering efficiency can be improved, and the real-time rendering requirement in the digital background shooting scene is more suitable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital background shooting, in particular to a three-dimensional scene rendering processing method and an electronic device. BACKGROUND

[0002] A digital background shooting scheme based on an LED (Light Emitting Diode) screen (also known as an electronic display screen or a flow display screen, which is composed of a plurality of small LED module panels and is used to display various information such as text, images and videos) is also known, that is, a digital background is displayed through the LED screen, and specific actors and characters can perform in front of the LED screen. In this way, when a camera shoots the performance process of the characters, the background content displayed in the LED screen can also be shot at the same time. Compared with the traditional green screen, the digital background rendered in real time on the LED screen can place real actors in a virtual scene and achieve the effect of shooting a scene in a studio.

[0003] In the above-mentioned shooting scene using an LED virtual background, the specific digital background can be a two-dimensional video, or a three-dimensional scene can also be projected onto the LED screen for display. In the case of using a three-dimensional scene as a digital background, in order to achieve a more realistic effect, the specific three-dimensional scene needs to be updated in real time following the camera position during projection onto the LED screen for display, so as to maintain the correct perspective relationship. For example, assuming that a three-dimensional scene is a sphere, when the camera shoots the LED screen directly, the front of the sphere can be displayed on the LED screen, and when the camera is turned to the left to shoot, the left side of the sphere should also be displayed on the LED screen. In this way, the camera can shoot the left side of the sphere, thereby achieving the effect of placing a sphere in the shooting scene.

[0004] Since the display content of the three-dimensional scene needs to be updated in real time following the camera position, the rendering task in this case is very heavy. Therefore, how to meet the real-time rendering requirements in the digital background shooting scene has become a technical problem to be solved by those skilled in the art. SUMMARY

[0005] The present application provides a three-dimensional scene rendering processing method and an electronic device, which can reduce hardware costs, improve rendering efficiency, and better meet the real-time rendering requirements in the digital background shooting scene.

[0006] The present application provides the following solutions:

[0007] A three-dimensional scene rendering processing method comprises the following steps:

[0008] determining the number of rendering devices required for digital background shooting based on a three-dimensional scene;

[0009] During the digital background shooting, pose information of a camera is acquired in units of frames, so as to determine total rendering task information of a same frame according to the pose information, the total rendering task information including: a total area range required to be rendered in the three-dimensional scene in a state of the pose information, and location information of the total area range in the three-dimensional scene;

[0010] The total rendering task of the same frame is evenly distributed to each rendering device according to the number of the rendering devices, so that the rendering devices respectively render different sub-area ranges in the total area range.

[0011] After the rendering devices respectively return rendering results in the corresponding sub-area ranges, the rendering results are spliced and combined to generate a target image frame, and the target image frame is displayed on a target screen associated with the digital background shooting.

[0012] The total rendering task of the same frame is evenly distributed to each rendering device according to the number of the rendering devices, including:

[0013] After the number of required rendering devices is determined, an area index is established for each rendering device according to a preset sub-area division manner, and the area index information of each rendering device is provided to each rendering device, so as to evenly distribute the total rendering task according to the area index information.

[0014] The total rendering task of the same frame is evenly distributed to each rendering device according to the number of the rendering devices, further including:

[0015] After the pose information of the camera is acquired, the pose information is broadcast to each rendering device, so that the rendering devices determine the total area range required to be rendered in the three-dimensional scene and the location information according to the pose information, and determine the sub-area range required to be responsible according to the total area range and the location information and the area index information of each rendering device.

[0016] The total rendering task of the same frame is evenly distributed to each rendering device according to the number of the rendering devices, further including:

[0017] After the pose information of the camera is acquired, the total area range required to be rendered in the three-dimensional scene and the location information are determined according to the pose information.

[0018] The total area range and the location information of the required rendering in the three-dimensional scene are broadcast to each rendering device, so that each rendering device determines the sub-area range required to be responsible for according to the total area range, the location information, and the area index information of each rendering device.

[0019] The rendering device returns the rendering result after encoding by the hardware coding manner based on a graphics processing unit (GPU).

[0020] Before the splicing and combination of the rendering result, the method further includes:

[0021] The rendering device returns the rendering result after encoding by the hardware coding manner based on a graphics processing unit (GPU).

[0022] The image display on the target screen associated with the digital background based on the target image frame includes:

[0023] The image display on the target screen associated with the digital background based on the target image frame includes:

[0024] A three-dimensional scene rendering processing method includes:

[0025] The area index information is established according to the number of rendering devices required for the digital background shooting based on a three-dimensional scene.

[0026] During the digital background shooting, after receiving the information broadcast frame by frame by the master control device, the total rendering task information in the pose state of the current frame of the camera is determined, and the total rendering task information includes the total area range required for rendering in the three-dimensional scene and the location information of the total area range in the three-dimensional scene.

[0027] The sub-area range required to be responsible for is determined from the total area range according to the total area range, the location information, and the area index information.

[0028] The three-dimensional scene content in the sub-area range is rendered, and the rendering result is returned to the master control device, so that after the master control device receives the rendering results in multiple sub-area ranges, the target image frame is generated by splicing and combination, and the image display on the target screen associated with the digital background based on the target image frame is performed.

[0029] The information broadcast frame by frame by the master control device includes the pose information of the current frame of the camera.

[0030] The determination of the total area range required to be rendered in the three-dimensional scene in the pose state of the camera in the current frame, and the location information of the total area range in the three-dimensional scene, comprises:

[0031] According to the pose information of the camera in the current frame, the total area range required to be rendered in the three-dimensional scene is determined, and the location information of the total area range in the three-dimensional scene is determined.

[0032] The rendering result returned to the master control device comprises:

[0033] The rendering result is encoded by GPU hardware coding and then returned to the master control device.

[0034] A three-dimensional scene rendering processing device comprises:

[0035] A rendering device number determination unit is configured to determine the number of rendering devices required for digital background shooting based on a three-dimensional scene.

[0036] A pose information acquisition unit is configured to acquire the pose information of a camera in units of frames during the digital background shooting process, so as to determine the total rendering task information of the same frame according to the pose information, wherein the total rendering task information comprises the total area range required to be rendered in the three-dimensional scene in the state of the pose information, and the location information of the total area range in the three-dimensional scene.

[0037] A rendering task allocation unit is configured to allocate the total rendering task of the same frame to each rendering device according to the number of rendering devices, so that the rendering devices render different sub-area ranges in the total area range respectively.

[0038] A rendering result processing unit is configured to splice and combine the rendering results after each rendering device returns the rendering result in the corresponding sub-area range, to generate a target image frame, and perform image display on a target screen associated with the digital background shooting according to the target image frame.

[0039] A three-dimensional scene rendering processing device comprises:

[0040] An area index information receiving unit is configured to receive area index information provided by a master control device, wherein the area index information is established according to the number of rendering devices required for digital background shooting based on a three-dimensional scene.

[0041] A total rendering task determination receiving unit is configured to, during the digital background shooting process, determine total rendering task information in a pose state of a current frame of the camera after receiving information broadcast frame by frame by the master device, wherein the total rendering task information comprises a total area range required to be rendered in the three-dimensional scene and location information of the total area range in the three-dimensional scene.

[0042] A rendering area determination unit is configured to determine a sub-area range required to be responsible from the total area range according to the total area range, the location information and the area index information.

[0043] A rendering unit is configured to render three-dimensional scene content in the sub-area range and return a rendering result to the master device, so that the master device stitches and combines the rendering results in the plurality of sub-area ranges to generate a target image frame and displays the target image frame on a target screen associated with the digital background shooting based on the target image frame.

[0044] A computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the method of any one of the preceding embodiments.

[0045] An electronic device comprising:

[0046] one or more processors; and

[0047] a memory associated with the one or more processors, the memory configured to store program instructions that, when executed by the one or more processors, perform the steps of the method of any one of the preceding embodiments.

[0048] According to the embodiments provided in the present application, the following technical effects are disclosed:

[0049] By the embodiment of the present application, the number of rendering devices required can be determined before the digital background shooting based on the three-dimensional scene is performed; during the digital background shooting, the pose information of the camera can be acquired in units of frames, so as to determine the total rendering task information of the same frame according to the pose information (this step can be completed by the rendering device or the master control device), the total rendering task information including the total area range and the location information of the three-dimensional scene required to be rendered in the same frame state; and the total rendering task of the same frame can be evenly distributed to each rendering device according to the number of rendering devices, so that the rendering device renders different sub-area ranges in the total area range respectively; after the rendering result in the corresponding sub-area range is returned by each rendering device, the rendering results can be spliced and combined to generate a target image frame, and the target image frame is projected to a target screen. In this way, the rendering task of each frame can be evenly distributed to multiple different rendering devices, and the rendering task completed by each rendering device is proportional, so that the balance of the rendering task between different rendering devices can be realized. In this way, since the rendering task of a frame is not concentrated on part or even one rendering device, the performance requirement of a single rendering device is reduced, which is conducive to reducing the hardware cost and improving the rendering efficiency, and is more suitable for the real-time rendering requirement in the digital background shooting scene.

[0050] In addition, in the optional implementation, the color depth precision of the rendering result can also be improved, GPU hardware encoding and decoding, parameter adjustment, post-quality improvement processing, and the like are used, so that on the basis of realizing the balance of the rendering task, the three-dimensional scene is projected to the LED screen at a stable frame rate and high quality through the high-fidelity image encoding, transmission and other mechanisms, the multi-machine load balancing is ensured, and the rendering quality and cost are considered.

[0051] Of course, implementing any product of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0053] Figure 1 is a schematic diagram of the system architecture provided by the embodiment of the present application;

[0054] Figure 2 is a flowchart of a first method provided by an embodiment of the present application;

[0055] Figure 3 is a sub-region rendering result and display schematic diagram provided by an embodiment of the present application;

[0056] Figure 4 is a flowchart of a second method provided by an embodiment of the present application;

[0057] Figure 5 is a schematic diagram of a first device provided by an embodiment of the present application;

[0058] Figure 6 is a schematic diagram of a second device provided by an embodiment of the present application;

[0059] Figure 7 is a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0061] In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, it is first necessary to explain that, for a three-dimensional scene, it is usually a 360-degree spherical scene. When rendering a three-dimensional scene, the model data, motion data, lighting data, etc. of the three-dimensional scene are generated into a 2D image with stereoscopic effect based on the observation position and angle, so as to display the 2D image to an LED screen, etc. The so-called "observation" position and angle here can be the position and angle of the camera when collecting images. Since the FOV (Field of View) of the camera is usually limited, when rendering a three-dimensional scene, only the content in a region range of the three-dimensional scene needs to be rendered. The size of the region range is related to the size of the camera FOV range. Correspondingly, only the rendering result of the three-dimensional scene in the corresponding region range needs to be displayed in the LED display screen.

[0062] The FOV is the range that can be covered by the camera lens, and is also the angle range of the image that can be received by the camera (objects exceeding the field of view angle will not be collected in the lens). The FOV is divided into horizontal FOV and vertical VFOV. For example, assuming that the horizontal FOV of a camera is 100 degrees and the vertical FOV is 50 degrees, it means that each frame of image that can be collected by the camera can only be the image within the range of 100 degrees from left to right and 50 degrees from top to bottom. Therefore, when rendering the three-dimensional scene, the range of the area to be rendered each time is also 100 degrees from left to right and 50 degrees from top to bottom. Of course, since the pose of the camera can be constantly changing, although the total span of the range of the area to be rendered each time can be the same, the position of the range of the area to be rendered in the three-dimensional scene (including the starting point or the ending point from left to right or from top to bottom) can be different.

[0063] For example, assuming that in the initial state, the camera is facing the screen, at this time, the range of the area to be rendered in the three-dimensional scene is 100 to 200 degrees in the horizontal direction and 0 to 50 degrees in the vertical direction. Then, the camera is turned 10 degrees to the left, and the range of the area to be rendered in the three-dimensional scene can become 90 to 190 degrees in the horizontal direction and 0 to 50 degrees in the vertical direction, and so on. It is also because the position of the area to be displayed in the three-dimensional scene is different each time that dynamic real-time rendering is needed, that is, the content in the range of the area to be displayed in the three-dimensional scene is rendered each time, and then projected into the LED display screen for display.

[0064] As described in the background section, due to the heavy task of real-time rendering of a three-dimensional scene, it is difficult to complete by a single machine, so a multi-machine collaborative rendering mode can be used to complete. Some technologies have a scheme that can support multi-machine collaborative rendering. Specifically, according to the complexity of the three-dimensional scene, etc., the LED screen can be divided into multiple blocks on the hardware, each screen is connected to a rendering server (which can be implemented by a powerful PC device, etc.), and then the rendering task is dynamically allocated according to the position and angle of the camera, etc. However, this mode may cause uneven distribution of tasks on different rendering servers. For example, assuming that a 10k resolution screen is divided into 5 blocks, each block is connected to a rendering server, and each rendering server is responsible for rendering the picture on the screen corresponding to itself. Since the camera itself is moving, not all areas of the screen can enter the shooting range of the camera. Assuming that at a certain moment, the camera can only shoot the pictures on three screens, then the other two screens do not need to be rendered, at this time, the rendering pressure is on the three rendering servers corresponding to the three screens that can be shot by the camera, and so on. At the same time, each rendering server needs to have high performance to ensure that it can complete the rendering task on time even when there are more rendering tasks. Therefore, the hardware cost of the rendering server will be high. In this case, the uneven distribution of rendering tasks will lead to waste of rendering resources. In addition, for more complex three-dimensional scenes, more rendering servers may be needed, at this time, the LED screen needs to be divided into more blocks on the hardware and connected to a rendering server, at this time, not only the cost of the rendering server will be higher, but also the cost of the modification of the LED screen will be higher.

[0065] Based on the above situation, in the embodiments of the present application, in order to make the tasks allocated to each rendering machine more balanced in a multi-machine rendering scene, and thus reduce the cost of the multi-machine rendering process, a corresponding solution is provided. In this solution, as shown in Figure 1 the hardware aspect, master devices and multiple rendering devices can be deployed, and these master devices and rendering devices can be deployed in a specific digital background shooting site and can be connected to the same local area network. The transmission bandwidth between the devices can be relatively high, for example, it can be higher than 10 Gb / s. This requirement for transmission bandwidth is mainly to ensure that the real-time rendered images can be transmitted quickly and stably. In terms of software, a master application (also referred to as a control terminal application) can be deployed in the master device, and a decoding transmission plug-in can also be deployed. Data can be transferred, recycled, and processed through these applications. In the rendering device, a rendering engine can be deployed, and a coding transmission plug-in, etc. can also be deployed to complete coding transmission tasks.

[0066] In a specific implementation, before digital background shooting is needed, the number of rendering devices to be used in the current scene can be determined by the master application in the master device according to the complexity of the specific three-dimensional scene, the resolution of the LED screen, the frame rate collected by the camera, and the like. Then, during the specific digital background shooting process by the camera, since the camera collects images frame by frame (for example, assuming that the frame rate is 60 frames per second, which means that the camera needs to collect 60 frames of images per second), and the pose (which can specifically include position, angle, focal length, and the like) of the camera can change during the collection, the pose information of the camera can be obtained frame by frame. Specifically, the pose information can be obtained by an inertial motion capture device (referred to as a motion capture device) and the like and delivered to the master device. After the pose information is obtained, the total rendering task information of the same frame can be determined, which includes the total area range to be rendered in the three-dimensional scene and the location information. In the embodiments of the present application, the total rendering task of the same frame can be evenly distributed to each rendering device (for example, rendering device-1, rendering device-2, rendering device-N), and a rendering engine can be deployed in the rendering device, and an encoding and transmission plug-in can also be provided for the rendering engine to perform the encoding and transmission task after the rendering engine completes the rendering of each frame. That is, after each rendering device completes the rendering of a region range in a frame, the image can be encoded and transmitted back to the master device. The control end application can be deployed in the master device, which can include a decoding and transmission plug-in to decode the rendering result returned by the rendering device. Then, image post-processing and screen projection splitting processing can be performed, and the specific image content can be displayed on the LED screen. The specific actor performance or scenery can be performed in front of the LED screen, and the camera can continue to perform the framing process. In this way, when rendering the same frame, the task amount of each rendering device is the same, thereby achieving task balancing. Moreover, since each rendering device can participate in rendering when rendering each frame, there is no situation that all rendering tasks are concentrated on a single rendering device at a certain moment, and therefore the performance requirement of a single rendering device is reduced, thereby reducing the hardware cost of the rendering device. In addition, in the embodiments of the present application, the LED display screen no longer needs to be physically divided (even if it is divided, it may be related to the LED itself and is irrelevant to the rendering complexity), and the specific rendering device no longer needs to be connected to each display screen, and therefore the cost is reduced from this perspective. In addition, in a specific implementation, after the master device receives multiple rendering results of the same frame and splices them into a complete frame of image, some post-processing can be performed to further improve the image quality, and then the specific image is projected onto the LED screen.

[0067] The specific implementation schemes provided by the embodiments of the present application are described in detail below.

[0068] Embodiment One

[0069] First, the embodiment first provides a three-dimensional scene rendering processing method from the perspective of the aforementioned master control device, referring to Figure 2 The method can specifically include the following steps.

[0070] S201: Determine the number of rendering devices required for digital background shooting based on a three-dimensional scene.

[0071] The number of specific rendering devices required can be determined according to the current shooting requirements. For example, specific influencing factors can include the complexity of the three-dimensional scene, the resolution of the LED display screen, the frame rate of the camera, and the like. Specifically, in the embodiments of the present application, since the rendering task of the same frame can be evenly distributed to multiple different rendering devices for execution, the number of rendering devices can be determined in combination with this feature. In addition, factors such as the performance parameters of each rendering device can also be considered, and the like.

[0072] S202: During the digital background shooting process, the pose information of the camera is acquired in units of frames, so as to determine the total rendering task information of the same frame according to the pose information, the total rendering task information including: the total area range required to be rendered in the three-dimensional scene in the state of the pose information, and the location information of the total area range in the three-dimensional scene.

[0073] Since the camera is collected frame by frame at a certain frame rate during the digital background shooting process, the pose information of the camera can also be acquired in units of frames. As described above, a motion compensation device can be connected to the camera, which can perceive the pose information of the camera and provide it to the master control device, and the like.

[0074] After the pose information of the camera is acquired, the total rendering task information of the same frame can be determined according to the pose information. In the specific implementation, after the pose information of the camera is acquired each time, it can be compared with the pose information of the last frame. If it is unchanged, it does not need to be re-rendered. Otherwise, if it is found that the pose information of the camera has changed, the total area range and position required to be rendered need to be determined from the three-dimensional scene again, and re-rendering is performed.

[0075] The process of determining the range and position of the region to be rendered in the three-dimensional scene according to the pose information of the camera can be completed by the master device or by the rendering device. If it is completed by the master device, the master device can determine the range and position of the region to be rendered in the three-dimensional scene each time the pose information of the camera is received, so as to provide the information to the rendering device. Alternatively, if it is completed by the rendering device, the master device can directly broadcast the pose information each time the pose information of the camera is received, and each rendering device can set the received camera pose parameter to the virtual camera by the rendering engine, and determine the range and position of the region to be rendered in the three-dimensional scene according to the pose information. In a specific implementation, the pose information can be compared with the pose information of the previous frame after being received, and if a change occurs, the above process is performed again. The specific implementation of determining the range and position of the region to be rendered in the three-dimensional scene according to the pose information is not the focus of the protection of the embodiments of the present application, and therefore will not be described in detail here.

[0076] S203: The total rendering task of the same frame is evenly distributed to each rendering device according to the number of the rendering devices, so that the rendering devices render different sub-regional ranges in the total regional range respectively.

[0077] After the total rendering task information of the same frame is determined according to the pose information of the camera, the total rendering task of the same frame can be evenly distributed to each rendering device according to the number of the rendering devices, so that the rendering devices render different sub-regional ranges in the total regional range respectively.

[0078] The specific way of evenly distributing the rendering task can be various, for example, in one way, if the master device calculates the total rendering task according to the pose information of the camera, the master device can also evenly divide the total regional range to be rendered in the three-dimensional scene into a plurality of sub-regions, and notify the range and position information of the sub-regions to each rendering device for rendering, so that each rendering device renders the content in one of the sub-regional ranges.

[0079] Alternatively, in another more preferred mode, since the number of rendering devices is fixed in the same shooting process, the sub-region division mode can also be preset, and at this time, the region index of each rendering device can also be established according to the preset sub-region division mode, and the region index information of each rendering device is provided respectively, so as to realize the average distribution of the total rendering task according to the region index information. For example, assuming that 5 rendering devices are provided, and need to be divided into 5 sub-regions in the horizontal direction, only the position serial number corresponding to the specific rendering device needs to be included in the region index. For example, the position serial number of a certain rendering device is 2, which means that the rendering device needs to render the 2nd sub-region from left to right in the current total rendering task. The span of each sub-region in the horizontal direction can be determined by the total span of the total region range in the horizontal direction and the number of rendering devices. For example, assuming that the horizontal FOV is 100 degrees, and the position of the current total region range is from the 10th degree to the 110th degree, the position of the above-mentioned 2nd sub-region from left to right is from the 30th degree to the 50th degree in the horizontal direction, and the like.

[0080] Of course, in specific implementation, if the three-dimensional scene is relatively complex, more rendering devices are needed for rendering, for example, 10, and the specific sub-region division mode can also be a multi-row and multi-column mode, for example, can be divided into two rows, each row has 5 sub-regions, and the like. In this case, the region index can include two information such as row serial number and column serial number, for example, the region index of a certain rendering device is (2, 3), which means that the rendering device needs to render the 3rd sub-region of the 2nd row, and the like. Of course, the total number of rendering devices can also be notified to the rendering device in advance.

[0081] In the case where the region index is provided to the rendering device as described above, the rendering device only needs to determine the total rendering task in the same frame, that is, the region range and position in the three-dimensional scene that needs to be rendered in the frame state, so as to determine the range and position of the specific sub-region to be rendered according to the region index information.

[0082] In specific implementation, as described above, the determination of the total rendering task can be performed by the master control device or the rendering device. For the former, after obtaining the pose information of the camera, the master control device can determine the total region range and position information to be rendered in the three-dimensional scene according to the pose information; then, the master control device can broadcast the total region range and position information to be rendered in the three-dimensional scene to each rendering device, so that each rendering device determines the sub-region range and position to be responsible according to the total region range and position information and the region index information of each rendering device.

[0083] For the latter, the master device can directly broadcast the pose information of the camera to each rendering device after obtaining the pose information. In this way, each rendering device can determine the total area range and location information of the required rendering in the three-dimensional scene according to the pose information, and then determine the sub-area range to be responsible for according to the total area range and location information and the area index information of each rendering device.

[0084] It should be noted that in actual application, since real-time rendering of the three-dimensional scene is required during image acquisition by the camera, and the images of the same frame are completed by multiple different rendering devices, a master node can be set in the rendering device cluster in specific implementation. The master node can communicate with other nodes to control the synchronous rendering of other nodes. For example, in specific implementation, the master device can broadcast the real-time pose information of the camera to the rendering device cluster through a transmission plug-in. The master node in the cluster controls other nodes to use the same camera pose information and perform rendering according to the same pace by communicating with other nodes.

[0085] After determining the sub-area to be rendered by each rendering device, each rendering device can render the sub-area to be responsible for. In order to present high image quality on the LED screen, some optimization can be performed in combination with the characteristics of the LED large screen during rendering. For example, the data format rendered is generally in a general manner according to a traditional scheme, that is, the color depth precision information is usually represented by RGB8bit (that is, RGB three primary color channels are represented by 8-bit precision). However, if this precision is used, color details may be lost when projected onto the LED large screen. Therefore, in the embodiment of the present application, the precision can be changed to 10bit to better preserve color details and avoid color step problems. Of course, in specific implementation, the color depth precision can also be set to other values, which can be determined according to the size of the screen and / or the requirement for preserving image details.

[0086] In addition, in the embodiment of the present application, after the rendering device completes rendering, the rendering result needs to be returned to the master device, which involves the coding and decoding of data. In a traditional manner, the encoding format mainly uses CPU (Central Processing Unit) for soft coding. However, in the scene of the embodiment of the present application, since real-time rendering and display are required during camera shooting, the coding and decoding efficiency is also very high, that is, the encoding needs to be completed quickly after rendering. Therefore, in the preferred embodiment of the present application, the rendering device can use GPU (Graphics Processing Unit) hardware coding.

[0087] Furthermore, although hardware encoding can be performed using GPUs, encoding parameters usually still need to be configured. Therefore, adjusting these parameters can further enhance the final display effect. Specific parameters include setting the keyframe interval, enabling lossless compression mode to maximize image quality, and so on.

[0088] S204: After each rendering device returns the rendering results within the corresponding sub-region, the rendering results are stitched together to generate a target image frame, and the target screen associated with the digital background is photographed based on the target image frame for image display.

[0089] After the main control device receives the rendering results from each sub-region returned by the various rendering devices, it can decode them and combine them according to the aforementioned sub-region division method to generate a complete image frame. Then, the image can be displayed to the target screen based on the image frame.

[0090] For example, suppose four rendering devices are used to render a 3D scene. Each rendering device is responsible for rendering one-quarter of the 3D scene's rendering task in the same frame. Each rendering device encodes and transmits its rendered image to the master device. After receiving the image, the master device reassembles it into a complete image, such as... Figure 3 The image frame shown in (A) (where the three vertical lines divide the image into four parts, representing the rendering results returned by four rendering devices respectively, indicating that each rendering device has the same rendering task ratio) can then be projected onto an LED screen. For example, assuming three mutually perpendicular LED screens (referred to as a tri-fold screen) are set up at the shooting location, the composite image frame can be distorted according to the relative positions of the camera and the screens (after distortion, it may appear tilted to the human eye, but the image captured by the camera will be normal), and then displayed on the LED screen, with the display effect as shown... Figure 3 As shown in (B).

[0091] As mentioned earlier, when encoding the rendering results, the rendering device can use GPU hardware encoding to speed up the process and better ensure real-time performance. Similarly, when decoding, the main control device can also use GPU-based hardware decoding of the information returned by the rendering device to speed up the process and further ensure real-time performance.

[0092] In addition, in an optional embodiment, specifically when image projection is performed on the target screen according to the target image frame, the target image frame can also be subjected to quality improvement processing before image projection is performed on the target screen. The specific quality improvement processing can include one or more of the following: de-color banding processing on color gradient conditions, blurring processing on long shots, modification of image saturation according to the on-site light environment, and the like. In this way, through the process of multi-machine collaborative rendering, encoding transmission, image post-processing, and synthesis of the screen, the rendering task can be evenly processed, and a high-fidelity image encoding and transmission mechanism is used to ensure that the three-dimensional scene is projected to the LED screen at a stable frame rate and high quality, ensuring that the multi-machine load is balanced while using the least number of devices to complete high-quality rendering, taking into account rendering quality and cost.

[0093] In summary, through the embodiments of the present application, the number of rendering devices required can be determined when digital background shooting based on a three-dimensional scene is performed; during the digital background shooting process, the pose information of the camera can be obtained in units of frames, so as to determine the total rendering task information of the same frame according to the pose information (this step can be completed by the rendering device or by the master control device), the total rendering task information including: the total area range in the three-dimensional scene that needs to be rendered in the same frame state, and the location information of the total area range in the three-dimensional scene; and the total rendering task of the same frame can be evenly distributed to each rendering device according to the number of rendering devices, so that the rendering devices render different sub-area ranges in the total area range respectively; after the rendering devices return the rendering results in the corresponding sub-area ranges respectively, the rendering results can be spliced and combined to generate a target image frame, and image display can be performed on a target screen according to the target image frame. In this way, the rendering task of each frame can be evenly distributed to multiple different rendering devices, and the rendering task is completed by multiple rendering devices, and the proportion of the task completed by each rendering device is the same, so that the rendering task can be balanced among different rendering devices. In this way, since the rendering task of a frame is not concentrated on part or even one rendering device, the performance requirement of a single rendering device is reduced, which is conducive to reducing hardware costs and improving rendering efficiency, and is more suitable for real-time rendering requirements in digital background shooting scenarios.

[0094] In addition, in an optional embodiment, the color depth precision of the rendering result can be improved, GPU hardware coding, parameter adjustment, post-processing, and the like are used, so that on the basis of realizing balanced rendering tasks, through high-fidelity image coding, transmission and other mechanisms, the three-dimensional scene is projected to the LED screen at a stable frame rate and high quality, ensuring that the multi-machine load balancing is realized while using the least equipment to complete high-quality rendering, and the rendering quality and cost are balanced.

[0095] Embodiment Two

[0096] In this embodiment two, a three-dimensional scene rendering processing method is provided from the perspective of a rendering device, which is described with reference to Figure 4 The method can specifically include the following steps.

[0097] S401: receiving region index information provided by a master device, wherein the region index information is established according to the number of rendering devices required for shooting based on a three-dimensional scene;

[0098] S402: during the digital background shooting process, after receiving information broadcast frame by frame by the master device, determining total rendering task information in the pose state of the current frame of the camera, wherein the total rendering task information includes a total region range required to be rendered in the three-dimensional scene and location information of the total region range in the three-dimensional scene;

[0099] S403: determining a sub-region range required to be responsible for from the total region range according to the total region range, the location information, and the region index information;

[0100] S404: rendering the three-dimensional scene content in the sub-region range, and returning the rendering result to the master device, so that after the master device receives the rendering results in multiple sub-region ranges, the master device splices and combines the rendering results to generate a target image frame, and displays the target image frame on a target screen associated with the digital background shooting based on the target image frame.

[0101] Specifically, the information broadcast frame by frame by the master device can include pose information of the current frame of the camera, at this time, the rendering device can determine the total region range required to be rendered in the three-dimensional scene and the location information based on the pose information of the camera in the current frame according to a rendering engine or the like.

[0102] In addition, when returning the rendering result to the master device, the rendering result can be encoded by GPU hardware coding and then returned to the master device.

[0103] For the parts not described in detail in this embodiment two, refer to the description in the embodiment one, which will not be repeated here.

[0104] It should be noted that the embodiments of the present application can involve the use of user data. In actual applications, user-specific personal data can be used in the schemes described herein in a manner that complies with applicable laws and regulations of the country (for example, with the explicit consent of the user, with the actual notification to the user, etc.) and within the scope permitted by applicable laws and regulations.

[0105] Corresponding to the first embodiment, the present application also provides a three-dimensional scene rendering processing device, which is described in detail with reference to Figure 5 The device can include:

[0106] A rendering device number determination unit 501 is configured to determine the number of rendering devices required for digital background shooting based on a three-dimensional scene.

[0107] A pose information acquisition unit 502 is configured to acquire the pose information of a camera in units of frames during the digital background shooting process, so as to determine the total rendering task information of the same frame according to the pose information, wherein the total rendering task information includes: the total area range required to be rendered in the three-dimensional scene in the state of the pose information, and the location information of the total area range in the three-dimensional scene.

[0108] A rendering task allocation unit 503 is configured to allocate the total rendering task of the same frame to each rendering device in an average manner according to the number of rendering devices, so that the rendering devices respectively render different sub-area ranges in the total area range.

[0109] A rendering result processing unit 504 is configured to splice and combine the rendering results after each rendering device returns the rendering result in the corresponding sub-area range, to generate a target image frame, and to perform image display on a target screen associated with the digital background shooting according to the target image frame.

[0110] The rendering task allocation unit can specifically include:

[0111] A region index information providing sub-unit is configured to establish a region index for each rendering device according to a preset sub-area division manner after determining the number of rendering devices required, and provide the region index information of each rendering device, so as to achieve average allocation of the total rendering task according to the region index information.

[0112] In addition, the rendering task allocation unit can specifically include:

[0113] The pose information broadcasting subunit is configured to broadcast the pose information of the camera to each rendering device after obtaining the pose information of the camera, so that each rendering device determines a total area range and a location information of a required rendering in the three-dimensional scene according to the pose information, and determines a sub-area range of a required responsibility of each rendering device according to the total area range and the location information and an area index information of each rendering device.

[0114] Alternatively, the rendering task allocation unit can further include:

[0115] The total area determination subunit is configured to determine a total area range and a location information of a required rendering in the three-dimensional scene according to the pose information of the camera after obtaining the pose information of the camera.

[0116] The total area information broadcasting subunit is configured to broadcast the total area range and the location information of the required rendering in the three-dimensional scene to each rendering device, so that each rendering device determines a sub-area range of a required responsibility of each rendering device according to the total area range and the location information and an area index information of each rendering device.

[0117] The color depth precision of the rendering result generated by the rendering device is equal to or higher than a threshold value, and the threshold value is determined according to a size of a screen in a digital background shooting process and / or a requirement of an image detail retention program.

[0118] In a specific implementation, the rendering device can encode the rendering result by using a hardware coding manner based on a graphics processing unit (GPU) and return the rendering result.

[0119] In this case, the device can further include:

[0120] The hardware decoding unit is configured to perform hardware decoding on the information returned by the rendering device before the rendering result is spliced and combined.

[0121] In addition, the device can further include:

[0122] The post-processing unit is configured to perform quality improvement processing on the target image frame and display the target image frame on the target screen. The quality improvement processing includes one or more of the following: de-gamma processing on a color gradient, blurring processing on a long shot, and image saturation modification processing according to a field light environment.

[0123] Corresponding to the second embodiment, the application also provides a three-dimensional scene rendering processing device, which is described below with reference to Figure 6 The device can include:

[0124] The regional index information receiving unit 601 is configured to receive regional index information provided by a master device, wherein the regional index information is established according to a number of rendering devices required for shooting a digital background based on a three-dimensional scene;

[0125] The total rendering task determining unit 602 is configured to, during the shooting of the digital background, determine total rendering task information in a pose state of a camera in a current frame after receiving information broadcast frame by frame by the master device, wherein the total rendering task information includes a total regional range required to be rendered in the three-dimensional scene and location information of the total regional range in the three-dimensional scene.

[0126] The rendering regional determining unit 603 is configured to determine a sub-regional range required to be responsible from the total regional range according to the total regional range, the location information and the regional index information.

[0127] The rendering unit 604 is configured to render three-dimensional scene content in the sub-regional range and return a rendering result to the master device, so that the master device stitches and combines rendering results in a plurality of sub-regional ranges to generate a target image frame and displays an image based on the target image frame on a target screen associated with the digital background shooting.

[0128] The information broadcast frame by frame by the master device includes pose information of the camera in the current frame.

[0129] The total rendering task determining unit 602 can be specifically configured to:

[0130] determine the total regional range required to be rendered in the three-dimensional scene and the location information of the total regional range in the three-dimensional scene according to the pose information of the camera in the current frame.

[0131] In addition, the embodiment of the present application further provides a computer readable storage medium having a computer program stored thereon, and the program is executed by a processor to implement steps of the method in any one of the preceding method embodiments.

[0132] An electronic device includes:

[0133] one or more processors; and

[0134] a memory associated with the one or more processors, the memory configured to store program instructions that, when executed by the one or more processors, perform steps of the method in any one of the preceding method embodiments.

[0135] wherein, Figure 7An exemplary architecture of the electronic device can include a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, and a memory 720. The processor 710, the video display adapter 711, the disk drive 712, the input / output interface 713, the network interface 714, and the memory 720 can be communicatively connected through a communication bus 730.

[0136] The processor 710 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the present application.

[0137] The memory 720 can be implemented in a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 720 can store an operating system 721 for controlling the operation of the electronic device 700, a BIOS (Basic Input Output System) for controlling the low-level operation of the electronic device 700. In addition, a web browser 723, a data storage management system 724, and a three-dimensional scene rendering processing system 725, and the like can also be stored. The three-dimensional scene rendering processing system 725 can be an application program for implementing the above-mentioned steps in the embodiments of the present application. In summary, when the technical solutions provided in the present application are implemented by software or firmware, the related program codes are stored in the memory 720 and executed by the processor 710.

[0138] The input / output interface 713 is configured to connect an input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, and the like, and the output device can include a display, a speaker, a vibrator, an indicator, and the like.

[0139] The network interface 714 is configured to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0140] Bus 730 includes a path for transferring information between the various components of the device, such as processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, and memory 720.

[0141] It should be noted that although the above device only shows the processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, memory 720, bus 730 and the like, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary to implement the scheme of the present application, and does not have to contain all the components shown in the figure.

[0142] From the description of the above embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and the necessary general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including a number of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.

[0143] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the different parts from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, it is described more simply, and the relevant parts can be referred to the part of the method embodiment. The above described system and system embodiment is only illustrative, and the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to the actual needs, some or all of the modules can be selected to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0144] The three-dimensional scene rendering processing method and the electronic device provided by the present application are described in detail above, and the principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method of rendering a three-dimensional scene, the method comprising: The method comprises the following steps: determining the number of rendering devices required for digital background shooting based on a three-dimensional scene; during the digital background shooting process, obtaining the pose information of the camera in units of frames, so as to determine the total rendering task information of the same frame according to the pose information, wherein the total rendering task information comprises the total area range required to be rendered in the three-dimensional scene in the state of the pose information, and the location information of the total area range in the three-dimensional scene; according to the number of rendering devices, establishing area indexes for each rendering device according to a preset sub-area division method, and providing each rendering device with the area index information where each rendering device is located, and distributing the total rendering task of the same frame to each rendering device according to the area index information, so that each rendering device determines the range and location of the sub-area in the total area range required to be rendered according to the area index information, and performs the same amount of rendering task during the rendering of each frame to complete real-time rendering of the sub-area required to be rendered by each rendering device, wherein each rendering device participates in the rendering and has the same amount of task during the rendering of the same frame; after each rendering device returns the rendering result of the sub-area required to be rendered by each rendering device, the rendering results are spliced and combined to generate a target image frame, and the target image frame is displayed on a target screen associated with the digital background shooting, wherein the target image frame is displayed in the area that can enter the camera shooting range of the target screen.

2. The method of claim 1, wherein the step of establishing area indexes for each rendering device according to the number of rendering devices and the preset sub-area division method, and providing each rendering device with the area index information where each rendering device is located, and distributing the total rendering task of the same frame to each rendering device according to the area index information further comprises: after obtaining the pose information of the camera, broadcasting the pose information to each rendering device, so that each rendering device determines the total area range required to be rendered in the three-dimensional scene and the location information according to the pose information, and determines the range and location of the sub-area in the total area range required to be rendered by each rendering device according to the total area range and the location information and the area index information where each rendering device is located.

3. The method of claim 1, wherein the step of establishing area indexes for each rendering device according to the number of rendering devices and the preset sub-area division method, and providing each rendering device with the area index information where each rendering device is located, and distributing the total rendering task of the same frame to each rendering device according to the area index information further comprises: after obtaining the pose information of the camera, determining the total area range required to be rendered in the three-dimensional scene and the location information according to the pose information. ​ ​ The total area range and the location information of the required rendering in the three-dimensional scene are broadcast to each rendering device, so that each rendering device determines the range and location of the sub-area in the total area range required for rendering according to the total area range and the location information, and the area index information of each rendering device.

4. The method of claim 1, wherein, The rendering device returns the rendering result after encoding by the hardware coding based on the graphic processing unit (GPU); Before the splicing and combining of the rendering results, the method further comprises: The rendering device returns the rendering result after encoding by the hardware coding based on the graphic processing unit (GPU); 5. The method of any one of claims 1 to 4, wherein, The image display on the target screen associated with the digital background based on the target image frame comprises: After the quality improvement processing of the target image frame, the image is displayed on the target screen; wherein the quality improvement processing comprises one or more of the following: color gradient de-scaling processing, blurring processing of long shots, and image saturation modification processing according to the on-site light environment.

6. A method of rendering a three-dimensional scene, the method comprising: The method further comprises: receiving the area index information provided by the master control device, wherein the area index information is established for each rendering device based on the number of rendering devices required for the digital background shooting of the three-dimensional scene and according to the preset sub-area division method; During the digital background shooting, after receiving the information broadcast by the master control device frame by frame, the total rendering task information in the pose state of the camera in the current frame is determined, and the total rendering task information comprises the total area range required for rendering in the three-dimensional scene and the location information of the total area range in the three-dimensional scene; According to the total area range, the location information, the area index information, and the number of rendering devices, the range and location of the sub-area in the total area range required for rendering by each rendering device are determined from the total area range. The three-dimensional scene content in the sub-regions is rendered in real time, and the rendering results are returned to the master device, so that the master device receives the rendering results in multiple sub-regions, assembles and combines the rendering results to generate a target image frame, and displays the target image frame on a target screen associated with the digital background based on the target image frame; wherein the target image frame is displayed in the area that can enter the camera shooting range of the target screen; the rendering result is that the master device establishes a region index for each rendering device according to a preset sub-region division method according to the number of rendering devices, and provides each rendering device with the region index information of the region where the rendering device is located, and distributes the total rendering task of the same frame to each rendering device according to the region index information, so that each rendering device determines the range and position of the sub-region in the total region range required to be rendered by each rendering device according to the region index information, and performs the same amount of rendering task during each frame rendering to complete the real-time rendering of the sub-region required to be rendered by each rendering device to obtain the rendering result, wherein each rendering device participates in the rendering and has the same amount of task during the rendering of the same frame.

7. The method of claim 6, wherein, The information broadcasted by the master device frame by frame includes pose information of the current frame of the camera. The determination of the total region range required to be rendered in the three-dimensional scene under the pose state of the current frame of the camera, and the location information of the total region range in the three-dimensional scene, comprises: According to the pose information of the camera in the current frame, the total region range required to be rendered in the three-dimensional scene, and the location information of the total region range in the three-dimensional scene are determined.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

9. An electronic device, comprising: Comprise: One or more processors; And The memory associated with the one or more processors is used to store program instructions, and the program instructions are read and executed by the one or more processors to perform the steps of the method of any one of claims 1 to 7.

Images