Rendering video acquisition method and related equipment

Abstract

The invention discloses a rendered video acquisition method and related equipment, and relates to the technical field of 3D rendering, and the method comprises the steps: receiving a rendered video instruction, selecting any constructed container based on the rendered video instruction, constructing a virtualized screen in the container by using an XVFB module, and obtaining a rendered video based on the virtualized screen. The method comprises the following steps: starting a Unity 3D engine, executing a preset rendering program, generating real-time rendering data, confirming the type of a rendering pipeline of the Unity 3D engine, acquiring real-time rendering image data from the real-time rendering data based on the pipeline type, and performing synthesis operation on the real-time rendering image data and audio data in the real-time rendering data, and obtaining a rendered video. According to the embodiment of the invention, the virtual screen is constructed for each container, the rendering data does not need to be displayed on the screen mounted on the Linux host, so that higher system authority does not need to be obtained, and the security risk problem is eliminated.

Description

Technical Field

[0001] The present application relates to the field of 3D rendering technology, and in particular to a rendering video acquisition method and related equipment. Background Art

[0002] With the rapid development of science and technology, the demand for 3D real-time rendering is gradually increasing. Through 3D real-time rendering technology, we can accurately simulate the light and shadow changes, texture details and dynamic effects of the real world, create highly realistic images or animations, and give users an immersive experience.

[0003] In the relevant 3D real-time rendering technology, the Linux host screen is mounted to the container of the rendering video acquisition device. When the Unity3D engine in the container recognizes the screen, the Unity3D engine in the container is started and the preset rendering program is executed to generate rendering data, and the rendering data is displayed on the mounted Linux host screen. Finally, the video displayed on the mounted Linux host screen is recorded to generate a rendered video.

[0004] In the actual rendering process, the rendering video acquisition device needs to obtain higher system permissions to display the rendering data on the screen of the mounted Linux host, which poses a security risk.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide a rendering video acquisition method and related equipment, aiming to solve the following security risks: in the actual rendering process, the rendering video acquisition device needs to obtain higher system permissions to display the rendering data on the screen of the mounted Linux host, which poses a security risk.

[0007] To achieve the above object, the present application proposes a method for obtaining a rendered video, the method comprising:

[0008] Receive a rendering video instruction, and select any constructed container based on the rendering video instruction, wherein the container is provided with an XVFB module and a Unity3D engine;

[0009] Use the XVFB module to build a virtualized screen in the container;

[0010] Based on the virtualized screen, start the Unity3D engine and execute a preset rendering program to generate real-time rendering data, wherein the preset rendering program is a rendering program associated with the rendered video that is set in advance;

[0011] Confirming the type of the rendering pipeline of the Unity3D engine, and acquiring real-time rendering image data from the real-time rendering data based on the pipeline type;

[0012] The real-time rendered image data and the audio data in the real-time rendered data are synthesized to obtain a rendered video.

[0013] In one embodiment, the step of confirming the type of the rendering pipeline of the Unity3D engine and obtaining the real-time rendering image data from the real-time rendering data based on the pipeline type further includes:

[0014] Determine the type of rendering pipeline of the Unity3D engine;

[0015] If the rendering pipeline is a built-in rendering pipeline, based on the Graphics Blit function, image texture data is obtained from the real-time rendering data, and the image texture data is copied to the custom rendering texture data using a shader; or, if the rendering pipeline is a universal rendering pipeline, based on the cmd Blit function, image texture data is obtained from the real-time rendering data, and the image texture data is copied to the custom rendering texture data using a shader;

[0016] Based on the custom rendered texture data after the data is copied, using a ReadPixels function and an Apply function, converting the custom rendered texture data after the data is copied into 2D texture data;

[0017] The GetRawTextureData function is used to convert the 2D texture data into real-time rendering image data.

[0018] In one embodiment, the step of obtaining image texture data from the real-time rendering data and copying the image texture data to the custom rendering texture data using a shader further includes:

[0019] Determine whether there is an instruction to execute a transparent background operation in the preset rendering program;

[0020] If there is an instruction to execute a transparent background operation, image texture data is obtained from the real-time rendering data, and pixels in the image texture data are determined, and the difference between the color value of the pixel and the color value of the background pixel is calculated in the shader. If the difference is less than or equal to a preset threshold, the pixel is adjusted to a transparent pixel, and the adjusted image texture data is copied to the custom rendering texture data using the shader;

[0021] If there is no instruction to perform a transparent background operation, the image texture data is copied to the custom rendering texture data using a shader.

[0022] In one embodiment, the step of performing a synthesis operation on the real-time rendered image data and the audio data in the real-time rendered data to obtain a rendered video further includes:

[0023] Adding the real-time rendered image data and the audio data in the real-time rendered data to a video queue and an audio queue respectively;

[0024] Send the image data in the video queue to the video named pipe, and send the audio data in the audio queue to the audio named pipe;

[0025] The first image data of the video named pipe and the first audio data of the audio named pipe are read, and a synthesis operation is performed on the first image data and the first audio data to obtain a rendered video.

[0026] In one embodiment, the step of adding the real-time rendered image data and the audio data in the real-time rendered data to a video queue and an audio queue respectively further includes:

[0027] Obtaining the length of the audio data in the real-time rendering data, and calculating the duration of writing the audio data based on the length to obtain a first duration;

[0028] Acquire a second duration for writing the real-time rendered image data, and calculate a difference between the first duration and the second duration to obtain a video frame difference;

[0029] Based on the video frame difference, performing a frame insertion or frame dropping operation to eliminate the video frame difference and synchronize the audio data with the real-time rendered image data;

[0030] The synchronized audio data and the real-time rendered image data are added to a video queue and an audio queue respectively.

[0031] In one embodiment, obtaining the rendered video includes obtaining a video file and obtaining a video stream.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a rendering video acquisition device, the rendering video acquisition device comprising:

[0033] A container selection module, the container selection module is used to receive a rendering video instruction, and based on the rendering video instruction, select any constructed container, wherein the container is provided with an XVFB module and a Unity3D engine;

[0034] A virtualized screen construction module, wherein the virtualized screen construction module is used to construct a virtualized screen in the container using an XVFB module;

[0035] A rendering data acquisition module, wherein the rendering data acquisition module is used to start the Unity3D engine and execute a preset rendering program based on the virtualized screen to generate real-time rendering data, wherein the preset rendering program is a rendering program associated with a rendering video that is set in advance;

[0036] A module for acquiring real-time rendering image data, wherein the module is used to confirm the type of the rendering pipeline of the Unity3D engine and acquire the real-time rendering image data from the real-time rendering data based on the pipeline type;

[0037] A data synthesis module is used to synthesize the real-time rendering image data and the audio data in the real-time rendering data to obtain a rendered video.

[0038] In one embodiment, the module for obtaining real-time rendering image data includes:

[0039] A first confirmation unit, used to confirm the type of the rendering pipeline of the Unity3D engine;

[0040] a copy unit, for obtaining image texture data from the real-time rendering data based on a Graphics Blit function, and copying the image texture data to the custom rendering texture data using a shader, if the rendering pipeline is a built-in rendering pipeline; or, for obtaining image texture data from the real-time rendering data based on a cmd Blit function, and copying the image texture data to the custom rendering texture data using a shader, if the rendering pipeline is a universal rendering pipeline;

[0041] A first data conversion unit is used to convert the custom rendered texture data after the data is copied into 2D texture data by using a ReadPixels function and an Apply function based on the custom rendered texture data after the data is copied;

[0042] The second data conversion unit is used to convert the 2D texture data into real-time rendering image data by using the GetRawTextureData function.

[0043] In one embodiment, the module for acquiring real-time rendering image data further includes:

[0044] A second confirmation unit is used to determine whether there is an instruction to execute a transparent background operation in the preset rendering program;

[0045] A pixel adjustment unit, if there is an instruction to execute a transparent background operation, obtains image texture data from the real-time rendering data, determines pixels in the image texture data, calculates the difference between the color value of the pixel and the color value of the background pixel in the shader, and if the difference is less than or equal to a preset threshold, adjusts the pixel to a transparent pixel, and uses the shader to copy the adjusted image texture data to the custom rendering texture data;

[0046] The data copying unit uses a shader to copy the image texture data into the custom rendering texture data if there is no instruction to execute the transparent background operation.

[0047] In one embodiment, the data synthesis module includes:

[0048] A first data processing unit, used for adding the real-time rendered image data and the audio data in the real-time rendered data into a video queue and an audio queue respectively;

[0049] A data sending unit, used for sending the image data in the video queue to a video named pipe, and sending the audio data in the audio queue to an audio named pipe;

[0050] The data reading unit is used to read the first image data of the video named pipe and the first audio data of the audio named pipe, perform a synthesis operation on the first image data and the first audio data, and obtain a rendered video.

[0051] In one embodiment, the data synthesis module further includes:

[0052] A duration acquisition unit, used to acquire the length of the audio data in the real-time rendering data, and based on the length, calculate the duration of writing the audio data to obtain a first duration;

[0053] A duration difference calculation unit, used for obtaining a second duration for writing the real-time rendered image data, and calculating a difference between the first duration and the second duration to obtain a video frame difference;

[0054] An audio and video synchronization unit, which performs a frame insertion or frame drop operation based on the video frame difference to eliminate the video frame difference and synchronize the audio data with the real-time rendered image data;

[0055] The second data processing unit adds the synchronized audio data and the real-time rendered image data into a video queue and an audio queue respectively.

[0056] In addition, to achieve the above-mentioned purpose, the present application also proposes a rendering video acquisition device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the rendering video acquisition method described above.

[0057] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the rendering video acquisition method described above are implemented.

[0058] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the rendering video acquisition method described above are implemented.

[0059] One or more technical solutions proposed in this application have at least the following technical effects:

[0060] The present application proposes a method for obtaining a rendered video and related equipment. In the related art, in the actual video rendering process, due to the limitation of the number of mounted Linux host screens, only a certain number of videos can be rendered each time, resulting in very low efficiency of video rendering. In the present application, first, any constructed container is selected, wherein the container is provided with an XVFB module and a Unity3D engine, and then, a virtualized screen is constructed in the container using the XVFB module. Further, based on the virtualized screen, the Unity3D engine is started and a preset rendering program is executed to generate real-time rendering data, and real-time audio data is obtained from the real-time rendering data, wherein the preset rendering program is a rendering program associated with the rendered video set in advance. After the real-time rendering data is generated, the type of the rendering pipeline of the Unity3D engine is confirmed, and based on the pipeline type, real-time rendering image data is obtained from the real-time rendering data. Finally, the real-time rendering image data and the real-time audio data are synthesized to obtain a rendered video.

[0061] It is understandable that for any built container, the XVFB module set in the container can be used to build a virtualized screen in the container. By building a virtualized screen for each container, it is not necessary to display the rendered data on the mounted Linux host screen, and thus it is not necessary to obtain higher system permissions, eliminating security risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 A flowchart of the first embodiment of the method for obtaining a rendered video according to the present application is provided;

[0065] Figure 2 A schematic diagram of the rendering program startup process provided for the rendering video acquisition method of this application;

[0066] Figure 3 A flowchart of Embodiment 2 of the method for obtaining rendered video provided in this application;

[0067] Figure 4 A brief flowchart of a real-time audio and video synthesis method provided for the video acquisition method of the present application;

[0068] Figure 5 A schematic diagram of the module structure of a video acquisition device rendered in an embodiment of the present application;

[0069] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the method for obtaining a rendered video in an embodiment of the present application.

[0070] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0071] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0072] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0073] The main solutions of the embodiments of this application are:

[0074] In this embodiment, for ease of description, the following description is made by taking the rendering video acquisition device as the execution subject.

[0075] In the actual rendering process of the prior art, the rendering video acquisition device needs to obtain higher system permissions to display the rendering data on the screen of the mounted Linux host, which poses a security risk.

[0076] The present application provides a solution, receiving a rendering video instruction, based on the rendering video instruction, selecting any constructed container, using the XVFB module to construct a virtualized screen in the container, based on the virtualized screen, starting the Unity3D engine and executing a preset rendering program, generating real-time rendering data, confirming the type of the rendering pipeline of the Unity3D engine, and based on the pipeline type, obtaining real-time rendering image data from the real-time rendering data, synthesizing the real-time rendering image data and the audio data in the real-time rendering data, and obtaining a rendered video. By constructing a virtualized screen for each container, the limitation on the number of mounted Linux host screens is avoided, multiple containers can be rendered, and multiple rendered videos can be obtained, thereby improving rendering efficiency.

[0077] It is understandable that for any built container, the XVFB module set in the container can be used to build a virtualized screen in the container. By building a virtualized screen for each container, it is not necessary to display the rendered data on the mounted Linux host screen, and thus it is not necessary to obtain higher system permissions, eliminating security risks.

[0078] Based on this, the embodiment of the present application provides a method for obtaining a rendered video. Figure 1 , Figure 1 This is a flowchart of the first embodiment of the method for acquiring rendered video in this application.

[0079] In this embodiment, the rendering video acquisition method includes steps S100 to S400:

[0080] Step S100, receiving a rendering video instruction, and selecting any constructed container based on the rendering video instruction, wherein the container is provided with an XVFB module and a Unity3D engine;

[0081] In this embodiment, the execution subject is a rendering video acquisition device, which can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, etc., or an electronic device capable of realizing the above functions. After the rendering video device detects that the staff clicks the "Get Rendering Video" button, it selects any constructed container. The rendering video acquisition device includes one or more containers, each of which is pre-installed with an XVFB module and a Unity3D engine.

[0082] It should be noted that containers are an important technology in modern software development and deployment. Containers can perform packaging and running operations in an efficient way. Containers have lightweight isolation characteristics. Based on this lightweight isolation feature, each container can run different applications without interfering with each other. Compared with traditional virtual machines, containers do not need to load a complete operating system, so they occupy fewer resources and start faster.

[0083] In addition, the XVFB module can be used to implement a virtual display function, so that the device installed with the XVFB module can run graphical interface applications. Based on the XVFB module, for any device without a physical display, after the XVFB module is installed on the device, the device can use the XVFB module to build a virtualized screen and perform operations to run graphical interface applications.

[0084] It should also be noted that the Unity3D engine is a powerful comprehensive game development tool. The Unity3D engine is developed by Unity Technologies. The Unity3D engine can run on multiple operating systems such as Windows, Mac and Linux (in this application, the operating system is the Linux system). The Unity3D engine not only supports 3D game development, but also supports 2D game development. The Unity3D engine can create various types of interactive content including architectural visualization and real-time 3D animation. The Unity3D engine has the function of producing real-time 3D video. Based on this function, the Unity3D engine is widely used in video production and film and television post-production.

[0085] Step S200, using the XVFB module to construct a virtualized screen in the container;

[0086] In this embodiment, after selecting a container, the rendering video acquisition device uses the XVFB module to construct a virtualized screen in the container.

[0087] It should be noted that in this application, the resolution of the virtualized screen is the same as the resolution of the rendered video, and the resolution of the rendered video can be changed by adjusting the resolution of the virtualized screen. And the size of the virtual screen is not affected by the display resolution.

[0088] Step S300, based on the virtualized screen, starting the Unity3D engine and executing a preset rendering program to generate real-time rendering data, wherein the preset rendering program is a rendering program associated with a rendered video that is set in advance;

[0089] It is understandable that, in this embodiment, the rendering video acquisition device first needs to build an environment that supports the operation of the Unity3D engine, that is, the virtualized screen. After the conditions for starting the Unity3D engine are met, the rendering video acquisition device starts the Unity3D engine in the container, and loads the preset rendering program to generate real-time rendering data. Among them, the preset rendering program is a rendering program associated with the rendering video set in advance, and the real-time rendering data includes real-time rendering image data and real-time audio data.

[0090] In addition, it can also be understood that if you want to start the Unity3D engine and execute the preset rendering program, the rendering video acquisition device needs to build an environment that supports the operation of the Unity3D engine. In the actual rendering process, if the rendering video acquisition device wants to start the Unity3D engine, it needs to meet two conditions: the rendering video acquisition device recognizes the screen; the rendering video acquisition device installs all dependent libraries in advance. Among them, the dependent libraries include the dynamic library of the Unity3D engine and the third-party library that the Unity3D engine depends on.

[0091] For the specific startup process, please refer to Figure 2 , the rendering video acquisition device installs all dependent libraries in advance. After receiving the rendering instruction, first, the rendering video acquisition device will identify whether there is a screen (virtualized screen) in the container. If there is a screen in the container, the rendering video acquisition device loads all dependent libraries installed in advance and starts the Unity3D engine to initialize the Unity3D engine. After completing the initialization of the Unity3D engine, based on the Unity3D engine, the rendering video acquisition device obtains a preset rendering program, and runs the preset rendering program to generate real-time rendering data including real-time rendering image data and real-time audio data. Among them, the virtualized screen provides a detection device for starting the Unity3D engine and executing the preset rendering program.

[0092] It is understandable that in the actual rendering process, the rendering is performed frame by frame. It is generally considered that rendering 30 frames per second or more is real-time rendering. The time consumed for each frame, in addition to the rendering time, also includes the time consumed by the rendering video acquisition device to generate real-time rendering data. Therefore, in the actual rendering process, GPU acceleration must be used to ensure real-time performance.

[0093] It should be noted that in the actual rendering process, the virtualized screen and the Unity3D engine can work together. After the rendering video acquisition device loads the dynamic library, the rendering video acquisition device will start the Unity3D engine and initialize the Unity3D engine. In the process of starting the Unity3D engine and initializing the Unity3D engine, the rendering video acquisition device will also call the GPU to output the rendered image to the virtualized screen.

[0094] Step S400, confirming the type of the rendering pipeline of the Unity3D engine, and acquiring real-time rendering image data from the real-time rendering data based on the pipeline type;

[0095] It should be noted that in this application, there are two types of rendering pipelines for the Unity3D engine, one is the built-in rendering pipeline and the other is the universal rendering pipeline. In the actual rendering process, generally only one type of rendering pipeline is used.

[0096] In this embodiment, the rendering video acquisition device first determines the rendering pipeline type of the Unity3D engine, and then based on different pipeline types, the rendering video acquisition device performs different operations to obtain real-time rendering image data from the real-time rendering data.

[0097] Specifically, the step of confirming the type of the rendering pipeline of the Unity3D engine and acquiring the real-time rendering image data from the real-time rendering data based on the pipeline type further includes steps S410 to S440:

[0098] Step S410, confirming the type of the rendering pipeline of the Unity3D engine;

[0099] In this embodiment, the rendering video acquisition device confirms that the rendering pipeline of the Unity3D engine is a built-in rendering pipeline or a universal rendering pipeline, and further determines which operation to perform.

[0100] Step S420, if the rendering pipeline is a built-in rendering pipeline, based on the Graphics Blit function, obtain image texture data from the real-time rendering data, and use a shader to copy the image texture data to the custom rendering texture data; or, if the rendering pipeline is a universal rendering pipeline, based on the cmd Blit function, obtain image texture data from the real-time rendering data, and use a shader to copy the image texture data to the custom rendering texture data;

[0101] In this embodiment, if the rendering pipeline is a built-in rendering pipeline, based on the Graphics Blit function, the rendering video acquisition device acquires image texture data from the real-time rendering data, and uses a shader to copy the image texture data to the custom rendering texture data.

[0102] If the rendering pipeline is a universal rendering pipeline, based on the cmd Blit function, the rendering video acquisition device acquires image texture data from the real-time rendering data, and uses a shader to copy the image texture data to the custom rendering texture data.

[0103] It is understandable that when using the Universal Render Pipeline (URP), the function OnRenderImage is invalid. Based on the function OnRenderImage, the rendering video acquisition device obtains image texture data from the real-time rendering data, and the rendering video acquisition device needs to rewrite the ScriptableRendererFeature class to achieve texture copying.

[0104] Specifically, the step of acquiring image texture data from the real-time rendering data and copying the image texture data to the custom rendering texture data using a shader further includes steps S421 to S423:

[0105] Step S421, determining whether there is an instruction to execute a transparent background operation in the preset rendering program;

[0106] In this embodiment, after the rendering video acquisition device acquires the image texture data from the real-time rendering data, the rendering video acquisition device searches the preset rendering program for whether there is an instruction to execute a transparent background operation.

[0107] It is understandable that in an actual rendering scenario, the rendering video acquisition device may need to perform a transparent operation on the background of the rendered image to highlight the rendered subject. Before rendering, if a transparent operation needs to be performed on the background of the rendered image, the staff will set a transparent background operation instruction in the preset rendering program.

[0108] Step S422, if there is an instruction to execute a transparent background operation, image texture data is obtained from the real-time rendering data, and pixels in the image texture data are determined, and the difference between the color value of the pixel and the color value of the background pixel is calculated in the shader. If the difference is less than or equal to a preset threshold, the pixel is adjusted to a transparent pixel, and the adjusted image texture data is copied to the custom rendering texture data using the shader;

[0109] In this embodiment, if there is an instruction to perform a transparent background operation in the preset rendering program, the rendering video acquisition device acquires image texture data from the real-time rendering data and determines the pixel points in the image texture data. After determining the pixel points, the rendering video acquisition device calculates the difference between the color value of the pixel point and the color value of the background pixel point in the shader. If the difference is less than or equal to the preset threshold, the pixel point corresponding to the difference is the background pixel point, and the rendering video acquisition device adjusts the pixel point to a transparent pixel point. The "the rendering video acquisition device calculates the difference between the color value of the pixel point and the color value of the background pixel point in the shader. If the difference is less than or equal to the preset threshold, the pixel point corresponding to the difference is the background pixel point, and the rendering video acquisition device adjusts the pixel point to a transparent pixel point" operation is performed on all pixel points in the image texture data. After completing the operation, the adjusted image texture data is obtained. Finally, the rendering video acquisition device uses the shader to copy the adjusted image texture data to the custom rendering texture data.

[0110] Specifically, the video acquisition device calculates the difference between the color value of the pixel and the color value of the background pixel in the shader. If the difference is less than or equal to a preset threshold, the video acquisition device adjusts the pixel to a transparent pixel (all four channels of RGBA are 0), and uses a shader to copy the adjusted image texture data to the custom rendering texture data.

[0111] It should also be noted that when the video acquisition device performs the transparent background operation, it is necessary to specify the output file suffix as ".webm" and add the output parameter "-c:v libvpx-vp9-c:a libvorbis-map0:v:0-map 1:a:0-crf 30-b:v 0-deadline realtime-cpu-used 8". Among them, libvpx-vp9 is an encoder that supports transparent background, and "-crf" and subsequent parameters are used to speed up data synthesis to ensure real-time performance.

[0112] It can be understood that for any two pixels, the rendering video acquisition device calculates the difference in color values ​​of the two pixels. If the difference is less than or equal to a preset threshold, the two pixels can be considered to have the same color. In this embodiment, the pixel corresponding to the difference is the background pixel.

[0113] It should be noted that the color value of a pixel is usually expressed in RGBA format, where each letter represents a color channel. R (Red): The value of the red channel, usually ranging from 0 to 255; G (Green): The value of the green channel, usually ranging from 0 to 255; B (Blue): The value of the blue channel, usually ranging from 0 to 255; A (Alpha): The value of the transparency channel, usually ranging from 0 to 1, where 0 means completely transparent and 1 means completely opaque.

[0114] For example, a pixel's color value could be (255, 0, 0, 1), which means the pixel is red and fully opaque. If the value of the transparency channel is 0.5, the pixel is semi-transparent red.

[0115] In different programming environments and image processing software, these values ​​may be expressed in different formats, such as hexadecimal (such as #FF0000FF represents red and completely opaque) or floating point numbers (such as (1.0, 0.0, 0.0, 1.0)). When processing pixel colors, you need to determine how to correctly set and obtain these values ​​based on the specific programming language or software API documentation.

[0116] Step S423: If there is no instruction to execute the transparent background operation, the image texture data is copied to the custom rendering texture data using a shader.

[0117] It is understandable that if there is no need to blur the background, the rendering video acquisition device can directly use a shader to copy the image texture data into the custom rendering texture data.

[0118] Step S430, based on the custom rendered texture data after the data is copied, using the ReadPixels function and the Apply function, converting the custom rendered texture data after the data is copied into 2D texture data;

[0119] In this embodiment, after obtaining the custom rendering texture data after data replication, the rendering video acquisition device uses the ReadPixels function and the Apply function to convert the custom rendering texture data into 2D texture data.

[0120] Step S440: Use the GetRawTextureData function to convert the 2D texture data into real-time rendering image data.

[0121] In this embodiment, after obtaining the 2D texture data, the rendering video acquisition device uses the GetRawTextureData function to convert the 2D texture data into real-time rendering image data.

[0122] Step S500: Perform a synthesis operation on the real-time rendered image data and the audio data in the real-time rendered data to obtain a rendered video.

[0123] In this embodiment, after obtaining the real-time rendering image data, the rendering video acquisition device obtains audio data from the real-time rendering data, and the rendering video acquisition device performs a synthesis operation on the real-time rendering image data and the audio data in the real-time rendering data to obtain a rendering video.

[0124] It should be noted that the rendered video obtained can be either a video file or a video stream. Video streaming is a technology for transmitting video data over the network, which allows workers to watch video content in real time without waiting for the entire video file to be downloaded. Video streaming is usually used in application scenarios such as online video services, live broadcasts, video conferencing, and remote monitoring.

[0125] In addition, it should be noted that the output audio and video files or audio and video streams can be controlled by specifying output parameters. For the generated rendered video file, the rendered video acquisition device sets the output parameter to the file name. When the video needs to be played, the video playback device can automatically match the encoder by the file suffix, such as ".mp4". For the generated video stream, the rendered video acquisition device sets the output parameter to the RTMP streaming address. In conjunction with the Nginx RTMP proxy, the video playback device can use FFmpeg or other software to open the RTMP stream for playback.

[0126] The present application proposes a method for obtaining a rendered video and related equipment. In the related art, in the actual video rendering process, due to the limitation of the number of mounted Linux host screens, only a certain number of videos can be rendered each time, resulting in very low efficiency of video rendering. In the present application, first, any constructed container is selected, wherein the container is provided with an XVFB module and a Unity3D engine, and then, a virtualized screen is constructed in the container using the XVFB module. Further, based on the virtualized screen, the Unity3D engine is started and a preset rendering program is executed to generate real-time rendering data, and real-time audio data is obtained from the real-time rendering data, wherein the preset rendering program is a rendering program associated with the rendered video set in advance. After the real-time rendering data is generated, the type of the rendering pipeline of the Unity3D engine is confirmed, and based on the pipeline type, real-time rendering image data is obtained from the real-time rendering data. Finally, the real-time rendering image data and the real-time audio data are synthesized to obtain a rendered video.

[0127] It is understandable that for any built container, the XVFB module set in the container can be used to build a virtualized screen in the container. By building a virtualized screen for each container, it is not necessary to display the rendered data on the mounted Linux host screen, and thus it is not necessary to obtain higher system permissions, eliminating security risks.

[0128] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 3 In step S500, the step of performing a synthesis operation on the real-time rendered image data and the audio data in the real-time rendered data to obtain a rendered video also includes steps S510 to S530:

[0129] For example, to help understand the implementation process of the rendering video acquisition method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 4 , Figure 4 A brief flowchart of a real-time audio and video synthesis method is provided.

[0130] Step S510, adding the real-time rendered image data and the audio data in the real-time rendered data to a video queue and an audio queue respectively;

[0131] In this embodiment, after obtaining the real-time rendering image data and the audio data in the real-time rendering data, the rendering video acquisition device adds the real-time rendering image data and the audio data in the real-time rendering data to a video queue and an audio queue respectively.

[0132] It is understandable that the real-time rendered image data and the real-time audio data may not be synchronized, so a synchronization algorithm needs to be executed cyclically to synchronize the real-time rendered image data and the real-time audio data.

[0133] Specifically, the step of adding the real-time rendered image data and the audio data in the real-time rendered data to the video queue and the audio queue respectively further includes steps S511 to S514:

[0134] Step S511, obtaining the length of the audio data in the real-time rendering data, and based on the length, calculating the duration of writing the audio data to obtain a first duration;

[0135] In this embodiment, the rendering video acquisition device acquires the length of the audio data, determines how much audio data is generated per second, calculates the duration of writing the audio data, and obtains a first duration.

[0136] Step S512, obtaining a second duration for writing the real-time rendered image data, and calculating a difference between the first duration and the second duration to obtain a video frame difference;

[0137] In this embodiment, the rendering video acquisition device acquires the second duration of writing the real-time rendering image data, and based on the first duration, the rendering video acquisition device calculates the difference between the first duration and the second duration to obtain the video frame difference.

[0138] It can be understood that in the actual rendering process, the format of the real-time rendering image data can be 30FPS / S, that is, 30 frames of images per second, and the length of the written real-time audio data can be 48K / S, that is, 48K audio data per second. 30 frames of images per second correspond to 48K audio data.

[0139] Step S513, based on the video frame difference, performing a frame insertion or frame drop operation to eliminate the video frame difference and synchronize the audio data with the real-time rendered image data;

[0140] It can be understood that in this embodiment, if there is 48K audio data at this time, but the image has only 29 frames, then 1 frame of image is inserted into the current image to complete 30 frames of image, so that the completed 30 frames of image are synchronized with the 48K audio data. Alternatively, if the image has 31 frames, then 1 frame of image is removed from the current image to obtain 30 frames of image, so that the 30 frames of image after the frame loss are synchronized with the 48K audio data.

[0141] Step S514: adding the synchronized audio data and the real-time rendered image data to a video queue and an audio queue respectively.

[0142] In this embodiment, after synchronizing the audio data and the real-time rendered image data, the rendered video acquisition device adds the audio data and the real-time rendered image data to the video queue and the audio queue respectively, so that the audio data and the image data can be synchronized in the subsequent process.

[0143] Step S520, sending the image data in the video queue to a video named pipe, and sending the audio data in the audio queue to an audio named pipe;

[0144] In this embodiment, after the audio data and the real-time rendered image data are added to the video queue and the audio queue respectively, the rendered video acquisition device sends the image data in the video queue to the video named pipe, and sends the audio data in the audio queue to the audio named pipe.

[0145] It should be noted that in order not to block the main thread of Unity3D, the acquired audio and video data needs to be managed through pipelines. Specifically, for audio, the rendering video acquisition device writes the content of each callback of the OnAudioFilterRead function into the audio queue; for video, the rendering video acquisition device writes the real-time rendering image data into the video queue according to the set video frame rate. Data can be cached through pipelines to avoid blocking problems.

[0146] It should be noted that in computer graphics, rendering refers to a series of steps used to convert a 3D model into a 2D image. This process usually includes steps such as geometry processing, rasterization, texture mapping, shading, and post-processing. Different graphics APIs (such as OpenGL, DirectX, and Vulkan) provide different rendering implementations.

[0147] Additionally, in this application, a pipe is a mechanism for transferring data between processes. Pipes follow a First-In-First-Out (FIFO) data processing method, which follows the principle that the first data that enters the pipe is processed first. This means that the first data written to the pipe will be the first data read. This FIFO behavior ensures that the data is transferred sequentially between processes, which is very important for communication scenarios where the order of data needs to be guaranteed.

[0148] Step S530: read the first image data of the video named pipe and the first audio data of the audio named pipe, perform a synthesis operation on the first image data and the first audio data, and obtain a rendered video.

[0149] In this embodiment, first, the rendering video acquisition device reads the first image data of the video named pipe and the first audio data of the audio named pipe, and then performs a synthesis operation on the first image data and the first audio data to obtain a rendering video, wherein the rendering video includes a rendering video file and a rendering video stream.

[0150] It can be understood that when the real-time rendered image data and the audio data in the real-time rendered data are added to the video queue and the audio queue respectively, the audio data and the image data have been synchronized, and the pipeline has the FIFO principle, so the first image data and the first audio data are synchronized.

[0151] It should be noted that the format of the data read in the two pipes must be consistent with the actual data. Specifically, for audio (first audio data), the input parameters are "-f f32le-ar sampling rate-ac number of channels-i audio named pipe name"; for video (first image data), the input parameters are "-r video frame rate-frawvideo-s video width x video height-pix_fmt image format-i video named pipe name". Among them, the video format is consistent with the format of the 2D texture data in step S430, such as

[0152] TextureFormat.BGRA32 corresponds to the video format bgra.

[0153] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method for obtaining rendered video in the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0154] This application also provides a rendering video acquisition device, please refer to Figure 5 , the rendered video acquisition device comprises:

[0155] A container selection module 10, the container selection module is used to receive a rendering video instruction, and based on the rendering video instruction, select any constructed container, wherein the container is provided with an XVFB module and a Unity3D engine;

[0156] A virtualized screen construction module 20, the virtualized screen construction module is used to construct a virtualized screen in the container using an XVFB module;

[0157] A rendering data acquisition module 30, wherein the rendering data acquisition module is used to start the Unity3D engine and execute a preset rendering program based on the virtualized screen to generate real-time rendering data, wherein the preset rendering program is a rendering program associated with a rendering video that is set in advance;

[0158] A module 40 for acquiring real-time rendering image data, wherein the module is used to confirm the type of the rendering pipeline of the Unity3D engine and acquire the real-time rendering image data from the real-time rendering data based on the pipeline type;

[0159] The data synthesis module 50 is used to synthesize the real-time rendering image data and the audio data in the real-time rendering data to obtain a rendered video.

[0160] In one embodiment, the module for obtaining real-time rendering image data includes:

[0161] A first confirmation unit, used to confirm the type of the rendering pipeline of the Unity3D engine;

[0162] a copy unit, for obtaining image texture data from the real-time rendering data based on a Graphics Blit function, and copying the image texture data to the custom rendering texture data using a shader, if the rendering pipeline is a built-in rendering pipeline; or, for obtaining image texture data from the real-time rendering data based on a cmd Blit function, and copying the image texture data to the custom rendering texture data using a shader, if the rendering pipeline is a universal rendering pipeline;

[0163] A first data conversion unit is used to convert the custom rendered texture data after the data is copied into 2D texture data by using a ReadPixels function and an Apply function based on the custom rendered texture data after the data is copied;

[0164] The second data conversion unit is used to convert the 2D texture data into real-time rendering image data by using the GetRawTextureData function.

[0165] In one embodiment, the module for acquiring real-time rendering image data further includes:

[0166] A second confirmation unit is used to determine whether there is an instruction to execute a transparent background operation in the preset rendering program;

[0167] A pixel adjustment unit, if there is an instruction to execute a transparent background operation, obtains image texture data from the real-time rendering data, determines pixels in the image texture data, calculates the difference between the color value of the pixel and the color value of the background pixel in the shader, and if the difference is less than or equal to a preset threshold, adjusts the pixel to a transparent pixel, and uses the shader to copy the adjusted image texture data to the custom rendering texture data;

[0168] The data copying unit uses a shader to copy the image texture data into the custom rendering texture data if there is no instruction to execute the transparent background operation.

[0169] In one embodiment, the data synthesis module includes:

[0170] A first data processing unit, used for adding the real-time rendered image data and the audio data in the real-time rendered data into a video queue and an audio queue respectively;

[0171] A data sending unit, used for sending the image data in the video queue to a video named pipe, and sending the audio data in the audio queue to an audio named pipe;

[0172] The data reading unit is used to read the first image data of the video named pipe and the first audio data of the audio named pipe, perform a synthesis operation on the first image data and the first audio data, and obtain a rendered video.

[0173] In one embodiment, the data synthesis module further includes:

[0174] A duration acquisition unit, used to acquire the length of the audio data in the real-time rendering data, and based on the length, calculate the duration of writing the audio data to obtain a first duration;

[0175] A duration difference calculation unit, used for obtaining a second duration for writing the real-time rendered image data, and calculating a difference between the first duration and the second duration to obtain a video frame difference;

[0176] An audio and video synchronization unit, which performs a frame insertion or frame drop operation based on the video frame difference to eliminate the video frame difference and synchronize the audio data with the real-time rendered image data;

[0177] The second data processing unit adds the synchronized audio data and the real-time rendered image data into a video queue and an audio queue respectively.

[0178] The rendered video acquisition device provided by the present application adopts the rendered video acquisition method in the above embodiment, which can solve the technical problem of rendered video acquisition. Compared with the prior art, the beneficial effects of the rendered video acquisition device provided by the present application are the same as the beneficial effects of the rendered video acquisition method provided by the above embodiment, and the other technical features in the rendered video acquisition device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0179] The present application provides a rendering video acquisition device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the rendering video acquisition method in the above-mentioned embodiment 1.

[0180] Reference below Figure 6, which shows a schematic diagram of the structure of a rendering video acquisition device suitable for implementing the embodiment of the present application. The rendering video acquisition device in the embodiment of the present application may include but is not limited to mobile terminals such as laptop computers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The rendered video acquisition device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0181] like Figure 6 As shown, the rendering video acquisition device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the rendering video acquisition device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the rendering video acquisition device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a rendering video acquisition device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0182] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0183] The rendered video acquisition device provided by the present application adopts the rendered video acquisition method in the above embodiment, which can solve the technical problem of rendered video acquisition. Compared with the prior art, the beneficial effects of the rendered video acquisition device provided by the present application are the same as the beneficial effects of the rendered video acquisition method provided by the above embodiment, and the other technical features in the rendered video acquisition device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0184] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0185] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0186] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the rendering video acquisition method in the above-mentioned embodiment.

[0187] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0188] The computer-readable storage medium may be included in the rendering video acquisition device; or may exist independently without being assembled into the rendering video acquisition device.

[0189] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the rendering video acquisition device, the rendering video acquisition device: selects any constructed container, wherein the container is provided with an XVFB module and a Unity3D engine;

[0190] Use the XVFB module to build a virtualized screen in the container;

[0191] Based on the virtualized screen, start the Unity3D engine and execute a preset rendering program to generate real-time rendering data, and obtain real-time audio data from the real-time rendering data, wherein the preset rendering program is a rendering program associated with the rendered video that is set in advance;

[0192] Confirming the type of the rendering pipeline of the Unity3D engine, and acquiring real-time rendering image data from the real-time rendering data based on the pipeline type;

[0193] A synthesis operation is performed on the real-time rendered image data and the audio data in the real-time rendered data to obtain a rendered video.

[0194] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, C#, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0195] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0196] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0197] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned rendering video acquisition method, and can solve the technical problem of rendering video acquisition. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the rendering video acquisition method provided in the above-mentioned embodiment, and will not be repeated here.

[0198] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned rendering video acquisition method when executed by a processor.

[0199] The computer program product provided by the present application can solve the technical problem of obtaining rendered video. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the method for obtaining rendered video provided by the above embodiment, which will not be described in detail here.

[0200] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for obtaining a rendered video, characterized in that: The rendering video acquisition method comprises: Receive a rendering video instruction, and select any constructed container based on the rendering video instruction, wherein the container is provided with an XVFB module and a Unity3D engine; Use the XVFB module to build a virtualized screen in the container; Based on the virtualized screen, start the Unity3D engine and execute a preset rendering program to generate real-time rendering data, wherein the preset rendering program is a rendering program associated with the rendered video that is set in advance; Confirming the type of the rendering pipeline of the Unity3D engine, and acquiring real-time rendering image data from the real-time rendering data based on the pipeline type; A synthesis operation is performed on the real-time rendered image data and the audio data in the real-time rendered data to obtain a rendered video.

2. The video rendering method according to claim 1, characterized in that: The step of confirming the type of the rendering pipeline of the Unity3D engine and acquiring the real-time rendering image data from the real-time rendering data based on the pipeline type further includes: Determine the type of rendering pipeline of the Unity3D engine; If the rendering pipeline is a built-in rendering pipeline, based on the Graphics Blit function, image texture data is obtained from the real-time rendering data, and the image texture data is copied to the custom rendering texture data using a shader; or, if the rendering pipeline is a universal rendering pipeline, based on the cmd Blit function, image texture data is obtained from the real-time rendering data, and the image texture data is copied to the custom rendering texture data using a shader; Based on the custom rendered texture data after the data is copied, using a ReadPixels function and an Apply function, converting the custom rendered texture data after the data is copied into 2D texture data; The GetRawTextureData function is used to convert the 2D texture data into real-time rendering image data.

3. The video rendering method according to claim 2, characterized in that: The step of acquiring image texture data from the real-time rendering data and copying the image texture data to the custom rendering texture data using a shader further includes: Determine whether there is an instruction to execute a transparent background operation in the preset rendering program; If there is an instruction to execute a transparent background operation, image texture data is obtained from the real-time rendering data, and pixels in the image texture data are determined, and the difference between the color value of the pixel and the color value of the background pixel is calculated in the shader. If the difference is less than or equal to a preset threshold, the pixel is adjusted to a transparent pixel, and the adjusted image texture data is copied to the custom rendering texture data using the shader; If there is no instruction to perform a transparent background operation, the image texture data is copied to the custom rendering texture data using a shader.

4. The video rendering method according to claim 1, characterized in that: The step of performing a synthesis operation on the real-time rendered image data and the audio data in the real-time rendered data to obtain a rendered video further includes: Adding the real-time rendered image data and the audio data in the real-time rendered data to a video queue and an audio queue respectively; Send the image data in the video queue to the video named pipe, and send the audio data in the audio queue to the audio named pipe; The first image data of the video named pipe and the first audio data of the audio named pipe are read, and a synthesis operation is performed on the first image data and the first audio data to obtain a rendered video.

5. The video rendering method according to claim 4, characterized in that: The step of adding the real-time rendered image data and the audio data in the real-time rendered data to a video queue and an audio queue respectively further includes: Obtaining the length of the audio data in the real-time rendering data, and calculating the duration of writing the audio data based on the length to obtain a first duration; Acquire a second duration for writing the real-time rendered image data, and calculate a difference between the first duration and the second duration to obtain a video frame difference; Based on the video frame difference, performing a frame insertion or frame dropping operation to eliminate the video frame difference and synchronize the audio data with the real-time rendered image data; The synchronized audio data and the real-time rendered image data are added to a video queue and an audio queue respectively.

6. The video rendering method according to claim 1, wherein: The obtaining of the rendered video includes obtaining a video file and obtaining a video stream.

7. A rendering video acquisition device, characterized in that: The rendered video acquisition device comprises: A container selection module, the container selection module is used to receive a rendering video instruction, and based on the rendering video instruction, select any constructed container, wherein the container is provided with an XVFB module and a Unity3D engine; A virtualized screen construction module, wherein the virtualized screen construction module is used to construct a virtualized screen in the container using an XVFB module; A rendering data acquisition module, wherein the rendering data acquisition module is used to start the Unity3D engine and execute a preset rendering program based on the virtualized screen to generate real-time rendering data, wherein the preset rendering program is a rendering program associated with a rendering video that is set in advance; A module for acquiring real-time rendering image data, wherein the module is used to confirm the type of the rendering pipeline of the Unity3D engine and acquire the real-time rendering image data from the real-time rendering data based on the pipeline type; A data synthesis module is used to synthesize the real-time rendering image data and the audio data in the real-time rendering data to obtain a rendered video.

8. A rendered video acquisition device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the rendering video acquisition method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the rendering video acquisition method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the rendered video acquisition method according to any one of claims 1 to 6 are implemented.

Images