A video rendering method and device, electronic equipment and storage medium

Abstract

The application discloses a video rendering method and device, electronic equipment and storage medium, and relates to the technical field of video processing. The method comprises the following steps: after obtaining encoded video frame data from the memory, performing hard decoding on the video frame data in the video memory to obtain texture data; performing color encoding format conversion operation on the texture data in the video memory to obtain target texture data; wherein the color encoding format of the target texture data is a basic color encoding format; in the video memory, determining a rendering resource pointer matched with the texture object type to which the target texture data belongs according to the texture object type, and performing video rendering operation on the target texture data according to the rendering resource pointer to obtain a target video, so as to reduce computer resource consumption in the three-dimensional video rendering process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video processing, in particular to a video rendering method and device, electronic equipment and storage medium. BACKGROUND

[0002] Three-dimensional video fusion technology is a branch of virtual reality technology, which can fuse virtual scenes and real-time videos to achieve a virtual-real fusion effect. It can rely on a separate three-dimensional engine to realize the fusion application of small-range or local three-dimensional scenes and videos, or rely on a three-dimensional geographic information system to realize the virtual-real fusion application of global wide-area three-dimensional geographic information visualization.

[0003] Currently, the three-dimensional video fusion process mainly includes video streaming, video decoding and video rendering. Video decoding and video rendering consume a lot of computer resources. If the computer resources are not used reasonably, video playback will be stuck, and the picture will be delayed, which will affect the normal operation of the business.

[0004] For example, in the security video monitoring scene, the picture resolution of the image acquisition device is basically 1080P or above, and 4K and 8K panoramic cameras are also common. In order to reduce the transmission bandwidth and storage space, a higher compression rate H265 encoding format is generally used. Therefore, such high video resolution and compression rate result in the consumption of a large amount of computer resources during video decoding and video rendering, especially when multiple high-definition three-dimensional videos are fused at the same time, which is more obvious. SUMMARY

[0005] The present application provides a video rendering method, device, electronic equipment and storage medium, which reduces the consumption of computer resources in the process of three-dimensional video rendering.

[0006] In a first aspect, a video rendering method is provided, comprising:

[0007] After obtaining the encoded video frame data from the memory, the video frame data is hard-decoded in the video memory to obtain texture data;

[0008] In the video memory, a color encoding format conversion operation is performed on the texture data to obtain target texture data; wherein the color encoding format of the target texture data is a basic color encoding format;

[0009] In the video memory, according to the texture object type to which the target texture data belongs, a rendering resource pointer matched with the texture object type is determined, and a video rendering operation is performed on the target texture data according to the rendering resource pointer to obtain target video frame data.

[0010] In the embodiments of the present application, decoding, color encoding format conversion and rendering of video frame data are all performed in the video memory, without frequent exchange between the memory and the video memory, and according to the texture object type to which the target texture data belongs, a rendering resource pointer matched with the texture object type is determined, so that the rendering operation can be optimized for a specific texture object type, thereby improving the rendering efficiency and quality; compared with the prior art, the parallel computing performance of the graphics processor can be utilized, and the data copying and carrying work of video frame data from the memory to the video memory can be reduced, the computer resource consumption is reduced in the three-dimensional video rendering process, and problems such as video playing lag and delay caused by high computer resource occupation are avoided; further, the decoding process and the rendering process are decoupled, as long as the decoding process obtains target texture data with a correct format, the target texture data can be rendered, and the flexibility of video rendering is increased.

[0011] In some embodiments, the color encoding format conversion operation is performed on the texture data to obtain target texture data, including:

[0012] The input view of the texture data is determined by calling a video processor input view creation function;

[0013] The output view of the target texture data is determined by calling a video processor output view creation function;

[0014] Based on the video processor instance, the input view, the output view, and a video processor bit block transfer function, the color encoding format conversion operation is performed on the texture data to obtain the target texture data.

[0015] In the above manner, the color space conversion process can be directly completed in the video memory, without occupying memory resources, and the three-dimensional graphics image parallel processing performance of the image processor is fully utilized.

[0016] In some embodiments, the rendering resource pointer matched with the texture object type is determined according to the texture object type to which the target texture data belongs, including:

[0017] When it is detected that the target texture data is updated, a texture rendering command is added for the target texture data;

[0018] According to the texture object type to which the target texture data belongs in the texture rendering command, a rendering resource pointer matched with the texture object type is obtained.

[0019] In the above manner, the rendering operation can be optimized for a specific texture object type, thereby improving the rendering efficiency and quality, so that subsequent target video frame data with high quality can be efficiently generated.

[0020] In some embodiments, the obtaining, according to the texture object type to which the target texture data belongs in the texture rendering command, a rendering resource pointer matched with the texture object type, comprises:

[0021] obtaining a resource pointer from the texture object pointer and converting the resource pointer into a target texture object pointer corresponding to the texture object type;

[0022] obtaining a rendering hardware interface pointer of an underlying layer from the target texture object pointer and obtaining a rendering resource pointer matched with the texture object type from the rendering hardware interface pointer.

[0023] In some embodiments, the performing, according to the rendering resource pointer, a video rendering operation on the target texture data to obtain target video frame data, comprises:

[0024] calling a GetDevice function to obtain a graphics processing application program interface corresponding to the rendering resource pointer;

[0025] calling a GetThreadContext function of the graphics processing application program interface to obtain a graphics processing application program context interface;

[0026] calling a ResourceCopy function of the graphics processing application program context interface to perform rendering on the target texture data to obtain the target video frame data.

[0027] In a second aspect, a video rendering apparatus is provided, comprising:

[0028] a decoding module configured to obtain encoded video frame data from a memory and perform hardware decoding on the video frame data in a display memory to obtain texture data;

[0029] a color encoding format conversion module configured to perform a color encoding format conversion operation on the texture data in the display memory to obtain target texture data, wherein a color encoding format of the target texture data is a basic color encoding format;

[0030] a rendering module configured to determine, according to a texture object type to which the target texture data belongs, a rendering resource pointer matched with the texture object type in the display memory and perform a video rendering operation on the target texture data according to the rendering resource pointer to obtain target video frame data.

[0031] In some embodiments, the color encoding format conversion module is specifically configured to:

[0032] determine an input view of the texture data by calling a CreateVideoProcessorInputView function;

[0033] determining an output view of the target texture data by calling a create video processor output view function;

[0034] performing a color encoding format conversion operation on the texture data based on the video processor instance, the input view, the output view, and calling a video processor bit block transfer function, to obtain the target texture data.

[0035] In some embodiments, the rendering module is specifically configured to:

[0036] adding a texture rendering command for the target texture data when it is detected that a flag field of the target texture data represents an update;

[0037] obtaining a rendering resource pointer matched with a texture object type to which the target texture data belongs in the texture rendering command.

[0038] In a third aspect, an electronic device is provided, comprising:

[0039] a memory for storing a computer program; and a processor for executing the computer program stored in the memory to implement the method of any one of the first aspect.

[0040] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the method of any one of the first aspect.

[0041] The technical effects of each of the above-mentioned second to fourth aspects and each of the possible solutions can be achieved are described above in the description of the first aspect or the possible solutions of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 An application scenario suitable for the embodiments of the present application is shown in the figure;

[0043] Figure 2 A flowchart of a video rendering method provided by the embodiments of the present application is shown in the figure;

[0044] Figure 3 A logic diagram of a video rendering provided by the embodiments of the present application is shown in the figure;

[0045] Figure 4 A structure diagram of a video rendering device provided by the embodiments of the present application is shown in the figure;

[0046] Figure 5 A structure diagram of an electronic device provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0047] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application. The embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily without conflict. Moreover, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that here.

[0048] The terms "first" and "second" in the description and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.

[0049] The exemplary embodiments of the present application are described below with reference to the accompanying drawings, which include various details of the embodiments of the present application to help understanding. They should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Also, for the sake of clarity and conciseness, the description below omits the description of well-known functions and structures. It should be noted that in the embodiments of the present application, some software, components, models and other industry solutions can be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solutions of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0050] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application will be described first.

[0051] (1) Unreal Engine (UE) is a game engine developed by Epic Games, which has become a popular virtual reality application engine at present due to its high flexibility and scalability, powerful rendering capability, and efficient resource management and optimization, and has been widely used in various industries. For example, in the field of security video monitoring, it is a typical application to fuse the video pictures collected by the monitoring camera into a three-dimensional scene.

[0052] (2) Memory is mainly used in computer systems, and is a storage area directly accessed by the central processing unit (CPU), used to store running programs and data.

[0053] (3) Video memory is a storage device dedicated to the graphics processing unit, and its main function is to store graphics data such as textures, vertex coordinates, and lighting information.

[0054] (4) High-performance three-dimensional graphics rendering framework (Direct3D, D3D) is the main underlying rendering interface of UE on platforms such as Microsoft, and through the D3D rendering interface, UE can use the computing power of the image processor to perform high-performance three-dimensional graphics rendering.

[0055] Currently, in the process of three-dimensional video rendering, one way is to obtain video frame data in a format such as H265 from an image acquisition device, then decode in the video memory using a hardware decoding method to obtain texture data in the format of NV12, then move the texture data to the memory, convert the texture data in the format of NV12 to texture data in the format of RGBA in the memory, further move the texture data in the format of RGBA to the video memory, and finally render the texture data in the video memory using the Unreal Engine. Another way is to obtain video frame data in a format such as H265 from an image acquisition device, then decode in the memory using a software decoding method to obtain texture data in the format of NV12, and convert the texture data in the format of NV12 to texture data in the format of RGBA in the memory, and finally move the texture data in the format of RGBA to the video memory, and render the texture data using the Unreal Engine. However, both of these two ways occupy too much CPU and memory resources in the conversion of NV12 to RGBA format and the mutual moving of data between the memory and the video memory.

[0056] In view of this, the embodiments of the present application provide a video rendering method and device, electronic equipment and storage medium, which involve format conversion, video decoding and rendering in the video memory, reducing the data copying and moving work from the memory to the video memory, and significantly reducing computer resource consumption.

[0057] First, the application scenarios to which the technical solutions of the embodiments of the present application can be applied will be briefly introduced. It should be noted that the following application scenarios are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0058] Figure 1An application scenario applicable to the embodiments of the present application is shown in the figure. As shown in the figure, the application scenario mainly includes: an image acquisition device 10, a server 11. Among them, the image acquisition device 10 and the server 11 can interact information through a communication network, and the communication network can adopt a communication mode including: a wireless communication mode and a wired communication mode. The number of the above-mentioned devices is not limited by the embodiments of the present application, as shown in Figure 1 , only the image acquisition device 10 and the server 11 are taken as examples for description, and the above-mentioned devices and their respective functions are briefly introduced as follows.

[0059] In some scenarios, the image acquisition device 10 is a device for acquiring image information, including but not limited to a surveillance camera, a laser radar, a scanner, etc.

[0060] In some scenarios, the server 11 can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0061] Further, in the embodiments of the present application, the above-mentioned server 11 can be equipped with a video rendering device, which is used for rendering processing various video frame data collected by the image acquisition device 10, and dynamically displaying the rendered video on the user interface for the user to browse, review, etc.

[0062] To further illustrate the technical solutions provided by the embodiments of the present application, the following will be described in detail in conjunction with the drawings and specific embodiments. Although the embodiments of the present application provide the following method operation steps as shown in the embodiments or the drawings, more or fewer operation steps can be included in the method based on conventional or non-creative labor. The execution order of these steps is not limited to the execution order provided by the embodiments of the present application in the logical sense that there is no necessary causal relationship between the steps. The method can be executed in the order shown in the embodiments or the drawings or in parallel during actual processing or device execution.

[0063] Figure 2 A flowchart of a video rendering method provided by the embodiments of the present application, which can be executed by a video rendering device to reduce computer resource consumption in a three-dimensional video rendering process. As shown in Figure 2 , the flowchart includes the following steps:

[0064] 201: After obtaining the encoded video frame data from memory, the video frame data is hardware decoded in the video memory to obtain texture data.

[0065] In some embodiments, before retrieving encoded video frame data from memory, the image acquisition device (such as...) can be accessed in real time. Figure 1 The image acquisition device 10 shown sends a streaming request to obtain the encoded video frame data, and then stores the encoded video frame data in memory to prevent video frame data loss. It should be noted that the format in which the image acquisition device encodes the video frame data can be in accordance with the H.264 format standard, the H.265 format standard, or other format standards. This embodiment of the application does not impose any restrictions on this.

[0066] After this step, the encoded video frame data is retrieved from memory and then decoded in video memory using hardware decoding to obtain the decoded texture data (also known as D3D texture data).

[0067] 202: In video memory, a color encoding format conversion operation is performed on the texture data to obtain the target texture data. The color encoding format of the target texture data is the basic color encoding format.

[0068] In some embodiments, since the image acquisition device generally uses the NV12 color encoding format, the decoded texture data is also in the NV12 color encoding format. In order for the rendering engine to perform the corresponding rendering on the texture data, a color encoding format conversion operation can be performed on the texture data to obtain the target texture data whose color encoding format is the base color encoding format (such as RGBA color encoding format).

[0069] In some embodiments, performing a color encoding format conversion operation on the texture data to obtain the target texture data may include the following process:

[0070] The input view for the texture data is determined by calling the CreateVideoProcessorInputView function; this input view may include parameters such as the pixel width, pixel height, and pixel data storage format of the texture data.

[0071] The output view of the target texture data is determined by calling the CreateVideoProcessorOutputView function; this output view may include parameters such as the pixel data width, pixel data height, and pixel data storage format of the target texture data;

[0072] Based on the video processor instance, the input view, the output view, and calling a video processor bit block transfer function (VideoProcessorBIt function), a color encoding format conversion operation is performed on the texture data to obtain target texture data.

[0073] In some embodiments, the video processor instance can be obtained by, for example, calling a CreateVideoProcessorEnumerator function of a D3D video device interface to obtain a video processing enumerator, and based on the video processing enumerator, calling a CreateVideoProcessor function of the D3D video device interface to create the corresponding video processor instance.

[0074] In some embodiments, the video processing enumerator can list information such as all input and output video formats supported by the device, the maximum and minimum resolutions and frame rates supported by the device, and video processing options supported by the device (such as color space conversion, scaling, rotation), etc.; the video processor instance is used to allow an application to configure and apply specific video processing operations, for example, the application can use the video processor to adjust the brightness, contrast, saturation, etc. of the video, or scale the video to adapt to different display resolutions, set the output video size ratio, the output video frame number, the video code rate, and some hardware configurations, etc.

[0075] In other embodiments, the converted width and height can also be set to the width and height of the texture data, and the converted color encoding format can be set to the RGBA color encoding format; then, a GetDevice function is called to obtain a D3D device interface, a GetImmediateContext function of the D3D device interface is called to obtain a D3D context interface, a QueryInterface function of the D3D device interface is called to obtain a D3D video device interface, and a QueryInterface function of the D3D context interface is called to obtain a D3D video context interface, so that in subsequent color encoding format conversion operations on the texture data, the color encoding format conversion process can be directly performed using these interfaces, improving the conversion efficiency.

[0076] 203: In the video memory, according to the texture object type to which the target texture data belongs, a rendering resource pointer matched with the texture object type is determined, and a video rendering operation is performed on the target texture data according to the rendering resource pointer to obtain target video frame data.

[0077] In some embodiments, determining the rendering resource pointer matched with the texture object type to which the target texture data belongs can include the following process:

[0078] First, when detecting that the flag field of the target texture data represents an update, add a texture rendering command for the target texture data.

[0079] In some embodiments, when detecting that the flag field of the target texture data represents an update, adding a texture rendering command for the target texture data can be: after converting the target texture data, setting the flag field of the target texture data to an update parameter (which can be represented in English (e.g., true) or a number (e.g., 1)) in the texture data callback thread; when detecting in the Unreal Engine game thread that the flag field is true or 1, indicating that the texture data has been updated, the texture rendering command can be pushed into the Unreal Engine rendering thread through the ENQUEUE_RENDER_COMMAND macro. The texture rendering command can include the command name and the texture object type, the new pixel data and the texture parameters, the rendering execution logic, etc.; when detecting in the Unreal Engine game thread that the flag field is false or 0, indicating that the texture data has not been updated, then wait for the detection of the flag field of the target texture data for the next frame.

[0080] Second, according to the texture object type to which the target texture data in the texture rendering command belongs, obtain a rendering resource pointer matching the texture object type.

[0081] In some embodiments, according to the texture object type to which the target texture data in the texture rendering command belongs, obtaining a rendering resource pointer matching the texture object type can be: obtaining a resource pointer from a texture object pointer (also known as an Unreal Engine texture object pointer, used to manage and operate texture resources in Unreal Engine), and converting the resource pointer to a target texture object pointer corresponding to the texture object type, obtaining a underlying rendering hardware interface pointer from the target texture object pointer, and obtaining a rendering resource pointer matching the texture object type from the rendering hardware interface pointer, to ensure that the rendering operation can be optimized for a specific texture object type, thereby improving rendering efficiency and quality. The rendering resource pointer usually points to a memory location storing resources required for rendering, which can include shader programs, texture data, vertex data, etc.

[0082] For example, a Resource pointer is obtained from the texture object pointer and converted into a target texture object pointer of the FTexture2DResource type (FTexture2DResource* Texture2DResource = (FTexture2DResource*) Texture2D->Resource, where Texture2D is a texture pointer of the UTexture2D type); a GetTexture2DRHI() function is called to obtain a Rendering Hardware Interface (RHI) texture object pointer (FTexture2DRHIRef Texture2DRHI = Texture2DResource->GetTexture2DRHI()) from the Texture2DResource; a GetNativeResource function is called to obtain a rendering resource pointer of the ID3D11Texture2D type (ID3D11Texture2D* D3D11Texture2D = Texture2DRHI->GetNativeResource()) from the Texture2DRHI.

[0083] In some embodiments, according to the rendering resource pointer, a video rendering operation is performed on the target texture data to obtain target video frame data, which can be: a GetDevice function is called to obtain a graphics processing application program interface (for example, D3D11Texture2D->GetDevice(&D3D11Device)) corresponding to the rendering resource pointer; a thread context function of the graphics processing application program interface is called to obtain a graphics processing application context interface (for example, D3D11Device->GetImmediateContext(&D3D11DeviceContext)); a resource copy function of the graphics processing application context interface is called (for example, D3D11DeviceContext->CopyResource(D3D11Texture2D, SrcD3D11Texture2D); where SrcD3D11Texture2D is a source texture object pointer called back), and the target texture data is rendered to obtain the target video frame data.

[0084] In some embodiments, after the target video frame data is rendered, the flag field thereof can be reset to false, and the texture is refreshed after new target texture data is called back, so as to complete the rendering of the entire continuous video picture texture data.

[0085] In the embodiment of the present application, decoding, color encoding format conversion and rendering of video frame data are all performed in the video memory, there is no frequent exchange between the memory and the video memory, and according to the texture object type to which the target texture data belongs, a rendering resource pointer matched with the texture object type is determined, so that the rendering operation can be optimized for a specific texture object type, thereby improving the rendering efficiency and quality; compared with the prior art, the parallel computing performance of the graphics processor can be utilized, and the data copying and carrying work of video frame data from the memory to the video memory can be reduced, in the three-dimensional video rendering process, the computer resource consumption is reduced, and then the problems of video playing lag and delay caused by high computer resource occupation are also avoided; further, the decoding process and the rendering process are decoupled, as long as the decoding process obtains the target texture data in a correct format, the target texture data can be rendered, and the flexibility of video rendering is increased.

[0086] Based on the above Figure 2 The method flow shown in the figure, Figure 3 A logical diagram of video rendering is provided in the embodiment of the present application. As Figure 3 shown, the video frame data encoded by the image acquisition device (such as Figure 1 the image acquisition device 10 shown in the figure) is stored in the memory, when video rendering is required, the video frame data is transmitted to the video memory, the video frame data is hard decoded in the video memory to obtain texture data in the NV12 color encoding format; then, the texture data in the NV12 color encoding format is converted into target texture data in the RGBA color encoding format by using a video hardware acceleration method in the video memory; finally, the target texture data is copied to the Unreal Engine rendering thread, and video rendering operation is performed on the target texture data according to the texture object type to which the target texture data belongs.

[0087] Based on the same technical concept, the embodiment of the present application also provides a video rendering device, which can realize the video rendering method flow described above in the embodiment of the present application.

[0088] Figure 4 A structural diagram of a video rendering device is provided in the embodiment of the present application. As Figure 4 shown, the device includes a decoding module 401, a color encoding format conversion module 402 and a rendering module 403.

[0089] The decoding module 401 is configured to obtain encoded video frame data from the memory, and perform hard decoding on the video frame data in the video memory to obtain texture data.

[0090] The color coding format conversion module 402 is configured to perform a color coding format conversion operation on the texture data in the video memory to obtain target texture data, wherein the color coding format of the target texture data is a basic color coding format.

[0091] In some embodiments, the rendering module 403 is configured to determine a rendering resource pointer matched with a texture object type to which the target texture data belongs according to the texture object type, and perform a video rendering operation on the target texture data according to the rendering resource pointer to obtain target video frame data.

[0092] The color coding format conversion module 402 is specifically configured to:

[0093] The input view of the texture data is determined by calling a create video processor input view function.

[0094] The output view of the target texture data is determined by calling a create video processor output view function.

[0095] The color coding format conversion operation is performed on the texture data based on the video processor instance, the input view, the output view, and a video processor bit block transmission function to obtain the target texture data.

[0096] In some embodiments, the rendering module 403 is specifically configured to:

[0097] When it is detected that the flag field of the target texture data represents an update, a texture rendering command is added for the target texture data.

[0098] According to the texture object type to which the target texture data belongs in the texture rendering command, a rendering resource pointer matched with the texture object type is obtained.

[0099] In some embodiments, the rendering module 403 is specifically configured to:

[0100] A resource pointer is obtained from a texture object pointer, and the resource pointer is converted into a target texture object pointer corresponding to the texture object type.

[0101] A rendering hardware interface pointer of an underlying layer is obtained from the target texture object pointer, and a rendering resource pointer matched with the texture object type is obtained from the rendering hardware interface pointer.

[0102] In some embodiments, the rendering module 403 is specifically configured to:

[0103] A graphics processing application program interface corresponding to the rendering resource pointer is obtained by calling a GetDevice function.

[0104] calling a get thread context function of the graphics processing application program interface, to obtain a graphics processing application program context interface;

[0105] calling a resource copy function of the graphics processing application program context interface, to render the target texture data, to obtain the target video frame data.

[0106] It should be noted that the above device provided by the embodiments of the present application can implement all the method steps in the method embodiments and achieve the same technical effects. Therefore, the same parts and beneficial effects of the method embodiments are not described in detail herein.

[0107] Based on the same technical concept, the embodiments of the present application also provide an electronic device, which can realize the functions of the above-mentioned video rendering device.

[0108] Figure 5 A structural schematic diagram of an electronic device provided by the embodiments of the present application.

[0109] The at least one processor 501 and the memory 502 connected with the at least one processor 501. In the embodiments of the present application, the specific connection medium between the processor 501 and the memory 502 is not limited, Figure 5 In the embodiments of the present application, the connection between the processor 501 and the memory 502 is taken as an example of connection through the bus 500. The bus 500 is represented by a thick line, Figure 5 In the embodiments of the present application, the connection between the processor 501 and the memory 502 is taken as an example of connection through the bus 500. The bus 500 is represented by a thick line, Figure 5 In the embodiments of the present application, the connection between the processor 501 and the memory 502 is taken as an example of connection through the bus 500. The bus 500 is represented by a thick line,

[0110] In the embodiments of the present application, the memory 502 stores instructions executable by the at least one processor 501. The at least one processor 501 can execute the video rendering method discussed above by executing the instructions stored in the memory 502. The processor 501 can realize the functions of each module in the device shown in the embodiments of the present application. Figure 4

[0111] The processor 501 is the control center of the device, and can connect each part of the entire control device through various interfaces and lines. By running or executing the instructions stored in the memory 502 and calling the data stored in the memory 502, the device can process various functions and data, thereby monitoring the entire device.

[0112] ​In the embodiments of the present application, the processor 501 can include one or more processing units, and the processor 501 can integrate an application processor and a modem processor, wherein the application processor mainly processes operating systems, user interfaces, and application programs, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 can be implemented on the same chip, and in some embodiments, they can also be implemented on separate chips respectively.

[0113] The processor 501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the video rendering method disclosed in the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0114] The memory 502 is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. The memory 502 can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, and the like. The memory 502 is any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto. The memory 502 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used for storing program instructions and / or data.

[0115] By designing and programming the processor 501, the code corresponding to the video rendering method introduced in the above embodiments can be fixed into the chip, so that the chip can execute the code when running Figure 2A video rendering method of the illustrated embodiment. How to design and program the processor 501 is a technology known to those skilled in the art, which will not be repeated here.

[0116] It should be noted that the above electronic device provided by the embodiments of the present application can realize all the method steps realized by the method embodiments and achieve the same technical effects. Therefore, the same parts and beneficial effects of the method embodiments will not be repeated here.

[0117] Based on the same technical concept, the embodiments of the present application provide a computer storage medium, which includes computer program code. When the computer program code runs on a computer, it makes the computer execute any one of the video rendering methods discussed in the foregoing. Since the principle of solving problems of the above computer storage medium is similar to that of the video rendering method, the implementation of the above computer storage medium can be referred to the implementation of the method, and the repeated parts will not be repeated.

[0118] In the specific implementation process, the computer storage medium can include a universal serial bus flash drive (USB, Universal Serial Bus Flash Drive), a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various storage media that can store program codes.

[0119] Based on the same technical concept, the embodiments of the present application also provide a computer program product, which includes computer program code. When the computer program code runs on a computer, it makes the computer execute any one of the video rendering methods discussed in the foregoing. Since the principle of solving problems of the above computer program product is similar to that of the video rendering method, the implementation of the above computer program product can be referred to the implementation of the method, and the repeated parts will not be repeated.

[0120] The computer program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0121] The methods in the present application can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the methods can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded and executed by a computer, the processes or functions described in the present application are performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, a core network device, an OAM, or other programmable devices.

[0122] The computer programs or instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer programs or instructions can be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center through a wired or wireless manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium, for example, a floppy disk, a hard disk, a magnetic tape; an optical medium, for example, a digital video disc; and a semiconductor medium, for example, a solid-state disk. The computer-readable storage medium can be a volatile or non-volatile storage medium, or can include both volatile and non-volatile storage media.

[0123] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage, etc.) containing computer-usable program code.

[0124] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks

[0125] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0127] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A video rendering method, characterized in that, include: After retrieving the encoded video frame data from memory, the video frame data is hardware-decoded in the video memory to obtain texture data; In the video memory, a color encoding format conversion operation is performed on the texture data to obtain target texture data; wherein, the color encoding format of the target texture data is the basic color encoding format; In the video memory, the Unreal Engine determines a rendering resource pointer that matches the texture object type to which the target texture data belongs, and performs video rendering operations on the target texture data according to the rendering resource pointer to obtain target video frame data; wherein, the rendering resource pointer is used to indicate the rendering of texture data of a specific texture object type; The step of using Unreal Engine to determine the rendering resource pointer that matches the texture object type to which the target texture data belongs includes: When a flag field is detected as true or 1 in the Unreal Engine game thread, a texture rendering command is pushed into the Unreal Engine rendering thread via the ENQUEUE_RENDER_COMMAND macro. A Resource pointer is obtained from the texture object pointer, and the Resource pointer is converted into a target texture object pointer of type FTexture2DResource. The GetTexture2DRHI() function is called to obtain the underlying rendering hardware interface RHI texture object pointer from Texture2DResource. The GetNativeResource function is called to obtain the underlying rendering resource pointer of type ID3D11Texture2D from Texture2DRHI.

2. The method as described in claim 1, characterized in that, The step of performing a color encoding format conversion operation on the texture data to obtain the target texture data includes: The input view for the texture data is determined by calling the function that creates the video processor input view; The output view of the target texture data is determined by calling the function to create the video processor output view; Based on the video processor instance, the input view, and the output view, and by calling the video processor bit block transfer function, a color encoding format conversion operation is performed on the texture data to obtain the target texture data.

3. The method as described in claim 1, characterized in that, The step of performing video rendering operations on the target texture data according to the rendering resource pointer to obtain target video frame data includes: Call the GetDevice function to obtain the graphics processing application interface corresponding to the rendering resource pointer; Call the thread context retrieval function of the graphics processing application interface to obtain the graphics processing application context interface; The resource copy function of the graphics processing application context interface is called to render the target texture data and obtain the target video frame data.

4. A video rendering apparatus, characterized in that, include: The decoding module is used to obtain the encoded video frame data from memory and then perform hardware decoding on the video frame data in the video memory to obtain texture data. A color encoding format conversion module is used to perform a color encoding format conversion operation on the texture data in the video memory to obtain target texture data; wherein the color encoding format of the target texture data is a basic color encoding format; The rendering module is used in the video memory to use Unreal Engine to determine a rendering resource pointer that matches the texture object type to which the target texture data belongs, and to perform video rendering operations on the target texture data according to the rendering resource pointer to obtain target video frame data; wherein, the rendering resource pointer is used to indicate the rendering of texture data of a specific texture object type; Specifically, the rendering module is used for: When a flag field is detected as true or 1 in the Unreal Engine game thread, a texture rendering command is pushed into the Unreal Engine rendering thread via the ENQUEUE_RENDER_COMMAND macro. A Resource pointer is obtained from the texture object pointer, and the Resource pointer is converted into a target texture object pointer of type FTexture2DResource. The GetTexture2DRHI() function is called to obtain the underlying rendering hardware interface RHI texture object pointer from Texture2DResource. The GetNativeResource function is called to obtain the underlying rendering resource pointer of type ID3D11Texture2D from Texture2DRHI.

5. The apparatus as described in claim 4, characterized in that, The color encoding format conversion module is specifically used for: The input view for the texture data is determined by calling the function that creates the video processor input view; The output view of the target texture data is determined by calling the function to create the video processor output view; Based on the video processor instance, the input view, and the output view, and by calling the video processor bit block transfer function, a color encoding format conversion operation is performed on the texture data to obtain the target texture data.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a computer program stored in the memory, implements the method of any one of claims 1-3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-3.

Images