Video rendering method and device, electronic equipment and storage medium

Abstract

The invention discloses a video rendering method and device, electronic equipment and a storage medium, and relates to the technical field of video processing. The method comprises the following steps: acquiring coded video frame data from a memory, and performing hard decoding on the video frame data in a video memory to obtain texture data; in the video memory, executing a color coding format conversion operation on the texture data to obtain target texture data; wherein the color coding format of the target texture data is a basic color coding format; and in the video memory, according to the texture object type to which the target texture data belongs, determining a rendering resource pointer matched with the texture object type, and according to the rendering resource pointer, executing a video rendering operation on the target texture data to obtain a target video, so that the target video can be rendered in a three-dimensional video rendering process. And computer resource consumption is reduced.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video rendering method, device, electronic device and storage medium. Background Art

[0002] 3D video fusion technology is a branch of virtual reality technology. It can fuse virtual scenes and real-time videos to achieve the effect of virtual-real integration. It can rely on a separate 3D engine to realize the fusion application of small-scale or local 3D scenes and video and other resources. It can also rely on a 3D geographic information system to realize the virtual-real integration application of 3D geographic information visualization in a global wide area.

[0003] At present, the 3D video fusion process mainly includes video streaming, video decoding and video rendering; among them, video decoding and video rendering consume more computer resources. If the computer resources are not used properly, problems such as video playback freeze and picture delay will occur, thus affecting the normal operation of the business.

[0004] For example, in security video surveillance scenarios, since the picture resolution of image acquisition devices is basically above 1080P, 4K and 8K panoramic cameras are also common. At the same time, in order to reduce transmission bandwidth and storage space, the H265 encoding format with a higher compression rate is generally adopted; therefore, such a high video resolution and compression rate requires a large amount of computer resources during video decoding and video rendering, especially when multiple high-definition 3D videos are integrated at the same time, the performance is more obvious. Summary of the invention

[0005] The present application provides a video rendering method, device, electronic device and storage medium, which are used to reduce computer resource consumption during three-dimensional video rendering.

[0006] In a first aspect, a video rendering method is provided, comprising: After obtaining the encoded video frame data from the memory, the video frame data is hard-decoded in the video memory to obtain texture data; In the video memory, a color coding format conversion operation is performed on the texture data to obtain target texture data; wherein the color coding format of the target texture data is a basic color coding format; In the video memory, according to the texture object type to which the target texture data belongs, a rendering resource pointer matching the texture object type is determined, and a video rendering operation is performed on the target texture data according to the rendering resource pointer to obtain target video frame data.

[0007] In the embodiment of the present application, decoding, color coding format conversion and rendering of video frame data are all performed in the video memory, and there is no frequent exchange between the internal memory and the video memory. In addition, according to the texture object type to which the target texture data belongs, a rendering resource pointer matching the texture object type is determined, thereby ensuring that the rendering operation can be optimized for a specific texture object type, thereby improving rendering efficiency and quality. Compared with the prior art, it can not only give full play to the parallel computing performance of the graphics processor, but also reduce the data copying and transportation work of the video frame data from the internal memory to the video memory. In the process of three-dimensional video rendering, the consumption of computer resources is reduced, thereby avoiding problems such as video playback freezes and high delays due to excessive computer resource usage. Furthermore, the decoding process and the rendering process are decoupled. As long as the decoding process obtains the target texture data in the format that meets the requirements, it can be rendered, thereby increasing the flexibility of video rendering.

[0008] In some embodiments, performing a color coding format conversion operation on the texture data to obtain target texture data includes: Determine the input view of the texture data by calling the create video processor input view function; Determine the output view of the target texture data by calling the create video processor output view function; Based on the video processor instance, the input view, the output view, and by calling the video processor bit block transfer function, a color coding format conversion operation is performed on the texture data to obtain the target texture data.

[0009] Through the above method, the color space conversion process can be completed directly in the video memory without occupying memory resources, and the three-dimensional graphics image parallel processing performance of the image processor can be fully utilized.

[0010] In some embodiments, determining, according to the texture object type to which the target texture data belongs, a rendering resource pointer matching the texture object type includes: When it is detected that the mark field representation of the target texture data is updated, adding a texture rendering command for the target texture data; According to the texture object type to which the target texture data in the texture rendering command belongs, a rendering resource pointer matching the texture object type is obtained.

[0011] Through the above method, it can be ensured that the rendering operation can be optimized for a specific texture object type, thereby improving the rendering efficiency and quality, so that high-quality target video frame data can be efficiently generated subsequently.

[0012] In some embodiments, obtaining a rendering resource pointer matching the texture object type according to the texture object type to which the target texture data in the texture rendering command belongs includes: Obtaining a resource pointer from a texture object pointer, and converting the resource pointer into a target texture object pointer corresponding to the texture object type; The underlying rendering hardware interface pointer is obtained from the target texture object pointer, and the rendering resource pointer matching the texture object type is obtained from the rendering hardware interface pointer.

[0013] In some embodiments, performing a video rendering operation on the target texture data according to the rendering resource pointer to obtain target video frame data includes: Calling GetDevice function to obtain the graphics processing application program interface corresponding to the rendering resource pointer; Calling the thread context function of the graphics processing application program interface to obtain the graphics processing application program context interface; The resource copy function of the graphics processing application context interface is called to render the target texture data to obtain the target video frame data.

[0014] In a second aspect, a video rendering device is provided, comprising: A decoding module, for obtaining the encoded video frame data from the memory, and then hard decoding the video frame data in the video memory to obtain texture data; A color coding format conversion module is used to perform a color coding format conversion operation on the texture data in the video memory to obtain target texture data; wherein the color coding format of the target texture data is a basic color coding format; A rendering module is used to determine a rendering resource pointer matching the texture object type in the video memory according to the texture object type to which the target texture data belongs, and perform a video rendering operation on the target texture data according to the rendering resource pointer to obtain target video frame data.

[0015] In some embodiments, the color coding format conversion module is specifically used to: Determine the input view of the texture data by calling the create video processor input view function; Determine the output view of the target texture data by calling the create video processor output view function; Based on the video processor instance, the input view, the output view, and by calling the video processor bit block transfer function, a color coding format conversion operation is performed on the texture data to obtain the target texture data.

[0016] In some embodiments, the rendering module is specifically used to: When it is detected that the mark field representation of the target texture data is updated, adding a texture rendering command for the target texture data; According to the texture object type to which the target texture data in the texture rendering command belongs, a rendering resource pointer matching the texture object type is obtained.

[0017] According to a third aspect, an electronic device is provided, including: A memory for storing a computer program; a processor for implementing any one of the methods described in the first aspect when executing the computer program stored in the memory.

[0018] According to a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of the first aspects is implemented.

[0019] For each aspect from the second to the fourth aspect and the technical effects that may be achieved by each aspect, please refer to the above description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of an application scenario applicable to the embodiments of the present application; Figure 2 A flowchart of a video rendering method provided in an embodiment of the present application; Figure 3 A logical schematic diagram of video rendering provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a video rendering device provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0022] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.

[0023] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0024] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are first explained below.

[0025] (1) Unreal Engine (UE) is a game engine developed by developers. With its high flexibility and scalability, powerful rendering capabilities, and efficient resource management and optimization, it has become a popular virtual reality application engine and has been widely used in various industries. For example, in the field of security video surveillance, integrating video images captured by surveillance cameras into three-dimensional scenes is a typical application.

[0026] (2) Memory is mainly used in computer systems. It is a storage area directly accessed by the Central Processing Unit (CPU) and is used to store running programs and data.

[0027] (3) Video memory is a storage device dedicated to the graphics processing unit. Its main function is to store graphics data, such as textures, vertex coordinates, and lighting information.

[0028] (4) High-performance 3D graphics rendering framework (Direct3D, D3D) is the main underlying rendering interface of UE on Microsoft and other platforms. Through the D3D rendering interface, UE can use the computing power of the image processor to perform high-performance 3D graphics rendering.

[0029] At present, in the process of 3D video rendering, one method is to obtain video frame data in the format of H265 or other encoding formats from an image acquisition device, and then use hard decoding to decode in the video memory to obtain texture data in the format of NV12, and then move the texture data to the memory, convert the texture data in the format of NV12 into texture data in the format of RGBA in the memory, further move the texture data in the format of RGBA to the video memory, and finally use the Unreal Engine to render the texture data in the video memory; the other method is to obtain video frame data in the format of H265 or other encoding formats from an image acquisition device, and then use soft decoding in the memory to obtain texture data in the format of NV12, and convert the texture data in the format of NV12 into texture data in the format of RGBA in the memory, and finally move the texture data in the format of RGBA to the video memory, and use the Unreal Engine to render the texture data. However, these two methods occupy too high CPU and memory resources in the conversion of NV12 to RGBA format and the mutual movement between data in the memory and data in the video memory.

[0030] In view of this, the embodiments of the present application provide a video rendering method, device, electronic device and storage medium, in which the format conversion, video decoding and rendering involved are all performed in the video memory, reducing the data copying and transportation work of video frame data from the internal memory to the video memory, and significantly reducing computer resource consumption.

[0031] The following first briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0032] Figure 1Schematic diagram of application scenarios applicable to the embodiment of the present application. As shown in the figure, the application scenario mainly includes: image acquisition device 10, server 11. Among them, the image acquisition device 10 and the server 11 can exchange information through a communication network, and the communication mode adopted by the communication network may include: wireless communication mode and wired communication mode. The embodiment of the present application does not impose any restrictions on the number of the above-mentioned devices, such as Figure 1 As shown, only the image acquisition device 10 and the server 11 are described as examples, and the above-mentioned devices and their respective functions are briefly introduced below.

[0033] In some scenarios, the image acquisition device 10 is a device for acquiring image information, including but not limited to a surveillance camera, a laser radar, a scanner, etc.

[0034] In some scenarios, server 11 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0035] Furthermore, in an embodiment of the present application, the server 11 may be equipped with a video rendering device, which is used to render and process various video frame data collected by the image acquisition device 10, and dynamically display the rendered video on the user interface for users to browse and view.

[0036] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application. The method may be executed or executed in parallel in the actual processing process or when the device is executed in accordance with the method sequence shown in the embodiment or drawings.

[0037] Figure 2 A flowchart of a video rendering method provided in an embodiment of the present application, which can be executed by a video rendering device to reduce computer resource consumption during 3D video rendering. Figure 2 As shown, the process includes the following steps: 201: After acquiring the encoded video frame data from the memory, hard decoding is performed on the video frame data in the video memory to obtain texture data.

[0038] In some embodiments, before obtaining the encoded video frame data from the memory, the image acquisition device (such as Figure 1 The image acquisition device 10 shown in the figure sends a stream pull request to obtain the video frame data encoded by the image acquisition device, and then stores the encoded video frame data in the memory to prevent the loss of the video frame data. It should be noted that the format in which the image acquisition device encodes the video frame data can be encoded according to the H.264 format standard, or according to the H.265 format standard, or according to other format standards, and the embodiment of the present application does not limit this.

[0039] After this step, after the encoded video frame data is obtained from the memory, the video frame data can be decoded in the video memory by hard decoding to obtain decoded texture data (also called D3D texture data).

[0040] 202: In the video memory, a color coding format conversion operation is performed on the texture data to obtain target texture data, where the color coding format of the target texture data is the basic color coding format.

[0041] In some embodiments, since the image acquisition device generally adopts the NV12 color coding format, the decoded texture data is also in the NV12 color coding format. In order for the rendering engine to subsequently render the texture data accordingly, a color coding format conversion operation can be performed on the texture data to obtain target texture data whose color coding format is a basic color coding format (such as an RGBA color coding format).

[0042] In some embodiments, performing a color coding format conversion operation on texture data to obtain target texture data may include the following process: By calling the CreateVideoProcessorInputView function, an input view of the texture data is determined; the input view may include parameters such as the pixel data width, pixel data height, and pixel data storage format of the texture data; By calling the CreateVideoProcessorOutputView function, the output view of the target texture data is determined; the output view may include parameters such as the pixel data width, pixel data height, and pixel data storage format of the target texture data; Based on the video processor instance, input view, output view, and calling the video processor bit block transfer function (VideoProcessorBIt function), a color coding format conversion operation is performed on the texture data to obtain the target texture data.

[0043] In some embodiments, the video processor instance can be obtained by calling the CreateVideoProcessorEnumerator function of the D3D video device interface to obtain a video processing enumerator, and based on the video processing enumerator, calling the CreateVideoProcessor function of the D3D video device interface to create a corresponding video processor instance.

[0044] In some embodiments, the video processing enumerator may list all input and output video formats supported by the device, the maximum and minimum resolutions and frame rates supported by the device, information about video processing options supported by the device (such as color space conversion, scaling, rotation), etc.; the video processor instance is used to allow applications to configure and apply specific video processing operations. For example, the application can use the video processor to adjust video parameters such as brightness, contrast, saturation, etc., or scale the video to adapt to different display resolutions, set the output video size ratio, the output video frame number, the video bit rate, and some hardware configurations.

[0045] In other embodiments, the converted width and height may be set in advance as the width and height of the texture data, and the converted color encoding format may be set to the RGBA color encoding format; then, the GetDevice function is called to obtain the D3D device interface, the GetImmediateContext function of the D3D device interface is called to obtain the D3D context interface, the QueryInterface function of the D3D device interface is called to obtain the D3D video device interface, and the QueryInterface function of the D3D context interface is called to obtain the D3D video context interface, so that when the color encoding format conversion operation is performed on the texture data later, these interfaces can be directly used to perform the color encoding format conversion process, thereby improving the conversion efficiency.

[0046] 203: In the video memory, according to the texture object type to which the target texture data belongs, a rendering resource pointer matching the texture object type is determined, and a video rendering operation is performed on the target texture data according to the rendering resource pointer to obtain target video frame data.

[0047] In some embodiments, according to the texture object type to which the target texture data belongs, determining a rendering resource pointer that matches the texture object type may include the following process: First, when it is detected that the flag field of the target texture data represents an update, a texture rendering command is added for the target texture data.

[0048] In some embodiments, when it is detected that the target texture data's tag field represents an update, a texture rendering command is added to the target texture data, which may be: after the target texture data is converted, the tag field of the target texture data may be set to an update parameter (which may be an English representation (e.g., true) or a number (e.g., 1)) in the texture data callback thread; when the tag field is detected as true or 1 in the Unreal Engine game thread, indicating that the texture data has been updated, the texture rendering command may be pushed into the virtual engine rendering thread through the ENQUEUE_RENDER_COMMAND macro. The texture rendering command may include the command name and texture object type, new pixel data and texture parameters, rendering execution logic, etc.; when the tag field is detected as false or 0 in the Unreal Engine game thread, indicating that the texture data has not been updated, the tag field of the target texture data of the next frame may be waited for to be detected.

[0049] Secondly, according to the texture object type to which the target texture data in the texture rendering command belongs, a rendering resource pointer matching the texture object type is obtained.

[0050] In some embodiments, according to the texture object type to which the target texture data in the texture rendering command belongs, obtaining a rendering resource pointer that matches the texture object type may be: obtaining a resource pointer from a texture object pointer (also called an Unreal Engine texture object pointer, used to manage and operate texture resources in the Unreal Engine), and converting the resource pointer to a target texture object pointer corresponding to the texture object type, obtaining an underlying rendering hardware interface pointer from the target texture object pointer, and obtaining a rendering resource pointer that matches the texture object type from the rendering hardware interface pointer, so as to ensure that the rendering operation can be optimized for a specific texture object type, thereby improving rendering efficiency and quality. The rendering resource pointer usually points to a memory location storing resources required for rendering, which may include shader programs, texture data, vertex data, etc.

[0051] For example, get the Resource pointer from the texture object pointer and convert it to a target texture object pointer of type FTexture2DResource (FTexture2DResource*Texture2DResource = (FTexture2DResource*)Texture2D->Resource, where Texture2D is a texture pointer of type UTexture2D); call the GetTexture2DRHI() function to get the underlying rendering hardware interface (RHI) texture object pointer from Texture2DResource (FTexture2DRHIRef Texture2DRHI = Texture2DResource->GetTexture2DRHI()); call the GetNativeResource function to get the underlying ID3D11Texture2D type rendering resource pointer from Texture2DRHI (ID3D11Texture2D* D3D11Texture2D = Texture2DRHI->GetNativeResource()). In some embodiments, according to the rendering resource pointer, a video rendering operation is performed on the target texture data to obtain the target video frame data, which can be: calling the GetDevice function to obtain the graphics processing application program interface corresponding to the rendering resource pointer (for example, D3D11Texture2D ->GetDevice(&D3D11Device)); calling the get thread context function of the graphics processing application program interface to obtain the graphics processing application program context interface (for example, D3D11Device->GetImmediateContext(&D3D11DeviceContext)); calling the resource copy function of the graphics processing application program context interface (for example, D3D11DeviceContext->CopyResource(D3D11Texture2D,SrcD3D11Texture2D); wherein SrcD3D11Texture2D is the source texture object pointer called back), rendering the target texture data to obtain the target video frame data.

[0052] In other embodiments, after the target video frame data is rendered, its mark field can be reset to false, and the texture can be refreshed after waiting for new target texture data to be called back, thereby completing the rendering of the entire continuous video screen texture data.

[0053] In the embodiment of the present application, decoding, color coding format conversion and rendering of video frame data are all performed in the video memory, and there is no frequent exchange between the internal memory and the video memory. In addition, according to the texture object type to which the target texture data belongs, a rendering resource pointer matching the texture object type is determined, thereby ensuring that the rendering operation can be optimized for a specific texture object type, thereby improving rendering efficiency and quality. Compared with the prior art, it can not only give full play to the parallel computing performance of the graphics processor, but also reduce the data copying and transportation work of the video frame data from the internal memory to the video memory. In the process of three-dimensional video rendering, the consumption of computer resources is reduced, thereby avoiding problems such as video playback freezes and high delays due to excessive computer resource usage. Furthermore, the decoding process and the rendering process are decoupled. As long as the decoding process obtains the target texture data in the format that meets the requirements, it can be rendered, thereby increasing the flexibility of video rendering.

[0054] Based on the above Figure 2 The method flow shown, Figure 3 A logic diagram of video rendering provided in an embodiment of the present application. Figure 3 As shown, the image acquisition device (such as Figure 1 The video frame data encoded by the image acquisition device 10) shown in the figure is stored in the memory. When video rendering is desired, the video frame data is transferred to the video memory, and the video frame data is hard-decoded in the video memory to obtain texture data in the color encoding format of NV12 color encoding format; then, the texture data in the NV12 color encoding format is converted into target texture data in the RGBA color encoding format in the video memory using video hardware acceleration; finally, the target texture data is copied to the Unreal Engine rendering thread, and video rendering operations are performed on the target texture data according to the texture object type to which the target texture data belongs. Based on the same technical concept, a video rendering device is also provided in an embodiment of the present application, and the device can implement the above-mentioned video rendering method process in the embodiment of the present application.

[0055] Figure 4 A schematic diagram of the structure of a video rendering device provided in an embodiment of the present application. Figure 4 As shown, the device includes: a decoding module 401, a color coding format conversion module 402, and a rendering module 403.

[0056] The decoding module 401 is used to obtain the encoded video frame data from the memory, and then perform hard decoding on the video frame data in the video memory to obtain texture data; A color coding format conversion module 402 is used to perform a color coding format conversion operation on the texture data in the video memory to obtain target texture data; wherein the color coding format of the target texture data is a basic color coding format; In some embodiments, the rendering module 403 is used to determine a rendering resource pointer that matches the texture object type in the video memory according to the texture object type to which the target texture data belongs, and perform a video rendering operation on the target texture data according to the rendering resource pointer to obtain target video frame data.

[0057] The color coding format conversion module 402 is specifically used for: Determine the input view of the texture data by calling the create video processor input view function; Determine the output view of the target texture data by calling the create video processor output view function; Based on the video processor instance, the input view, the output view, and by calling the video processor bit block transfer function, a color coding format conversion operation is performed on the texture data to obtain the target texture data.

[0058] In some embodiments, the rendering module 403 is specifically configured to: When it is detected that the mark field representation of the target texture data is updated, adding a texture rendering command for the target texture data; According to the texture object type to which the target texture data in the texture rendering command belongs, a rendering resource pointer matching the texture object type is obtained.

[0059] In some embodiments, the rendering module 403 is specifically configured to: Obtaining a resource pointer from a texture object pointer, and converting the resource pointer into a target texture object pointer corresponding to the texture object type; The underlying rendering hardware interface pointer is obtained from the target texture object pointer, and the rendering resource pointer matching the texture object type is obtained from the rendering hardware interface pointer.

[0060] In some embodiments, the rendering module 403 is specifically configured to: Calling GetDevice function to obtain the graphics processing application program interface corresponding to the rendering resource pointer; Calling the thread context function of the graphics processing application program interface to obtain the graphics processing application program context interface; The resource copy function of the graphics processing application context interface is called to render the target texture data to obtain the target video frame data.

[0061] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps in the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0062] Based on the same technical concept, an electronic device is also provided in an embodiment of the present application, and the electronic device can realize the functions of the aforementioned video rendering device.

[0063] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0064] At least one processor 501, and a memory 502 connected to the at least one processor 501. The specific connection medium between the processor 501 and the memory 502 is not limited in the embodiment of the present application. Figure 5 In the example, the processor 501 and the memory 502 are connected via a bus 500. The bus 500 is Figure 5 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 500 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 501 can also be called a controller, and there is no limitation on the name.

[0065] In the embodiment of the present application, the memory 502 stores instructions that can be executed by at least one processor 501. The at least one processor 501 can execute a video rendering method discussed above by executing the instructions stored in the memory 502. The processor 501 can implement Figure 4 The functions of each module in the device shown.

[0066] Among them, the processor 501 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 502 and calling data stored in the memory 502, the various functions of the device and processing data, the device can be monitored as a whole.

[0067] In the embodiment of the present application, the processor 501 may include one or more processing units, and the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0068] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of a video rendering method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware processor execution, or a combination of hardware and software modules in the processor.

[0069] The memory 502 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 502 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 502 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0070] By programming the processor 501, the code corresponding to the video rendering method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 2 The embodiment shown is a video rendering method. How to design and program the processor 501 is a technique known to those skilled in the art and will not be described in detail here.

[0071] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0072] Based on the same technical concept, an embodiment of the present application provides a computer storage medium, the computer storage medium includes: a computer program code, when the computer program code is executed on a computer, the computer executes any of the methods for video rendering discussed above. Since the principle of solving the problem by the above-mentioned computer storage medium is similar to that of a method for video rendering, the implementation of the above-mentioned computer storage medium can refer to the implementation of the method, and the repeated parts will not be repeated.

[0073] In a specific implementation process, computer storage media may include: Universal Serial Bus Flash Drive (USB), mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other storage media that can store program codes.

[0074] Based on the same technical concept, the embodiment of the present application also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, the computer executes any one of the video rendering methods discussed above. Since the principle of solving the problem by the above computer program product is similar to that of a video rendering method, the implementation of the above computer program product can refer to the implementation of the method, and the repeated parts will not be repeated.

[0075] The computer program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0076] The method in the present application can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the present application is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, an OAM or other programmable device.

[0077] The computer program or instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disk; it may also be a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0078] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0079] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that has the functions specified in one or more boxes.

[0080] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0081] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0082] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A video rendering method, characterized in that: include: After obtaining the encoded video frame data from the memory, the video frame data is hard-decoded in the video memory to obtain texture data; In the video memory, a color coding format conversion operation is performed on the texture data to obtain target texture data; wherein the color coding format of the target texture data is a basic color coding format; In the video memory, according to the texture object type to which the target texture data belongs, a rendering resource pointer matching the texture object type is determined, and a video rendering operation is performed on the target texture data according to the rendering resource pointer to obtain target video frame data.

2. The method according to claim 1, characterized in that The performing a color coding format conversion operation on the texture data to obtain target texture data includes: Determine the input view of the texture data by calling the create video processor input view function; Determine the output view of the target texture data by calling the create video processor output view function; Based on the video processor instance, the input view, the output view, and calling the video processor bit block transfer function, a color coding format conversion operation is performed on the texture data to obtain the target texture data.

3. The method according to claim 1, characterized in that The step of determining, according to the texture object type to which the target texture data belongs, a rendering resource pointer that matches the texture object type comprises: When it is detected that the mark field representation of the target texture data is updated, adding a texture rendering command for the target texture data; According to the texture object type to which the target texture data in the texture rendering command belongs, a rendering resource pointer matching the texture object type is obtained.

4. The method according to claim 3, characterized in that The step of obtaining a rendering resource pointer matching the texture object type according to the texture object type to which the target texture data in the texture rendering command belongs comprises: Obtaining a resource pointer from a texture object pointer, and converting the resource pointer into a target texture object pointer corresponding to the texture object type; The underlying rendering hardware interface pointer is obtained from the target texture object pointer, and the rendering resource pointer matching the texture object type is obtained from the rendering hardware interface pointer.

5. The method according to claim 1, characterized in that The step of performing a video rendering operation on the target texture data according to the rendering resource pointer to obtain target video frame data includes: Calling GetDevice function to obtain the graphics processing application program interface corresponding to the rendering resource pointer; Calling the thread context function of the graphics processing application program interface to obtain the graphics processing application program context interface; The resource copy function of the graphics processing application context interface is called to render the target texture data to obtain the target video frame data.

6. A video rendering device, characterized in that: include: A decoding module, for obtaining the encoded video frame data from the memory, and then hard decoding the video frame data in the video memory to obtain texture data; A color coding format conversion module is used to perform a color coding format conversion operation on the texture data in the video memory to obtain target texture data; wherein the color coding format of the target texture data is a basic color coding format; A rendering module is used to determine a rendering resource pointer matching the texture object type in the video memory according to the texture object type to which the target texture data belongs, and perform a video rendering operation on the target texture data according to the rendering resource pointer to obtain target video frame data.

7. The device according to claim 6, characterized in that The color coding format conversion module is specifically used for: Determine the input view of the texture data by calling the create video processor input view function; Determine the output view of the target texture data by calling the create video processor output view function; Based on the video processor instance, the input view, the output view, and calling the video processor bit block transfer function, a color coding format conversion operation is performed on the texture data to obtain the target texture data.

8. The device according to claim 6, characterized in that The rendering module is specifically used for: When it is detected that the mark field representation of the target texture data is updated, adding a texture rendering command for the target texture data; According to the texture object type to which the target texture data in the texture rendering command belongs, a rendering resource pointer matching the texture object type is obtained.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to implement the method according to any one of claims 1 to 5 when executing a computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images