Video Frame Processing Method and Apparatus

Abstract

Embodiments of the present disclosure provide a video frame processing method and apparatus. In this method, transcoding data of the video frame is obtained. The transcoding data is stored in a specified address space in the GPU. After that, a first mapping relationship is established between a pointer pointing to the specified address space and an index number of a target texture to be created in the GPU. Then, according to the first mapping relationship, the transcoding data stored in the specified address space is created into a target texture.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly, to a video frame processing method and apparatus. Background Art

[0002] With the development of multimedia technology, various processing requirements for video files are increasing in many application fields such as teaching, entertainment, and communication. Sometimes, it is necessary to transcode video files to adapt to different network bandwidths or picture quality requirements. In some cases, it may also be necessary to render video files to achieve better display effects. For example, in some application scenarios, it may be necessary to make certain objects in video frames more realistic. In a video frame showing the exterior wall of a house, it may be necessary to make the bricks on the exterior wall of the house more three-dimensional through rendering. In a video frame showing a lake, it may be necessary to make the lake appear sparkling through rendering. In a video frame showing a human face, it may be necessary to make the human face look softer or fairer through rendering. In other application scenarios, effects such as mirror effects can also be produced through rendering. Such rendering processing may require a large amount of computation and consume a large amount of system resources.

[0003] Common video encoding and decoding methods are performed on a central processing unit (CPU), and this encoding and decoding method is also called software encoding and decoding. However, the speed of software encoding and decoding is relatively slow. To improve the speed of video encoding and decoding, a GPU (commonly known as a graphics card) may be used to participate in video encoding and decoding, so that some of the video encoding and decoding work is completed by the hardware part. Summary of the Invention

[0004] Embodiments described herein provide a video frame processing method and a video frame processing apparatus.

[0005] According to a first aspect of the present disclosure, there is provided a video frame processing method executed by a GPU. In this method, transcoding data of the video frame is obtained. The transcoding data is stored in a specified address space in the GPU. Then, a first mapping relationship is established between a pointer pointing to the specified address space and an index number of a target texture to be created in the GPU. Then, according to the first mapping relationship, the transcoding data stored in the specified address space is created into a target texture.

[0006] In some embodiments of the present disclosure, the GPU is an NVIDIA GPU. The NVIDIA GPU includes CUDA (Compute Unified Device Architecture). The first mapping relationship is established by the CUDA.

[0007] In some embodiments of the present disclosure, the GPU includes OpenGL. The target texture is created by the OpenGL.

[0008] In some embodiments of the present disclosure, in the step of obtaining the transcoded data of the video frame, the demultiplexed compressed data of the video frame is obtained. Then, the compressed data is decoded into the original data of the video frame. Next, the original data is transcoded into the transcoded data of the video frame.

[0009] In some embodiments of the present disclosure, the GPU is an NVIDIA GPU. The NVIDIA GPU includes NVDEC. The compressed data is decoded into the original data by the NVDEC.

[0010] In some embodiments of the present disclosure. The GPU is an NVIDIA GPU. The NVIDIA GPU includes CUDA. The original data is transcoded into the transcoded data by the CUDA.

[0011] In some embodiments of the present disclosure, the GPU is an NVIDIA GPU. The NVIDIA GPU includes CUDA and a video encoder. The method further includes: rendering the target texture to obtain a rendered texture; enabling the CUDA to share the storage space of the rendered texture; establishing a second mapping relationship between the index number of the rendered texture and the pointer of the CUDA pointing to the storage space; obtaining rendered data from the rendered texture according to the second mapping relationship; copying the rendered data into the input buffer of the video encoder; and encoding the rendered data in the input buffer into encoded data by the video encoder.

[0012] In some embodiments of the present disclosure, the GPU includes OpenGL. The step of rendering the target texture to obtain a rendered texture is performed by the OpenGL.

[0013] In some embodiments of the present disclosure, the step of enabling the CUDA to share the storage space of the rendered texture is performed by the CUDA.

[0014] In some embodiments of the present disclosure, the step of establishing a second mapping relationship between the index number of the rendered texture and the pointer of the CUDA pointing to the storage space is performed by the CUDA.

[0015] In some embodiments of the present disclosure, the step of obtaining rendered data from the rendered texture according to the second mapping relationship is performed by the CUDA.

[0016] In some embodiments of the present disclosure, the step of copying the rendered data into the input buffer of the video encoder is performed by the CUDA.

[0017] According to a second aspect of the present disclosure, a video frame processing device including a GPU is provided. The GPU is configured to: obtain transcoding data of the video frame; store the transcoding data in a specified address space in the GPU; establish a first mapping relationship between a pointer to the specified address space and an index number of a target texture to be created in the GPU; and create the transcoding data stored in the specified address space into a target texture according to the first mapping relationship.

[0018] In some embodiments of the present disclosure, the GPU is an NVIDIA GPU. The NVIDIA GPU includes CUDA. The first mapping relationship is established by the CUDA.

[0019] In some embodiments of the present disclosure, the GPU includes OpenGL. The target texture is created by the OpenGL.

[0020] In some embodiments of the present disclosure, the GPU is configured to obtain the transcoding data of the video frame by the following operations: obtain the decompackaged compressed data of the video frame; decode the compressed data into the original data of the video frame; and transcode the original data into the transcoding data of the video frame.

[0021] In some embodiments of the present disclosure, the GPU is an NVIDIA GPU. The NVIDIA GPU includes NVDEC. The compressed data is decoded into original data by the NVDEC.

[0022] In some embodiments of the present disclosure. The GPU is an NVIDIA GPU. The NVIDIA GPU includes CUDA. The original data is transcoded into transcoding data by the CUDA.

[0023] In some embodiments of the present disclosure, the GPU is an NVIDIA GPU. The NVIDIA GPU includes CUDA and a video encoder. The GPU is further configured to: render the target texture to obtain a rendered texture; enable the CUDA to share the storage space of the rendered texture; establish a second mapping relationship between the index number of the rendered texture and a pointer of the CUDA pointing to the storage space; obtain rendering data from the rendered texture according to the second mapping relationship; copy the rendering data into an input buffer of the video encoder; and encode the rendering data in the input buffer into encoded data by the video encoder.

[0024] In some embodiments of the present disclosure, the device further includes at least one processor; and at least one memory storing a computer program. When the computer program is executed by the at least one processor, the device performs a decompackaging operation on the video frame.

[0025] In some embodiments of the present disclosure, the apparatus further includes at least one processor; and at least one memory storing a computer program. When the computer program is executed by the at least one processor, the apparatus performs an encapsulation operation on the encoded data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below. It should be understood that the following described drawings only relate to some embodiments of the present disclosure and do not limit the present disclosure, where:

[0027] Figure 1 is an exemplary flowchart of a method for processing video frames executed by a GPU according to an embodiment of the present disclosure;

[0028] Figure 2 is in Figure 1 an exemplary flowchart of steps for obtaining transcoded data of a video frame in the embodiment shown;

[0029] Figure 3 is Figure 1 an exemplary flowchart of further steps included in the method of the embodiment shown; and

[0030] Figure 4 is a schematic diagram of a process for processing video frames according to an embodiment of the present disclosure.

[0031] The elements in the drawings are schematic and not drawn to scale. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of the present disclosure without creative efforts also fall within the scope of protection of the present disclosure.

[0033] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the subject matter of the present disclosure pertains. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the specification and the relevant art, and will not be interpreted in an idealized or overly formal form unless otherwise clearly defined herein. As used herein, terms such as "first" and "second" are only used to distinguish one element (or a part of an element) from another element (or another part of an element).

[0034] "Video" in this article refers to the digital representation of a sequence of consecutive images. Each image in the sequence of consecutive images can be referred to as a video frame. Although the general meaning of "video" may also include an audio part, in this article, the image part in the video, that is, the video frame, is mainly discussed.

[0035] Generally, the video encoding and decoding process includes performing the following operations on video frames: demultiplexing -> decoding -> transcoding -> rendering -> encoding -> multiplexing. As mentioned above, in order to improve the speed of video encoding and decoding, a GPU (commonly known as a graphics card) may be used to participate in video encoding and decoding, so that some of the work of video encoding and decoding is completed by the hardware part. In some applications where a GPU is used to participate in video encoding and decoding, the CPU demultiplexes the video frames to obtain the demultiplexed compressed data of the video frames. Then the CPU sends the demultiplexed compressed data to the GPU, and the GPU decodes the compressed data to obtain the original data of the video frames. The original data of the video frames is generally in the YUV format, while the rendering operation needs to be performed on data in the RGB format. Therefore, transcoding is also required before rendering to convert the data in the YUV format into data in the RGB format. Generally, transcoding can be performed in the form of FFmpeg or libyuv transcoding. FFmpeg is an open-source computer program that can be used to record, convert digital audio and video, and convert them into streams. libyuv is a library open-sourced by Google that implements various conversions, rotations, and scalings between YUV and RGB. Such transcoding methods are performed on the CPU. Therefore, after the GPU decodes to obtain the original data of the video frames, the GPU needs to send the original data of the video frames to the CPU for transcoding. After the CPU obtains the data in the RGB format through transcoding, the CPU sends the data in the RGB format to the GPU for rendering operations in the GPU. The rendered data (which can be referred to as "rendered data" in the context of this disclosure) can be encoded in the GPU. The encoded data can be sent to the CPU for multiplexing processing.

[0036] In the above process, since the video frame data needs to be transmitted between the CPU and the GPU multiple times, and the transcoding operation is completed by software, this video encoding and decoding method (which can also be referred to as "semi-hardware encoding and decoding method" in the context of this disclosure) is still relatively time-consuming.

[0037] If the transcoding operation is also performed by the GPU, the number of times the video frame data is transferred between the CPU and the GPU can be reduced. The way in which the decoding operation, the transcoding operation, and the rendering operation are all performed in the GPU can be referred to as the "fully hardware decoding mode" in the context of the present disclosure. The way in which the decoding operation, the transcoding operation, the rendering operation, and the encoding operation are all performed in the GPU can be referred to as the "fully hardware encoding and decoding mode" in the context of the present disclosure. In the case of adopting the fully hardware decoding mode, how to efficiently combine the hardware components for performing the transcoding operation and the hardware components for performing the rendering operation is an aspect studied by the embodiments of the present disclosure. In the case of adopting the fully hardware encoding and decoding mode, how to efficiently combine the hardware components for performing the rendering operation and the hardware components for performing the encoding operation is another aspect studied by the embodiments of the present disclosure.

[0038] Figure 1 FIG. 4 shows an exemplary flowchart of a method 100 for processing video frames performed by a GPU according to an embodiment of the present disclosure. The following will describe the method 100 for processing video frames performed by the GPU with reference to Figure 1 FIG. 4.

[0039] At Figure 1 block S102, transcoding data of a video frame is obtained. In some embodiments of the present disclosure, the transcoding data is generated in the GPU. Figure 2 FIG. 5 shows an exemplary flowchart of the step of obtaining the transcoding data of the video frame.

[0040] At Figure 2 block S202, the demultiplexed compressed data of the video frame is obtained. The video file may include a file header for defining some parameters and / or information of the video file, such as sequence parameter set (SPS) / picture parameter set (PPS) information. According to the parameters and / or information included in the file header, information such as the start position of the video frame and the total length of the video frame can be known. According to such information, the specific storage position of the video frame data in the video file can be known. To save the storage space of the video, the video frame data is usually compressed data, which can also be understood as a compressed package of the original data. The process of obtaining the compressed data from the video file is called demultiplexing. Since the demultiplexing operation takes less time, the demultiplexing operation is usually performed in the CPU. After the CPU performs the demultiplexing operation on the video, the CPU can obtain the demultiplexed compressed data of the video frame. Then, the CPU sends the compressed data to the GPU. In this case, the process of obtaining the demultiplexed compressed data of the video frame can be understood as: the GPU receives the demultiplexed compressed data of the video frame from the CPU. In some embodiments, the demultiplexing operation can also be performed by the GPU to obtain the demultiplexed compressed data of the video frame.

[0041] At block S204, the compressed data is decoded into the original data of the video frame. In some embodiments of the present disclosure, after the GPU obtains the decompackaged compressed data, the decompackaged compressed data is sent to a decoder for decoding. After sending, for example, 4 frames of data, the decoder can return the decoded original data. The original data is generally in the YUV format. In some embodiments of the present disclosure, the GPU is, for example, an NVIDIA GPU (a GPU produced by the graphics card manufacturer NVIDIA). The compressed data can be decoded into the original data by the video decoder engine NVDEC in the NVIDIA GPU. Those skilled in the art should understand that the GPU can also be a GPU with a video decoder produced by other manufacturers.

[0042] At block S206, the original data is transcoded into the transcoded data of the video frame. Since the rendering operation needs to be performed on data in the RGB format, the original data in the YUV format also needs to be transcoded before rendering to convert the data in the YUV format into data in the RGB format. As described above, in some embodiments of the present disclosure, the GPU is, for example, an NVIDIA GPU. The original data in the YUV format can be transcoded into the transcoded data in the RGB format by CUDA in the NVIDIA GPU. CUDA is a general-purpose parallel computing architecture introduced by the graphics card manufacturer NVIDIA, which enables the GPU to solve complex computing problems. It includes the CUDA instruction set architecture (ISA) and the parallel computing engine inside the GPU. Those skilled in the art should understand that the GPU can also be a GPU with a transcoding function produced by other manufacturers.

[0043] Returning to Figure 1 , at block S104, the transcoded data is stored in the specified address space in the GPU. In some embodiments, a pointer to the specified address space can be obtained. In an example where the GPU is an NVIDIA GPU and the transcoding operation is performed by CUDA, the transcoded data can be stored in CUDA.

[0044] At block S106, a first mapping relationship is established between a pointer to a specified address space and an index number of a target texture to be created in the GPU. A texture can be one or more graphics representing the surface details of an object, which is essentially an array of data, such as color data, brightness data, etc. A single value in the texture array is usually called a texture unit, also known as a texel. The index number of the texture is the index number at which the texture is stored in the memory. In some graphics processing applications, such as OpenGL, OpenGLES, etc., textures are stored according to the index numbers of unsigned int type data. For example, if there are 10 textures, the ten numbers from 0 to 9 can be used as the index numbers respectively to represent the storage addresses of each texture. Before creating the target texture, the index number of the target texture can be determined first, so as to reserve the storage space pointed to by the index number for creating the target texture in this storage space in the future.

[0045] In an example where the GPU is an NVIDIA GPU, the first mapping relationship can be established by CUDA in the NVIDIA GPU.

[0046] At block S108, the transcoded data stored in the specified address space is created into a target texture according to the first mapping relationship. Since the operation object of the rendering tool (which can also be called a "renderer") is a texture rather than image data, it is necessary to create a target texture from the transcoded data for the rendering tool to perform the rendering operation. The target texture contains the information of the transcoded data. Since the first mapping relationship between the pointer to the specified address space and the index number of the target texture is established at block S106, it is not necessary to copy the transcoded data into the rendering tool. The target texture can be quickly created through the first mapping relationship between the pointer to the specified address space and the index number of the target texture.

[0047] In this way, the method for processing video frames according to the embodiments of the present disclosure can efficiently combine the hardware components for performing transcoding operations and the hardware components for performing rendering operations.

[0048] Next, further discussion will be made on how to efficiently combine the hardware components for performing rendering operations and the hardware components for performing encoding operations in the case of adopting a full hardware encoding and decoding method.

[0049] Figure 3 Shown Figure 1 An exemplary flowchart of further steps included in the method of the illustrated embodiment.

[0050] At block S302, the target texture is rendered to obtain a rendered texture. As described above, the texture array includes texture units. A texture unit is a reference to a texture object that can be sampled by a shader in a rendering tool. The texture object itself includes the data required by the texture, such as image data. The rendering tool can render the texture through the texture unit to form a rendered texture. The rendered texture is stored in a storage space different from that of the target texture, corresponding to a different index number from that of the target texture.

[0051] In an example where the GPU is an NVIDIA GPU and the NVIDIA GPU includes CUDA, at block S304, CUDA is enabled to share the storage space of the rendered texture. In some embodiments of the present disclosure, the rendered texture can be stored in an address space in the rendering tool. By registering this address space as a graphics resource cudaResource of CUDA, CUDA can be enabled to share this storage space.

[0052] At block S306, a second mapping relationship is established between the index number of the rendered texture and the pointer of CUDA pointing to the above storage space. In some embodiments of the present disclosure, the second mapping relationship can be established by CUDA.

[0053] At block S308, rendering data is obtained from the rendered texture according to the second mapping relationship. As mentioned before, a texture is essentially an array of data, such as color data, brightness data, etc. Image (video frame) data after rendering processing can be obtained from this data array, which can also be referred to as rendering data in the context. Since the operation object of the video encoder is image data rather than a texture, it is necessary to extract the information of the image data from the rendered texture. Since the pointer of CUDA points to the storage space shared by CUDA and the rendered texture, the rendering data corresponding to the texture stored in this storage space can be obtained through this pointer. Since the second mapping relationship between the index number of the rendered texture and the pointer of CUDA pointing to the storage space is established at block 306, it is convenient to obtain the rendering data from the rendered texture.

[0054] At block S310, the rendering data is copied into the input buffer of the video encoder. In some embodiments of the present disclosure, after each rendering data is obtained through this pointer, the rendering data can be copied into the input buffer of the video encoder.

[0055] At block S312, the video encoder encodes the rendering data in the input buffer into encoded data. In some embodiments, the encoding operation starts when the rendering data in the input buffer exceeds, for example, 3 frames. In an example where the GPU is an NVIDIA GPU, the image data can be encoded into encoded data by the video encoder engine NVENC in the NVIDIA GPU. Those skilled in the art should understand that the GPU can also be a GPU with a video encoder produced by other manufacturers.

[0056] To more clearly describe the process of processing video frames according to embodiments of the present disclosure, a schematic diagram of the entire video encoding and decoding process is shown in Figure 4 The dotted line in is used to divide the operations performed by the CPU and the GPU respectively. The operations in the upper half of the divided by the dotted line are performed by the CPU. The operations in the lower half of the divided by the dotted line are performed by the GPU. Figure 4 The dotted line in is used to divide the operations performed by the CPU and the GPU respectively. The operations in the upper half of the divided by the dotted line are performed by the CPU. The operations in the lower half of the divided by the dotted line are performed by the GPU. Figure 4 The dotted line in is used to divide the operations performed by the CPU and the GPU respectively. The operations in the upper half of the divided by the dotted line are performed by the CPU. The operations in the lower half of the divided by the dotted line are performed by the GPU. Figure 4 The operations in the lower half of the divided by the dotted line are performed by the GPU.

[0057] In Figure 4 In the example of, an NVIDIA GPU is taken as an example for illustration. Among them, the NVIDIA GPU may include: a video decoder engine NVDEC, CUDA, OpenGL, and a video encoder engine NVENC. NVDEC can perform decoding operations. CUDA can perform transcoding operations. OpenGL can perform rendering operations. NVENC can perform encoding operations.

[0058] As Figure 4 shown, at block 402, the CPU obtains the input video file. In some embodiments, the CPU can detect the information of the GPU in the current system to obtain GPU-related data. GPU-related data is, for example, information such as the name, model, and memory of the GPU. According to this information, it can be known whether the GPU supports decoding, transcoding, rendering, and encoding functions. In the case where the GPU supports decoding, transcoding, rendering, and / or encoding functions, the GPU can perform the operations of processing video frames according to embodiments of the present disclosure.

[0059] At block 404, the CPU creates a context pointer for demultiplexing by, for example, calling the FFmpeg API. At block 406, based on this context pointer, the CPU uses the FFmpeg API to perform demultiplexing of the video file. Taking a file in MP4 format as an example for illustration. The SPS / PPS information of the MP4 file is stored in the file header. Therefore, the SPS / PPS information can be extracted from the file header to perform the demultiplexing operation according to this SPS / PPS information. After the demultiplexing operation, the CPU obtains the compressed package of the original data (i.e., the demultiplexed compressed data of the video frame).

[0060] Then, at block 408, the CPU sends the compressed package of the acquired raw data to the GPU for decoding operations in the decoder (NVDEC) of the GPU. After the decoder receives, for example, 4 frames of compressed data, the decoded raw data (i.e., the raw data of the video frames) can be returned. The decoded raw data cannot be directly rendered because the input data of the rendering tool OpenGL needs to be in RGB format, while the raw data is generally in YUV format. Therefore, the raw data needs to be transcoded before being sent to OpenGL for rendering, converting the YUV format data into RGB format data. In Figure 4 In the example of

[0061] at block 410, the transcoding operation is performed on the GPU by CUDA. The transcoded data is stored in the specified address space in CUDA.

[0062] Then, at block 412, OpenGL that performs the rendering operation is mapped to CUDA that performs the transcoding operation. In some embodiments, a pointer of type CUdeviceptr can be used to point to the address space where the transcoded data is stored. Then, CUDA can use a mapping method to associate the index number of the target texture to be created in the GPU (which can be referred to as the "OpenGL texture ID") with this CUdeviceptr type pointer. OpenGL can quickly generate an OpenGL texture by virtue of the mapping relationship between the OpenGL texture ID and this CUdeviceptr type pointer. In this way, CUDA does not need to copy the transcoded data to OpenGL for OpenGL to generate the texture, thus saving the time and resources occupied by the copy operation. And each time the transcoded data is obtained, the OpenGL texture can be quickly updated, thereby accelerating the rendering speed. Figure 4 In

[0063] In some embodiments, in the case where the rendered video frames need to be stored in a compressed format, an encoding operation may be required for the rendered video frames. The encoding operation needs to be performed on image data, and as described above, the output of the rendering tool OpenGL is a texture rather than image data. Therefore, the output of OpenGL cannot be directly used as the input of the encoder. To address this problem, in the embodiments of the present disclosure, it is proposed to obtain data that can be processed by the encoder from the output of OpenGL with the help of CUDA for the encoder to perform the encoding operation.

[0064] At block 416, another mapping relationship is established between OpenGL and CUDA. Specifically, the rendered texture is stored in a certain address space in OpenGL. In one example, this address space can be registered as a CUDA graphics resource cudaResource. In this way, OpenGL and CUDA can share this address space. A pointer of cudaArrary type is generated in CUDA, and this pointer of cudaArrary type points to the registered cudaResource. CUDA can use a mapping method to associate the index number of the rendered texture (which can be referred to as the "OpenGL rendered texture ID") with this pointer of cudaArrary type. Through the mapping relationship between the OpenGL rendered texture ID and this pointer of cudaArrary type, the rendered data (in the context of the present disclosure, can be referred to as the rendering data) can be obtained through this pointer of cudaArrary type. In this way, each time the rendered data is obtained, this pointer of cudaArrary type will be updated synchronously. CUDA can copy the data pointed to by this pointer of cudaArrary type to the input buffer of the encoder. In this way, the encoding operation and the rendering operation can be combined in the GPU.

[0065] At block 418, the encoder encodes the data stored in the input buffer. In some embodiments, the encoding operation starts when the rendering data in the input buffer exceeds, for example, 3 frames. The encoded data can be stored in a two-dimensional array and sent to the CPU.

[0066] At block 420, the CPU can create an object of AvPacket type, and read the two-dimensional array in a loop to copy the encoded data into the data part of the object of AvPacket type.

[0067] At block 422, the CPU can call the FFmpeg API to encapsulate the video file, and then obtain or output the encapsulated output video file at block 424.

[0068] Embodiments of the present disclosure perform decoding, transcoding, and encoding on the GPU, and bind the transcoding and encoding with OpenGL rendering, which improves the overall codec speed and reduces the resource occupancy of the CPU.

[0069] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatuses and methods according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0070] Unless the context clearly indicates otherwise, the singular forms of words used in this specification and the appended claims include the plural, and vice versa. Thus, when referring to the singular, the corresponding plural is generally included. Similarly, the terms "comprising" and "including" are to be construed as inclusive rather than exclusive. Likewise, the term "including" and "or" should be interpreted as inclusive, unless such an interpretation is explicitly prohibited herein. Where the term "exemplary" is used in this specification, particularly when it is followed by a list of terms, the "exemplary" is merely illustrative and explanatory and should not be considered exclusive or extensive.

[0071] Further aspects and scopes of adaptability become apparent from the description provided herein. It should be understood that the various aspects of the present application may be implemented individually or in combination with one or more other aspects. It should also be understood that the description herein and the specific embodiments are for illustrative purposes only and are not intended to limit the scope of the present application.

[0072] The above has described in detail several embodiments of the present disclosure. However, it is obvious that those skilled in the art can make various modifications and variations to the embodiments of the present disclosure without departing from the spirit and scope of the present disclosure. The protection scope of the present disclosure is defined by the appended claims.

Claims

1. A video frame processing method, executed by a GPU, the video frame processing method comprises: obtaining transcoding data of the video frame; storing the transcoding data in a specified address space in the GPU; establishing a first mapping relationship between a pointer pointing to the specified address space and an index number of a target texture to be created in the GPU; and creating the target texture from the transcoding data stored in the specified address space according to the first mapping relationship; rendering the target texture to obtain a rendered texture; enabling CUDA to share the storage space of the rendered texture, wherein the GPU is an NVIDIA GPU, and the NVIDIA GPU includes the CUDA and a video encoder; establishing a second mapping relationship between the index number of the rendered texture and a pointer of the CUDA pointing to the storage space; obtaining rendering data from the rendered texture according to the second mapping relationship; copying the rendering data into an input buffer of the video encoder; and encoding the rendering data in the input buffer into encoded data by the video encoder.

2. The video frame processing method according to claim 1, wherein the first mapping relationship is established by the CUDA.

3. The video frame processing method according to claim 1 or 2, wherein the GPU includes OpenGL, and the target texture is created by the OpenGL.

4. The video frame processing method according to claim 1, wherein obtaining the transcoding data of the video frame includes: obtaining the decompressed compressed data of the video frame; decoding the compressed data into the original data of the video frame; and transcoding the original data into the transcoding data of the video frame.

5. The video frame processing method according to claim 4, wherein the NVIDIA GPU includes NVDEC, and the compressed data is decoded into the original data by the NVDEC.

6. The video frame processing method according to claim 4, wherein the original data is transcoded into the transcoding data by the CUDA.

7. A video frame processing device, comprising a GPU, the GPU being configured to: obtain the transcoding data of the video frame; store the transcoding data in a specified address space in the GPU; establish a first mapping relationship between a pointer pointing to the specified address space and an index number of a target texture to be created in the GPU ； and create the target texture from the transcoding data stored in the specified address space according to the first mapping relationship; render the target texture to obtain a rendered texture; enable CUDA to share the storage space of the rendered texture, wherein the GPU is an NVIDIA GPU, and the NVIDIA GPU includes the CUDA and a video encoder; establish a second mapping relationship between the index number of the rendered texture and a pointer of the CUDA pointing to the storage space; obtain rendering data from the rendered texture according to the second mapping relationship; Copy the rendering data into the input buffer of the video encoder; and Encode the rendering data in the input buffer into encoded data by the video encoder.

8. The video frame processing device according to claim 7, wherein, The first mapping relationship is established by the CUDA.

9. The video frame processing device according to claim 7 or 8, wherein, The GPU includes OpenGL, and the target texture is created by the OpenGL.

Images