Video image display method and device, computing device and readable storage medium

Abstract

The present invention discloses a video image display method, including: applying for a frame memory, converting a frame memory address into a frame memory identifier; parsing video frame data and placing it in the frame memory; passing the frame memory identifier to an application program that plays the video, calling a hardware rendering language through the application program, and the application program transmitting the frame memory identifier to the hardware rendering language; and using the frame data to perform data rendering through the hardware rendering language. According to the embodiments of the present invention, it is possible to smoothly watch movies without occupying too much memory, and the problem of poor viewing experience when the CPU computing power is poor is also solved.

Description

Technical Field

[0001] The present invention relates to the field of video playback technology, and in particular to a video image display method and device, a computing device and a readable storage medium. Background Art

[0002] Currently, there are two copies of video content when playing videos. One is when FFmpeg applies for memory and transmits it to the player. Among them, when FFmpeg plays the video, it initializes the video memory in the data driver module (DRM), then applies for memory and fills it with frame data. In the process of filling the frame data, there will be a large data copy operation of the CPU. After copying, the frame data will be transmitted to the player. Finally, the player will send the frame data to the DRM through the rendering language OPenGL EGL to perform the underlying driver work of the display, and finally the synthesized image will be sent to the display to display the video. There will also be data copying in the process of transmitting to the DRM. The other time is in the process of receiving texture data when the GPU performs rendering.

[0003] Copying large data will take up a lot of CPU. If the CPU performance is insufficient, this will cause playback to freeze, affecting the video viewing experience. In addition to unsmooth playback, it may also cause serious frame loss and severe CPU chip heating. Summary of the invention

[0004] To this end, the present invention provides a video image display method and apparatus, a computing device and a readable storage medium, in an effort to solve or at least alleviate at least one of the above problems.

[0005] According to one aspect of the present invention, a video image display method is provided, comprising: applying for a frame memory, converting a frame memory address into a frame memory identifier; parsing video frame data and placing it in the frame memory; passing the frame memory identifier to an application that plays the video, calling a hardware rendering language through the application, and the application transmitting the frame memory identifier to the hardware rendering language; using the frame data through the hardware rendering language to perform data rendering, the data rendering comprising: creating an original data texture; activating and binding the original data texture, preparing texture data for subsequent refreshing of the texture; using a texture refresh tool to share the frame data, generating a color space image for refreshing the texture data, and refreshing the texture data; sending the texture data to a graphics processor through a graphics processor raw channel interface; using the frame memory identifier through the graphics processor to access the data placed in the frame memory, and using the texture data to generate an image for display by the application.

[0006] According to another aspect of the present invention, a video image display device is provided, comprising: a frame memory application module, used for applying for frame memory, converting the frame memory address into a frame memory identifier; a parsing module, used for parsing video frame data and placing it in the frame memory; a transmission module, used for transmitting the frame memory identifier to an application for playing the video, calling a hardware rendering language through the application, and the application transmitting the frame memory identifier to the hardware rendering language; a rendering module, used for performing data rendering using the frame data through the hardware rendering language, the rendering module comprising: a creation unit, creating an original data texture; a texture data preparation unit, activating and binding the original data texture, preparing texture data for subsequent refreshing of the texture; a refresh unit, using a texture refresh tool to share the frame data, generating a color space image for refreshing the texture data, and refreshing the texture data; a texture data sending unit, sending the texture data to a graphics processor through a graphics processor raw channel interface; an image generation unit, used for using the texture data through the graphics processor to generate an image for display by the application.

[0007] According to another aspect of the present invention, there is provided a computing device, comprising: at least one processor and a memory storing program instructions; when the program instructions are read and executed by the processor, the computing device executes the above-mentioned video image display method.

[0008] According to another aspect of the present invention, a readable storage medium storing program instructions is provided. When the program instructions are read and executed by a computing device, the computing device executes the above-mentioned video image display method.

[0009] According to the implementation mode of the present invention, it is possible to watch movies smoothly without occupying too much memory, and the problem of poor viewing experience when the CPU computing power is poor is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] To achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The above and other purposes, features and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout the present disclosure, the same reference numerals generally refer to the same parts or elements.

[0011] Figure 1 A schematic diagram of a computing device 100 according to one embodiment of the present invention is shown.

[0012] Figure 2 A schematic block diagram showing the technical concept of the zero-copy video playback process according to the present invention.

[0013] Figure 3 A flowchart of a video image display method 300 according to an embodiment of the present invention is shown;

[0014] Figure 4 The process of data rendering in method 300 is shown.

[0015] Figure 5 A block diagram of a video image display device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0016] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0017] In view of the problems of playback freeze, frame loss and so on that may be caused by the existing video playback technology, the present invention provides a video image display method, which can realize zero-copy video playback and avoid the problems of playback freeze, frame loss and so on.

[0018] The video image display method of the present invention is executed in a computing device. The computing device can be any device with storage and computing capabilities, which can be implemented as a server, a workstation, etc., or as a personal computer such as a desktop computer, a notebook computer, or as a terminal device such as a mobile phone, a tablet computer, a smart wearable device, an Internet of Things device, but is not limited thereto.

[0019] Figure 1 FIG. 1 is a schematic diagram of a computing device 100 according to an embodiment of the present invention. It should be noted that: Figure 1 The computing device 100 shown is only an example. In practice, the computing device used to implement the video image display method of the present invention can be any type of device, and its hardware configuration can be different from the above. Figure 1 The computing device 100 shown is the same as Figure 1 The computing device 100 shown is different. In practice, the computing device used to implement the video image display method of the present invention can be Figure 1 The hardware components of the computing device 100 shown are increased or deleted, and the present invention does not limit the specific hardware configuration of the computing device.

[0020] like Figure 1As shown, in a basic configuration 102, computing device 100 typically includes system memory 106 and one or more processors 104. A memory bus 108 may be used for communication between processor 104 and system memory 106.

[0021] Depending on the desired configuration, the processor 104 can be any type of processor, including but not limited to: a microprocessor (μP), a microcontroller (μC), a digital information processor (DSP), or any combination thereof. The processor 104 can include one or more levels of cache such as a first-level cache 110 and a second-level cache 112, a processor core 114, and registers 116. An example processor core 114 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 118 can be used with the processor 104, or in some implementations, the memory controller 118 can be an internal part of the processor 104.

[0022] Depending on the desired configuration, the system memory 106 can be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. The physical memory in the computing device generally refers to the volatile memory RAM, and the data in the disk needs to be loaded into the physical memory before it can be read by the processor 104. The system memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some embodiments, the application 122 may be arranged to execute instructions on the operating system by one or more processors 104 using program data 124. The operating system 120 may be, for example, Linux, Windows, etc., which includes program instructions for processing basic system services and performing tasks that depend on hardware. The application 122 includes program instructions for implementing various user-desired functions, and the application 122 may be, for example, a browser, instant messaging software, a software development tool (such as an integrated development environment IDE, a compiler, etc.), etc., but is not limited thereto. When the application 122 is installed in the computing device 100, a driver module may be added to the operating system 120.

[0023] When the computing device 100 starts running, the processor 104 reads and executes the program instructions of the operating system 120 from the memory 106. The application 122 runs on the operating system 120 and uses the interfaces provided by the operating system 120 and the underlying hardware to implement various functions desired by the user. When the user starts the application 122, the application 122 is loaded into the memory 106, and the processor 104 reads and executes the program instructions of the application 122 from the memory 106.

[0024] Computing device 100 also includes a storage device 132 including a removable storage 136 and a non-removable storage 138 , both of which are connected to storage interface bus 134 .

[0025] The computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to the basic configuration 102 via the bus / interface controller 130. Example output devices 142 include a graphics processing unit 148 and an audio processing unit 150. They can be configured to facilitate communication with various external devices such as a display or speakers via one or more A / V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices) or other peripherals (e.g., printers, scanners, etc.) via one or more I / O ports 158. Example communication devices 146 may include a network controller 160, which may be arranged to facilitate communication with one or more other computing devices 162 via a network communication link via one or more communication ports 164.

[0026] A network communication link can be an example of a communication medium. Communication media can generally be embodied as computer-readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A "modulated data signal" can be a signal in which one or more of its data sets or its changes can be carried out in a manner that encodes information in the signal. As non-limiting examples, communication media can include wired media such as a wired network or a dedicated line network, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR) or other wireless media. The term computer-readable medium used herein can include both storage media and communication media.

[0027] In the computing device 100 according to the present invention, the application 122 includes instructions for executing the video image display method 300 of the present invention, and the instructions can instruct the processor 104 to execute the video image display method 300 of the present invention.

[0028] Before describing method 300, an overview of the inventive concept is provided below. As described in the background technology section, in the prior art, the rendering of the image is completed through frame storage data. In the whole process, there are multiple copies of large data. What the present invention aims to achieve is "zero copy", which will convert the data into a corresponding frame storage identifier at the application layer. The frame storage identifier is used to identify the frame storage address, which is equivalent to the house number of the frame storage address, referred to as fd (file descriptor). Instead of transferring large frame storage data, the solution of the present invention implements the transfer of frame storage data by transferring the frame storage identifier. The so-called "zero copy" is a technology based on the fd to complete video playback.

[0029] Figure 2 Schematic block diagram showing the "zero copy" technical concept of the present invention. Figure 2 As shown, in the implementation process, first apply for a section of space (referred to as frame storage) from the cache to store the data of the video frame (referred to as frame data). According to the size of the memory space and the computing power, you can apply for a space that can only store one frame of data, or you can apply for a space that can store multiple frames of data. In an optional implementation, in order to support smooth playback, you may apply for enough space to store 10-20 frames of data. Then, the frame storage address is converted into a frame storage identifier, referred to as fd, which is used to identify the address of the frame storage applied for. When rendering, fd is transmitted to the GPU, and the GPU recognizes fd and automatically completes the rendering process. In this process, it is no longer necessary to copy the large frame storage data to the corresponding driver module, nor is it necessary to retransmit the large data back to the GPU after decoding is completed. In the technical solution of this application, only the frame storage identifier fd needs to be transmitted to realize the rotation of data-the underlying driver module DRM, the graphics processor GPU, and the audio and video interface obtain the frame storage data through the fd transmitted by the application layer, and the frame storage fd only occupies a few bytes, which greatly saves the computing space. Using the zero-copy solution of this application, the viewing work can be completed even on a platform with poor CPU computing performance.

[0030] Figure 3 1 is a flowchart of a video image display method 300 according to an embodiment of the present invention. The method 300 is executed in a computing device (such as the aforementioned computing device 100). Figure 3 Method 300 is described. Figure 3 The method can be executed by a program in an application (such as a player) that plays the video, and the program can mobilize relevant hardware, interfaces, etc. to complete the corresponding operation. In the following description, when it is described that, for example, "A executes..." (A refers to a hardware, interface, rendering language, etc.), it means that the program mobilizes A, etc. to perform the corresponding operation. Figure 3 In the embodiment, the method 300 includes:

[0031] S310, applying for a frame memory, and converting the frame memory address into a frame memory identifier.

[0032] In step S310, the audio and video decoding tool will be mobilized to complete the audio and video decoding work and the frame storage application work. In the implementation mode of the present application, the FFmpeg interface can be used to complete the audio and video decoding work and the frame storage application work. For ease of explanation, the implementation mode of the present application is also described by taking FFmpeg as an example. FFmpeg is a set of open source computer programs that can be used to record and convert digital audio and video and convert them into streams. However, it should be understood that the tools for audio and video decoding are not limited to this.

[0033] FFmpeg is responsible for applying for frame memory. The number of frame memories applied for depends on the computing power and the size of the memory space, such as one, dozens, or dozens. Then, the frame memory is converted into a frame memory identifier through a mapping tool. The mapping tool can use dma_buf technology. dma_buf is an independent subsystem in the kernel, which provides a unified framework for sharing caches between different devices and subsystems. The cache here usually refers to the memory that interacts with the hardware and is accessed through dma_buf. The main application scenario of the dma_buf mechanism in the kernel is to support the core mechanism in the GPU driver, but as a general module in the kernel, it has a wide range of applications.

[0034] The emergence of dma_buf is to solve the problem of cache sharing between various drivers. Therefore, it is essentially a combination of cache and file, that is, dma_buf is both a physical cache and a system file. The cache stores content, and the file is the medium. Only through the medium of the file can the same cache be transferred between different drivers.

[0035] The frame buffer identifier is referred to as fd (file descriptor), and its number corresponds to the number of frame buffers applied for. Each applied frame buffer corresponds to an fd. An fd can be understood as the address number of the frame buffer. In the subsequent rendering and synthesis process of the video frame, an fd corresponds to the transmission of a frame of data. The number of fds and frame buffers is not limited, as long as it can support the smooth playback of the entire video in the relevant computing environment.

[0036] The process of converting the frame memory address into fd can be: each frame memory has its own address number, and the address number is converted into a corresponding value through mapping technology. The value can be the first / last / middle digit / several digits of the frame memory address number. fd can be an integer value.

[0037] S320, parsing the video frame data and storing it in a frame memory.

[0038] The parsing can be done using the FFmpeg mentioned above.

[0039] In the case that more than one frame store is applied for at the same time in step S310, and a frame store identifier fd corresponding to the number of frame stores is generated, the parsing step of step S320 may include: parsing the video frame data corresponding to the number of frame store identifiers. For example, through the above application, N frame stores are applied for. Here, N video frames are parsed and placed in the corresponding frame stores.

[0040] Optionally, the video frame data may be stored in a frame memory in a YUV format such as NV21, so that in a subsequent rendering step, it does not need to be converted into other formats, thereby reducing CPU space utilization.

[0041] S330, passing the frame buffer identifier to an application program for playing the video, calling a hardware rendering language through the application program, and the application program transmitting the frame buffer identifier to the hardware rendering language.

[0042] In the implementation manner of the present application, OPenGL EGL is taken as an example of a hardware rendering language for description.

[0043] OpenGL ES (OpenGL for Embedded Systems) is a subset of the OpenGL 3D graphics API (application programming interface), designed for embedded devices such as mobile phones, PDAs, and game consoles. Graphics card manufacturers and system manufacturers implement this set of APIs. EGL is an intermediate interface layer between the OpenGL ES rendering API and the native window system (native platform window system), which is mainly implemented by system manufacturers. The introduction of EGL is to shield the differences between different platforms. EGL provides the following mechanisms: communicating with the native window system of the device; querying the available types and configurations of the drawing surface; creating a drawing surface; synchronizing rendering between OpenGL ES and other graphics rendering APIs; and managing rendering resources such as texture maps.

[0044] Through this step, fd is transmitted to the application, and the application calls the hardware rendering language so that the hardware rendering language knows which frame to render. For this step, within the system, the bottom layer operation can be: the program mobilizes the bottom layer driver module (DRM) to perform the fd transmission operation to the upper layer. When the GPU is executed, the application needs to call the hardware rendering language to perform rendering. In step S330, the application calls OPenGL EGL, and the graphics processor (GPU) described later can recognize the hardware rendering language for subsequent processing.

[0045] Among them, DRM is a bottom-level data driver module used to perform bottom-level work of display driver.

[0046] S340, performing data rendering using the frame data through the hardware rendering language.

[0047] The data rendering process is performed in applications such as players. Figure 4 As shown, it may include:

[0048] S3410, create raw data texture.

[0049] Create a texture object in the data rendering program. Then the texture will be input to the GPU to perform subsequent refresh operations. Texture is an ID of an image recognized by the GPU, and the GPU can synthesize images through textures. The original texture is the texture that the GPU can directly recognize.

[0050] The original data texture only needs to be created once. After a frame of image is synthesized and displayed, the new frame of data will continue to be drawn on the previous texture.

[0051] S3420, activate and bind the original data texture, and prepare the texture data to facilitate subsequent refreshing of the texture.

[0052] By binding the original data texture, the GPU will know that this is the object that needs to be drawn. Texture data refers to the value of the texture. Preparing texture data is equivalent to creating an object for initialization, preparing for the subsequent refresh of texture data.

[0053] S3430, using a texture refresh tool to share the frame data, generating a color space image for refreshing the texture data, and refreshing the texture data.

[0054] After the player transmits fd to the texture refresh tool, the texture refresh tool can share the frame data in the frame memory pointed to by fd. In this embodiment, the texture refresh tool can use EGLImage, and this article also uses the refresh tool as an example for description. Sharing frame data means placing the frame data saved in the frame memory pointed to by fd into the EGLImage object. EGLImage can generate specific pictures through fd. Here, two pictures can be generated, one is a Y picture and the other is a UV picture.

[0055] Among them, Y and UV are types of color space in a color encoding method YUV. Among them, Y represents brightness, which is a grayscale value. U and V represent chromaticity, which describes the color and saturation of the image and is used to specify the color of the pixel. This article takes a YUV picture as an example to illustrate the refresh process, but it should be understood that the present invention is not limited thereto.

[0056] Using the image to refresh the above texture data refers to binding the image and data in the EGLImage to the original data texture of step S3420.

[0057] S3440: Send the texture data to the graphics processor through the graphics processor raw channel interface.

[0058] S3450: Generate an image for display by the application program using the texture data through the GPU. Specifically, when the GPU receives the texture data, it also receives the fd through the EGLImage, and can access the data stored in the frame memory through the fd. The graphics processor GPU can know the address of the specific data through the Y picture and the UV picture, and find the YUV data stored in the frame memory accordingly.

[0059] The GPU binds the data to the received texture data to perform image synthesis and generate an image that can be displayed on an application such as a player. In this process, only the fd is passed, and there is no need to copy the YUV data, thus achieving zero copy and reducing CPU usage.

[0060] At this point, the zero-copy video playback process is completed. During the entire process, fd is exported through the mapping tool (which can be dma_buf technology) in the underlying driver module (DRM), and fd is passed in the application layer. The underlying DRM obtains the address of the data in the cache through fd, and the GPU synthesizes the image through the data and outputs it to the display, realizing a zero-copy playback process.

[0061] The above generates an image display for one frame. In the case of generating a video playback for multiple frames, after generating the image for displaying the application, you can switch to the next frame memory identifier and repeat steps S3420-S3440. After generating the texture data for each frame, generate an image of the texture data for the frame through the GPU for display on the application.

[0062] After generating an image display for one frame, you can also release objects in EGLImage, including fd, generated color space images, etc., to save memory space.

[0063] In an optional implementation, the method 300 may further include:

[0064] After the image for display by the application is generated for the frame data in all the frame stores, it is determined whether all the frames of the video have been processed. When the determination result is no, the process starting from step S310 is continued.

[0065] After the images for display by the application are generated for all frame data of the video, the texture data may be released to save memory space.

[0066] After the image displayed by the application is generated for all frame data of the video, a notification may be made to close the fd data transmission channel.

[0067] According to the implementation manner of the present invention, 4K and 8K videos can be directly played when the CPU computing power is poor, which can directly improve the viewing experience of normal videos.

[0068] According to an embodiment of the present invention, a video image display device is also provided. Figure 5 FIG. 4 is a schematic block diagram of the video image display device. Figure 5 As shown, the device includes: a frame memory application module 510, which is used to apply for frame memory and convert the frame memory address into a frame memory identifier; a parsing module 520, which is used to parse the video frame data and place it in the frame memory; a transmission module 530, which is used to transmit the frame memory identifier to the application program for playing the video, and transmit the frame memory identifier from the application program to the hardware rendering language to call the hardware rendering language; a rendering module 540, which is used to perform data rendering using the frame data through the hardware rendering language, and the rendering module includes: a creation unit 5410, which creates an original data texture; a texture data preparation unit 5420, which activates and binds the original data texture, and prepares texture data for subsequent refreshing of the texture; a refresh unit 5430, which uses a texture refresh tool to share the frame data, generates a color space image for refreshing the texture data, and refreshes the texture data; a texture data sending unit 5440, which sends the texture data to the graphics processor through the graphics processor raw channel interface; an image generation unit 5450, which uses the texture data through the graphics processor to generate an image for display by the application.

[0069] Optionally, the frame memory application module 510 may apply for more than one frame memory at the same time and generate frame memory identifiers corresponding to the number of frame memory. The parsing module 520 may parse the video frame data corresponding to the number of frame memory identifiers.

[0070] Optionally, Figure 5 The device shown may also include a judgment module for judging whether all frames of the video have been processed after generating images for display by the application for frame data in all frame stores, and when the judgment result is no, continuing to execute the video image display method for unprocessed frames.

[0071] Optionally, the rendering module 540 may further include a switching module: after generating an image for display by the application, switching to a next frame buffer identifier and repeating the data rendering process.

[0072] Optionally, the rendering module 540 may further include a releasing module for releasing the object in the texture rendering tool after generating the image for display by the application.

[0073] For the parts of the device that are not described in detail, please refer to the above description of the method implementation method, and no further details will be given here.

[0074] The various techniques described herein may be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present invention, or some aspects or parts of the method and apparatus of the present invention may be in the form of program codes (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present invention.

[0075] In the case where the program code is executed on a programmable computer, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store the program code; the processor is configured to execute the video image display method of the present invention according to the instructions in the program code stored in the memory.

[0076] By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer readable instructions, data structures, program modules or other data. Communication media generally embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information transfer medium. Any combination of the above is also included within the scope of readable media.

[0077] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with examples of the present invention. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to implement the content of the present invention described herein, and the above description of specific languages ​​is for disclosing preferred embodiments of the present invention.

[0078] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0079] Similarly, it should be understood that in order to simplify the present disclosure and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features clearly recorded in each claim. It should be understood by those skilled in the art that the modules or units or components of the devices in the examples disclosed herein can be arranged in the devices described in the embodiments, or alternatively can be located in one or more devices different from the devices in the examples. The modules in the aforementioned examples can be combined into one module or can be divided into multiple submodules in addition.

[0080] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition they can be divided into multiple submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0081] In addition, it will be appreciated by those skilled in the art that, although some embodiments described herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments is meant to be within the scope of the present invention and to form different embodiments. In addition, some of the embodiments described herein are described as a combination of methods or method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiment are examples of devices that are used to implement the functions performed by the elements for the purpose of implementing the invention.

[0082] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved, and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.

[0083] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is primarily selected for readability and instructional purposes, rather than for explaining or limiting the subject matter of the present invention.

Claims

1. A video image display method, comprising: Use audio and video decoding tools to apply for more than one frame buffer; Through the mapping tool, each frame memory address is converted into a frame memory identifier, and one frame memory identifier corresponds to the transmission of one frame of data; Parse the video frame data and place it into the corresponding frame memory; The frame memory identifier is passed to an application program for playing the video, and the hardware rendering language is called by the application program, and the application program transmits the frame memory identifier to the hardware rendering language so that the hardware rendering language knows the frame to be rendered; wherein, within the system, a bottom-layer driver module for display driving is mobilized to perform a transmission operation of the frame memory identifier to an upper layer; The hardware rendering language is used to perform data rendering using the frame data, the data rendering comprising: Create raw data textures; Activate and bind the original data texture, and prepare the texture data for subsequent texture refresh; Using a texture refresh tool to share the frame data, generate a color space image for refreshing the texture data, and refresh the texture data; Sending the texture data to a graphics processor through a graphics processor raw channel interface; Accessing the data stored in the frame memory by using the frame memory identifier through a graphics processor, and using the texture data to generate an image for display by the application program; wherein, after generating an image display for one frame, releasing the frame memory identifier and the generated color space image in the texture refresh tool; After the image for display by the application is generated by the graphics processor, the next frame memory identifier is switched to, and the steps after the step of "creating the original data texture" in the data rendering process are repeated.

2. The video image display method according to claim 1, wherein: Parsing the video frame data includes: parsing the video frame data corresponding to the number of the frame memory identifier.

3. The video image display method as described in claim 2, wherein the number of frame memories is 10-20.

4. The video image display method according to claim 2, wherein: After data rendering is performed on the frame data in all frame stores, it is determined whether all frames of the video have been processed. When the determination result is no, the video image display method is continued to be performed on the unprocessed frames.

5. A video image display device, comprising: A frame memory application module is used to apply for one or more frame memories using an audio and video decoding tool, and convert each frame memory address into a frame memory identifier through a mapping tool. One frame memory identifier corresponds to the transmission of one frame of data. A parsing module, used to parse the video frame data using an audio and video decoding tool and store it in a corresponding frame memory; A transmission module is used to transmit the frame storage identifier to an application program for playing a video, and the application program calls a hardware rendering language, and the application program transmits the frame storage identifier to the hardware rendering language, so that the hardware rendering language knows the frame to be rendered; wherein, within the system, a bottom-layer driver module for display driving is mobilized to perform a frame storage identifier transmission operation to an upper layer; A rendering module, used to perform data rendering using the frame data through the hardware rendering language; the rendering module includes: Create unit, create raw data texture; Texture data preparation unit, activates and binds the original data texture, and prepares the texture data to facilitate subsequent texture refresh; A refresh unit, using a texture refresh tool to share the frame data, generate a color space image for refreshing the texture data, and refresh the texture data; A texture data sending unit; sending the texture data to a graphics processor through a graphics processor raw channel interface; An image generation unit, configured to generate an image for display by the application program using the texture data through a graphics processor; wherein after the image display for one frame is generated, the frame memory identifier and the generated color space image in the texture refresh tool are released; The rendering module is also used to: after generating an image for display by the application through a graphics processor, switch to the next frame memory identifier and repeat the actions performed by the texture data preparation unit, the refresh unit, the texture data sending unit and the image generation unit.

6. The video image display device according to claim 5, wherein: The parsing module parses the video frame data corresponding to the number of the frame memory identifier.

7. A computing device comprising: at least one processor and a memory storing program instructions; When the program instructions are read and executed by the processor, the computing device executes the video image display method according to any one of claims 1 to 4.

8. A readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, the computing device executes the video image display method according to any one of claims 1 to 4.

Images