Video processing method and device, and computer readable storage medium

CN119096546AActive Publication Date: 2024-12-06HUAWEI TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202280095226.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-06-30
Publication Date
2024-12-06
Estimated Expiration
2042-06-30

AI Technical Summary

Technical Problem

Smart terminal devices consume too much power when watching high-resolution, high-frame-rate videos for a long time, resulting in shortened battery life and affecting user experience.

Method used

By parsing the video code stream, image frames that are referenced by subsequent image frames are processed differently. Unreferenced frames are directly transmitted to the display module online, avoiding transfer through high-power-consuming off-chip memory. The referenced frames are stored in the off-chip memory for later use. Ensures stable output and enables diameter transfer in on-chip memory to reduce space requirements.

Benefits of technology

It effectively reduces the power consumption of terminal equipment, extends battery life, improves user viewing experience, and optimizes video data transmission efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119096546A_ABST
    Figure CN119096546A_ABST
Patent Text Reader

Abstract

The invention relates to a video processing method and device and a computer readable storage medium. The method comprises: analyzing a code stream of a video, and determining information indicating whether at least one image frame is referenced by a subsequent image frame (301); decoding (304) the code stream to obtain at least one image frame when the information indicates that the at least one image frame is not referenced by the subsequent image frame; and transmitting the at least one image frame to a display module to display the at least one image frame without storing the at least one image frame in the off-chip memory. According to the scheme, the power consumption of the processing equipment in the video processing process can be effectively reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Method, device and computer-readable storage medium for processing video Technical Field

[0001] The present disclosure relates to the technical field of video processing, and more particularly to a method, apparatus, and computer-readable storage medium for processing a video. Background Art

[0002] As the performance of smart devices such as mobile phones and tablets continues to improve, people can watch local videos or online videos on these devices anytime and anywhere. As video quality continues to improve, high-resolution, high-frame-rate, and high-bitrate videos continue to emerge. While this provides users with a better viewing experience, it also places higher demands on the performance of terminal devices.

[0003] Currently, due to limitations in device size and battery technology, terminal devices have very stringent requirements for power consumption and battery life. Prolonged video viewing can lead to rapid power consumption on smart terminal devices, shortening battery life and causing significant user anxiety. Therefore, reducing power consumption for video playback services and improving battery life are key technical issues in improving the user experience with smart terminal devices.

[0004] Summary of the Invention

[0005] In view of the above problems, at least in order to reduce the impact of video processing on the battery life of terminal devices, embodiments of the present disclosure provide a method, apparatus, and computer-readable storage medium for processing videos.

[0006] In a first aspect of the present disclosure, a method for processing a video is provided. The method includes: parsing a video code stream to determine information indicating whether at least one image frame is referenced by a subsequent image frame; when the information indicates that the at least one image frame is not referenced by the subsequent image frame, decoding the code stream to obtain the at least one image frame; and transmitting the at least one image frame to a display module to display the at least one image frame without storing the at least one image frame in an off-chip memory.

[0007] According to an embodiment of the present disclosure, image frames are processed differently based on the actual situation of whether the image frame being processed will be referenced by subsequent image frames. Image frames that will not be referenced by subsequent image frames can be directly transmitted online to the display module without the need for transit through an off-chip memory with slower reading and writing speeds and higher power consumption, thereby reducing the power consumption of the terminal device.

[0008] In one implementation of the first aspect, the method further includes: when information indicates that at least one image frame is referenced by a subsequent image frame, decoding the code stream to obtain the at least one image frame; transferring the at least one image frame to an off-chip memory; and reading the at least one image frame from the off-chip memory to a display module to display the at least one image frame. In this manner, for image frames that will be referenced by subsequent image frames, the larger off-chip memory can be used to ensure stable video output.

[0009] In an implementation of the first aspect, when the information indicates that the at least one image frame is not referenced by the subsequent image frame, the method further includes: pre-synchronizing the image frame and the audio before decoding the bitstream. In this way, a poor user experience caused by audio and image asynchrony can be avoided.

[0010] In an implementation of the first aspect, at least one image frame is stored in an on-chip memory without storing the at least one image frame in an off-chip memory. In this way, the on-chip memory can be used to achieve a complete transmission of video data.

[0011] In an implementation of the first aspect, the image frames are stored in the on-chip memory in the form of blocks or strips. Since the space of the on-chip memory is relatively limited, this method can reduce the space requirement of the on-chip memory.

[0012] In an implementation of the first aspect, the method further includes: synchronizing the image frame with the audio after storing the image frame in the off-chip memory. In this way, the user's viewing experience can be improved.

[0013] In a second aspect of the present disclosure, a device for processing video is provided. The device includes a video decoding module, a display module, and a central processing unit, wherein the central processing unit is configured to perform the following operations: parsing a video code stream to determine information indicating whether at least one image frame is referenced by a subsequent image frame; when the information indicates that the at least one image frame is not referenced by the subsequent image frame, decoding the code stream to obtain the at least one image frame; and transmitting the at least one image frame to the display module for display without storing the at least one image frame in an off-chip memory.

[0014] In one implementation of the second aspect, the device also includes an off-chip memory, wherein the central processing unit is configured to perform the following operations: when information indicates that at least one image frame is referenced by a subsequent image frame, decoding the code stream to obtain at least one image frame; transmitting the at least one image frame to the off-chip memory; and reading the at least one image frame from the off-chip memory to the display module to display the at least one image frame.

[0015] In an implementation of the second aspect, the central processing unit is configured to implement the following operations: when the information indicates that the at least one image frame is not referenced by the subsequent image frame, pre-synchronize the image frame and audio before decoding the code stream.

[0016] In an implementation of the second aspect, the apparatus further comprises an on-chip memory, wherein the central processing unit is configured to implement the following operation: at least one image frame is stored in the on-chip memory without storing the at least one image frame in an off-chip memory.

[0017] In an implementation of the second aspect, the image frames are stored in an on-chip memory in the form of blocks or strips.

[0018] In an implementation of the second aspect, the central processing unit is configured to implement the following operation: after storing the image frame in the off-chip memory, synchronizing the image frame and the audio.

[0019] In a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method of the first aspect of the present disclosure is implemented.

[0020] It should be understood that the contents described in the Summary of the Invention are not intended to limit the key or important features of the implementation of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the various implementations of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0022] FIG1 shows a schematic usage environment in which the method according to an embodiment of the present disclosure may be applied;

[0023] FIG2 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

[0024] FIG3 shows a schematic flow chart of a method for processing a video according to an embodiment of the present disclosure;

[0025] FIG4 shows a data flow diagram according to an embodiment of the present disclosure; and

[0026] 5A and 5B respectively show schematic diagrams of image frame blocking according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following describes implementations of the present disclosure in more detail with reference to the accompanying drawings. Although certain implementations of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the implementations described herein. Rather, these implementations are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and implementations of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0028] In the description of the implementations of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "an implementation" or "the implementation" should be understood as "at least one implementation". The terms "first", "second", etc. can refer to different or the same objects. The term "and / or" means at least one of the two items it associates. For example, "A and / or B" means A, B, or A and B. Other explicit and implicit definitions may also be included below.

[0029] It should be understood that the technical solutions provided by the implementation of this application may not be repeated in some of the following specific implementations, but these specific implementations should be considered as having been referenced to each other and can be combined with each other.

[0030] Some exemplary implementations of the present disclosure are described below with reference to the accompanying drawings.

[0031] FIG1 illustrates an exemplary application environment in which the methods according to the embodiments of the present disclosure can be applied. As shown, the methods according to the embodiments of the present disclosure can be used in a terminal device 100, such as a mobile phone. Users can play videos on the terminal device 100. Such videos can be stored locally on the terminal device 100 or played online using various types of networks. The embodiments of the present disclosure do not impose any particular restrictions on this. As previously mentioned, with the gradual increase in video resolution, frame rate, and bit rate, video playback and processing consume a significant amount of power on the terminal device 100, resulting in a shortened battery life of the terminal device 100. To at least address the aforementioned issues, the embodiments of the present disclosure provide a video processing solution. This solution can effectively reduce the power consumption of the terminal device 100 for video processing, thereby improving the user experience. It should be understood that although FIG1 illustrates the terminal device 100 as a mobile phone, this is merely illustrative and not restrictive. The terminal device 100 may also take other forms, such as a tablet computer, laptop computer, desktop computer, in-vehicle smart device, etc., and its specific form is not limited by the embodiments of the present disclosure.

[0032] In the context of the present disclosure, a video may be composed of a plurality of images arranged in a certain time sequence and audio associated with these images. These images may sometimes also be referred to as "frames" or "image frames". A video may contain a string of code streams (also referred to as code stream segments), each of which may correspond to an image frame. Taking a video with a frame rate of 60 frames per second as an example, it includes 60 frames of images arranged in a specific order within 1 second. It should be noted that the method of the embodiments of the present disclosure is applicable to processing videos of various frame rates and encoded based on different protocols, and there is no particular limitation on the specific frame rate and encoding protocol of the video to be processed.

[0033] A block diagram of an apparatus 200 according to an embodiment of the present disclosure is described below with reference to FIG2 . The apparatus 200 may be integrated into the terminal device 100 shown in FIG1 . As shown in FIG2 , the apparatus 200 may include a chip 210 and an off-chip memory 206 located outside the chip 210. As shown, the chip 210 includes a central processing unit (CPU) 201. Program code for executing the method according to the embodiment of the present disclosure may be stored in a non-volatile memory 208 outside the chip 210. The CPU 201 may communicate with the non-volatile memory 208, thereby reading the program code from the non-volatile memory 208 to execute each method step.

[0034] 2 , the central processing unit 201 can be coupled to the video decoding module 202, the display module 203 and the on-chip memory 205 integrated into the chip 210 via the bus 204. It should be noted that, in some embodiments, the central processing unit 201, the video decoding module 202, the display module 203 and the on-chip memory 205 can be integrated into the same chip 210 and coupled to each other via the bus 204. In other embodiments, the central processing unit 201, the video decoding module 202, the display module 203 and the on-chip memory 205 can be located on different chips. In other words, the chip 210 can be in the form of a system on chip (SoC) or other multi-chip combination forms. The embodiments of the present disclosure do not particularly limit the specific form of the chip 210, as long as it can achieve mutual transmission and communication between different modules on the device 200.

[0035] The following is a brief introduction to the various modules on the device 200. The central processing unit 201 can perform various appropriate actions and processes based on computer program instructions stored in a read-only memory or loaded into a random access memory. The video decoding module 202 can be configured to decode a standard compressed video stream. The display module 203 can be configured to overlay the various layers of image frames to be displayed on the screen of the terminal device 100, thereby displaying a video containing the various image frames on the screen. The on-chip memory 205 can be a cache or static random access memory (SRAM). The off-chip memory 206 can be a dynamic random access memory (DRAM). In comparison, although the on-chip memory 205 has a smaller storage space than the off-chip memory 206, its read and write speeds are much faster than those of the off-chip memory 206, and its read and write power consumption is also much lower than that of the off-chip memory 206. As shown in FIG. 2 , the on-chip memory 205 and the off-chip memory 206 may be coupled to the video decoding module 202 and the display module 203 via a bus 204 .

[0036] Referring now to FIG3 , a method 300 for processing a video according to an embodiment of the present disclosure is described. As shown in FIG3 , in block 301, one or more codestreams included in a video are parsed to determine information indicating whether the image frame corresponding to the codestream segment to be processed is referenced by subsequent image frames. It should be noted that this information determination is not required for all image frames included in a codestream; it can be performed for the image frames corresponding to a segment of the codestream. For a continuous video, the content reflected by the preceding and following image frames often has a strong correlation, especially when the scene changes slightly. Therefore, within the set of image frames that comprise the video, some image frames can serve as references for subsequent image frames during decoding. Of course, some image frames will not serve as references for subsequent image frames. The video codestream includes information indicating whether the image frame corresponding to the codestream segment to be processed in the video will be referenced by subsequent image frames. This information about the codestream segments in the video can be obtained based on software syntax parsing. It should be understood that in some embodiments, the image frame corresponding to the codestream segment to be processed may be referenced by a subsequent image frame, while in other embodiments, it may be referenced by multiple subsequent image frames. These image frames may be adjacent or separated by other image frames. In other words, the image frame to be processed may be referenced during the decoding process of subsequent image frames. The specific reference form is not particularly limited.

[0037] In block 302, different processing methods are determined for the video bitstream based on the information indicating whether the image frame needs to be referenced by subsequent image frames, as determined in block 301. Accordingly, corresponding processing methods can be applied to the bitstream based on the different determination results. If, as determined in block 302, the image corresponding to the bitstream segment being processed does not need to be referenced by subsequent image frames, method 300 proceeds to block 303. In block 303, the video can be pre-synchronized before decoding the bitstream. In some embodiments, the current frame sequence number and timestamp information can be reported to the audio and video synchronization module of device 200 to pre-synchronize the audio and image frames. Only when the audio and video synchronization point is reached does method 300 proceed to block 304, triggering the video decoding module 202 to perform the decoding process. In block 304, the bitstream can be decoded using various decoding methods to obtain image frames. The decoded image frames are then directly transmitted to the display module 203 via the on-chip memory 205 within chip 210 for display, as shown in block 309. Thus, the display of the image frame on the device 200 is completed.

[0038] FIG4 shows a data flow diagram 400 according to an embodiment of the present disclosure. In conjunction with FIG4 , it can be seen that according to an embodiment of the present disclosure, if it is determined that the image corresponding to the code stream segment being processed does not need to be referenced by subsequent image frames, then there is no need for the participation of the off-chip memory 206. Only the on-chip memory 205 of the chip 210 can be used to transmit the data decoded by the video decoding module 202 to the on-chip memory 205, and finally the display module 203 reads the data from the on-chip memory 205 to realize the display of the image frame. This method can be understood as online transmission. In this way, the on-chip memory 205 with faster reading and writing speed and lower power consumption can be fully utilized, thereby avoiding excessive power consumption of the terminal device 100 during the video processing process.

[0039] In some embodiments, if it is determined at block 302 that the image frame corresponding to the codestream segment being processed will be referenced by subsequent image frames during decoding, method 300 branches to block 306. At block 306, the codestream can be decoded using various decoding methods to obtain image frames. Subsequently, at block 307, the obtained image frames are transferred to off-chip memory 206 located outside chip 210. Furthermore, via off-chip memory 206, the image frames are transferred to display module 203 for display. In some embodiments, at block 308, before the image frames are transferred to display module 203 for display, video and audio synchronization can be performed to enhance the user's viewing experience.

[0040] According to an embodiment of the present disclosure, in conjunction with Figure 4 , if the image frame corresponding to the code stream segment being processed will be referenced by subsequent image frames during the parsing process, after the video decoding module 202 decodes the code stream being processed, the resulting image frame data is written to the off-chip memory 206. The display module 303 then reads the data from the off-chip memory 306 for display. This method can be understood as offline transmission. In this case, the larger storage space of the off-chip memory 306 is utilized to ensure normal video display.

[0041] Therefore, the subsequent hardware decoding method can be determined based on the information of the image frame corresponding to the code stream segment being processed. By making reasonable distinctions based on the actual situation of the code stream being processed, a balance can be achieved between low power consumption and stable video display.

[0042] In some embodiments, at block 304, when decoding the code stream, the image frame can be output in a certain manner. Figures 5A and 5B respectively illustrate schematic diagrams of dividing the output image frame according to embodiments of the present disclosure. In some embodiments, as shown in Figure 5A, the image frame 500 can be divided horizontally to form a plurality of strips 502-1, 502-2, ..., 502-N. In other embodiments, as shown in Figure 5B, the image frame 410 can be divided both horizontally and vertically to form a plurality of strips 512-1, 512-2, ..., 512-M. After decoding, the data of a strip 502-1, 502-2, ..., 502-N or a strip 512-1, 512-2, ..., 512-M is written directly to the on-chip memory 205 instead of being written to the off-chip memory 206, thereby reducing the bandwidth of the off-chip memory 206. In this manner, image frames that will not be referenced by subsequent image frames are partitioned at a relatively coarse granularity, since they are written to the smaller on-chip memory 205. This reduces the space required for the on-chip memory 205. It should be understood that while four strips are shown in FIG5A, this is merely illustrative and non-restrictive. Depending on the actual usage scenario, the image frame 500 may be partitioned into a greater or lesser number of strips. Similarly, the number of strips 512-1, 512-2, ..., 512-M shown in FIG5B is merely illustrative; other numbers of strips may be formed in other embodiments. Furthermore, in some embodiments, the image frame 500 may be partitioned only vertically. It should also be understood that such partitioning may be equally spaced or unequally spaced. That is, the strips 502-1, 502-2, ..., 502-N in FIG5A may be of the same size or of different sizes. Similarly, the sizes of the blocks 512-1, 512-2, ... 512-M in FIG5B can be the same or different. This application does not impose any limitation on this, as long as it can segment the image frame and thus reduce the video processing time.

[0043] In some embodiments, since the data of the blocks or strips decoded by the video decoding module 202 has already undergone audio and video pre-synchronization in block 303, it can now be directly transmitted to the display module 203 for display on the display of the terminal device 100. In some embodiments, when a predetermined number of blocks or strips are transmitted to the on-chip memory 205, the central processing unit 201 can notify the display module 203 so that the image frames in the on-chip memory 205 can be transmitted to the display module 203 in a timely manner. Referring back to FIG. 2 , a communication module 207 provided between the video decoding module 202 and the display module 203 can be used to achieve timely communication between the two, thereby effectively reducing communication latency. In some embodiments, after the video decoding module 202 finishes writing the data of a predetermined number of strips 512-1, 512-2, ..., 512-M or strips 502-1, 502-2, ..., 502-N to the on-chip memory 205, it can notify the display module 203 via the communication module 207 that the data of the predetermined number of strips 512-1, 512-2, ..., 512-M or strips 502-1, 502-2, ..., 502-N has been decoded, and inform the display module 203 of the storage address of the data. In this way, the display module 203 reads the data of the strips 512-1, 512-2, ..., 512-M or strips 502-1, 502-2, ..., 502-N in the on-chip memory 205 from the address, and then superimposes the image frame with the image frames of other layers and transmits the result to the display of the terminal device 100 for display. After the above-mentioned reading step is completed, the video decoding module 202 can also be informed via the communication module 207, so that the video decoding module 202 can reclaim the address to free up space on the on-chip memory 205 for subsequent decoding. Since the space of the on-chip memory 205 is usually very limited, it is impossible to store too much data. Using the method in the embodiment of the present disclosure, through direct communication between the video decoding module 202 and the display module 203, the space of the on-chip memory 205 can be released in time to ensure efficient data processing. As a result, the power consumption of the terminal device 100 can be further reduced.

[0044] In some embodiments, the predetermined number of blocks or strips can be one. This means that as long as the video decoding module 302 finishes writing the data of a block or strip to the on-chip memory 205, it can notify the display module 203 via the communication module 207 that the data of the block or strip has been decoded, and then read the data in the block or strip through the storage address for display, and once the reading of the block or strip is completed, the space on the on-chip memory 205 will be released for subsequent decoding. This can make data processing more efficient. It should be understood that, depending on the requirements for the endurance performance of the terminal device 100, the predetermined number can also be other numbers, such as two, three, or more.

[0045] In some embodiments, the communication module 207 may be a direct hardware connection or a hardware scheduling accelerator provided between the video decoding module 202 and the display module 203 .

[0046] In some embodiments, to ensure that decoded image frames are accurately and efficiently transmitted to display module 203, a synchronization signal can be used to enable display module 203 to control the operating rhythm of video decoding module 202, thereby ensuring that the operating rhythm of video decoding module 202 is consistent with that of display module 203. For example, if the refresh rate of the display screen of device 200 is 60 frames per second, display module 203 will notify video decoding module 202 via communication module 207 every 16.6 milliseconds (1000 / 60). In this case, if the video decoder module 202 decodes video at 30 frames per second, the decoding process is triggered every time two synchronization signals are received; if the video decoder module 202 decodes video at 60 frames per second, the decoding process is triggered every time one synchronization signal is received. In this way, the synchronous operation between video decoding module 202 and display module 203 can be ensured, thereby ensuring that the video can be correctly displayed on device 200. It should be understood that the numerical values ​​here are merely exemplary and not limiting.

[0047] The various processes and procedures described herein, such as method 300, may be performed by the central processing unit 201. For example, in some embodiments, method 300 may be implemented as a computer software program tangibly embodied in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 300 via a read-only memory and / or a communication unit. When the computer program is loaded into a random access memory and executed by the central processing unit 301, one or more actions of method 300 described above may be performed.

[0048] The present disclosure may be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium, which may be the non-volatile memory 208 described above, carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0049] Computer-readable storage media can be a tangible device that can hold and store instructions used by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. The computer-readable storage media used herein is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by a fiber optic cable), or an electrical signal transmitted by a wire.

[0050] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0051] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0052] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0053] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0054] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0055] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0056] Compared to existing solutions, according to the implementation method of the present disclosure, decoded image frames that do not need to be referenced are transmitted online to the display module 203 without being transferred through the off-chip memory 206, which can achieve the purpose of reducing the power consumption of the terminal device 100. For most videos, due to the need to save network transmission bandwidth or video storage space, a large number of frames that are not referenced are usually used. Therefore, the embodiments of the present disclosure have a very good performance. The more frames that are not referenced, the greater the power consumption benefits of the embodiments of the present disclosure.

[0057] It should be understood that the solution according to the embodiments of the present disclosure can also be applied to scenarios that require multi-address collaborative processing of video data, such as the online data transmission solution in the camera recording scenario, thereby reducing the power consumption of the camera device in the recording scenario.

[0058] It should also be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0059] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for processing a video, the method comprising: Parsing the code stream of the video to determine information indicating whether at least one image frame is referenced by a subsequent image frame; When the information indicates that the at least one image frame is not referenced by the subsequent image frame, Decoding the code stream to obtain the at least one image frame; as well as Without storing the at least one image frame in an off-chip memory, the at least one image frame is transmitted to a display module to display the at least one image frame.

2. The method according to claim 1, further comprising: When the information indicates that at least one image frame is referenced by the subsequent image frame, Decoding the code stream to obtain the at least one image frame; transferring the at least one image frame to an off-chip memory; as well as The at least one image frame is read from the off-chip memory to the display module to display the at least one image frame.

3. The method according to claim 1 or 2, wherein when the information indicates that the at least one image frame is not referenced by the subsequent image frame, the method further comprises: Before decoding the code stream, the image frame and audio are pre-synchronized.

4. The method according to any one of claims 1 to 3, Without storing the at least one image frame in the off-chip memory, the at least one image frame is stored in the on-chip memory. The method of claim 4 , wherein the image frames are stored in the on-chip memory in the form of blocks or strips.

6. The method according to claim 2, further comprising: After the image frame is stored in the off-chip memory, the image frame and audio are synchronized.

7. A video processing device, comprising a video decoding module, a display module, and a central processing unit, wherein the central processing unit is configured to implement the following operations: Parsing the code stream of the video to determine information indicating whether at least one image frame is referenced by a subsequent image frame; When the information indicates that the at least one image frame is not referenced by the subsequent image frame, decoding the code stream to obtain the at least one image frame; and Without storing the at least one image frame in an off-chip memory, the at least one image frame is transmitted to the display module to display the at least one image frame.

8. The apparatus according to claim 7, further comprising an off-chip memory, wherein the central processing unit is configured to implement the following operations: When the information indicates that at least one image frame is referenced by the subsequent image frame, Decoding the code stream to obtain the at least one image frame; transferring the at least one image frame to the off-chip memory; as well as The at least one image frame is read from the off-chip memory to the display module to display the at least one image frame.

9. The apparatus according to claim 7 or 8, wherein the central processing unit is configured to implement the following operation: when the information indicates that the at least one image frame is not referenced by the subsequent image frame, pre-synchronize the image frame and audio before decoding the code stream.

10. The apparatus according to any one of claims 7 to 9, further comprising an on-chip memory, wherein the central processing unit is configured to implement the following operations: Without storing the at least one image frame in the off-chip memory, the at least one image frame is stored in the on-chip memory. The apparatus according to claim 10 , wherein the image frames are stored in the on-chip memory in the form of blocks or strips.

12. The apparatus according to claim 8, wherein the central processing unit is configured to implement the following operations: After the image frame is stored in the off-chip memory, the image frame and audio are synchronized.

13. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Method for improving display efficiency in video decoding

    CN102103848A

  • Method and system of frame re-ordering for video coding

    CN107925777A

  • Methods and apparatuses for temporal synchronisation between the video bit stream and the output video sequence

    US20130208814A1

  • Multi-media reproduction processing apparatus

    US5542023A