Electronic device and method for managing buffer for image frames, and non-transitory computer-readable recording medium

Abstract

This electronic device comprises at least one processor, and a memory, which includes at least one storage medium that stores a decoder, a post-processing module, a renderer and instructions, and which stores the instructions. The decoder decodes a frame from among the plurality of frames of video stored in an input buffer, so as to generate a decoded frame. The post-processing module can generate a post-processed frame by post-processing the decoded frame. The renderer can render the decoded frame or the post-processed frame stored in an output buffer. When executed by the at least one processor individually or collectively, the instructions can cause the electronic device to: configure the output buffer on the basis of the time required for the post-processing; and adjust, on the basis of the speed at which the rendered frame is played back through a display, the speed at which the plurality of frames are input into the input buffer.

Description

Electronic device, method, and non-transitory computer-readable recording medium for managing a buffer for an image frame

[0001] The following descriptions relate to electronic devices, methods, and non-transitory computer-readable recording media for managing a buffer for image frames.

[0002] An electronic device can play a video. To play the video, the electronic device can decode and render frames of the video. The electronic device can provide one or more post-processing functions related to the video. The post-processing functions can include screen stabilization, frame interpolation, or scaling.

[0003] The above information may be provided as background information (related art) to aid in understanding this document. None of the above-described content is claimed as prior art related to this document or can be used to determine prior art.

[0004] An electronic device, a method, and a computer-readable recording medium for managing a buffer for an image frame are provided.

[0005] According to an aspect of the disclosure, an electronic device includes at least one processor, and a decoder, a post-processing module, a renderer, and one or more storage media storing instructions, and a memory configured to store instructions, wherein the decoder is configured to decode a frame among the plurality of frames of video stored in an input buffer to generate a decoded frame, the post-processing module is configured to post-process the decoded frame to generate a post-processed frame, and the renderer is configured to render the decoded frame stored in an output buffer, or the post-processed frame, wherein the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to set the output buffer based on a required time for the post-processing and to adjust a speed of inputting the plurality of frames to the input buffer based on a speed at which the rendered frame is played back through the display.

[0006] According to an aspect of the disclosure, an electronic device includes a memory including a decoder, a post-processing module, and one or more storage media storing a decoder, a post-processing module, and a renderer, the memory including the decoder, the memory including the decoder, the memory including the decoder, and the renderer ...

[0007] The above and other aspects, features, and advantages of specific embodiments of the disclosure will become apparent from the following description taken in conjunction with the accompanying drawings.

[0008] Figure 1 illustrates a block diagram of the order in which one or more frames included in an image are rendered.

[0009] Figure 2a illustrates a timing diagram regarding when a post-processed frame is rendered in response to a post-processing request.

[0010] Figure 2b illustrates a timing diagram regarding when a post-processed frame is rendered in response to a post-processing request.

[0011] FIG. 3 illustrates a block diagram of an electronic device according to one embodiment.

[0012] FIG. 4 illustrates a block diagram of an order in which an electronic device renders one or more frames included in an image, according to one embodiment.

[0013] FIG. 5A illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment.

[0014] FIG. 5b illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment.

[0015] FIG. 6A illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment.

[0016] FIG. 6b illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment.

[0017] FIG. 7 illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment.

[0018] FIG. 8A illustrates a block diagram of an order in which an electronic device renders one or more frames included in an image, according to one embodiment.

[0019] FIG. 8B illustrates a block diagram of an order in which an electronic device renders one or more frames included in an image, according to one embodiment.

[0020] FIG. 9 illustrates a flowchart of operations of an electronic device according to an embodiment.

[0021] FIG. 10 illustrates a flowchart of operations of an electronic device according to one embodiment.

[0022] FIG. 11 illustrates a flowchart of operations of an electronic device according to one embodiment.

[0023] FIG. 12 illustrates a flowchart of operations of an electronic device according to one embodiment.

[0024] FIG. 13 illustrates a flowchart of operations of an electronic device according to one embodiment.

[0025] FIG. 14 illustrates a flowchart of operations of an electronic device according to one embodiment.

[0026] FIG. 15 illustrates a flowchart of operations of an electronic device according to one embodiment.

[0027] FIG. 16 illustrates a flowchart of operations of an electronic device according to one embodiment.

[0028] FIG. 17 illustrates a flowchart of operations of an electronic device according to one embodiment.

[0029] FIG. 18 illustrates a flowchart of operations of an electronic device according to one embodiment.

[0030] FIG. 19 illustrates a block diagram of an electronic device within a network environment according to one or more embodiments.

[0031] The terminology used in this disclosure is not intended to limit the scope of other embodiments, but is provided solely to describe specific embodiments. Unless the context clearly dictates otherwise, singular forms include plural references. Terms and words, including technical or scientific terms, used herein have the same meanings as commonly understood by those skilled in the art. Terms commonly defined in dictionaries may be interpreted as having the same or similar meanings as the contextual meanings of the relevant art. Unless otherwise defined, terms should not be construed as idealistic or overly formal. Even if a term is defined in this disclosure, the term should not be construed to exclude embodiments of the disclosure depending on the circumstances.

[0032] Before embarking on the detailed description below, it may be helpful to provide definitions of certain words and phrases used throughout the disclosure. The term "combine" and its derivatives refer to any direct or indirect communication between two or more elements, whether in physical contact with each other. The terms "transmit," "receive," and "communicate," and their derivatives, encompass both direct and indirect communication. The terms "include," "comprise," and "comprises," and their derivatives, refer to an unlimited inclusion. The term "or" is an inclusive term meaning "and / or." The phrase "associated with" and its derivatives refer to include, be included in, interconnect, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be in proximity to, be bound to or with, have, have characteristics of, have a relationship to or with, etc. The term "controller" refers to any device, system, or part thereof that controls at least one operation. The functionality associated with any particular controller may be centralized or distributed, local or remote. The phrase "at least one" when used with a list of items means one or more different combinations of the listed items, and as may be required, only one item in the list. For example, "at least one of A, B, and C" includes any one of the following combinations: A, B, C, A and B, A and C, B and C, A and B and C and any variations thereof.As a further example, the expression "at least one of a, b, or c" can refer to only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof. Similarly, the term "set" means one or more. Thus, a set of items can be a single item or a collection of two or more items.

[0033] Additionally, various functions described below may be implemented or supported by one or more computer programs, each of which is formed of computer-readable program code and implemented on a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, sets of instructions, procedures, functions, objects, classes, instances, related data, or portions thereof adapted for implementation in suitable computer-readable program code. The phrase "computer-readable program code" includes any type of computer code, including source code, object code, and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as Read Only Memory (ROM), Random Access Memory (RAM), a hard disk drive, a Compact Disc (CD), a Digital Video Disc (DVD), or any other type of memory. A "non-transitory" computer-readable medium excludes wired, wireless, optical, or other communication links that transport transitory electrical or other signals. A non-transitory computer-readable medium includes media capable of storing data permanently and media that can be stored and later overwritten (e.g., a rewritable optical disc or an erasable memory device).

[0034] The term "module" as used in connection with one or more embodiments of the disclosure may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as logic, logic block, unit, portion, part, or circuit. A module may be a single integrated component or minimal unit or portion thereof adapted to perform one or more functions. For example, according to an embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0035] Figure 1 illustrates a block diagram of the rendering order of one or more frames included in an image. Figure 2a illustrates a timing diagram of the time at which a post-processed frame is rendered in response to a post-processing request. Figure 2b illustrates a timing diagram of the time at which a post-processed frame is rendered in response to a post-processing request.

[0036] Referring to FIG. 1, one or more frames included in a video may be sequentially stored in an input buffer (20). In one embodiment, a demuxer (15) may perform demuxing of a (media) file for a video. In one embodiment, the demuxer (15) may store a frame to be played from a currently playing track of the video in an input buffer (20). The input buffer (20) may be accessible to a decoder (25). The decoder (25) may decode one or more frames stored in the input buffer (20).

[0037] According to one embodiment, the decoder (25) can pass the decoded frame to the post-processing module (30) when post-processing is activated. The post-processing module (30) can post-process the decoded frame according to the requested post-processing function.

[0038] According to one embodiment, the decoder (25) can store the decoded frame in the output buffer (35) when post-processing is disabled. The output buffer (35) can store a specified number of decoded frames. The output buffer (35) can be accessed by the renderer (40). The renderer (40) can render the frame stored in the output buffer (35).

[0039] In the above-described order, when one or more frames included in an image are processed, the processing speed of the decoder (25) is faster than the speed at which the frames are rendered by the renderer (40) and then output through the display. Accordingly, all storage space of the output buffer (35) can be filled with frames produced (or decoded) by the decoder (25).

[0040] In the above-described order, while one or more frames included in the video are being processed (or while the video is being played back), the change between activating or deactivating post-processing can be handled while maintaining the video's playback.

[0041] In Fig. 2a, when post-processing is requested at time point (t0), the post-processing module (30) may be loaded (or initialized) (or set) during a designated time interval (50). During this time interval (50), decoded frames may be rendered without post-processing. Thereafter, when the post-processing module (30) is loaded (or initialized) (or set) at time point (t1), the post-processing module (30) may sequentially post-process the frames decoded by the decoder (25). That is, when the post-processing module (30) is loaded (or initialized) (or set) at time point (t1), the frames stored in the input buffer (20) may be decoded and post-processed. Accordingly, post-processed frames may not be played during a time interval (60) in which all decoded frames already stored in the output buffer (35) are played. Accordingly, there may be a time difference between the time at which post-processing is requested (t0) and the time at which the post-processed frame is rendered (t2).

[0042] In one embodiment, when post-processing is requested, it may include a case where conversion for post-processing (e.g., frame interpolation according to frame rate conversion (FRC), frame shake correction, playback speed change, resolution change, frame image filter, and / or deflicker) and / or correction processing is requested.

[0043] In some embodiments, while one or more frames included in a video are being processed (or during video playback), a change between activating or deactivating post-processing may be processed while playback of the video is paused, and playback may then be initiated.

[0044] For example, referring to FIG. 2B, if post-processing is requested at time point (t0), playback of the video may be stopped. In addition, if post-processing is requested at time point (t0), loading (or initialization) (or setting) of the post-processing module (30) may be performed during a designated time interval (70). In addition, if post-processing is requested at time point (t0), seeking for a frame to be post-processed may be performed during a designated time interval (80). For example, the seeking operation during the time interval (80) may include decoding of frames necessary for decoding of a frame for which post-processing is requested, and dropping (or deleting) (or removing) post-processed frames that are not related to post-processing of the frame for which post-processing is requested among the decoded frames. Here, the frames required for decoding the frame for which post-processing is requested may be an instantaneous decoder refresh (IDR) frame (or an intra-coded (I) frame), a prediction (P) frame, and / or a bi-directional prediction (B) frame. Here, the frames required for decoding the frame for which post-processing is requested may be frames that are decoded before the frame for which post-processing is requested among the frames in a group of pictures (GOP). In addition, when post-processing is requested at time point (t0), the decoded frames stored in the output buffer (35) may be removed (or deleted) (or dropped).

[0045] After the post-processing module (30) has completed loading (t1), post-processing may not be performed until the search is completed. After the search is completed and the frame to be post-processed has been decoded, the post-processing module (30) may post-process the decoded frame during a time interval (90), and the renderer (40) may perform rendering. Accordingly, when a search is required, there may be a time difference between the time (t0) at which post-processing is requested and the time (t2) at which the post-processed frame is rendered. The time required for a search operation to a location requested by a user may include the time required for decoding from a previous synchronous frame (e.g., an I-frame) to the frame at the location requested by the user and for dropping the decoded frame. Accordingly, the time required for decoding to a frame at the location requested by the user may vary depending on the location of the previous synchronous frame.

[0046] Accordingly, when post-processing is activated (or deactivated) while maintaining video playback as in Fig. 2a, the response time for post-processing may be delayed until the exhaustion time of the decoded frames previously stored in the output buffer (35) in addition to the time interval (50) for post-processing. Accordingly, a timestamp difference may exist between the frames displayed on the display when the user requests post-processing and the frames actually displayed on the display after post-processing is applied.

[0047] Additionally, when post-processing is activated (or deactivated) while stopping playback of the video as in FIG. 2b, playback may be stopped by a time interval (70) for post-processing, a time interval (80) for searching, and / or a time interval (90).

[0048] Therefore, high responsiveness to post-processing and / or seamless image playback may be required. Below, the configuration and / or operation of an electronic device for high responsiveness to post-processing and / or seamless image playback is described.

[0049] FIG. 3 illustrates a block diagram of an electronic device according to an embodiment. In an embodiment, the electronic device (101) of FIG. 3 may correspond to the electronic device (101) of FIG. 19. Referring to FIG. 3, the electronic device (101) may include a processor (120), a memory (130), and a display (260).

[0050] In one embodiment, the processor (120) may be used to execute the operations of the electronic device (101) exemplified in the descriptions of FIGS. 9 to 18. For example, the processor (120) may include at least a portion of the processor (120) of FIG. 19 or may correspond to at least a portion of the processor (120) of FIG. 19. For example, the processor (120) may include one or more processors, including an application processor (AP) and / or a communication processor (CP). For example, the processor (120) may be implemented as a single chip, such as a system on chip (SoC), or may be implemented as multiple chips. For example, the processor (120) may be implemented as a single integrated circuit or may be implemented as multiple integrated circuits. For example, the processor (120) may be distributedly arranged within the electronic device (101).

[0051] In one embodiment, the memory (130) may (at least temporarily) store instructions for executing operations of the electronic device (101) exemplified in the descriptions of FIGS. 9 to 18. The instructions may be executed by the processor (120). The instructions may be included in one or more programs (140) stored in the memory (130) (e.g., a playback module (310), a demuxer (320), a decoder (330), a post-processing module control unit (340), a post-processing module (350), and a renderer (360)). For example, the memory (130) may include at least a portion of the memory (130) of FIG. 19 (or at least a portion of the non-volatile memory (134)) or may correspond to at least a portion of the memory (130) of FIG. 19 (or at least a portion of the non-volatile memory (134)). For example, the memory (130) may include a main memory (e.g., a random access memory (RAM)) within the electronic device (101), a register for the processor (120), a cache for the processor (120), a communication circuit (e.g., a register for the communication module (190) of FIG. 19), a buffer (or soft buffer) for the communication module (190) of FIG. 19), and / or an auxiliary memory (e.g., a hard disk drive (HDD), a solid state drive (SSD)) of the electronic device (101). For example, the memory (130) may be implemented as a single chip or may be implemented as multiple chips. For example, the memory (130) may be implemented as one integrated circuit or may be implemented as multiple integrated circuits. For example, the memory (130) may be distributedly arranged within the electronic device (101).

[0052] In one embodiment, each of the playback module (310), the demuxer (320), the codec unit (331), the post-processing module control unit (340), the post-processing module (350), and the renderer (360) may include instructions. In one embodiment, each of the playback module (310), the demuxer (320), the decoder (330), the post-processing module control unit (340), the post-processing module (350), and the renderer (360) may be executed by the processor (120). In one embodiment, the codec unit (331) may include a decoder (330) and an encoder (335).

[0053] In one embodiment, the playback module (310) can control the operation of the demuxer (320), the codec unit (331), the post-processing module control unit (340), the post-processing module (350), and the renderer (360). In one embodiment, the playback module (310) can control the playback of a video (or image) by controlling the operation of the demuxer (320), the codec unit (331), the post-processing module control unit (340), the post-processing module (350), and the renderer (360). For example, the playback module (310) can select a first path (or a path with post-processing disabled) or a second path (or a path with post-processing enabled) for playing a plurality of frames included in the video (or image). For example, the playback module (310) may control the decoder (330) and / or the renderer (360) to render frames decoded through the decoder (330) of the codec unit (331) through the renderer (360) based on selecting the first path. For example, the playback module (310) may control the decoder (330), the post-processing module (350) and / or the renderer (360) to render frames decoded through the decoder (330) of the codec unit (331) through the post-processing module (350) after being post-processed through the post-processing module (350) based on selecting the second path. The first path (or the path with post-processing disabled) or the second path (or the path with post-processing enabled) may be selected (or changed) based on the input. Hereinafter, operations when changing from the first path (or the path with post-processing disabled) to the second path (or the path with post-processing enabled) can be described with reference to FIGS. 5A, 5B, 6A, and 6B. Operations when changing from the second path (or the path with post-processing enabled) to the first path (or the path with post-processing disabled) can be described with reference to FIG. 7. In one embodiment, the encoder (335) can encode an image (or frames of a video).

[0054] In one embodiment, the playback module (310) may obtain input related to video playback. For example, the playback module (310) may obtain input for playing or stopping the video. For example, the playback module (310) may obtain input for adjusting the video playback speed.

[0055] For example, the playback module (310) may obtain an input for applying (or activating) or deactivating (or disabling) post-processing of the video. For example, the playback module (310) may obtain an input for changing the playback position of the video (or an input for seeking). For example, the playback module (310) may obtain an input for settings related to video processing. In one embodiment, the settings related to video processing may be related to one or more threshold values. For example, the one or more threshold values ​​may be values ​​for determining whether to stop playback of the video based on an input related to video playback.

[0056] For example, one or more threshold values ​​may represent a reference threshold value (or reference time) for the search time. For example, one or more threshold values ​​may represent a reference threshold value (or reference timestamp difference) for a timestamp difference. In one embodiment, the timestamp difference may be the timestamp difference between a frame for which a post-processing request (application) change is requested based on the input and the earliest frame for which the post-processing request (application) has been changed.

[0057] In one embodiment, the playback module (310) may process input related to video playback. In one embodiment, the playback module (310) may transmit the input related to video playback to at least one of the demuxer (320), the decoder (330), the post-processing module control unit (340), the post-processing module (350), or the renderer (360) so that the input is processed. For example, in the case of an input for applying or de-applying post-processing to a video, the playback module (310) may transmit the input to the post-processing module control unit (340) and / or the post-processing module (350). Hereinafter, operations of the playback module (310) for processing input related to video playback may be described with reference to FIGS. 9 to 16.

[0058] In one embodiment, the demuxer (320) can perform demuxing of a (media) file for a video. In one embodiment, the demuxer (320) can identify the location of a specified (image) frame in the video. In one embodiment, the demuxer (320) can identify the location of a specified frame (e.g., a sync frame) in a currently playing track of the video. In one embodiment, the demuxer (320) can extract a frame to be played from a currently playing track of the video. In one embodiment, the specified frame can be an instantaneous decoder refresh (IDR) frame. In one embodiment, an IDR frame can be decoded without reference to other frames. In one embodiment, frames between an IDR frame and a P (prediction) frame preceding the next IDR frame can constitute one GOP (group of pictures). In one embodiment, a P frame can be decoded with reference to a preceding P frame and / or an IDR frame. In one embodiment, a P frame can be decoded with reference to a preceding P frame and / or an IDR frame within its GOP, but is not limited thereto. For example, frames of a video may include a B (bi-directional prediction) frame. In one embodiment, a B frame can be decoded with reference to a subsequent P frame and / or a preceding IDR frame. In one embodiment, a B frame can be decoded with reference to a subsequent B or P frame and / or a preceding IDR frame within its GOP.

[0059] In one embodiment, the demuxer (320) may provide a frame to be played from a currently playing track of a video to the decoder (330). In one embodiment, the demuxer (320) may store a frame to be played from a currently playing track of a video in an input buffer (410 of FIG. 4) allocated to the decoder (330).

[0060] In one embodiment, the decoder (330) can decode frames and / or audio according to a specified codec. In one embodiment, the decoder (330) can provide the decoded frames according to the specified codec to the post-processing module (350) and / or the renderer (360). In one embodiment, the decoder (330) can store the decoded frames according to the specified codec in a backup buffer (430 of FIG. 4) allocated to the post-processing module (350) and / or an output buffer (450 of FIG. 4) allocated to the renderer (360).

[0061] In one embodiment, the post-processing module control unit (340) may load the post-processing module (350) into the memory (130) (or volatile memory of the memory (130)). In one embodiment, the post-processing module control unit (340) may initialize and / or configure the post-processing module (350). In one embodiment, the configuration of the post-processing module (350) may include loading the post-processing module (350) into the memory (130) (or volatile memory of the memory (130)) based on configuration values ​​for processing the requested post-processing. In one embodiment, the configuration values ​​may include configuration values ​​for frame interpolation according to frame rate conversion (FRC), frame shake correction, playback speed change, resolution change, and / or image filter of the frame, but are not limited thereto.

[0062] In one embodiment, the post-processing module control unit (340) can manage a backup buffer (430 in FIG. 4) allocated to the post-processing module (350).

[0063] In one embodiment, the post-processing module (350) may post-process a frame according to a requested post-processing function. In one embodiment, the post-processing function may include frame interpolation according to frame rate conversion (FRC), frame shake correction, playback speed change, resolution change, and / or image filtering of the frame.

[0064] In one embodiment, the decoder (330) may provide the post-processed frame to the renderer (360). In one embodiment, the decoder (330) may store the post-processed frame in an output buffer (450 of FIG. 4) allocated to the renderer (360). In one embodiment, the renderer (360) may render the frame stored in the output buffer (450 of FIG. 4).

[0065] In one embodiment, the display (260) can display visual content. In one embodiment, the display (260) can display a rendered frame. For example, the display (260) can include at least a portion of the display module (160) of FIG. 19 or correspond to at least a portion of the display module (160) of FIG. 19.

[0066] According to one embodiment, at least one module among the playback module (310), the demuxer (320), the decoder (330), the post-processing module control unit (340), the post-processing module (350), or the renderer (360) may be configured as hardware (e.g., a processor, an integrated circuit (IC) chip, a graphic processing unit (GPU)). However, the present invention is not limited thereto. At least one module among the playback module (310), the demuxer (320), the decoder (330), the post-processing module control unit (340), the post-processing module (350), or the renderer (360) may be implemented as a software module (or a program), one or more hardware blocks / components, or a combination thereof.

[0067] FIG. 4 illustrates a block diagram of a sequence in which an electronic device renders one or more frames included in an image, according to one embodiment. FIG. 3 may be referred to for the description of FIG. 4.

[0068] Referring to FIG. 4, the input buffer (410) can store a specified first number of frames (415). In one embodiment, the input buffer (410) can be accessed by the decoder (330). In one embodiment, the specified first number of frames (415) stored in the input buffer (410) can be accessed (or decoded) by the decoder (330).

[0069] In one embodiment, the first number may be changed by the playback module (310). For example, the size (or first number) of the input buffer (410) may be changed by the playback module (310). For example, the playback module (310) may adjust the size (or first number) of the input buffer (410) based on the decoding speed of the decoder (330) and / or the rendering speed of the renderer (360). For example, the playback module (310) may adjust the size (or first number) of the input buffer (410) based on the playback speed of the video (e.g., playback time magnification, or playback FPS (frames per second)). However, the present invention is not limited thereto. The change of the first number may be performed by a module other than the playback module (310). For example, the change of the first number may be performed by the post-processing module control unit (340).

[0070] In one embodiment, the playback module (310) may store a first number of frames in the input buffer (410) based on a decoding speed of the decoder (330) and / or a speed at which the renderer (360) reads (or renders) frames from the output buffer (450). For example, the playback module (310) may set the first number of frames stored in the input buffer (410) to be less than a third number of frames stored in the output buffer (450) if the decoding speed of the decoder (330) is faster than a speed at which the renderer (360) reads (or renders) frames from the output buffer (450).

[0071] In one embodiment, the backup buffer (430) can store a specified second number of frames (435). In one embodiment, the backup buffer (430) can be accessed by the post-processing module (350). In one embodiment, the specified second number of frames (435) stored in the backup buffer (430) can be accessed (or post-processed) by the post-processing module (350). Depending on the embodiment, the backup buffer (430) may not be provided to the post-processing module (350). In this case, the backup buffer (430) can perform post-processing without storing the decoded frames obtained from the decoder (330) in the backup buffer (430).

[0072] In one embodiment, the second number can be changed by the playback module (310) and / or the post-processing module control unit (340). For example, the size (or second number) of the backup buffer (430) can be changed by the playback module (310) and / or the post-processing module control unit (340). For example, the playback module (310) and / or the post-processing module control unit (340) can adjust the size (or second number) of the backup buffer (430) based on the number of frames required for the post-processing function. For example, the playback module (310) and / or the post-processing module control unit (340) can adjust the size (or second number) of the backup buffer (430) so that fewer frames can be stored than the number of frames required for the post-processing function. In one embodiment, when the number of frames required for the post-processing function is four, the size (or second number) of the backup buffer (430) may be a size in which three frames can be stored. However, this is not limited thereto. The change in the second number may be performed by a module (or computer code) other than the playback module (310). For example, the change in the second number may be performed by the post-processing module control unit (340).

[0073] In one embodiment, the second number may be set within a size that can be allocated to the output buffer (450) for the size of the backup buffer (430). In one embodiment, the second number may be set within a size of an area that is not allocated to the output buffer (450) among the areas that can be allocated to the output buffer (450). However, the present invention is not limited thereto.

[0074] In one embodiment, the output buffer (450) can store a specified third number of frames (455). In one embodiment, the output buffer (450) can be accessed by the renderer (360). In one embodiment, the specified third number of frames (455) stored in the output buffer (450) can be accessed (or rendered) by the renderer (360).

[0075] In one embodiment, the third number can be changed by the playback module (310). For example, the size (or third number) of the output buffer (450) can be changed by the playback module (310). For example, the playback module (310) can adjust the size (or third number) of the output buffer (450) based on the first required time for preparation of the post-processing module (350). For example, the playback module (310) can adjust the size (or third number) of the output buffer (450) based on the second required time for post-processing of the post-processing module (350). However, the present invention is not limited thereto. The change in the third number can be performed by a module other than the playback module (310). For example, the change in the third number can be performed by the post-processing module control unit (340).

[0076] For example, the playback module (310) can adjust the size (or the third number) of the output buffer (450) based on the rendering speed of the renderer (360). For example, the playback module (310) can adjust the size (or the third number) of the output buffer (450) based on the playback speed (e.g., playback time multiplier, or playback FPS) of the video. For example, the playback module (310) can allocate to the output buffer (450) a size equal to the number based on the following mathematical expression 1.

[0077]

[0078] In Equation 1, N may represent a third number. In Equation 1, D1 may represent a first required time. In Equation 1, D2 may represent a second required time. For example, D1 may include the time required for initialization or configuration changes to prepare for post-processing. For example, D2 may include the time required until the first frame to which post-processing is applied is generated. N may represent the number of frames corresponding to the minimum size of the output buffer (450).

[0079] In mathematical expression 1, FPS may represent the number of frames played per second through the display (260) (or the number of frames rendered per second through the renderer (360). In one embodiment, the playback module (310) may allocate to the output buffer (450) a value greater than or equal to the product of the size (or capacity) of each frame by N. Hereinafter, operations for allocating the backup buffer (430) and / or the output buffer (450) may be described with reference to FIGS. 11 and 18.

[0080] In one embodiment, the playback module (310) may select (or change) a first path (or a path with post-processing disabled) or a second path (or a path with post-processing enabled) for playing a plurality of frames included in a video (or image) based on an input related to video playback. In one embodiment, through the first path, frames are processed in the order of an input buffer (410), a decoder (330), an output buffer (450), and a renderer (360). In one embodiment, through the second path, frames are processed in the order of an input buffer (410), a decoder (330), a post-processing module (350), an output buffer (450), and a renderer (360).

[0081] In one embodiment, the playback module (310) (or the post-processing module control unit (340) (or the processor (120) (or the electronic device (101)) may determine whether to stop playback of the video when changing between the first path (or the path with post-processing disabled) or the second path (or the path with post-processing enabled). In the following, the operations performed by the playback module (310) may be performed by the post-processing module control unit (340), the processor (120), and the electronic device (101).

[0082] In one embodiment, the playback module (310) may determine whether to stop playback of the video based on an input requesting a change between the first path or the second path. For example, the playback module (310) may determine whether to stop playback of the video based on a post-processing function requested by the input. For example, if a post-processing function requested for changing from the first path to the second path requires application to the requested frame for post-processing (e.g., frame interpolation according to frame rate conversion (FRC) or an image filter of the frame), the playback module (310) may determine to stop playback of the video. For example, if a post-processing function requested for changing from the first path to the second path does not require application to the requested frame for post-processing (e.g., frame shake correction, change in playback speed, or change in resolution), the playback module (310) may determine not to stop playback of the video. For example, if the FPS drops below the reference FPS due to a post-processing function requested for changing from the first path to the second path, the playback module (310) may decide to stop playing the video.

[0083] In one embodiment, the playback module (310) may determine whether to stop playback of the video based on one or more threshold values. For example, the one or more threshold values ​​may represent a reference threshold value (or reference time) for the seek time. For example, the one or more threshold values ​​may represent a reference threshold value (or reference timestamp difference) for a timestamp difference. In one embodiment, the timestamp difference may be between a frame for which a change in a post-processing request (application) is requested and the earliest frame for which a change in the post-processing request (application) has been requested. According to one embodiment, the playback module (310) may select (or switch) a post-processing application method based on content characteristics (e.g., I-frame interval) and / or application characteristics (e.g., a threshold value specified for the application).

[0084] In one embodiment, if the expected seek time is less than the reference seek time in response to a request for a post-processing function to change from the first path to the second path, the playback module (310) may determine to stop playback of the video. In one embodiment, if the expected seek time is greater than or equal to the reference seek time, the playback module (310) may determine not to stop playback of the video. In one embodiment, the expected seek time may include the time required to decode, drop, and / or post-process frames required to post-process the first frame for which post-processing is requested.

[0085] In one embodiment, if a difference between a timestamp of the earliest frame to which a post-processing function is applied and a reference timestamp is greater than or equal to the reference timestamp difference in response to a request for a post-processing function to change from a first path to a second path, the playback module (310) may determine to stop playback of the video. In one embodiment, if a difference between a timestamp of the earliest frame to which a post-processing function is applied and a reference timestamp is less than or equal to the reference timestamp difference in response to a request for a post-processing function to change from a first path to a second path, the playback module (310) may determine not to stop playback of the video.

[0086] In one embodiment, the playback module (310) may stop playback of the video based on a determination that the video playback is to be stopped. In one embodiment, the playback module (310) may perform post-processing (or operations for setting post-processing) while the video playback is stopped. Hereinafter, an operation of performing post-processing while the video playback is stopped may be described with reference to FIGS. 6A and 6B.

[0087] In one embodiment, the playback module (310) may drop (or delete) (or remove) decoded frames stored in the output buffer (450) while playback of the video is paused.

[0088] In one embodiment, the playback module (310) may load the post-processing module (350) into the memory (130) (or the volatile memory of the memory (130)) while the video playback is paused. In one embodiment, the playback module (310) may initialize (or configure) the post-processing module (350) while the video playback is paused. In one embodiment, initializing (or configuring) the post-processing module (350) may include loading the post-processing module (350) into the memory (130) (or the volatile memory of the memory (130)) based on setting values ​​for processing the requested post-processing. In one embodiment, the setting values ​​may include, but are not limited to, setting values ​​for frame interpolation according to frame rate conversion (FRC), frame shake correction, playback speed change, resolution change, and / or image filter of the frame. In some embodiments, if the post-processing module (350) is already loaded, loading of the post-processing module (350) may be omitted.

[0089] In one embodiment, the playback module (310) may perform a seek for a frame to be post-processed while playback of the video is paused. In one embodiment, the seek operation may include an operation of identifying a post-processing requested frame (or a first frame for which post-processing is requested) indicated by the input. In one embodiment, the seek operation may include an operation of decoding one or more frames for decoding the post-processing requested frame (or the first frame for which post-processing is requested) through the decoder (330). In one embodiment, the seek operation may include an operation of dropping (or deleting) (or removing) decoded frames that are not related to playback (or not requested for post-processing) among the one or more frames decoded through the decoder (330). Here, the frames required for decoding the post-processing requested frame may be IDR frames, P frames, and / or B frames. Here, the frames required for decoding the post-processing-requested frame may be frames that are decoded before the post-processing-requested frame among the frames within the GOP. Here, the decoded frames being dropped (or deleted) (or removed) may include that the frames decoded by the decoder (330) are not stored in the output buffer (450).

[0090] In one embodiment, the playback module (310) can post-process the decoded frame through the post-processing module (350). In one embodiment, the playback module (310) can resume video playback based on the post-processing of the decoded frame through the post-processing module (350). For example, the playback module (310) can store the post-processed frame through the post-processing module (350) in the output buffer (450). For example, the playback module (310) can render the post-processed frame stored in the output buffer (450) through the post-processing module (350). For example, the playback module (310) can display the rendered frame through the display (260).

[0091] In one embodiment, the playback module (310) may perform post-processing (or operations for setting post-processing) during playback of the video based on a determination that the video playback will not be stopped. Here, the operation of performing post-processing (or operations for setting post-processing) during playback of the video may be performed in parallel with the operation of rendering decoded frames stored in the output buffer (450) through the renderer (360) and displaying the rendered frames through the display (260). Hereinafter, the operation of performing post-processing while the video playback is maintained may be described with reference to FIGS. 5A and 5B.

[0092] In one embodiment, the post-processing module control unit (340) may load the post-processing module (350) into the memory (130) (or the volatile memory of the memory (130)) during playback of the video. In one embodiment, the post-processing module control unit (340) may initialize (or set) the post-processing module (350) during playback of the video. In one embodiment, the initialization (or setting) of the post-processing module (350) may include loading the post-processing module (350) into the memory (130) (or the volatile memory of the memory (130)) based on setting values ​​for processing the requested post-processing. In one embodiment, the setting values ​​may include setting values ​​for frame interpolation according to frame rate conversion (FRC), frame shake correction, playback speed change, resolution change, and / or image filter of the frame, but are not limited thereto. In some embodiments, if the post-processing module (350) is already loaded, loading of the post-processing module (350) may be omitted.

[0093] In one embodiment, the playback module (310) (or the post-processing module control unit (340)) can post-process frames sequentially decoded by the decoder (330) through the post-processing module (350). In one embodiment, the first post-processed frame can be rendered by the renderer (360) after all decoded frames stored in the output buffer (450) have been played back. In one embodiment, the playback module (310) (or the renderer (360)) can play back the post-processed frame based on the post-processing of the decoded frame through the post-processing module (350). For example, the playback module (310) (or the post-processing module control unit (340)) can store the post-processed frame through the post-processing module (350) in the output buffer (450). For example, the playback module (310) (or renderer (360)) can render the post-processed frame stored in the output buffer (450) through the decoder (330). For example, the playback module (310) (or renderer (360)) can display the rendered frame through the display (260).

[0094] As described above, the electronic device (101) can manage the size of the output buffer (450) to the minimum size required for the preparation time for post-processing. Accordingly, in response to a post-processing request, the post-processed frames generated after the frames previously stored in the output buffer (450) are output can be output through the display (260).

[0095] Additionally, the electronic device (101) can minimize the response to post-processing or maintain uninterrupted playback by determining whether to stop playback based on the type of post-processing function, the time required for post-processing, and / or the timestamp of the first frame on which post-processing is performed.

[0096] Hereinafter, an operation for performing post-processing while video playback is maintained is described with reference to FIGS. 5A and 5B. FIG. 5A illustrates an operation for performing post-processing when a backup buffer (430) exists, and FIG. 5B illustrates an operation for performing post-processing when a backup buffer (430) does not exist.

[0097] FIG. 5A illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment. For the description of FIG. 5A, reference may be made to FIGS. 3 and 4.

[0098] Referring to FIG. 5A, a point in time (501) may be a point in time (510) when a renderer (360) renders a frame (1) and / or a point in time (540) when a decoder (330) decodes a frame (5). In one embodiment, a point in time (501) may be a point in time when frames (1 to 4) among frames (1 to 16) of a video (500) are stored in an output buffer (520). At a point in time (501), the frames (1 to 4) stored in the output buffer (520) may be frames decoded by the decoder (330). At a point in time (501), the frames (2 to 4) decoded by the decoder (330) may be stored in a backup buffer (530). At point (501), frames (5 to 8) can be stored in an input buffer (550) for decoding by a decoder (330).

[0099] In one embodiment, at point (501), the playback module (310) may obtain an input requesting post-processing. In one embodiment, the requested post-processing may require at least four decoded frames, but is not limited thereto.

[0100] In one embodiment, the playback module (310) may request post-processing from the post-processing module (350) based on an input requesting post-processing. In one embodiment, the post-processing module (350) may generate a post-processed frame (5) based on the decoded frames (2 to 4) stored in the backup buffer (530) and the frame (5) newly decoded by the decoder (330). In one embodiment, the post-processing module (350) may store the post-processed frame (5) in the output buffer (520). In one embodiment, the post-processing module (350) may not store the un-post-processed frames (2 to 4) in the output buffer (520). In one embodiment, the post-processing module (350) may discard the un-post-processed frames (2 to 4).

[0101] Referring to FIG. 5A, a point in time (502) may be a point in time (510) at which a renderer (360) renders a frame (2) and / or a point in time (540) at which a decoder (330) decodes a frame (5). In one embodiment, a point in time (502) may be a point in time at which frames (2 to 5) among frames (1 to 16) of a video (500) are stored in an output buffer (520). At a point in time (502), frames (2 to 4) among frames (2 to 5) stored in the output buffer (520) may be frames decoded by the decoder (330). At a point in time (502), frame (5) among frames (2 to 5) stored in the output buffer (520) may be a frame post-processed by a post-processing module (350). At point (502), frames (3 to 5) decoded by the decoder (330) may be stored in a backup buffer (530). At point (502), frames (6 to 9) may be stored in an input buffer (550) for decoding by the decoder (330).

[0102] In one embodiment, the post-processing module (350) may sequentially generate post-processed frames based on decoded frames stored in the backup buffer (530) and frames newly decoded by the decoder (330). In one embodiment, the post-processing module (350) may sequentially store the post-processed frames in the output buffer (520).

[0103] Referring to FIG. 5A, point in time (503) may be point in time (510) when the renderer (360) renders frame (5) and / or point in time (540) when the decoder (330) decodes frame (9). In one embodiment, point in time (503) may be point in time when frames (5 to 8) among frames (1 to 16) of video (500) are stored in output buffer (520). At point in time (502), frames (5 to 8) stored in output buffer (520) may be frames post-processed by post-processing module (350). At point in time (503), frames (6 to 8) decoded by decoder (330) may be stored in backup buffer (530). At point (503), frames (9 to 12) can be stored in an input buffer (550) for decoding by a decoder (330).

[0104] FIG. 5b illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment. FIG. 3 and FIG. 4 may be referenced for explanation of FIG. 5b.

[0105] Referring to FIG. 5B, point in time (504) may be point in time (510) when the renderer (360) renders frame (1) and / or point in time (540) when the decoder (330) decodes frame (5). In one embodiment, point in time (504) may be point in time when frames (1 to 4) among frames (1 to 16) of video (500) are stored in output buffer (520). At point in time (504), frames (1 to 4) stored in output buffer (520) may be frames decoded by decoder (330). At point in time (504), frames (5 to 8) may be stored in input buffer (550) for decoding by decoder (330).

[0106] In one embodiment, at point (504), the playback module (310) may obtain an input requesting post-processing. In one embodiment, the requested post-processing may require at least four decoded frames, but is not limited thereto.

[0107] In one embodiment, the playback module (310) may request post-processing from the post-processing module (350) based on an input requesting post-processing. In one embodiment, the post-processing module (350) may generate a post-processed frame (8) based on frames (5 to 8) newly decoded by the decoder (330). In one embodiment, the post-processing module (350) may store the post-processed frame (8) in the output buffer (520).

[0108] Referring to FIG. 5B, the point in time (505) may be the point in time (510) when the renderer (360) renders the frame (5) and / or the point in time (540) when the decoder (330) decodes the frame (9). In one embodiment, the point in time (505) may be the point in time when frames (5 to 8) among frames (1 to 16) of the video (500) are stored in the output buffer (520). At the point in time (505), frames (5 to 7) among the frames (5 to 8) stored in the output buffer (520) may be frames decoded by the decoder (330). At the point in time (505), frame (8) among the frames (5 to 8) stored in the output buffer (520) may be a frame post-processed by the post-processing module (350). At point (505), frames (9 to 12) may be stored in an input buffer (550) for decoding by a decoder (330).

[0109] Referring to FIG. 5B, point in time (506) may be point in time (510) when the renderer (360) renders frame (8) and / or point in time (540) when the decoder (330) decodes frame (12). In one embodiment, point in time (506) may be point in time when frames (8 to 11) among frames (1 to 16) of the video (500) are stored in the output buffer (520). At point in time (505), frames (8 to 11) stored in the output buffer (520) may be frames post-processed by the post-processing module (350). At point in time (506), frames (12 to 15) may be stored in the input buffer (550) for decoding by the decoder (330).

[0110] As described with reference to FIGS. 5A and 5B as examples, the electronic device (101) can reproduce post-processed frames at a faster time by managing the size of the output buffer (450) to the minimum size required for the preparation time for post-processing. In addition, as described with reference to FIG. 5A as an example, the electronic device (101) can reproduce post-processed frames at a faster time by responding more quickly to post-processing requests by storing (or managing) decoded frames in the backup buffer (430).

[0111] Hereinafter, an operation for performing post-processing while video playback is stopped is described with reference to FIGS. 6A and 6B. FIG. 6A illustrates an operation for performing post-processing when a backup buffer (430) exists, and FIG. 6B illustrates an operation for performing post-processing when a backup buffer (430) does not exist.

[0112] FIG. 6A illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment. FIG. 3 and FIG. 4 may be referenced for explanation of FIG. 6A.

[0113] Referring to FIG. 6A, a point in time (601) may be a point in time (610) when a renderer (360) renders a frame (5) and / or a point in time (640) when a decoder (330) decodes a frame (10). In one embodiment, a point in time (601) may be a point in time when frames (6 to 9) among frames (1 to 16) of a video (600) are stored in an output buffer (620). At a point in time (601), the frames (6 to 9) stored in the output buffer (620) may be frames decoded by the decoder (330). At a point in time (601), the frames (3 to 9) decoded by the decoder (330) may be stored in a backup buffer (630). At point (601), frames (10 to 13) can be stored in an input buffer (650) for decoding by a decoder (330).

[0114] Compared to FIG. 5A, the backup buffer (530) of FIG. 6A may further store the output frame (5) and frames (1 to 4) preceding the output frame. In one embodiment, the number of frames (1 to 4) preceding the output frame may correspond to the number of frames required for post-processing.

[0115] In one embodiment, at point (601), the playback module (310) may obtain an input requesting post-processing during playback of a frame (5). In one embodiment, the requested post-processing may require at least four decoded frames, but is not limited thereto.

[0116] In one embodiment, the playback module (310) may stop playback of a video based on an input requesting post-processing. In one embodiment, the playback module (310) may identify a frame (6) to be post-processed and frames (e.g., frames (3 to 6)) required for decoding the frame (6) to be post-processed. In one embodiment, the playback module (310) may post-process the frame (6) through the frames (3 to 6) via the post-processing module (350).

[0117] Referring to FIG. 6A, at point (602), the renderer (360) may stop rendering the frame. In one embodiment, at point (602), based on the stopping of rendering the frame (or the stopping of playback of the video), the playback module (310) may delete (or discard) (or remove) the decoded frames (6 to 9) stored in the output buffer (620).

[0118] In one embodiment, at point (602), the playback module (310) may delete (or discard) (or remove) some frames (e.g., frame (3)) among the decoded frames (3 to 9) stored in the backup buffer (630) based on the generation of the post-processed frame (6) (or based on the storage of the post-processed frame (6) in the output buffer (620). For example, some of the decoded frames stored in the backup buffer (630) that are deleted may be decoded frames that are no longer needed for post-processing.

[0119] In one embodiment, at point (602), the playback module (310) can generate a post-processed frame (6) based on the decoded frames (3 to 6) stored in the backup buffer (430). For example, the post-processing module (350) can post-process the frame (6) based on the decoded frames (3 to 6) stored in the backup buffer (430) and store the post-processed frame (6) in the output buffer (620).

[0120] Referring to FIG. 6A, point in time (603) may be point in time (610) when the renderer (360) renders the post-processed frame (7) and / or point in time (640) when the decoder (330) decodes the frame (11). In one embodiment, point in time (603) may be point in time when frames (7 and 8) among frames (1 to 16) of the video (600) are stored in the output buffer (620). At point in time (602), frames (7 and 8) stored in the output buffer (620) may be frames post-processed by the post-processing module (350). At point in time (603), frames (6 to 10) decoded by the decoder (330) may be stored in the backup buffer (630). At point (603), frames (4 and 5) that are no longer needed for post-processing among the decoded frames (4 to 9) stored in the backup buffer (630) at the previous point (602) may be deleted (or discarded) (or removed) from the backup buffer (630). At point (603), frames (11 to 14) may be stored in the input buffer (650) for decoding by the decoder (330).

[0121] Referring to FIG. 6A, point in time (604) may be point in time (610) when the renderer (360) renders the post-processed frame (8) and / or point in time (640) when the decoder (330) decodes the frame (13). In one embodiment, point in time (604) may be point in time when frames (9 to 12) among frames (1 to 16) of the video (600) are stored in the output buffer (620). At point in time (602), frames (9 to 12) stored in the output buffer (620) may be frames post-processed by the post-processing module (350). At point in time (604), frames (9 to 12) decoded by the decoder (330) may be stored in the backup buffer (630). At point (604), among the decoded frames (6 to 10) stored in the backup buffer (630) at the previous point (602), a frame (9) that is no longer needed for post-processing may be deleted (or discarded) (or removed) from the backup buffer (630). At point (604), frames (13 to 16) may be stored in the input buffer (650) for decoding by the decoder (330).

[0122] Referring to FIG. 6A, the speed at which a frame post-processed by the renderer (360) is rendered in the output buffer (620) may be faster than the speed at which a frame processed by the decoder (330) (and the post-processing module (350)) in the input buffer (650) is stored in the output buffer (620). Accordingly, in order to optimize the size of the output buffer (620), the size of the backup buffer (630) may be gradually reduced. However, the present invention is not limited thereto.

[0123] FIG. 6b illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment. FIG. 3 and FIG. 4 may be referenced for explanation of FIG. 6b.

[0124] Referring to FIG. 6B, point in time (605) may be point in time (610) when the renderer (360) renders frame (4) and / or point in time (640) when the decoder (330) decodes frame (10). In one embodiment, point in time (604) may be point in time when frames (6 to 9) among frames (1 to 16) of the video (600) are stored in the output buffer (620). At point in time (605), frames (6 to 9) stored in the output buffer (620) may be frames decoded by the decoder (330). At point in time (605), frames (10 to 13) may be stored in the input buffer (650) for decoding by the decoder (330).

[0125] In one embodiment, at point (605), the playback module (310) may obtain an input requesting post-processing while playing (or rendering) a frame (5). In one embodiment, the requested post-processing may require at least four decoded frames, but is not limited thereto.

[0126] In one embodiment, the playback module (310) may stop playback of the video based on an input requesting post-processing obtained while playing (or rendering) frame (5). In one embodiment, the playback module (310) (or decoder (330)) may re-decode frames (3 to 6) necessary to post-process frame (6) in order to play back the frame (6) following frame (5) after post-processing it. In addition, the playback module (310) may drop (or delete) (or remove) decoded frames (6 to 9) stored in the output buffer (620) that are no longer being played back.

[0127] Referring to FIG. 6B, at point (606), the playback module (310) can identify a frame (6) to be post-processed and frames (e.g., frames (3 to 6)) required for decoding the frame (6) to be post-processed while the video playback is paused. In one embodiment, the playback module (310) can decode the frame (6) to be post-processed and frames (e.g., frames (3 to 6)) required for decoding the frame (6) to be post-processed through the decoder (330).

[0128] In one embodiment, the playback module (310) may drop (or delete) (or remove) decoded frames (3 to 5) that are not related to playback (or not requested for post-processing) among one or more frames (3 to 6) decoded by the decoder (330). Here, dropping (or deleting) (or removing) the decoded frames (3 to 5) may include that the frames (3 to 5) decoded by the decoder (330) are not stored in the output buffer (620).

[0129] Referring to FIG. 6B, at point (607), the playback module (310) may generate a post-processed frame (6) based on the newly decoded frames (3 to 6) by the decoder (330). In one embodiment, the post-processing module (350) may store the generated post-processed frame (6) based on the newly decoded frames (3 to 6) in the output buffer (620).

[0130] Referring to FIG. 6B, point in time (608) may be point in time (610) when the renderer (360) renders the post-processed frame (6) and / or point in time (640) when the decoder (330) decodes the frame (11). In one embodiment, point in time (608) may be point in time when frames (7 to 10) among frames (1 to 16) of the video (600) are stored in the output buffer (620). At point in time (608), frames (7 to 10) stored in the output buffer (620) may be frames post-processed by the post-processing module (350). At point in time (608), frames (11 to 14) may be stored in the input buffer (650) for decoding by the decoder (330).

[0131] As described with reference to FIGS. 6A and 6B as examples, the electronic device (101) can reproduce post-processed frames at a faster time by managing the size of the output buffer (450) to the minimum size required for the preparation time for post-processing. In addition, as described with reference to FIG. 6A as an example, the electronic device (101) can reproduce post-processed frames at a faster time by responding to post-processing requests at a faster time (e.g., without repeating decoding for decoded frames (4) already stored in the backup buffer (430)) by storing (or managing) decoded frames in the backup buffer (430).

[0132] FIG. 7 illustrates a block diagram of the order of frames processed by an electronic device according to one embodiment. For the description of FIG. 7, reference may be made to FIGS. 3 and 4.

[0133] Referring to FIG. 7, a point in time (701) may be a point in time (710) when a renderer (360) renders a post-processed frame (1) and / or a point in time (740) when a decoder (330) decodes a frame (5). In one embodiment, a point in time (701) may be a point in time when frames (1 to 4) among frames (1 to 16) of a video (700) are stored in an output buffer (720). At a point in time (701), the frames (1 to 4) stored in the output buffer (720) may be frames post-processed by a post-processing module (350). At a point in time (701), frames (2 to 4) decoded by the decoder (330) may be stored in a backup buffer (730). At point (701), frames (5 to 8) can be stored in an input buffer (750) for decoding by a decoder (330).

[0134] In one embodiment, at time point (701), the playback module (310) may obtain an input requesting the disabling of post-processing (or the deactivation of post-processing). In one embodiment, the playback module (310) may stop the playback of the video based on the input requesting the disabling of post-processing (or the deactivation of post-processing). In one embodiment, the playback module (310) may drop (or delete) (or remove) the post-processed frames (1 to 4) stored in the output buffer (720) based on the input requesting the disabling of post-processing (or the deactivation of post-processing). In one embodiment, the playback module (310) may store the decoded frames (2 to 4) stored in the backup buffer (730) in the output buffer (720) based on the input requesting the disabling of post-processing (or the deactivation of post-processing).

[0135] Referring to FIG. 7, a point in time (702) may be a point in time (710) at which a renderer (360) renders a decoded frame (2) and / or a point in time (740) at which a decoder (330) decodes a frame (6). In one embodiment, a point in time (702) may be a point in time at which frames (2 to 5) among frames (1 to 16) of a video (700) are stored in an output buffer (720). At a point in time (702), frames (2 to 4) among frames (2 to 5) stored in the output buffer (720) may be frames previously stored in a backup buffer (730). At point (702), among the frames (2 to 5) stored in the output buffer (720), frame (5) may be a newly decoded frame by the decoder (330) after an input requesting release of post-processing (or deactivation of post-processing).

[0136] As described with reference to FIG. 7 as an example, the electronic device (101) can reproduce a decoded frame rather than a post-processed frame at a faster time by managing the size of the output buffer (450) to the minimum size necessary for releasing post-processing. In addition, the electronic device (101) can react more quickly to a request for releasing post-processing by storing (or managing) the decoded frame in the backup buffer (430) (e.g., storing the decoded frames (2 to 4) previously stored in the backup buffer (430) in the output buffer (720), thereby reproducing the decoded frame at a faster time.

[0137] FIG. 8A illustrates a block diagram of a sequence in which an electronic device renders one or more frames included in an image, according to one embodiment. FIG. 3 and FIG. 4 may be referenced for explanation of FIG. 8A.

[0138] In one embodiment, the playback module (310) may select (or change) one of the paths for playing a plurality of frames included in the video (or image) based on an input related to playing the video. In one embodiment, the paths for playing the plurality of frames may be different from each other depending on the selective application of post-processing functions according to the plurality of post-processing modules (350, 810, 820). For example, the post-processing module (350) may perform frame interpolation according to frame rate conversion (FRC), the post-processing module (810) may perform resolution change, and the post-processing module (820) may apply an image filter.

[0139] The size of the backup buffers (430, 811, 821) of each of the plurality of post-processing modules (350, 810, 820) can be adjusted according to the required post-processing function. The size of the backup buffers (430, 811, 821) can be set based on the number of frames required for the post-processing function. The sizes of the backup buffers (430, 811, 821) can be different depending on the post-processing function. For example, when the number of frames required for the post-processing function processed by the post-processing module (350) is four, the size (or second number) of the backup buffer (430) can be a size in which three frames can be stored. For example, when the number of frames required for the post-processing function processed by the post-processing module (810) is three, the size (or second number) of the backup buffer (811) can be a size in which two frames can be stored. For example, if the number of frames required for the post-processing function processed by the post-processing module (820) is 5, the size (or second number) of the backup buffer (821) may be a size in which 4 frames can be stored.

[0140] For example, as shown in FIG. 8A, all of the post-processing modules (350, 810, 820) may be used to play back multiple frames included in a video (or image). In this case, the path may be a path in which frames are processed in the following order: input buffer (410), decoder (330), post-processing modules (350, 810, 820), output buffer (450), and renderer (360).

[0141] Referring to FIG. 8A, the backup buffer (430) of the post-processing module (350) can store the output (e.g., a decoded frame) of the decoder (330). The backup buffer (811) of the post-processing module (810) can store the output (e.g., a post-processed frame) of the post-processing module (350). The backup buffer (821) of the post-processing module (820) can store the output (e.g., a post-processed frame) of the post-processing module (810). In one embodiment, the output buffer (450) can store the output (e.g., a post-processed frame) of the post-processing module (820).

[0142] For example, to play back multiple frames included in a video (or image), not all of the post-processing modules (350, 810, 820) may be used. In this case, the path may be a path in which frames are processed in the order of the input buffer (410), the decoder (330), the output buffer (450), and the renderer (360). In this case, the output buffer (450) may store the output of the decoder (330) (e.g., a decoded frame).

[0143] For example, some of the post-processing modules (350, 810, 820) may be used to play back multiple frames included in a video (or image) as in FIG. 8B. In this case, the path (850) may be a path in which frames are processed in the following order: an input buffer (410), a decoder (330), some of the post-processing modules (350, 810, 820) (e.g., a post-processing module (350), a post-processing module (820)), an output buffer (450), and a renderer (360).

[0144] Referring to FIG. 8B, the backup buffer (430) of the post-processing module (350) can store the output (e.g., a decoded frame) of the decoder (330). For the path (850), the backup buffer (821) of the post-processing module (820) can store the output (e.g., a post-processed frame) of the post-processing module (350). In one embodiment, the output buffer (450) can store the output (e.g., a post-processed frame) of the post-processing module (820).

[0145] As described above, the electronic device (101) may include a plurality of post-processing modules (350, 810, 820) depending on the post-processing function. In addition, the device (101) may use some of the backup buffers (430, 811, 821) of the plurality of post-processing modules (350, 810, 820) depending on the selected post-processing function to respond more quickly to an input requesting activation or deactivation of the post-processing function.

[0146] FIG. 9 illustrates a flowchart of operations of an electronic device according to an embodiment. For the description of FIG. 9, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0147] The operations of FIG. 9 can be performed by the electronic device (101). The operations of FIG. 9 can be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130).

[0148] Referring to FIG. 9, in operation 910, the electronic device (101) may identify an input related to post-processing. In one embodiment, the electronic device (101) may obtain the input related to post-processing during playback of a video. In one embodiment, the electronic device (101) may obtain the input related to post-processing through an application for playback of a video. In one embodiment, the input related to post-processing may include an input requesting activation of at least one function among one or more post-processing-related functions or an input requesting deactivation of at least one function. The one or more post-processing-related functions may include frame interpolation according to frame rate conversion (FRC), frame shake correction, playback speed change, resolution change, and / or frame image filter.

[0149] In operation 920, the electronic device (101) may determine whether a pause in video playback is required. In one embodiment, the electronic device (101) may determine whether a pause in video playback is required for post-processing.

[0150] In one embodiment, the electronic device (101) may determine whether a pause in video playback is necessary based on the time for generating a frame to which post-processing is applied (hereinafter, referred to as a generation time). In one embodiment, the generation time may include a time for searching for a frame to be post-processed, a time for decoding the searched frame, and / or a time for initializing (or setting) (or loading) post-processing. Hereinafter, an operation for determining whether a pause in video playback is necessary based on the generation time may be described with reference to FIG. 10, FIG. 11, FIG. 13, or FIG. 14.

[0151] At operation 920, based on determining that pausing the video playback is not necessary, the electronic device (101) may perform operation 930. At operation 920, based on determining that pausing the video playback is necessary, the electronic device (101) may perform operation 940.

[0152] In operation 930, the electronic device (101) can perform post-processing without pausing. In one embodiment, the electronic device (101) can perform at least one operation for post-processing while playing the video without pausing.

[0153] In one embodiment, the electronic device (101) may load the post-processing module (350) into the memory (130) (or the volatile memory of the memory (130)) during video playback. In one embodiment, the electronic device (101) may post-process frames sequentially decoded by the decoder (330) during video playback. In one embodiment, the first post-processed frame may be rendered by the renderer (360) after all decoded frames stored in the output buffer (450) have been played. In one embodiment, the electronic device (101) may play back the post-processed frame based on whether the decoded frame is post-processed after being input. In one embodiment, if the post-processing module (350) is already loaded, loading the post-processing module (350) may be omitted.

[0154] In one embodiment, the electronic device (101) may perform post-processing using decoded frames stored in the backup buffer (430) while playing a video if the backup buffer (430) is available. In one embodiment, the electronic device (101) may perform post-processing using newly decoded frames after an input related to post-processing while playing a video if the backup buffer (430) is not available. Hereinafter, an operation of performing post-processing without pause may be described with reference to FIG. 16. Hereinafter, an operation of performing post-processing depending on whether the backup buffer (430) is available may be described with reference to FIG. 12.

[0155] In operation 940, the electronic device (101) may perform post-processing during the pause. In one embodiment, the electronic device (101) may perform at least one operation for post-processing during the pause.

[0156] In one embodiment, the electronic device (101) can drop (or delete) (or remove) decoded frames stored in the output buffer (450) while the video playback is paused. In one embodiment, the electronic device (101) can load the post-processing module (350) into the memory (130) (or the volatile memory of the memory (130)) while the video playback is paused. In one embodiment, the electronic device (101) can perform a seek for a frame to be post-processed while the video playback is paused. In one embodiment, the electronic device (101) can post-process the decoded sought frame. In one embodiment, the electronic device (101) can resume the video playback based on the post-processing of the decoded sought frame. In some embodiments, if the post-processing module (350) is already loaded, loading of the post-processing module (350) may be omitted.

[0157] In one embodiment, the electronic device (101) may perform post-processing using decoded frames stored in the backup buffer (430) while the video is paused, if the backup buffer (430) is available. In one embodiment, the electronic device (101) may perform post-processing using newly decoded frames after an input related to post-processing while the video is paused, if the backup buffer (430) is not available. Hereinafter, an operation of performing post-processing during a pause may be described with reference to FIG. 15. Hereinafter, an operation of performing post-processing depending on whether the backup buffer (430) is available may be described with reference to FIG. 12.

[0158] FIG. 10 illustrates a flowchart of operations of an electronic device according to an embodiment. For the description of FIG. 10, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0159] The operations of FIG. 10 can be performed by the electronic device (101). The operations of FIG. 10 can be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130).

[0160] Operations 910, 930, and 940 of FIG. 10 may correspond to operations 910, 930, and 940 of FIG. 9. Operations 1010 and 1020 of FIG. 10 may include operation 920 of FIG. 9.

[0161] Referring to FIG. 10, in operation 910, the electronic device (101) may identify an input related to post-processing. In one embodiment, the electronic device (101) may obtain the input related to post-processing through an application for playing a video.

[0162] In operation 1010, the electronic device (101) may identify a seek time. In one embodiment, the seek time may include the time required to decode, drop, and / or post-process frames required to post-process the first frame for which post-processing is requested.

[0163] In operation 1020, the electronic device (101) may determine whether the search time is less than a reference search time. In one embodiment, the reference search time may be preset. For example, the reference search time may be preset for a video. For example, the reference search time may be preset for an application that plays a video.

[0164] In operation 1020, based on the determination that the search time is greater than or equal to the reference search time, the electronic device (101) may perform operation 930. In operation 1020, based on the determination that the search time is less than or equal to the reference search time, the electronic device (101) may perform operation 940.

[0165] In operation 930, the electronic device (101) can perform post-processing without pausing. In one embodiment, the electronic device (101) can perform at least one operation for post-processing while playing the video without pausing.

[0166] In operation 940, the electronic device (101) may perform post-processing during the pause. In one embodiment, the electronic device (101) may perform at least one operation for post-processing during the pause.

[0167] FIG. 11 illustrates a flowchart of operations of an electronic device according to an embodiment. For the description of FIG. 11, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0168] The operations of FIG. 11 can be performed by the electronic device (101). The operations of FIG. 11 can be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130).

[0169] Operations 910, 930, and 940 of FIG. 11 may correspond to operations 910, 930, and 940 of FIG. 9. Operations 1110 and 1120 of FIG. 11 may include operation 920 of FIG. 9.

[0170] Referring to FIG. 11, in operation 910, the electronic device (101) may identify an input related to post-processing. In one embodiment, the electronic device (101) may obtain the input related to post-processing through an application for playing a video.

[0171] In operation 1110, the electronic device (101) may identify a timestamp difference. In one embodiment, the timestamp difference may be between a frame in which a request for post-processing has been changed based on input and the most recent frame to which post-processing is applied.

[0172] In operation 1120, the electronic device (101) can determine whether the timestamp difference is greater than or equal to a reference timestamp difference. In one embodiment, the timestamp difference may be preset. For example, the timestamp difference may be preset for a video. For example, the timestamp difference may be preset for an application that plays the video.

[0173] In operation 1120, based on determining that the timestamp difference is less than the reference timestamp difference, the electronic device (101) may perform operation 930. In operation 1120, based on determining that the timestamp difference is greater than or equal to the reference timestamp difference, the electronic device (101) may perform operation 940.

[0174] In operation 930, the electronic device (101) can perform post-processing without pausing. In one embodiment, the electronic device (101) can perform at least one operation for post-processing while playing the video without pausing.

[0175] In operation 940, the electronic device (101) may perform post-processing during the pause. In one embodiment, the electronic device (101) may perform at least one operation for post-processing during the pause.

[0176] FIG. 12 illustrates a flowchart of operations of an electronic device according to an embodiment. For the description of FIG. 12, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0177] The operations of FIG. 12 can be performed by the electronic device (101). The operations of FIG. 12 can be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130).

[0178] Operation 910 of FIG. 12 may correspond to operation 910 of FIG. 9. Operations 1210 and 1220 of FIG. 12 may be performed independently of operations 920 of FIG. 9. For example, operations 1210 and 1220 of FIG. 12 may be performed sequentially or in parallel with operations 920 of FIG. 9. Operations 1230 and 1240 of FIG. 12 may be performed independently of operations 930 or 940 of FIG. 9, respectively. For example, operations 1230 and 1240 of FIG. 12 may be performed sequentially or in parallel with operations 930 or 940 of FIG. 9, respectively. For example, operation 1230 or operation 1240 of FIG. 12 may be performed sequentially or in parallel with operation 930 of FIG. 9. For example, operation 1230 or operation 1240 of FIG. 12 may be performed sequentially or in parallel with operation 940 of FIG. 9.

[0179] Referring to FIG. 12, in operation 910, the electronic device (101) may identify an input related to post-processing. In one embodiment, the electronic device (101) may obtain the input related to post-processing through an application for playing a video.

[0180] In operation 1210, the electronic device (101) may identify a backup buffer (430). In one embodiment, the electronic device (101) may identify a backup buffer (430) assigned to a post-processing module (350).

[0181] In operation 1220, the electronic device (101) may determine whether a backup buffer (430) is available. In one embodiment, the availability of the backup buffer (430) may include that the backup buffer (430) is allocated to the post-processing module (350). In one embodiment, the availability of the backup buffer (430) may include that decoded frames are stored in the backup buffer (430) allocated to the post-processing module (350). However, the present invention is not limited thereto.

[0182] At operation 1220, based on determining that the backup buffer (430) is unavailable, the electronic device (101) may perform operation 1230. At operation 1220, based on determining that the backup buffer (430) is available, the electronic device (101) may perform operation 1240.

[0183] In operation 1230, the electronic device (101) can perform post-processing without a backup buffer. For example, the electronic device (101) can generate post-processed frames based on frames that are decoded after input related to post-processing is input.

[0184] For example, the electronic device (101) can perform post-processing without a backup buffer while the video is being played. For example, the electronic device (101) can perform post-processing without a backup buffer while the video is being played.

[0185] In operation 1240, the electronic device (101) may perform post-processing using a backup buffer. For example, the electronic device (101) may generate a post-processed frame based on decoded frames stored in the backup buffer (430) before input related to post-processing is input.

[0186] For example, the electronic device (101) may perform post-processing using a backup buffer while the video is being played. For example, the electronic device (101) may perform post-processing using a backup buffer while the video is being played.

[0187] FIG. 13 illustrates a flowchart of operations of an electronic device according to an embodiment. For the description of FIG. 13, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0188] The operations of FIG. 13 can be performed by the electronic device (101). The operations of FIG. 13 can be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130).

[0189] Operation 910 of FIG. 13 may correspond to operation 910 of FIG. 9. Operations 1310 to 1360 of FIG. 13 may correspond to operations 920, 930, or 940 of FIG. 9.

[0190] Referring to FIG. 13, in operation 910, the electronic device (101) may identify an input related to post-processing. In one embodiment, the electronic device (101) may obtain the input related to post-processing through an application for playing a video.

[0191] In operation 1310, the electronic device (101) may determine whether a reference search time has been set. In one embodiment, the reference search time may be preset. For example, the reference search time may be preset for a video. For example, the reference search time may be preset for an application that plays a video.

[0192] In operation 1320, the electronic device (101) may determine whether the search time is less than a reference search time. In one embodiment, the search time may include the time required to decode, drop, and / or post-process frames required to post-process the first frame for which post-processing is requested.

[0193] In operation 1320, based on the determination that the search time is less than the reference search time, the electronic device (101) may perform operation 1330. In operation 1320, based on the determination that the search time is greater than the reference search time, the electronic device (101) may perform operation 1340.

[0194] In operation 1330, the electronic device (101) may identify a post-processing application frame based on the search time. In one embodiment, the electronic device (101) may identify a post-processing application frame within the search time. In one embodiment, the electronic device (101) may identify a post-processing application frame within the search time without a pause. In one embodiment, the post-processing application frame may include the first frame requested by the post-processing-related input and subsequent frames.

[0195] In operation 1340, the electronic device (101) may determine whether a reference timestamp difference has been set. In one embodiment, the timestamp difference may be between a frame in which a request for post-processing has been changed based on the input and the most recent frame to which post-processing is applied.

[0196] In operation 1340, based on determining that the reference timestamp difference is set, the electronic device (101) may perform operation 1350. In operation 1340, based on determining that the reference timestamp difference is not set, the electronic device (101) may perform operation 1360.

[0197] In operation 1350, the electronic device (101) can identify a frame to which post-processing is applied based on a reference timestamp difference. The electronic device (101) can identify a frame to which post-processing is applied based on a comparison of the timestamp difference with the reference timestamp difference. For example, if the timestamp difference is less than the reference timestamp difference, the electronic device (101) can perform post-processing without a pause. In one embodiment, if the timestamp difference is greater than or equal to the reference timestamp difference, the electronic device (101) can perform at least one operation for post-processing during the pause.

[0198] In operation 1360, the electronic device (101) can perform post-processing without pausing.

[0199] FIG. 14 illustrates a flowchart of operations of an electronic device according to an embodiment. For the description of FIG. 14, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0200] The operations of FIG. 14 can be performed by the electronic device (101). The operations of FIG. 14 can be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130).

[0201] Operation 910 of FIG. 14 may correspond to operation 910 of FIG. 9. Operations 1330, 1350, 1360, 1410, 1420, or 1440 of FIG. 14 may correspond to operations 920, 930, or 940 of FIG. 9. Operations 1330, 1350, and 1360 of FIG. 14 may correspond to operations 1330, 1350, and 1360 of FIG. 13.

[0202] Referring to FIG. 14, in operation 910, the electronic device (101) may identify an input related to post-processing. In one embodiment, the electronic device (101) may obtain the input related to post-processing through an application for playing a video.

[0203] In operation 1410, the electronic device (101) may determine whether a reference timestamp difference has been set. In one embodiment, the timestamp difference may be preset. For example, the timestamp difference may be preset for a video. For example, the timestamp difference may be preset for an application that plays the video.

[0204] In operation 1420, the electronic device (101) may determine whether the timestamp difference is greater than or equal to a reference timestamp difference. In one embodiment, the timestamp difference may be between a frame in which a request for post-processing has been changed based on the input and the most recent frame to which post-processing is applied.

[0205] In operation 1440, the electronic device (101) may determine whether a reference search time has been set. In one embodiment, the reference search time may be preset. For example, the reference search time may be preset for a video. For example, the reference search time may be preset for an application that plays a video.

[0206] In operation 1440, based on determining that the reference search time is set, the electronic device (101) may perform operation 1330. In operation 1440, based on determining that the reference search time is not set, the electronic device (101) may perform operation 1360.

[0207] In operation 1330, the electronic device (101) may identify a frame to which post-processing is applied based on the search time required. The electronic device (101) may identify a frame to which post-processing is applied based on a comparison of the search time required and a reference search time required. For example, if the search time required is less than the reference search time required, the electronic device (101) may perform post-processing without a pause. In one embodiment, if the search time required is greater than or equal to the reference search time required, the electronic device (101) may perform at least one operation for post-processing during the pause.

[0208] In operation 1350, the electronic device (101) can identify a post-processing application frame based on a difference in reference timestamps. In one embodiment, the electronic device (101) can identify a post-processing application frame within the difference in reference timestamps. In one embodiment, the electronic device (101) can identify a post-processing application frame within the difference in reference timestamps without a pause. In one embodiment, the post-processing application frame can include a first frame requested by an input related to post-processing and subsequent frames.

[0209] In operation 1360, the electronic device (101) can perform post-processing without pausing.

[0210] FIG. 15 illustrates a flowchart of operations of an electronic device according to one embodiment. For the description of FIG. 15, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0211] The operations of FIG. 15 may be performed by the electronic device (101). The operations of FIG. 15 may be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130). The operations of FIG. 15 may correspond to operation 940 of FIG. 9.

[0212] Referring to FIG. 15, in operation 1510, the electronic device (101) may pause playback of the video. In one embodiment, the electronic device (101) may drop (or delete) (or remove) decoded frames stored in the output buffer (450) while playback of the video is paused.

[0213] In operation 1520, the electronic device (101) may identify a frame. The electronic device (101) may include an operation for identifying a frame for which post-processing is requested (or, the first frame for which post-processing is requested) indicated by the input.

[0214] In operation 1530, the electronic device (101) may decode a frame. The electronic device (101) may decode one or more frames for decoding a frame for which post-processing has been requested (or a first frame for which post-processing has been requested) through the decoder (330). Here, the frames required for decoding the frame for which post-processing has been requested may be an IDR frame, a P frame, and / or a B frame. Here, the frames required for decoding the frame for which post-processing has been requested may be frames that are decoded before the frame for which post-processing has been requested among the frames in the GOP. In an embodiment, if the frame for which post-processing has been requested (or the first frame for which post-processing has been requested) is stored in the backup buffer (430), decoding of the frame for which post-processing has been requested (or the first frame for which post-processing has been requested) may be omitted.

[0215] In one embodiment, operations 1520 and 1530 and operation 1540 may be performed in parallel.

[0216] In operation 1540, the electronic device (101) may prepare post-processing. In one embodiment, the electronic device (101) may load the post-processing module (350) into the memory (130) (or volatile memory of the memory (130)) while the video playback is stopped.

[0217] In operation 1550, the electronic device (101) may perform post-processing. The electronic device (101) may perform post-processing on a frame for which post-processing has been requested (or the first frame for which post-processing has been requested).

[0218] In one embodiment, the electronic device (101) can store a post-processed frame in an output buffer (450) through a post-processing module (350).

[0219] In operation 1560, the electronic device (101) may play a video. In one embodiment, the electronic device (101) may render a post-processed frame stored in the output buffer (450). For example, the electronic device (101) may display the rendered frame through the display (260).

[0220] FIG. 16 illustrates a flowchart of operations of an electronic device according to an embodiment. For the description of FIG. 16, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0221] The operations of FIG. 16 may be performed by the electronic device (101). The operations of FIG. 16 may be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130). The operations of FIG. 16 may correspond to operation 930 of FIG. 9.

[0222] Referring to FIG. 16, in operation 1610, the electronic device (101) may prepare post-processing. In one embodiment, the electronic device (101) may load the post-processing module (350) into the memory (130) (or volatile memory of the memory (130)) during playback of the video.

[0223] In operation 1620, the electronic device (101) may perform post-processing. The electronic device (101) may perform post-processing on a frame for which post-processing has been requested (or the first frame for which post-processing has been requested).

[0224] FIG. 17 illustrates a flowchart of operations of an electronic device according to one embodiment. For the description of FIG. 17, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0225] The operations of FIG. 17 can be performed by the electronic device (101). The operations of FIG. 17 can be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130).

[0226] Referring to FIG. 17, in operation 1710, the electronic device (101) may set the size of the output buffer (450). In one embodiment, the electronic device (101) may set the size of the output buffer (450) based on a request for video playback. In one embodiment, the electronic device (101) may set the size of the output buffer (450) based on information for video playback (e.g., resolution, FPS). In one embodiment, the electronic device (101) may set the size of the output buffer (450) based on an initial setting value for video playback.

[0227] For example, the electronic device (101) can set the size (or the third number) of the output buffer (450) based on the first required time for preparation of the post-processing module (350). For example, the electronic device (101) can set the size (or the third number) of the output buffer (450) based on the second required time for post-processing of the post-processing module (350). For example, the electronic device (101) can set the size (or the third number) of the output buffer (450) based on the rendering speed of the renderer (360). For example, the electronic device (101) can set the size (or the third number) of the output buffer (450) based on the playback speed of the video (e.g., playback time magnification, or playback FPS).

[0228] In operation 1720, the electronic device (101) may set the size of the backup buffer (430). In one embodiment, the electronic device (101) may set the size of the backup buffer (430) based on a request for video playback. In one embodiment, the electronic device (101) may set the size of the backup buffer (430) within a size allocable to the output buffer (450). In one embodiment, the electronic device (101) may set the size of the backup buffer (430) within a size of an area not allocated to the output buffer (450) among areas allocable to the output buffer (450). However, the present invention is not limited thereto.

[0229] In one embodiment, the electronic device (101) may set the size (or second number) of the backup buffer (430) based on the number of frames required for the post-processing function. For example, the playback module (310) and / or the post-processing module control unit (340) may adjust the size (or second number) of the backup buffer (430) so that a number of frames less than the number of frames required for the post-processing function (or as many as the number of frames required for the post-processing function) can be stored. When the number of frames required for the post-processing function is four, the size (or second number) of the backup buffer (430) may be a size in which three frames can be stored.

[0230] In one embodiment, the electronic device (101) may set the size of the output buffer (450) based on information for playing the video (e.g., resolution, FPS). In one embodiment, the electronic device (101) may set the size of the output buffer (450) based on an initial setting value for playing the video.

[0231] At operation 1730, the electronic device (101) may determine whether adjustment of the buffer size is required. For example, the electronic device (101) may determine whether adjustment of the size of the backup buffer (430) and / or the output buffer (450) is required.

[0232] For example, the electronic device (101) can determine whether adjustment of the buffer size is necessary depending on a change in the playback speed of the video (e.g., playback time magnification, or playback FPS). For example, when the playback speed of the video increases, the electronic device (101) can determine that adjustment of the buffer size is necessary as the number of frames played within the first required time for preparation of the post-processing module (350) and the second required time for post-processing of the post-processing module (350) increases. For example, when the playback speed of the video decreases, the electronic device (101) can determine that adjustment of the buffer size is necessary as the number of frames played within the first required time for preparation of the post-processing module (350) and the second required time for post-processing of the post-processing module (350) decreases.

[0233] For example, the electronic device (101) can determine whether adjustment of the backup buffer (430) is necessary depending on whether post-processing is activated or deactivated. For example, the electronic device (101) can determine that adjustment of the backup buffer (430) is necessary as the number of decoded frames required by the post-processing function increases. For example, the electronic device (101) can determine that adjustment of the backup buffer (430) is necessary as the number of decoded frames required by the post-processing function decreases.

[0234] At operation 1730, based on determining that adjustment of the buffer size is necessary, the electronic device (101) may perform operation 1740. At operation 1730, based on determining that adjustment of the buffer size is not necessary, the electronic device (101) may perform operation 1730 again.

[0235] In operation 1740, the electronic device (101) may adjust the buffer size. For example, the electronic device (101) may adjust the output buffer (450) buffer size based on a change in the video playback speed (e.g., playback time magnification, or playback FPS). For example, the electronic device (101) may adjust the size of the backup buffer (430) based on the activation or deactivation of post-processing. Detailed operations of operation 1740 may be described with reference to FIG. 18.

[0236] FIG. 18 illustrates a flowchart of operations of an electronic device according to an embodiment. For the description of FIG. 18, reference may be made to FIGS. 3, 4, 5a, 5b, 6a, 6b, 7, 8a, and 8b.

[0237] The operations of FIG. 18 may be performed by the electronic device (101). The operations of FIG. 18 may be performed by the electronic device (101) by the processor (120) of the electronic device (101) executing instructions stored in the memory (130). The operations of FIG. 18 may correspond to operation 1740 of FIG. 17. Referring to FIG. 18, in operation 1810, the electronic device (101) may identify a difference in the size of the output buffer (450) and a difference in the size of the backup buffer (430).

[0238] For example, the electronic device (101) can identify a first size difference between a preset first size of the output buffer (450) (or the number of output buffers (450) before adjustment) and a second size required for the output buffer (450) (or the number of output buffers (450) after adjustment). For example, the second size required for the output buffer (450) may be a size required for the output buffer (450) according to a change in the playback speed of the video (e.g., playback time multiplier, or playback FPS). For example, the second size required for the output buffer (450) may be a size required for the output buffer (450) according to a change in the number of frames played back within a first required time for preparing the post-processing module (350) and a second required time for post-processing of the post-processing module (350).

[0239] For example, the electronic device (101) may identify a second size difference between a third size of the backup buffer (430) before activation or deactivation of post-processing (or the number of backup buffers (430) before adjustment) and a fourth size required for the backup buffer (430) after activation or deactivation of post-processing (or the number of backup buffers (430) after adjustment). The fourth size required for the backup buffer (430) after activation or deactivation of post-processing may be a size required for the backup buffer (430) as the number of decoded frames required by the post-processing function changes.

[0240] In operation 1820, the electronic device (101) may determine whether a switch between the output buffer (450) and the backup buffer (430) is required. For example, the electronic device (101) may determine whether a switch between the output buffer (450) and the backup buffer (430) is required based on the first size difference and / or the second size difference. For example, the electronic device (101) may determine that a switch between the output buffer (450) and the backup buffer (430) is required when a size reduction of the output buffer (450) is required and an increase in the size of the backup buffer (430) is required. For example, the electronic device (101) may determine that a switch between the output buffer (450) and the backup buffer (430) is required when a size increase of the output buffer (450) is required and a size decrease of the backup buffer (430) is required. For example, the electronic device (101) may determine that switching between the output buffer (450) and the backup buffer (430) is not necessary when an increase in the size of the output buffer (450) and the backup buffer (430) is required. For example, the electronic device (101) may determine that switching between the output buffer (450) and the backup buffer (430) is not necessary when a decrease in the size of the output buffer (450) and the backup buffer (430) is required.

[0241] At operation 1820, based on determining that switching between the output buffer (450) and the backup buffer (430) is not necessary, the electronic device (101) may perform operation 1830. At operation 1820, based on determining that switching between the output buffer (450) and the backup buffer (430) is necessary, the electronic device (101) may perform operation 1870.

[0242] In operation 1830, the electronic device (101) may determine whether additional allocation is required. For example, if an increase in the size of the output buffer (450) and the backup buffer (430) is required, the electronic device (101) may determine that additional allocation is required for the output buffer (450) and / or the backup buffer (430). For example, if a decrease in the size of the output buffer (450) and the backup buffer (430) is required, the electronic device (101) may determine that no additional allocation is required for the output buffer (450) and / or the backup buffer (430).

[0243] At operation 1830, based on determining that additional allocation is required, the electronic device (101) may perform operation 1840. At operation 1830, based on determining that additional allocation is not required, the electronic device (101) may perform operation 1850.

[0244] In operation 1840, the electronic device (101) may additionally allocate a buffer. For example, the electronic device (101) may add a buffer to a buffer requiring additional allocation. For example, the electronic device (101) may additionally allocate a buffer within a first size difference or a second size difference to at least one of the output buffer (450) or the backup buffer (430).

[0245] In operation 1850, the electronic device (101) may determine whether deallocation is required. For example, if a size reduction of the output buffer (450) and the backup buffer (430) is required, the electronic device (101) may determine that deallocation of the output buffer (450) and / or the backup buffer (430) is required. For example, if a size reduction and increase of the output buffer (450) and the backup buffer (430) are not required, the electronic device (101) may determine that deallocation of the output buffer (450) and / or the backup buffer (430) is not required.

[0246] At operation 1850, based on determining that deallocation is required, the electronic device (101) may perform operation 1860. At operation 1850, based on determining that deallocation is not required, the electronic device (101) may terminate the operations according to FIG. 18.

[0247] In operation 1860, the electronic device (101) may deallocate the remaining buffer. In one embodiment, the electronic device (101) may deallocate the remaining buffer exceeding the second size among the buffers allocated to the output buffer (450). In one embodiment, the electronic device (101) may deallocate the remaining buffer exceeding the fourth size among the buffers allocated to the backup buffer (430). In one embodiment, the remaining buffer of the output buffer (450) may be a buffer exceeding the second size among the buffers allocated to the output buffer (450). In one embodiment, the remaining buffer of the backup buffer (430) may be a buffer exceeding the fourth size among the buffers allocated to the backup buffer (430).

[0248] In operation 1870, the electronic device (101) may determine whether a transition from the backup buffer (430) to the output buffer (450) is required. For example, if an increase in the size of the output buffer (450) is required and a decrease in the size of the backup buffer (430) is required, the electronic device (101) may determine that a transition from the backup buffer (430) to the output buffer (450) is required. For example, if a decrease in the size of the output buffer (450) is required and an increase in the size of the backup buffer (430) is required, the electronic device (101) may determine that a transition from the backup buffer (430) to the output buffer (450) is not required.

[0249] At operation 1870, based on determining that a transition from the backup buffer (430) to the output buffer (450) is required, the electronic device (101) may perform operation 1880. At operation 1870, based on determining that a transition from the backup buffer (430) to the output buffer (450) is not required (or determining that a transition from the output buffer (450) to the backup buffer (430) is required), the electronic device (101) may perform operation 1890.

[0250] In operation 1880, the electronic device (101) may convert a portion of the backup buffer (430) to an output buffer (450). In one embodiment, the electronic device (101) may allocate a portion of the backup buffer (430) exceeding a fourth size from the size allocated to the backup buffer (430) to the output buffer (450). For example, the electronic device (101) may convert a portion of the backup buffer (430) to an output buffer (450) within the remaining buffer exceeding a fourth size from the size allocated to the backup buffer (430).

[0251] In one embodiment, the electronic device (101) may perform operation 1830 after operation 1880. For example, the electronic device (101) may determine to perform operation 1850 if there is still a remaining buffer in the backup buffer (430) in operation 1830 (or if the size allocated to the backup buffer (430) exceeds the fourth size). For example, the electronic device (101) may determine to perform operation 1840 if additional buffer allocation is required in the output buffer (450) in operation 1830.

[0252] In operation 1890, the electronic device (101) may convert a portion of the output buffer (450) to a backup buffer (430). In one embodiment, the electronic device (101) may allocate a portion of the output buffer (450) that exceeds a second size from the size allocated to the output buffer (450) to the backup buffer (430). For example, the electronic device (101) may convert a portion of the output buffer (450) to a backup buffer (430) within the remaining buffer that exceeds the second size from the size allocated to the output buffer (450).

[0253] In one embodiment, the electronic device (101) may perform operation 1830 after operation 1880. For example, the electronic device (101) may determine to perform operation 1850 if there is still a remaining buffer in the output buffer (450) in operation 1830 (or if the size allocated to the output buffer (450) exceeds the second size). For example, the electronic device (101) may determine to perform operation 1840 if additional buffer allocation is required in the backup buffer (430) in operation 1830.

[0254] FIG. 19 illustrates a block diagram of an electronic device within a network environment according to one or more embodiments.

[0255] Referring to FIG. 19, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0256] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0257] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In some embodiments, the artificial intelligence model may include a software structure in addition to a hardware structure.

[0258] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0259] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0260] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0261] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0262] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0263] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0264] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0265] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0266] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0267] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0268] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0269] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0270] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0271] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0272] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for realizing eMBB, a loss coverage (e.g., 664 dB or less) for realizing mMTC, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 6 ms or less for round trip) for realizing URLLC.

[0273] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0274] According to one or more embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0275] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0276] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0277] As described above, the electronic device (101) may include a display (260). The electronic device (101) may include at least one processor (120). The electronic device (101) may include a memory (130) including a decoder (330), a post-processing module (350), a renderer (360), and one or more storage media storing instructions. The decoder (330) may decode a frame stored in an input buffer (410) among the plurality of frames of a video. The post-processing module (350) may post-process the decoded frame. The renderer (360) may render a frame stored in an output buffer (450). The frame stored in the output buffer (450) may be the decoded frame or the post-processed frame. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to configure the output buffer (450) based on a time required for configuring the post-processing. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to adjust a speed at which the plurality of frames are input to the input buffer (410) based on a speed at which the rendered frames are played back through the display (260).

[0278] The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to obtain an input requesting the post-processing. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify a generation time for generating the post-processed frame based on the input. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine, based on the generation time, whether to stop playback of the rendered frame through the display (260). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to set the post-processing while the playback of the video is paused based on a determination to pause the playback. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to set the post-processing while the video is played based on a determination not to pause the playback.

[0279] The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to search for a frame to post-process based on the input. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to compare the generation time with a reference time, which includes a time for decoding the searched frame and a time for setting the post-processing. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to decide to stop the playback based on the generation time being less than the reference time. The above instructions, when executed individually or collectively by the at least one processor (120), may cause the electronic device (101) to determine not to stop the playback based on the generation time being greater than or equal to the reference time.

[0280] The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine not to stop the playback if the generation time is greater than or equal to the reference time and no reference timestamp is set. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine whether to stop the playback of the rendered frame based on comparing a timestamp difference between the frame for which the post-processing is requested and the earliest frame for which the post-processing is possible with the reference timestamp, if the generation time is greater than or equal to the reference time and the reference timestamp is set.

[0281] The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to search for a frame for which post-processing is requested based on the input. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify the earliest frame for which post-processing is possible by the post-processing settings. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to identify a timestamp difference between the frame for which post-processing is requested and the earliest frame for which post-processing is possible. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to compare the identified timestamp and the reference timestamp. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine not to stop the playback based on the identified timestamp being greater than or equal to the reference timestamp. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to determine to stop the playback based on the identified timestamp being less than the reference timestamp.

[0282] The instructions, when executed individually or collectively by the at least one processor (120), may cause the electronic device (101) to determine not to stop the playback if the identified timestamp is less than the reference timestamp and the reference time is not set. If the identified timestamp is less than the reference timestamp and the reference time is set, the electronic device (101) may determine whether to stop the playback of the rendered frame based on comparing the generation time, which includes a time for decoding the searched frame and a time for setting the post-processing, with the reference time.

[0283] The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to set up a backup buffer (430) in which frames decoded by the decoder (330) are stored and accessed by the post-processing module (350). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to generate the post-processed frame based on one or more decoded frames stored in the backup buffer (430) by the post-processing module (350).

[0284] The size of the backup buffer (430) may correspond to the number of one or more decoded frames required according to the post-processing function provided by the post-processing module (350).

[0285] The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to obtain an input requesting the release of the post-processing. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to store, based on the input, one or more decoded frames stored in the backup buffer (430) in the output buffer (450). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to render, by the renderer (360), one or more decoded frames from the backup buffer (430) to the output buffer (450).

[0286] The memory (130) may include another post-processing module (350) that further post-processes the post-processed frame. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to set another backup buffer (430) in which the frame post-processed by the post-processing module (350) is stored and accessed by the other post-processing module (350). The post-processed frame may be stored in the other backup buffer (430) while the other post-processing module (350) is activated. The post-processed frame may be stored in the output buffer (450) while the other post-processing module (350) is deactivated.

[0287] As described above, the method may be performed in an electronic device (101) including a memory (130) including one or more storage media storing a decoder (330), a post-processing module (350), and a renderer (360). The method may include an operation of decoding, through the decoder (330), a frame stored in an input buffer (410) among the plurality of frames of a video. The method may include an operation of post-processing, through the post-processing module (350), the decoded frame. The method may include an operation of rendering, through the renderer (360), a frame stored in an output buffer (450). The frame stored in the output buffer (450) may be the decoded frame or the post-processed frame. The output buffer (450) may be set based on a required time required for setting the post-processing. The speed at which the plurality of frames are input into the input buffer (410) can be adjusted based on the speed at which the rendered frames are played back through the display (260).

[0288] The method may include an operation of obtaining an input requesting the post-processing. The method may include an operation of identifying a generation time for generating the post-processed frame based on the input. The method may include an operation of determining whether to stop playback of the rendered frame through the display (260) based on the generation time. The method may include an operation of setting the post-processing while the playback of the video is stopped based on a determination to stop the playback. The method may include an operation of setting the post-processing while the video is being played based on a determination not to stop the playback.

[0289] The method may include an operation of searching for a frame to be post-processed based on the input. The method may include an operation of comparing the generation time, which includes a time for decoding the searched frame and a time for setting the post-processing, with a reference time. The method may include an operation of determining to stop the playback based on the generation time being less than the reference time. The method may include an operation of determining not to stop the playback based on the generation time being greater than or equal to the reference time.

[0290] The method may include an operation of determining not to stop the playback if the generation time is greater than or equal to the reference time and the reference timestamp is not set. The method may include an operation of determining whether to stop the playback of the rendered frame based on a timestamp difference between the frame for which the post-processing is requested and the earliest frame for which the post-processing is possible and the reference timestamp, if the generation time is greater than or equal to the reference time and the reference timestamp is set.

[0291] The method may include an operation of searching for a frame for which post-processing is requested based on the input. The method may include an operation of identifying the earliest frame for which post-processing is possible based on the post-processing settings. The method may include an operation of identifying a timestamp difference between the frame for which post-processing is requested and the earliest frame for which post-processing is possible. The method may include an operation of comparing the identified timestamp with the reference timestamp. The method may include an operation of determining to stop the playback based on whether the identified timestamp is greater than or equal to the reference timestamp. The method may include an operation of determining not to stop the playback based on whether the identified timestamp is less than the reference timestamp.

[0292] The method may include an operation of determining not to stop the playback if the identified timestamp is less than the reference timestamp and the reference time is not set. The method may include an operation of determining whether to stop the playback of the rendered frame based on comparing the generation time, which includes a time for decoding the searched frame and a time for setting the post-processing, with the reference time, if the identified timestamp is less than the reference timestamp and the reference time is set.

[0293] The method may include an operation of setting a backup buffer (430) in which frames decoded by the decoder (330) are stored and accessed by the post-processing module (350). The method may include an operation of the post-processing module (350) generating the post-processed frame based on one or more decoded frames stored in the backup buffer (430).

[0294] The size of the backup buffer (430) may correspond to the number of one or more decoded frames required according to the post-processing function provided by the post-processing module (350).

[0295] The method may include an operation of obtaining an input requesting release of the post-processing. The method may include an operation of storing one or more decoded frames stored in the backup buffer (430) in the output buffer (450) based on the input. The method may include an operation of rendering one or more decoded frames from the backup buffer (430) to the output buffer (450) by the renderer (360).

[0296] The memory (130) may include another post-processing module (350) that further post-processes the post-processed frame. The memory (130) may store the post-processed frame by the post-processing module (350), and another backup buffer (430) that is accessed by the other post-processing module (350) may be set. The post-processed frame may be stored in the other backup buffer (430) while the other post-processing module (350) is activated. The post-processed frame may be stored in the output buffer (450) while the other post-processing module (350) is deactivated.

[0297] As described above, a non-transitory computer readable storage medium can store a program including a decoder (330), a post-processing module (350), a renderer (360), and instructions. The decoder (330) can decode a frame stored in an input buffer (410) among the plurality of frames of the video. The post-processing module (350) can post-process the decoded frame. The renderer (360) can render a frame stored in an output buffer (450). The frame stored in the output buffer (450) can be the decoded frame or the post-processed frame. The instructions, when individually or collectively executed by at least one processor (120) of the electronic device (101), may cause the electronic device (101) to configure the output buffer (450) based on a time required for configuring the post-processing. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101) to adjust a speed at which the plurality of frames are input to the input buffer (410) based on a speed at which the rendered frames are played back through the display (260).

[0298] Electronic devices according to one or more embodiments disclosed herein may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the aforementioned devices.

[0299] It should be understood that one or more embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but rather to encompass various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said item, unless the context clearly dictates otherwise. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish the corresponding component from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a (e.g., first) component is referred to as "coupled" or "connected" to another (e.g., second) component, with or without the terms "functionally" or "communicatively," it means that the corresponding component can be connected to the corresponding other component directly (e.g., wired), wirelessly, or through a third component.

[0300] One or more embodiments of the present invention may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0301] According to one or more embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to one or more embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. In some embodiments, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component of the plurality of components prior to the integration. According to one or more embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0302] Embodiments may be described and illustrated in terms of blocks that perform the function or functions described, as illustrated in the drawings. These blocks, which may be referred to herein as a post-processing module (30) in the demuxer (15) of FIG. 1, a renderer (360) in the processor (120) of FIG. 3, and other similar blocks in other drawings, may be physically implemented by analog and / or digital circuits including one or more of logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, and the like, and may also be implemented or driven by software and / or firmware (configured to perform the functions or operations described herein). For example, a circuit may be implemented on one or more semiconductor chips or on substrate supports such as a printed circuit board. Circuits included in a block may be implemented by dedicated hardware or processors (e.g., one or more programmed microprocessors and associated circuitry), or a combination of dedicated hardware that performs some functions of the block and processors that perform other functions of the block. Each block of the embodiments may be physically separated into two or more interacting and separate blocks. Similarly, the blocks of the embodiments may be physically combined into more complex blocks.

Claims

1. In an electronic device (101), Display (260), at least one processor (120); and A decoder (330), a post-processing module (350), a renderer (360), a memory (130) configured to store instructions, and including one or more storage media, The above decoder (330) is configured to decode a frame among the plurality of frames of the video stored in the input buffer (410) to generate a decoded frame, The above post-processing module (350) is configured to post-process the decoded frame to generate a post-processed frame. The above renderer (360) is configured to render the decoded frame or the post-processed frame stored in the output buffer (450), The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: The above output buffer (450) is set based on the time required for the post-processing, Causing the speed at which the plurality of frames are input into the input buffer (410) to be adjusted based on the speed at which the rendered frames are played back through the display (260). Electronic devices.

2. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Obtain an input requesting the above post-processing, Based on the above input, identify the generation time for generating the post-processed frame, Based on the above generation time, determine whether to stop playing the rendered frame through the display (260), Based on the decision to stop the above playback, if the above playback of the above video is stopped, the post-processing is set, Based on the decision not to stop the above playback, further causing the above post-processing to be set if the above video is played, Electronic devices.

3. In claim 2, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Based on the above input, search for a frame to be post-processed, Compare the generation time and the reference time, including the time for decoding the searched frame and the time for setting the post-processing, Based on the above generation time being less than the above reference time, the above playback is stopped, Causing a decision to continue playing the video based on the above generation time being greater than or equal to the above reference time; Electronic devices.

4. In claim 3, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: If the above creation time is greater than or equal to the above reference time and the reference timestamp is not set, it is determined to maintain playback of the video; If the generation time is greater than or equal to the reference time and the reference timestamp is set, determining whether to stop the playback of the rendered frame based on comparing the timestamp difference between the frame for which the post-processing is requested and the earliest frame for which the post-processing is possible and the reference timestamp. Electronic devices.

5. In claim 2, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Based on the above input, the post-processing is requested to search for the frame, Identify the most advanced frame that can be post-processed by setting the above post-processing, Identify the timestamp difference between the frame for which the post-processing is requested and the earliest frame for which the post-processing is possible, Compare the identified timestamp and the reference timestamp above, Based on the identified timestamp being less than the reference timestamp, it is determined to maintain playback of the video, Further causing a decision to stop playback of said video based on the identified timestamp being greater than or equal to said reference timestamp; Electronic devices.

6. In claim 5, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: If the identified timestamp is less than the reference timestamp and the reference time is not set, it is determined to maintain playback of the video; Further causing a determination to be made as to whether to stop the playback of the rendered frame based on comparing the generation time including the time for decoding the searched frame and the time for setting the post-processing with the reference time, if the identified timestamp is less than the reference timestamp and the reference time is set. Electronic devices.

7. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: A frame decoded by the above decoder (330) is stored and a backup buffer (430) accessed by the above post-processing module (350) is set up. Further causing the post-processed frame to be generated based on one or more decoded frames stored in the backup buffer (430) by the post-processing module (350). Electronic devices.

8. In claim 7, The size of the backup buffer (430) corresponds to the number of one or more decoded frames required according to the post-processing function provided by the post-processing module (350). Electronic devices.

9. In claim 7, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: Obtain an input requesting release of the above post-processing, Based on the above input, one or more decoded frames stored in the backup buffer (430) are stored in the output buffer (450), further causing the renderer (360) to render one or more decoded frames from the backup buffer (430) to the output buffer (450); Electronic devices.

10. In claim 7, The above memory (130) includes another post-processing module (350) that further post-processes the post-processed frame, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101) to: The frame post-processed by the above post-processing module (350) is stored and further causes another backup buffer (430) to be set up that is accessed by the other post-processing module (350). The above post-processed frame is stored in the other backup buffer (430) while the other post-processing module (350) is activated, The above post-processed frame is stored in the output buffer (450) while the other post-processing module (350) is inactive. Electronic devices.

11. A method of an electronic device (101) including a memory (130) including one or more storage media storing a decoder (330), a post-processing module (350), and a renderer (360), An operation of decoding a frame stored in an input buffer (410) among the plurality of frames of the video through the above decoder (330) to generate a decoded frame. An operation of generating a post-processed frame by post-processing the decoded frame through the post-processing module (350), and An operation of rendering the decoded frame or the post-processed frame stored in the output buffer (450) through the renderer (360) is included, The above output buffer (450) is set based on the time required to set the post-processing, The speed at which the above multiple frames are input into the input buffer (410) is adjusted based on the speed at which the rendered frames are played through the display (260). method.

12. In claim 11, An action for obtaining input requesting the above post-processing; An action for identifying a generation time for generating the post-processed frame based on the above input; An action for determining whether to stop playing the rendered frame through the display (260) based on the above generation time; Based on a decision to stop the said playback, if the said playback of the said video is stopped, an operation to set the said post-processing, and Based on a decision not to stop the above playback, if the above video is played, the action of setting the post-processing is included. method.

13. In claim 12, Based on the above input, an action is taken to search for a frame to be post-processed. An operation of comparing the generation time with a reference time, including a time for decoding the searched frame and a time for setting the post-processing; An action to determine to stop playing the video based on the above generation time being less than the above reference time, and An action including determining to maintain playback of the video based on the above generation time being greater than or equal to the above reference time. method.

14. In claim 13, An action to determine to maintain playback of the video if the above generation time is greater than or equal to the above reference time and the reference timestamp is not set, and If the generation time is greater than or equal to the reference time and the reference timestamp is set, an operation for determining whether to stop the playback of the rendered frame is included based on comparing the timestamp difference between the frame for which the post-processing is requested and the earliest frame for which the post-processing is possible and the reference timestamp. method.

15. In claim 12, Based on the above input, an action is taken to search for the frame for which post-processing is requested. An operation to identify the most advanced frame that can be post-processed by setting the above post-processing. An operation for identifying the timestamp difference between the frame for which the post-processing is requested and the earliest frame for which the post-processing is possible; An action to compare the identified timestamp and a reference timestamp; If the identified timestamp is greater than or equal to the reference timestamp, an action to maintain playback of the video; and If the identified timestamp above is less than the reference timestamp above, stop playing the video. contains actions method.

Images