Electronic device and video processing method using same

Abstract

According to various embodiments of the present disclosure, an electronic device may comprise: a display; a memory for storing instructions; and a processor. According to one embodiment, when executed by the processor, the instructions enable the electronic device to: detect an input of requesting playback of a first time point of a video while playing back the video; identify, on the basis of a table related to frames of the video, the number of frames from the first time point to a second time point indicating the previous sync frame; calculate, on the basis of the identified number of frames and the maximum performance of a codec according to the resolution of the video, the time required for decoding; and decode the video on the basis of the maximum performance of the codec according to the resolution of the video and the frames from the first time point to the second time point if it is identified that the calculated time required for decoding is within a reference time. In addition to various embodiments disclosed in the present document, various other embodiments are possible.

Description

Electronic device and image processing method using the same

[0001] Embodiments of the present disclosure relate to an electronic device and an image processing method using the same.

[0002] As various electronic devices such as smart phones, tablet PCs, portable multimedia players (PMPs), personal digital assistants (PDAs), laptop personal computers (PCs), and wearable devices become more widespread, interest in technologies for viewing or editing videos using various electronic devices is increasing.

[0003] When an input requesting playback of a specific point in time of a video is detected during video playback, the electronic device may decode and output a frame (e.g., a sync frame) (or a random access picture) at a specific point in time to output the specific point in time of the video. For example, the electronic device may decode and output a frame at a specific point in time based on codec information and resolution information of the video.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0005] However, if a frame at a specific point in the video is not a sync frame, the decoding must be sequentially decoded from the previous sync frame at the specific point in time, so the time required for decoding may vary depending on the interval between the specific point in time and the previous sync frame and the decoding speed. If the interval between sync frames is wide and the frame at a specific point in time is far from the sync frame, the time required for decoding may take a long time, which may cause motion delay. In addition, if the time required for decoding a video with a variable fps is calculated using only static information such as the average fps, codec type, and / or resolution, the accuracy may be low.

[0006] An electronic device according to one embodiment of the present disclosure, when an input requesting playback of a first point of view of an image is detected, calculates a time required for decoding using information related to the image and the number of frames from a first point of view identified based on a table related to frames of the image to a second point of view indicating a previous sync frame, and when the calculated time required for decoding is confirmed to be within a reference time, the electronic device can decode the image.

[0007] According to one embodiment of the present disclosure, an electronic device may include a display, a memory storing instructions, and a processor. According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to detect, while playing back a video, an input requesting playback of a first point in time of the video. According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to determine, based on a table associated with frames of the video, the number of frames from the first point in time to a second point in time representing a previous sync frame. According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to calculate a time required for decoding based on the maximum performance of a codec according to a resolution of the video and the determined number of frames. According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to decode the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if the time required for the calculated decoding is confirmed to be within a reference time.

[0008] According to one embodiment of the present disclosure, an image processing method of an electronic device may include an operation of detecting an input requesting playback of a first point in time of an image while the image is being played back. According to one embodiment, the image processing method of an electronic device may include an operation of checking the number of frames from the first point in time to a second point in time representing a previous sync frame, based on a table related to frames of the image. According to one embodiment, the image processing method of an electronic device may include an operation of calculating a time required for decoding, based on the maximum performance of a codec according to a resolution of the image and the number of frames checked. According to one embodiment, if the calculated time required for decoding is confirmed to be within a reference time, the image processing method of an electronic device may include an operation of decoding the image, based on the maximum performance of a codec according to a resolution of the image and the frames from the first point in time to the second point in time.

[0009] According to one embodiment of the present disclosure, a non-transitory computer-readable storage medium (or a computer program product) storing one or more programs may be described. The one or more programs according to one embodiment may include instructions that, when executed by a processor of an electronic device, detect an input requesting playback of a first time point of an image while playing back an image. The one or more programs according to one embodiment may include instructions that, when executed by the processor of the electronic device, determine, based on a table associated with frames of the image, the number of frames from the first time point to a second time point representing a previous sync frame. The one or more programs according to one embodiment may include instructions that, when executed by the processor of the electronic device, calculate a time required for decoding based on the maximum performance of a codec according to the resolution of the image and the determined number of frames. One or more programs according to one embodiment may include instructions for decoding the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first time point to the second time point, when the time required for the calculated decoding is confirmed to be within the reference time when executed by the processor of the electronic device.

[0010] An electronic device according to an embodiment of the present disclosure can provide fast output of a frame corresponding to a first time point by accurately calculating the time required for decoding by using information related to the video and the number of frames from a first time point identified based on a table related to frames of the video to a second time point indicating a previous sync frame. In addition, when the video has a variable fps, the electronic device can accurately calculate the time required for decoding by using not only static information (e.g., average fps, codec type, and / or resolution) but also dynamic information, e.g., a table related to frames of the video, to calculate the time required for decoding.

[0011] An electronic device according to one embodiment of the present disclosure can quickly determine whether decoding can be performed within a reference time by accurately calculating the time required for decoding, thereby reducing the time a user has to wait to output a frame corresponding to a first time point.

[0012] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment of the present disclosure.

[0013] FIG. 2 is a block diagram illustrating an electronic device according to one embodiment of the present disclosure.

[0014] FIG. 3 is a flowchart illustrating a method for obtaining a table related to frames of an image according to one embodiment of the present disclosure.

[0015] FIG. 4 is a diagram for explaining information included in a header of an image according to one embodiment of the present disclosure.

[0016] FIG. 5 is a diagram for explaining a method of updating a second table based on operation 320 of FIG. 3 described above, according to one embodiment of the present disclosure.

[0017] FIG. 6 is a diagram for explaining a method of updating a second table based on operations 325 to 335 of FIG. 3 described above, according to one embodiment of the present disclosure.

[0018] FIG. 7 is a flowchart illustrating an image processing method of an electronic device according to one embodiment of the present disclosure.

[0019] FIG. 8 is a flowchart illustrating an image processing method of an electronic device according to one embodiment of the present disclosure.

[0020] FIG. 9 is a diagram for explaining weight information set in an image according to one embodiment of the present disclosure.

[0021] FIG. 10 is a flowchart illustrating a method for setting a reference time based on weight information of an image according to one embodiment of the present disclosure.

[0022] FIG. 11 is a flowchart illustrating an image processing method of an electronic device according to one embodiment of the present disclosure.

[0023] FIG. 12 is a flowchart illustrating an image processing method of an electronic device according to one embodiment of the present disclosure.

[0024] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0025] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to one embodiment of the present disclosure.

[0026] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0027] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0028] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0029] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0030] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0031] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0032] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0033] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0034] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0035] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0036] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0037] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0038] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0039] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0040] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0041] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0042] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0043] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0044] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a printed circuit board (PCB)). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0045] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0046] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0047] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0048] FIG. 2 is a block diagram illustrating an electronic device (101) according to one embodiment of the present disclosure.

[0049] Referring to FIG. 2, an electronic device (101) (e.g., the electronic device (101) of FIG. 1) may include a communication circuit (210) (e.g., the communication module (190) of FIG. 1), a memory (220) (e.g., the memory (130) of FIG. 1), a touchscreen display (230) (e.g., the display module (160) of FIG. 1), and / or a processor (240) (e.g., the processor (120) of FIG. 1).

[0050] According to one embodiment of the present disclosure, a communication circuit (210) (e.g., a communication module (190) of FIG. 1) may establish a communication channel with an external electronic device (e.g., an electronic device (102) of FIG. 1) and support transmission and reception of various data with the external electronic device. For example, the communication circuit (210) may support establishment of a wireless communication channel with the external electronic device and performance of communication through the established communication channel.

[0051] According to one embodiment of the present disclosure, the memory (220) (e.g., the memory (130) of FIG. 1) performs a function of storing a program (e.g., the program (140) of FIG. 1) for processing and controlling the processor (240) of the electronic device (101), an operating system (OS) (e.g., the operating system (142) of FIG. 1), various applications, and / or input / output data, and may store a program that controls the overall operation of the electronic device (101). The memory (220) may store various instructions that may be executed by the processor (240).

[0052] In one embodiment, the memory (220) may, under the control of the processor (240), store instructions for calculating the time required to decode frames from the first time point to the second time point when an input requesting playback of the first time point is detected during playback of a video. The memory (220) may, under the control of the processor (240), store instructions for determining whether to perform decoding based on whether the calculated time required to decode frames from the first time point to the second time point is confirmed to be within a reference time.

[0053] According to one embodiment of the present disclosure, a touchscreen display (230) (e.g., display module (160) of FIG. 1) is implemented as an integrated unit including a display (231) and a touch panel (233), and can display an image under the control of a processor (240).

[0054] In one embodiment, the display (231) may be implemented as any one of a liquid crystal display (LCD), a light-emitting diode (LED) display, a micro LED (μLED) display, an organic light-emitting diode (OLED) display, an active matrix organic light-emitting diode (AMOLED) display, a micro electro mechanical systems (MEMS) display, an electronic paper display, a flexible display, a foldable display, or a rollable display, but is not limited thereto.

[0055] In one embodiment, the touch panel (233) may be a composite touch panel including a hand touch panel that detects hand gestures and a pen touch panel that detects pen gestures.

[0056] In one embodiment, the touchscreen display (230) can output an image under the control of the processor (240).

[0057] According to one embodiment of the present disclosure, a processor (240) (e.g., processor (120) of FIG. 1) may include, for example, a microcontroller unit (MCU) and may control a plurality of hardware components connected to the processor (240) by running an operating system (OS) or an embedded software program. The processor (240) may control a plurality of hardware components according to, for example, instructions stored in a memory (220) (e.g., program (140) of FIG. 1).

[0058] In various embodiments, the processor (240) may include at least one component (or module) for operations according to embodiments of the present disclosure. For example, the processor (240) may include at least one functional unit, such as an image playback unit (241), an image information collection and analysis unit (243), and / or a demuxer (245). According to one embodiment, at least some of the functional units may be included in the processor (240) as a hardware module (e.g., circuitry) and / or may be implemented as software including one or more instructions executable by the processor (240). For example, operations performed by the processor (240) may be stored in the memory (220) and, when executed, may be executed by instructions that cause the processor (240) to operate.

[0059] In one embodiment, the video playback unit (241) may play a video, and during the video playback, detect an input requesting playback of a first point in time of the video. Based on the video playback, the video playback unit (241) may initialize (e.g., set to a ready state) the video information collection and analysis unit (243) and the demuxer unit (245).

[0060] In one embodiment, the video information collection and analysis unit (243) may obtain information for calculating the time required to decode frames from a first time point to a second time point, or for adjusting a reference time for determining whether to perform decoding. For example, the information may include information related to the video (e.g., resolution and frames per second (fps) of the video, and / or maximum performance fps of the codec for each resolution), information on the number of frames from the first time point to a second time point representing a previous sync frame, information related to the usage status of the codec, information on the reading speed of the video, weight information of each frame of the video, performance information of the processor (240), usage information of the processor (240), remaining capacity information of the battery (e.g., battery (189) of FIG. 1), information related to whether the power saving mode is in operation, temperature information of the electronic device (101) (or temperature of the battery (189)), and / or concentration information of the user.

[0061] In one embodiment, the processor (240) (e.g., the video playback unit (241)) may set a reference time related to the time required for changing the video playback time point during video playback. For example, the reference time related to the time required for changing the video playback time point may be a time for determining whether to perform decoding based on frames from the first time point to the second time point representing the previous sync frame when an input requesting playback of a first time point different from the current playback time point is detected during video playback. The processor (240) (e.g., the video playback unit (241)) may reset the video information collection and analysis unit (e.g., the video information collection and analysis unit (243)) during video playback. The reset video information collection and analysis unit (243) may obtain information related to the video (e.g., the resolution and fps (frames per second) of the video and / or the maximum performance fps of the codec for each resolution). The processor (240) (e.g., the image information collection and analysis unit (243)) can check whether a first table related to frames of the image exists in the demuxer (245). The demuxer (245) can obtain (read) first information indicating whether each of all frames of the image corresponds to a sync frame among a plurality of pieces of information related to the image stored in the header of the image, and second information indicating time information (e.g., timestamp) of each frame. Based on the obtained first information and second information, the demuxer (245) can change each frame into a specific unit (e.g., millisecond). The demuxer (245) can indicate frames corresponding to sync frames as negative numbers (e.g., negative time information), and indicate frames not corresponding to sync frames among the frames of the image as positive numbers (e.g., positive time information), and store them in the first table.

[0062] In one embodiment, if it is determined that a first table related to frames of an image exists in the demuxer (245), the processor (240) (e.g., the image information collection and analysis unit (243)) may receive the first table from the demuxer (245) and update the second table. For example, the second table may include time information of each frame of the image and information indicating an interval between each frame (e.g., distance information or count information). The information indicating the interval between each frame may mean information in which frames corresponding to sync frames among a plurality of frames are indicated with a first value (e.g., 0), and at least one frame existing between sync frames (e.g., at least one frame that is not a sync frame) is indicated with a second value (e.g., 1) to an n-th value (e.g., n-1) by increasing the first value (e.g., 0) indicating the sync frame by a specified value (e.g., 1).

[0063] In one embodiment, if it is determined that the first table related to frames of the image does not exist in the demuxer (245), the processor (240) (e.g., the image information collection and analysis unit (243)) may sequentially request and receive information on frames of the image from the demuxer (245). The processor (240) (e.g., the image information collection and analysis unit (243)) may update the second table based on the frame information sequentially acquired from the demuxer (245).

[0064] In one embodiment, the processor (240) (e.g., the video playback unit (241)) may detect an input requesting playback of a first point in time of the video while playing back the video. Requesting playback of the first point in time of the video may mean decoding a frame corresponding to the first point in time for which the user requested playback and frames from the frame corresponding to the first point in time to a previous sync frame, and requesting an output from the decoded frame of the first point in time. The processor (240) (e.g., the video information collection and analysis unit (243)) may determine the number of frames from the first point in time to a second point in time representing the previous sync frame based on a table related to frames of the video. The processor (240) (e.g., the video information collection and analysis unit (243)) may calculate the time required for decoding based on the maximum performance of the codec according to the resolution of the video and the number of confirmed frames. If the time required for the generated decoding is confirmed to be within the reference time, the processor (240) can decode the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time.

[0065] An electronic device (101) according to one embodiment of the present disclosure may include a display (231), a memory (220) storing instructions, and a processor (240). The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to detect, while playing back an image, an input requesting playback of a first point in time of the image. The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to determine, based on a table associated with frames of the image, the number of frames from a first point in time of the image to a second point in time representing a previous sync frame. The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to calculate a time required for decoding based on the maximum performance of the codec according to the resolution of the image and the number of frames from a first point in time to a second point in time representing a previous sync frame. The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to decode the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if it is confirmed that the calculated time required for decoding is within a reference time.

[0066] In one embodiment, the reference time may be a time for determining whether to perform decoding based on frames from the first time point to the second time point when an input requesting playback of the first time point of the video is detected.

[0067] In one embodiment, the reference time may be a time obtained from the application playing the video, or a time set as a default.

[0068] The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to check whether a table related to frames of an image exists while playing back an image. The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to obtain a table related to frames of an image, if a table related to frames of an image exists.

[0069] The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to create a table related to frames of an image if a table related to frames of an image does not exist. The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to sequentially acquire frame information of each frame of an image. The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to update a table related to frames of an image that is generated based on frame information of each of the sequentially acquired frames.

[0070] In one embodiment, the frame information of each of the sequentially acquired frames may include information related to a sync frame and information indicating an interval between a sync frame and a non-sync frame.

[0071] Instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to decode a sync frame adjacent to the first time point if it is determined that the time required for the resulting decoding exceeds a reference time.

[0072] Instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to calculate the time required for decoding, taking into account the fps of the image.

[0073] The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to obtain reading speed information of an image while playing back the image. The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to calculate a second time required for decoding based on the obtained reading speed information of the image and the sizes of frames from a first time point of the image to a second time point representing a previous sync frame. The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to decode the image based on the maximum performance of a codec according to the resolution of the image and the frames from the first time point to the second time point if it is confirmed that the calculated second time required for decoding is within a reference time.

[0074] The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to check weight information of frames of an image while playing back the image. The instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to set a second reference time related to decoding of frames of the image based on the checked weight information of frames of the image.

[0075] Instructions according to one embodiment, when executed by the processor (240), may cause the electronic device (101) to decode an image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if the time required for the resulting decoding is confirmed to be within the second reference time.

[0076] FIG. 3 is a flowchart illustrating a method for obtaining a table related to frames of an image according to one embodiment of the present disclosure.

[0077] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0078] According to one embodiment, operations 305 to 335 may be understood to be performed by a processor (e.g., a processor (240) of FIG. 2) of an electronic device (e.g., an electronic device (101) of FIG. 1) (e.g., an image playback unit (241) and / or an image information collection and analysis unit (243) of FIG. 2).

[0079] Referring to FIG. 3, the processor (240) (e.g., the image playback unit (241) of FIG. 2) may play an image in operation 305. In one embodiment, the image may be an image stored in the memory of the electronic device (101) (e.g., the memory (220) of FIG. 2), an image stored in an external storage device (e.g., an SD device), or a streaming image provided from an external electronic device or server. However, the present invention is not limited thereto.

[0080] In one embodiment, the processor (240) (e.g., the video playback unit (241) of FIG. 2) may set a reference time related to the time required for changing the video playback time point in operation 310.

[0081] In one embodiment, the reference time associated with the time required for changing the playback point of the video may be a time for determining whether to perform decoding based on frames from the first point of time to a second point of time representing a previous sync frame when an input requesting playback of a first point of time different from the currently played point of time is detected while playing the video.

[0082] In one embodiment, the processor (240) (e.g., the video playback unit (241) of FIG. 2) may obtain a reference time related to the time required for changing the video playback point from the application that plays the video. However, this is not limited thereto, and if the processor (240) (e.g., the video playback unit (241) of FIG. 2) cannot obtain a reference time related to the time required for changing the video playback point from the application that plays the video, the processor (240) may use a reference time set as a default.

[0083] In one embodiment, the processor (240) (e.g., the image playback unit (241) of FIG. 2) may reset the image information collection and analysis unit (e.g., the image information collection and analysis unit (243) of FIG. 2) during image playback. For example, the operation of resetting the image information collection and analysis unit (243) may be an operation of preparing before acquiring information related to the image in order to determine the time required to perform image decoding when an input requesting playback of the first point in time is detected.

[0084] In one embodiment, the image information collection and analysis unit (243) may acquire image-related information after being reset. For example, the image-related information may include at least one of the image resolution, frames per second (fps), and the maximum performance fps of the codec for each resolution. However, this is not limited thereto.

[0085] In one embodiment, the processor (240) (e.g., the image information collection and analysis unit (243)) may, in operation 315, check whether a first table related to frames of an image exists in the demuxer (e.g., the demuxer (245) of FIG. 2).

[0086] In one embodiment, a header of an image may store a plurality of pieces of information related to the image. The demuxer (245) may acquire (read) first information indicating whether each of all frames of the image corresponds to a sync frame and / or second information indicating time information (e.g., timestamp) of each frame from among the plurality of pieces of information related to the image stored in the header. For example, the demuxer (245) may acquire first information indicating whether each of all frames of the image corresponds to a sync frame and / or second information indicating time information from the header of the image during the process of resetting the image information collection and analysis unit (243).

[0087] In one embodiment, the demuxer (245) may change the time information of each frame to a specific unit (e.g., milliseconds) based on the acquired first and second information. For example, if the expression range of the time information is changed to milliseconds, thereby increasing the time size, the amount of data that can be expressed per unit of time may increase. Accordingly, more data can be expressed during a specific time period.

[0088] In one embodiment, the demuxer (245) may change the time information into a specific unit (e.g., millisecond), and may indicate frames corresponding to sync frames among frames of the video as negative numbers (e.g., indicated with negative time information), and may indicate frames not corresponding to sync frames among frames of the video as positive numbers (e.g., indicated with positive time information). The demuxer (245) may store time information in a specific unit in which frames corresponding to sync frames among frames of the video are indicated as negative numbers, and time information in a specific unit in which frames not corresponding to sync frames among frames of the video are indicated as positive numbers, as a first table.

[0089] In one embodiment, if it is determined that the demuxer (245) has a first table related to frames of the image (e.g., YES in operation 315), the processor (240) (e.g., the image information collection and analysis unit (243)) may receive the first table from the demuxer (245) and update the second table in operation 320. For example, the processor (240) (e.g., the image information collection and analysis unit (243)) may generate a second table related to frames of the image. If the demuxer (245) has a first table related to frames related to the image, the processor (240) (e.g., the image information collection and analysis unit (243)) may update the generated second table based on the first table received from the demuxer (245).

[0090] In one embodiment, the second table may include time information of each frame of the video and information (e.g., distance information or count information) indicating an interval between each frame (e.g., an interval from a previous sync frame). The information indicating an interval between each frame may mean information in which frames corresponding to sync frames among a plurality of frames are indicated with a first value (e.g., 0), and each of at least one frame existing between the sync frames (e.g., at least one frame that is not a sync frame) is indicated with a second value (e.g., 1) to an n-th value (e.g., n-1) by increasing the first value (e.g., 0) indicating the sync frame by a specified value (e.g., 1). For example, it is described that the plurality of sync frames include a first sync frame, a second sync frame, and three frames between the first sync frame and the second sync frame, for example, a 1-1-th frame, a 1-2-th frame, and a 1-3-th frame. In this case, the processor (240) (e.g., the image information collection and analysis unit (243)) can update the second table by indicating the first sync frame and the second sync frame with a first value (e.g., 0), indicating the 1-1 frame existing between the first sync frame and the second sync frame with a second value (e.g., 1), indicating the 1-2 frame with a third value (e.g., 2), and indicating the 1-3 frame with a fourth value (e.g., 3).

[0091] In one embodiment, if it is determined that the first table related to frames of the image does not exist in the demuxer (245) (e.g., NO in operation 315), the processor (240) (e.g., the image information collection and analysis unit (243)) may request information on frames of the image from the demuxer (245) in operation 325. For example, the processor (240) (e.g., the image information collection and analysis unit (243)) may sequentially request information on each frame of the image from the demuxer (245). The demuxer (245) may sequentially transmit information on each frame of the image requested by the processor (240) (e.g., the image information collection and analysis unit (243)) to the processor (240) (e.g., the image information collection and analysis unit (243)). For example, the demultiplexer (245) can acquire (read) first information indicating whether a frame of the image corresponds to a sync frame and second information indicating time information of the frame among a plurality of pieces of information related to the image stored in the header of the image, and sequentially transmit them to the processor (240) (e.g., image information collection and analysis unit (243)).

[0092] In one embodiment, the processor (240) (e.g., the image information collection and analysis unit (243)) may update the second table based on the frame information obtained from the demuxer (245) in operation 330. For example, the processor (240) (e.g., the image information collection and analysis unit (243)) may extract first information (e.g., information indicating whether a frame of the image corresponds to a sync frame (e.g., true / false)) and second information (e.g., time information of the frame) from each frame information of the image sequentially obtained from the demuxer (245), and may update the second table by indicating the time information of each frame of the image and information indicating the interval between each frame (e.g., the interval with the previous sync frame).

[0093] In one embodiment, the processor (240) (e.g., the image information collection and analysis unit (243)) may end the operation of requesting information on frames of the image to the demuxer (245) and updating the second table when the frame information acquired from the demuxer (245) is confirmed as the last frame (end of stream, EOS) in operation 335 (e.g., YES in operation 335). For example, when the processor (240) (e.g., the image information collection and analysis unit (243)) is confirmed as the last frame (end of stream, EOS), the processor (240) may determine that information on all frames of the image has been acquired, and end the operation of requesting information on frames of the image to the demuxer (245) and updating the second table.

[0094] In one embodiment, if the frame information acquired from the demuxer (245) is not confirmed as the last frame (e.g., NO in operation 335), the processor (240) (e.g., image information collection and analysis unit (243)) branches to operation 325, requests information on frames of the image from the demuxer (245), and repeatedly performs an operation of updating the second table based on the frame information acquired from the demuxer (245). In one embodiment, the operation of updating the second table described above may be performed asynchronously.

[0095] FIG. 4 is a diagram for explaining information included in a header of an image according to one embodiment of the present disclosure.

[0096] Referring to FIG. 4, according to one embodiment, the header of an image may store information related to the image. For example, the information related to the image stored in the header may include first information (410) indicating whether each of all frames of the image corresponds to a sync frame and second information (420) indicating time information (e.g., timestamp) of each frame. In one embodiment, the first information (410) indicating whether each of all frames of the image corresponds to a sync frame may be set to a first value (e.g., true) for frames corresponding to sync frames, and may be set to a second value (e.g., false) for frames that do not correspond to sync frames. Texts such as true and false described as examples of the first and second values ​​described above are, in one embodiment, not limited thereto. For example, the first and second values ​​may be set to various values ​​such as Boolean values ​​(e.g., 0 and 1).

[0097] The information related to the image stored in the header according to one embodiment is not limited to the first information (410) and second information (420) described above, and the information related to the image stored in the header may further include at least one of the creation time of the image, the editing time of the image, the shooting location of the image, and information on the device that shot the image.

[0098] FIG. 5 is a diagram for explaining a method of updating a second table based on operation 320 of FIG. 3 described above, according to one embodiment of the present disclosure.

[0099] Reference numerals of FIG. 5 according to various embodiments <510> is a drawing showing the first table stored in the demuxer (e.g., the demuxer (245) of Fig. 2). Reference number <550> is a diagram showing a second table updated by an image information collection and analysis unit (e.g., the image collection and analysis unit (243) of FIG. 2) based on the first table.

[0100] Reference number according to one embodiment <510> Referring to , the demuxer (245) obtains first information (e.g., true, false) indicating whether each of all frames of the video corresponds to a sync frame among a plurality of pieces of information related to the video stored in the header, and second information indicating time information (e.g., timestamp) of each frame, and can change each frame into a specific unit (e.g., millisecond). For example, the demuxer (245) can change the time information into a specific unit (e.g., millisecond), and indicate frames corresponding to sync frames among the frames of the video with negative time information (511, 515), and indicate frames not corresponding to sync frames among the frames of the video with positive time information (512, 513, 514, 516, 517). The demultiplexer (245) can store each frame of the video as a first table by indicating it with negative time information (511, 515) (e.g., frames corresponding to a sync frame) and positive time information (512, 513, 514, 516, 517) (e.g., frames not corresponding to a sync frame).

[0101] Reference number according to one embodiment <550> Referring to , the image information collection and analysis unit (243) can obtain the first table from the demultiplexer unit (245) and update the second table. For example, the second table can include time information of each frame of the image and information indicating the interval between each frame (e.g., distance information or count information). The video information collection and analysis unit (243) obtains the first information (410) and the second information from the first table, and, together with the time information of each frame of the video, marks the first sync frame (551), the second sync frame (555), and the n-th sync frame (559) with a first value (e.g., 0), marks the 1-1 frame (552) existing between the first sync frame (551) and the second sync frame (555) with a second value (e.g., 1) (e.g., a value increased by the value (e.g., 1) specified in the first value (e.g., 0) indicating the 1-1st sync frame (551), marks the 1-2 frame (553) with a third value (e.g., 2) (e.g., a value increased by the value (e.g., 1) specified in the second value (e.g., 1) indicating the 1-1st sync frame (552), and marks the 1-3 frame (554) with a fourth value (e.g., 2). The second table can be updated by indicating the value (e.g., 3) (e.g., a value increased by the value (e.g., 1) specified in the third value (e.g., 2) representing the 1st-2nd sync frame (553). The 2nd-1 frame (556) and the 2nd-2nd frame (557) can also be indicated by the second value (e.g., 1) and the third value (e.g., 2) in the above-described manner, thereby updating the second table.

[0102] FIG. 6 is a diagram for explaining a method of updating a second table based on operations 325 to 335 of FIG. 3 described above, according to one embodiment of the present disclosure.

[0103] In one embodiment, a processor (e.g., processor (240) of FIG. 2), for example, an image information collection and analysis unit (e.g., image information collection and analysis unit (243) of FIG. 2), may request frame information of an image from the demuxer (e.g., demuxer (245) of FIG. 2) when it is determined that a first table related to frames of an image does not exist in the demuxer.

[0104] Referring to FIG. 6, the image information collection and analysis unit (243) may request the first frame information of the image from the demuxer (245) in operation 611. The demuxer (245) may obtain the first frame information (613) (e.g., sync frame: true, timestamp: 1000000) from the header of the image. In operation 615, the demuxer (245) may transmit the first frame information (613) (e.g., sync frame: true, timestamp: 1000000) to the image information collection and analysis unit (243). In operation 617, the video information collection and analysis unit (243) may extract information (e.g., true) indicating that the first frame is a sync frame and information (e.g., 1000000) indicating time information (timestamp) of the first frame based on the first frame information (613) (e.g., sync frame: true, timestamp: 1000000) received (or acquired) from the demuxer (245), and update (619) a second table based on the extracted information (e.g., true, 1000000) of the first frame. The updated second table may include time information (e.g., 1000) of the first frame (e.g., time information converted from the time information of the first frame to a specific unit (e.g., millisecond)) and distance information indicating an interval with the first frame, which is a sync frame (e.g., 0, which is information indicating that the first frame is a sync frame).

[0105] In one embodiment, the image information collection and analysis unit (243) may request second frame information of the image from the demuxer (245) in operation 621. The demuxer (245) may obtain second frame information (623) (e.g., sync frame: false, timestamp: 1033000) from the header of the image. In operation 625, the demuxer (245) may transmit the second frame information (623) (e.g., sync frame: false, timestamp: 1033000) to the image information collection and analysis unit (243). In operation 627, the video information collection and analysis unit (243) may extract information indicating that the second frame is not a sync frame (e.g., false) and information indicating the time information of the second frame (e.g., 1033000), based on the second frame information (623) (e.g., sync frame: false, timestamp: 1033000) received (or acquired) from the demuxer (245), and update (629) the second table based on the extracted information of the second frame (e.g., false, 1033000). For example, the image information collection and analysis unit (243) can update the second table (629) by adding the time information of the second frame (e.g., 1033) (e.g., time information converted to a specific unit (e.g., millisecond) of the time information of the second frame) to the information of the first frame described above (time information: 1000, information indicating that the first frame is a sync frame (e.g., 0)) and the distance information (e.g., 1) indicating the interval from the first frame, which is a sync frame, due to it not being a sync frame.

[0106] In one embodiment, the image information collection and analysis unit (243) may request third frame information of the image from the demuxer (245) in operation 631. The demuxer (245) may obtain third frame information (633) (e.g., sync frame: false, timestamp: 1066000) from the header of the image. In operation 635, the demuxer (245) may transmit the third frame information (633) (e.g., sync frame: false, timestamp: 1066000) to the image information collection and analysis unit (243). In operation 637, the video information collection and analysis unit (243) may extract information indicating that the third frame is not a sync frame (e.g., false) and information indicating the time information of the third frame (e.g., 1066000), based on the third frame information (633) (e.g., sync frame: false, timestamp: 1066000) received (or acquired) from the demuxer (245), and update (639) the second table based on the extracted information of the third frame (e.g., false, 1066000). For example, the image information collection and analysis unit (243) can update the second table (639) by adding time information (e.g., 1066) of the third frame (e.g., time information converted to a specific unit (e.g., millisecond) of the time information of the third frame) and distance information (e.g., 2) of the distance information of the first frame, which is a sync frame, to the information of the first frame described above (time information: 1000, information indicating that the first frame is a sync frame (e.g., 0)) and the information of the second frame (e.g., time information: 1033, distance information indicating an interval from the first frame, which is a sync frame).

[0107] In one embodiment, the image information collection and analysis unit (243) may request the 4th frame information of the image from the demuxer (245) in operation 641. The demuxer (245) may obtain the 4th frame information (643) (e.g., sync frame: false, timestamp: 1100000) from the header of the image. In operation 645, the demuxer (245) may transmit the 4th frame information (643) (e.g., sync frame: false, timestamp: 1100000) to the image information collection and analysis unit (243). In operation 647, the video information collection and analysis unit (243) may extract information indicating that the fourth frame is not a sync frame (e.g., false) and information indicating the time information of the fourth frame (e.g., 1100000), based on the fourth frame information (643) (e.g., sync frame: false, timestamp: 1100000) received (or acquired) from the demuxer (245), and update (649) the second table based on the extracted information of the fourth frame (e.g., false, 1100000). For example, the image information collection and analysis unit (243) can update the second table (649) by adding time information (e.g., 1100) of the fourth frame (e.g., time information of the fourth frame converted into a specific unit (e.g., millisecond)) and distance information (e.g., 3) indicating the interval from the first frame, which is a sync frame, to the information of the first frame described above (e.g., time information: 1000, information indicating that the first frame is a sync frame (e.g., 0)), information of the second frame (e.g., time information: 1033, distance information indicating the interval from the first frame, which is a sync frame (e.g., 1)), and information of the third frame (e.g., time information: 1066, distance information indicating the interval from the first frame, which is a sync frame (e.g., 2)).

[0108] In one embodiment, the image information collection and analysis unit (243) may request the fifth frame information of the image from the demuxer (245) in operation 651. The demuxer (245) may obtain the fifth frame information (653) (e.g., sync frame: true, timestamp: 1133000) from the header of the image. In operation 655, the demuxer (245) may transmit the fifth frame information (653) (e.g., sync frame: true, timestamp: 1133000) to the image information collection and analysis unit (243). In operation 657, the video information collection and analysis unit (243) may extract information indicating that the fifth frame is a sync frame (e.g., true) and information indicating time information of the fifth frame (e.g., 1133000), based on the fifth frame information (653) (e.g., sync frame: true, timestamp: 1133000) received (or acquired) from the demuxer (245), and update (659) the second table based on the extracted information of the fifth frame (e.g., true, 1133000). For example, the image information collection and analysis unit (243) can update the second table (659) by adding time information of the fifth frame (e.g., 1133) (e.g., time information of the fifth frame converted to a specific unit (e.g., millisecond)) and information of the fifth frame (e.g., 0) to the information of the first frame described above (e.g., time information: 1000, information indicating that the first frame is a sync frame (e.g., 0)), information of the second frame (e.g., time information: 1033, distance information indicating an interval from the first frame, which is a sync frame (e.g., 1)), information of the third frame (e.g., time information: 1066, distance information indicating an interval from the first frame, which is a sync frame (e.g., 2)), and information of the fourth frame (e.g., time information: 1100, distance information indicating an interval from the first frame, which is a sync frame (e.g., 3)).

[0109] The above-described operation according to one embodiment may be terminated when the frame information obtained from the demuxer (245) is confirmed as the last frame (end of stream, EOS).

[0110] In FIG. 6 according to various embodiments, it is assumed that information on each frame of the video is sequentially requested to the demuxer (245), but it is not limited thereto.

[0111] FIG. 7 is a flowchart for explaining an image processing method of an electronic device (101) according to one embodiment of the present disclosure.

[0112] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0113] According to one embodiment, operations 705 to 720 may be understood to be performed by a processor (e.g., a processor (240) of FIG. 2) of an electronic device (e.g., an electronic device (101) of FIG. 1) (e.g., an image playback unit (241) of FIG. 2, and / or an image information collection and analysis unit (243) of FIG. 2).

[0114] Referring to FIG. 7, the processor (240) (e.g., the video playback unit (241)) may, in operation 705, detect an input requesting playback of a first point in time of the video while playing the video.

[0115] In one embodiment, an input requesting playback of a first point in time of a video may include an input requesting playback of a first point in time (e.g., a playback point before or after the current playback point in time) while the video being played is being output to a display (e.g., the display (231) of FIG. 2). The input requesting playback of a first point in time of a video may also include an input requesting playback of a video through a second application (e.g., a video playback application) while the video is being previewed and displayed through a first application (e.g., a photo application). In addition, the input requesting playback of a first point in time of a video may also include an input requesting output of a video (e.g., video data and audio data) through a second application (e.g., a video playback application) while audio data of the video is being output (e.g., audio data is output in the background). However, the input is not limited thereto.

[0116] In one embodiment, requesting playback of a first point in time of a video may mean requesting output from the frame corresponding to the first point in time for which the user requested playback.

[0117] In one embodiment, the processor (240) (e.g., the image information collection and analysis unit (243)) may, in operation 710, determine the number of frames from the first point in time to the second point in time representing the previous sync frame based on a table related to frames of the image. For example, the processor (240) (e.g., the image information collection and analysis unit (243)) may determine the number of frames from the first point in time to the second point in time representing the previous sync frame based on the table related to frames of the image. <550> Based on the second table illustrated in or the second table illustrated in FIG. 6, the number of frames from the first point in time to the second point in time representing the previous sync frame can be confirmed.

[0118] For example, reference number of Fig. 5 <550> Referring to the second table shown in , when an input requesting playback of a first point in time of an image, for example, a first point in time where time information corresponds to “1100”, is detected, the processor (240) (e.g., the image information collection and analysis unit (243)) can confirm that the number of frames from the first point in time (e.g., the time information is “1100”) to the second point in time (e.g., the time information is “1000”) representing the previous sync frame is 4.

[0119] In one embodiment, the processor (240) (e.g., the image information collection and analysis unit (243)) may calculate the time required for decoding based on the maximum performance of the codec according to the resolution of the image and the number of confirmed frames in operation 715. However, this is not limited thereto, and the processor (240) (e.g., the image information collection and analysis unit (243)) may also calculate the time required for decoding by further considering the frame rates of the image.

[0120] As an example of calculating the time required for decoding according to one embodiment, it is assumed that the maximum performance fps of the codec according to the resolution of the image is 300 fps, and the number of frames from the first time point to the second time point representing the previous sync frame is 30. The processor (240) (e.g., the image information collection and analysis unit (243)) can calculate the time required for decoding based on 300 fps (e.g., the maximum performance fps of the codec) and 30 frames (e.g., the frames from the first time point to the second time point). For example, the processor (240) (e.g., the image information collection and analysis unit (243)) can calculate the time required for decoding as a value obtained by dividing 30 frames by 300 fps (e.g., 30 / 300 fps=0.1 second).

[0121] In one embodiment, the processor (240) can decode the video based on the maximum performance of the codec according to the resolution of the video and the frames from the first time point to the second time point, if the time required for the calculated decoding is confirmed to be within the reference time in the 720 operation.

[0122] In one embodiment, the reference time may be a time for determining whether to perform decoding based on frames from the first time point to the second time point representing the previous sync frame, when an input requesting playback of the first time point, such as the operation 705 described above, is detected.

[0123] In one embodiment, assuming a reference time of about 1 second, if the time required for decoding (e.g., 0.1 second) calculated in the above-described operation 715 is confirmed to be within the reference time (e.g., 1 second), the processor (240) can decode the image based on 300 fps (e.g., the maximum performance fps of the codec) and 30 frames (e.g., frames from the first time point to the second time point). The processor (240) can display the decoded image of the first time point on the display (231).

[0124] In various embodiments, although not shown, if it is determined that the time required for the produced decoding exceeds the reference time, the processor (240) may determine that it is not possible to decode the image in the section corresponding to the first time point to the second time point.

[0125] As an example, it is assumed that the maximum performance fps of the codec according to the resolution of the video is 60 fps, and the number of frames from the first time point to the second time point representing the previous sync frame is 100. The processor (240) can calculate the time required for decoding based on 60 fps (e.g., the maximum performance fps of the codec) and 100 frames (e.g., frames from the first time point to the second time point). For example, the processor (240) can calculate the time required for decoding as a value obtained by dividing the 100 frames by 60 fps (e.g., 100 / 60 fps=approximately 1.67 seconds). When the processor (240) determines that the time required for decoding (e.g., approximately 1.67 seconds) exceeds the reference time (e.g., 1 second), the processor (240) can determine that the video corresponding to the section from the first time point to the second time point cannot be decoded. In this case, the processor (240) can display (or play, or output) an image of a sync frame adjacent to a frame corresponding to the first point in time at which playback is requested.

[0126] In one embodiment, although not shown, a table (e.g., reference numeral 5) associated with frames of the image <550> The second table illustrated in FIG. 6 or the second table illustrated in FIG. 6) may not contain information about the number of frames from the first time point to the second time point representing the previous sync frame. For example, if playback of the first time point that is not included in the second table is requested while updating the second table, the second table may not contain information about the number of frames from the first time point to the second time point representing the previous sync frame. In this case, the processor (240) may determine that it is not possible to decode the video in the section corresponding to the first time point to the second time point.

[0127] In FIG. 7 according to various embodiments, the time required to decode frames from the first point in time to the second point in time is calculated based on the maximum performance of the codec according to the resolution of the image and the number of frames from the first point in time to the second point in time, but is not limited thereto. For example, the processor (240) may also calculate the time required to decode frames from the first point in time to the second point in time based on the reading speed of the image and the size of the frames from the first point in time to the second point in time. In this regard, various embodiments will be described in FIG. 8 described below.

[0128] FIG. 8 is a flowchart for explaining an image processing method of an electronic device (101) according to one embodiment of the present disclosure.

[0129] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0130] According to one embodiment, operations 805 to 825 may be understood to be performed by a processor (e.g., a processor (240) of FIG. 2) of an electronic device (e.g., an electronic device (101) of FIG. 1) (e.g., an image playback unit (241) of FIG. 2, and / or an image information collection and analysis unit (243) of FIG. 2).

[0131] Referring to FIG. 8, the processor (240) (e.g., the video playback unit (241) of FIG. 2) may detect, in operation 805, an input requesting playback of a first point in time of the video while playing back the video. For example, the input requesting playback of the first point in time of the video may include an input requesting playback of the first point in time (e.g., a playback point before or after the current playback point in time) while outputting the video being played to a display (e.g., the display (231) of FIG. 2), an input requesting playback of the video through a second application (e.g., a video playback application) while previewing and displaying the video through a first application (e.g., a photo application), or an input requesting output of the video (e.g., video data and audio data) through a second application (e.g., a video playback application) while outputting audio data of the video (e.g., outputting audio data in the background).

[0132] In one embodiment, the processor (240) (e.g., the image information collection and analysis unit (243)) may obtain the reading speed of the image in operation 810. For example, when streaming an image, the processor (240) (e.g., the image information collection and analysis unit (243)) may obtain the reading speed of the image corresponding to the network transmission speed based on the network status from a communication circuit (e.g., the communication circuit (210) of FIG. 2). In another example, when playing an image stored in an external storage device (e.g., an SD card), the processor (240) (e.g., the image information collection and analysis unit (243)) may obtain the reading speed of the image from the external storage device.

[0133] In one embodiment, the processor (240) (e.g., the image information collection and analysis unit (243)) may, in operation 815, determine the size of frames from the first point in time to the second point in time representing the previous sync frame based on a table related to frames of the image. For example, the processor (240) (e.g., the image information collection and analysis unit (243)) may ... second point in time to the second point in time representing the previous sync frame based on a table related to frames of the image. For example, the processor (240) may determine the size of frames from the first point in time to the second point in time representing the previous sync frame based on a table related to frames of the image. For example, the processor (240) may determine the size of frames from <550> Based on the second table illustrated in FIG. 6 or the second table illustrated in FIG. 6, the number of frames from the first time point to the second time point representing the previous sync frame can be confirmed. The processor (240) (e.g., the image information collection and analysis unit (243)) can confirm the size of the frames from the first time point to the second time point representing the previous sync frame based on the number of frames from the first time point to the second time point representing the previous sync frame.

[0134] In one embodiment, the processor (240) (e.g., the image information collection and analysis unit (243)) may calculate the time required for decoding based on the reading speed of the acquired image and the size of the confirmed frames in operation 820.

[0135] As an example of calculating the time required for decoding according to one embodiment, it is assumed that the reading speed of the video is 312 Mbps and the size of the frames from the first time point to the second time point representing the previous sync frame is 100 Mb. The processor (240) (e.g., the video information collection and analysis unit (243)) can calculate the time required for decoding based on 312 Mbps (e.g., the reading speed of the video) and 100 Mb (e.g., the size of the frames from the first time point to the second time point). For example, the processor (240) (e.g., the video information collection and analysis unit (243)) can calculate the time required for decoding as the value obtained by dividing the size of the frames, which is 100 Mb, by 312 Mbps, which is the reading speed of the video (e.g., 100 Mb / 312 Mbps=approximately 0.32 seconds).

[0136] In one embodiment, the processor (240) may decode the video based on the maximum performance of the codec according to the resolution of the video and the frames from the first time point to the second time point representing the previous sync frame, if the time required for the calculated decoding is confirmed to be within the reference time in operation 825.

[0137] In one embodiment, the reference time may be a time for determining whether to perform decoding based on frames from the first time point to the second time point representing the previous sync frame when an input requesting playback of the first time point, such as the operation 805 described above, is detected.

[0138] In one embodiment, assuming the reference time to be about 1 second, since the time required for decoding (e.g., about 0.32 seconds) calculated in the above-described operation 820 is confirmed to be within the reference time (e.g., 1 second), the processor (240) can decode the video based on the maximum performance of the codec according to the resolution of the video and the frames from the first time point to the second time point representing the previous sync frame. The processor (240) can display the decoded first time point video on the display (231).

[0139] FIG. 9 is a diagram for explaining weight information set in an image according to one embodiment of the present disclosure.

[0140] Referring to FIG. 9, a processor (e.g., processor (240) of FIG. 2) may further include an image weight information management unit. The image weight information management unit may manage (or store) weight information for frames of a currently playing image.

[0141] In one embodiment, the frames set as weight information among the frames of the video may include, but are not limited to, at least one frame corresponding to a time point set as a bookmark by the user, frames corresponding to a slow playback section, and / or at least one frame determined as a key scene based on image analysis.

[0142] In one embodiment, the image weight information management unit can map the frame number (910) and the weight value (915) of the image (905) and store (or manage) them as weight information. For example, the image weight information management unit can map the frame number (910) of the image (905) 5 and the weight value (915) 10 and store them as first weight information (920) of the image (905). The image weight information management unit can map the frame numbers (910) of the image (905) 8 to 10 and the weight value (915) 5 and store them as second weight information (925). The frame numbers (910) of the image (905) 13 to 15 and the weight value (915) 4 can be mapped and stored as third weight information (930). The frame number (910) and weight value (915) of the image (905) stored as the first weight information (920), second weight information (925), and third weight information (930) described above are just one example for ease of explanation and are not limited thereto.

[0143] In one embodiment, when the image weight information management unit receives a request for image weight information from the image information collection and analysis unit (e.g., the image information collection and analysis unit (243) of FIG. 2), the image weight information management unit may transmit weight information for frames of the image (e.g., frame numbers 1 to 19) to the image information collection and analysis unit (243). For example, the image weight information management unit may transmit weight information for frames of the image (e.g., frame numbers 1 to 19) in the form of a table to the image information collection and analysis unit (243). The present invention is not limited thereto, and the image weight information management unit may transmit only weight information for specific frames (e.g., frame numbers 5, 8 to 10, 13 to 15) for which weight information is set among the frames of the image to the image information collection and analysis unit (243). Alternatively, the image weight information management unit may transmit only weight information for a frame corresponding to the first point in time at the request of the image information collection and analysis unit (243).

[0144] FIG. 10 is a flowchart illustrating a method for setting a reference time based on weight information of an image according to one embodiment of the present disclosure.

[0145] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0146] According to one embodiment, operations 1005 and 1010 may be understood to be performed in a processor (e.g., processor (240) of FIG. 2) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0147] Referring to FIG. 10, the processor (240) may check weight information of frames of a video in operation 1005. For example, the processor (240) may further include an image weight information management unit that manages (or stores) weight information for frames of a currently playing video. The processor (240) (e.g., the image information collection and analysis unit (243) of FIG. 2) may receive (or obtain) weight information for frames of a video (e.g., frame numbers 1 to 19 of FIG. 9) in a table format from the image weight information management unit, or may receive (or obtain) only weight information for specific frames for which weight information is set (e.g., frame numbers 5, 8 to 10, 13 to 15 of FIG. 9). The present invention is not limited thereto, and the processor (240) (e.g., the image information collection and analysis unit (243) of FIG. 2) may request weight information for a frame corresponding to a first point in time from the image weight information management unit. In this case, the processor (240) (e.g., the image information collection and analysis unit (243) of FIG. 2) may receive (or acquire) only the weight information of the frame corresponding to the first point in time (e.g., frame number 8 when frame number 8 of FIG. 9 is the frame corresponding to the first point in time) from the image weight information management unit.

[0148] In one embodiment, the processor (240) may, in operation 1010, set a reference time associated with the decoding of each frame of the video based on the identified weight information.

[0149] In one embodiment, the reference time associated with the decoding of each frame of the video can be changed or maintained according to weight information of each frame of the video.

[0150] In one embodiment, the processor (240) may determine a first state (e.g., high priority) when the weight value of the frame exceeds a first weight value (e.g., 7), determine a second state (e.g., medium priority) when the weight value of the frame exceeds a second weight value (e.g., 3), and determine a third state (e.g., low priority) when the weight value of the frame is less than or equal to the second weight value.

[0151] In one embodiment, the processor (240) can set a reference time for each frame of the image by multiplying the weight ratio value set in the determined state by “1 second” set as the reference time. For example, in the following embodiment, it is assumed that the weight ratio value set in the first state (e.g., high priority) is “5”, the weight ratio value set in the second state (e.g., medium priority) is “3”, and the weight ratio value set in the third state (e.g., low priority) is “1”.

[0152] For example, as shown in FIG. 9, the reference time for a frame (e.g., frame number (910) 5) of an image (905) having a weight value (915) of 10 as the first weight information (920) can be changed from 1 second to 5 seconds (e.g., 1 second x 5.0 = 5 seconds). As another example, the reference time for a frame (e.g., frame numbers (910) 8 to 10) of an image (905) having a weight value (915) of 5 as the second weight information (925) can be changed from 1 second to 3 seconds (e.g., 1 second x 3.0 = 3 seconds). As another example, the reference time for frames (e.g., frame numbers (910) 13 to 15) of an image (905) having a weight value (915) of 4 as the third weight information (925) may be changed from 1 second to 4 seconds (e.g., 1 second x 3.0 = 4 seconds). As another example, the reference time for frames (e.g., frame numbers (910) 0 to 4, 6, 7, 11, 12, 16 to 19) of an image (905) having a weight value (915) of 1 other than the aforementioned frames may be maintained at 1 second (e.g., 1 second x 1.0 = 1 second).

[0153] FIG. 11 is a flowchart for explaining an image processing method of an electronic device (101) according to one embodiment of the present disclosure.

[0154] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0155] According to one embodiment, operations 1105 and 1120 may be understood to be performed in a processor (e.g., processor (240) of FIG. 2) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0156] Since operations 1105 to 1115 of FIG. 11 according to various embodiments are substantially the same as operations 705 to 715 of FIG. 7 described above, detailed descriptions thereof may be replaced with the description of FIG. 7.

[0157] Referring to FIG. 11, in operation 1105, the processor (240) may detect an input requesting playback of a first point in time of the video while playing back the video. In operation 1110, the processor (240) may determine the number of frames from the first point in time to the second point in time representing the previous sync frame based on a table related to frames of the video. In operation 1115, the processor (240) may calculate the time required for decoding based on the maximum performance of the codec according to the resolution of the video and the number of confirmed frames.

[0158] In one embodiment, the processor (240) may decode the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if the time required for the calculated decoding is confirmed to be within the reference time set based on the weight information in operation 1120.

[0159] As an example, assuming that the maximum performance fps of the codec according to the resolution of the video is 60 fps, the number of frames from the first time point to the second time point representing the previous sync frame is 100, the reference time related to the time required for changing the video playback time point is 1 second, and since the weight value of the frames from the first time point to the second time point is set to 3, the reference time set based on the weight information is assumed to be 3 seconds (e.g., 1 second x 3.0 = 3 seconds). The processor (240) can calculate the time required for decoding based on 60 fps (e.g., the maximum performance fps of the codec) and 100 frames (e.g., frames from the first time point to the second time point). For example, the processor (240) can calculate the time required for decoding as a value obtained by dividing 100 frames by 60 fps (e.g., 100 / 60 fps = approximately 1.67 seconds). When the processor (240) determines that the time required for decoding (e.g., about 1.67 seconds) is less than the reference time (e.g., 3 seconds) set based on the weight information, the processor (240) can decode the image and display it on a display (e.g., the display (231) of FIG. 2) based on 60 fps (e.g., the maximum performance of the codec) and 100 frames (e.g., frames from the first point in time to the second point in time).

[0160] In various embodiments, as shown in FIGS. 9 to 11, the reference time for determining whether to perform decoding based on frames from a first time point to a second time point representing a previous sync frame is adjusted based on weight information of at least one frame set in the video, so that, for example, the probability of performing decoding for a frame section having large weight information may be high. Accordingly, the electronic device (101) may provide a user with the ability to accurately reproduce a video at a desired time point.

[0161] In FIG. 7 according to various embodiments, the maximum performance of the codec according to the resolution of the video, the number of frames from the first time point to the second time point representing the previous sync frame when playback is requested, and in FIG. 8, the time required to decode the frames from the first time point to the second time point is calculated based on the reading speed of the video, and in FIGS. 9 to 11, it is described that whether to perform decoding of the frames from the first time point to the second time point is determined based on a reference time set based on weight information, but the present invention is not limited thereto. For example, the processor (240) may calculate the time required to decode the frames from the first time point to the second time point by considering all of the maximum performance of the codec according to the resolution of the video, the number of frames from the first time point to the second time point representing the previous sync frame, information related to the usage status of the codec, the reading speed of the video, and the weight information, and determine whether to decode. In this regard, various embodiments will be described in FIG. 12 described below.

[0162] FIG. 12 is a flowchart for explaining an image processing method of an electronic device (101) according to one embodiment of the present disclosure.

[0163] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0164] According to one embodiment, operations 1205 and 1255 may be understood to be performed in a processor (e.g., processor (240) of FIG. 2) (e.g., image information collection and analysis unit (243)) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0165] Referring to FIG. 12, the processor (240) may detect, in operation 1205, an input requesting playback of a first point in time of the video while playing back the video. For example, the input requesting playback of the first point in time of the video may include an input requesting playback of the first point in time (e.g., a playback point before or after the current playback point in time) while outputting the video being played to a display (e.g., the display (231) of FIG. 2), an input requesting playback of the video through a second application (e.g., a video playback application) while previewing and displaying the video through a first application (e.g., a photo application), or an input requesting output of the video (e.g., video data and audio data) through a second application (e.g., a video playback application) while outputting audio data of the video (e.g., outputting audio data in the background).

[0166] In one embodiment, the processor (240) may obtain weight information of frames of an image in operation 1210. In operation 1215, the processor (240) may set a reference time associated with decoding of each frame of the image based on the weight information.

[0167] For example, as examined in the aforementioned FIG. 10, based on the weight information for the frames of the image obtained from the image weight information management unit (not shown), the processor (240) (e.g., the image information collection and analysis unit (243) of FIG. 2) can set a reference time related to the decoding of each frame of the image. For example, the image weight information management unit can map the frame number and the weight value of the image and store (or manage) them as weight information. The processor (240) (e.g., the image information collection and analysis unit (243) of FIG. 2) can set a reference time related to the decoding of each frame of the image (e.g., change (or maintain) the reference time) by applying a weight ratio corresponding to the weight value of the frame number to the reference time.

[0168] In one embodiment, the processor (240) may obtain information related to the usage status of the codec at operation 1220. The processor (240) may obtain maximum performance information of the codec at operation 1225 (e.g., maximum performance fps of the codec according to the resolution of the image). At operation 1230, the processor (240) may calculate and store the maximum performance of an available codec based on the obtained information related to the usage status of the codec and the obtained maximum performance information of the codec.

[0169] In one embodiment, the maximum performance fps of the codec may vary depending on the usage status of the codec used in the electronic device (101). For example, if no application uses the codec in the electronic device (101), the codec may perform at its maximum performance, but if at least one application uses the codec, the processor (240) may perform video decoding at a performance that excludes the performance of the codec used by at least one application from the maximum performance of the codec.

[0170] In one embodiment, although not shown, the middleware (e.g., the middleware (144) of FIG. 1) of the electronic device (101) may include a resource management unit. The processor (240) may receive information related to the usage status of the codec from the resource management unit. The processor (240) may calculate the maximum performance of an available codec based on the maximum performance information of the codec and the information related to the usage status of the codec received from the resource management unit. For example, the maximum performance of an available codec may be calculated based on the following <Mathematical Formula 1>.

[0171]

[0172] ( : resolution of the video, : Codec maximum performance fps depending on video resolution, : Codec maximum performance figures, : sum of the performance of the currently used codecs, n: number of codecs used, fps: other video fps in use)

[0173] As an example, if the resolution of the video is “1920x1080”, and the maximum performance that the codec can process the video with the “1920x1080” resolution is 240 fps, and the performance of the codec being used in the electronic device (101) is, for example, “1920x1080x30 fps”, “1280x720x60 fps”, “3840x2160x24 fps”, then the maximum performance of the available codec can be calculated to be approximately 87.33 fps, as in <Mathematical Expression 2> below. Although the maximum performance that the codec can process the video with the “1920x1080” resolution is 240 fps, the maximum performance of the available codec calculated according to <Mathematical Expression 2> can be calculated to be approximately 87.33 fps. The processor (240) can store about 87.33 fps, which is the maximum performance of the available codec produced.

[0174]

[0175] Not limited thereto, if information related to the usage status of the codec is not obtained (e.g., if the video can be processed at the maximum performance of the codec due to the absence of running applications), operations 1220 and 1230 may be omitted. For example, the processor (240) may use the maximum performance information of the codec obtained in operation 1225 to calculate the time required for decoding, which will be described later.

[0176] In one embodiment, the processor (240) may obtain the reading speed of the image in operation 1235, and store the reading speed of the obtained image in operation 1240. For example, as described in FIG. 8, when playing a streaming image, the processor (240) may obtain the reading speed of the image corresponding to the network transmission speed based on the network status from a communication circuit (e.g., the communication circuit (210) of FIG. 2). The present invention is not limited thereto, and when playing an image stored in an external storage device (e.g., an SD card), the processor (240) may obtain the reading speed of the image from the external storage device. The processor (240) may store the reading speed (e.g., network transmission speed) of the image obtained from the communication circuit (210) or the reading speed of the image obtained from the external storage device.

[0177] In one embodiment, the processor (240) may, in operation 1245, determine the number of frames from a first point in time to a second point in time representing a previous sync frame based on a table related to frames of the video. For example, the table related to frames of the video may be referred to as the reference number of FIG. 5 described above. <550> The processor (240) may include a second table illustrated in FIG. 6 or a second table illustrated in FIG. 6. The processor (240) may determine the number of frames from a first point in time to a second point in time representing a previous sync frame based on a table (e.g., the second table) related to frames of the image.

[0178] In one embodiment, although not shown, if there is no information about the number of frames from the first time point to the second time point representing the previous sync frame in a table (e.g., the second table) related to frames of the video, the processor (240) may determine that it is not possible to perform decoding of frames from the first time point to the second time point representing the previous sync frame, and may terminate the corresponding operation (e.g., the operation of FIG. 12). The present invention is not limited thereto, and the processor (240) may also reproduce a video of a sync frame adjacent to a frame corresponding to the first time point for which reproduction is requested.

[0179] In one embodiment, the processor (240) may calculate the time required for decoding, at operation 1250, based on the maximum performance of the available codec, the read speed of the video, and the number of frames from the first time point to the second time point representing the previous sync frame. If the calculated time required for decoding is confirmed to be within a reference time set based on the weight information, the processor (240) may decode the video, at operation 1255.

[0180] Although not shown, the performance and / or usage rate of the processor (240) may further be included as a condition based on which the time required to decode frames from the first time point to the second time point according to various embodiments is calculated. For example, since the performance and / or usage rate of the processor (240) may affect the time required for decoding, the processor (240) may set the time required for decoding to be greater than or less than a reference time by taking into account the performance (e.g., clock speed and / or number of cores) and / or usage rate of the processor (240).

[0181] It is not limited thereto, and may further include information on the remaining capacity of a battery (e.g., battery (189) of FIG. 1), whether a power saving mode is in operation, the temperature of the electronic device (101) (or the temperature of the battery (189)), and / or the concentration of the user as a condition based on which the time required to decode frames from the first point in time to the second point in time is calculated.

[0182] For example, if the first time point is not a sync frame, decoding of frames from the first time point to the second time point representing the previous sync frame may consume more battery (189) because the number of frames is greater than decoding of the sync frame if the first time point is a sync frame. In consideration of this, if the remaining amount of the battery (189) is less than a specified remaining amount or the electronic device (101) operates in a power saving mode, the processor (240) may set the time required for decoding to exceed a reference time in order to limit the decoding of frames from the first time point to the second time point representing the previous sync frame.

[0183] For another example, since decoding of frames from a first time point to a second time point representing a previous sync frame may affect the heat generation of the electronic device (101) (or the battery (189)), the temperature of the electronic device (101) (or the temperature of the battery (189)) may increase. In consideration of this, if the temperature of the electronic device (101) (or the temperature of the battery (189)) exceeds a specified temperature value, the processor (240) may set the time required for decoding to exceed a reference time in order to limit the decoding of frames from the first time point to a second time point representing a previous sync frame. The present invention is not limited thereto, and if the temperature of the electronic device (101) (or the temperature of the battery (189)) exceeds a specified temperature value, the processor (240) may determine that it is impossible to perform decoding of frames from the first time point to a second time point representing a previous sync frame, and may terminate the corresponding operation.

[0184] As another example, when performing decoding of frames from a first point in time to a second point in time representing a previous sync frame, the user's concentration level may be used as weight information. For example, if it is determined that the user is looking at a display (e.g., display (231) of FIG. 2) outputting a specific frame of an image and / or a touch interaction (e.g., a gesture for enlarging the screen or a gesture for tracking a specific object within the image) is detected while outputting a specific frame of an image, the processor (240) may recognize that the user's concentration level is high and may set the weight value of the specific frame of the image to be higher than a designated weight value. As another example, if it is determined that the user is not looking at the display (231) outputting a specific frame of an image and / or a touch interaction such as skipping a specific frame of an image is detected, the processor (240) may recognize that the user's concentration level is low and may set the weight value of the specific frame of the image to be lower than a designated weight value.

[0185] In FIGS. 3 to 12 according to various embodiments, the electronic device (101) can accurately calculate the time required for decoding to output an image at a specific point in time by further considering not only information related to the image (e.g., resolution and fps of the image and / or maximum performance fps of the codec for each resolution), but also the number of frames required for decoding based on a table related to frames of the image, information related to the usage status of the codec, the reading speed of the image, and / or weight information. Accordingly, an operation on whether to perform decoding to output an image at a specific point in time can also be quickly determined, so that the waiting time for the user can also be reduced.

[0186] An image processing method of an electronic device (101) according to an embodiment of the present disclosure may include an operation of detecting an input requesting playback of a first point of time of an image while playing back an image. An image processing method of an electronic device (101) according to an embodiment may include an operation of checking the number of frames from a first point of time of an image to a second point of time representing a previous sync frame, based on a table related to frames of the image. An image processing method of an electronic device (101) according to an embodiment may include an operation of calculating a time required for decoding, based on the maximum performance of a codec according to the resolution of the image and the number of frames from the first point of time of the image to the second point of time representing a previous sync frame. An image processing method of an electronic device (101) according to an embodiment may include an operation of decoding an image, based on the maximum performance of a codec according to the resolution of the image and the frames from the first point of time to the second point of time, if it is confirmed that the calculated time required for decoding is within a reference time.

[0187] In one embodiment, the reference time may be a time for determining whether to perform decoding based on frames from the first point in time of the video to the second point in time representing the previous sync frame when an input requesting playback of the first point in time of the video is detected.

[0188] In one embodiment, the reference time may be a time obtained from the application playing the video, or a time set as a default.

[0189] An image processing method of an electronic device (101) according to one embodiment may include an operation of checking whether a table related to frames of an image exists while playing back an image. If a table related to frames of an image exists, the image processing method of an electronic device (101) according to one embodiment may include an operation of obtaining a table related to frames of an image.

[0190] An image processing method of an electronic device (101) according to one embodiment may include an operation of generating a table related to frames of an image if a table related to frames of an image does not exist. An image processing method of an electronic device (101) according to one embodiment may include an operation of sequentially acquiring frame information of each frame of an image. An image processing method of an electronic device (101) according to one embodiment may include an operation of updating a table related to frames of the generated image based on frame information of each of the sequentially acquired frames.

[0191] In one embodiment, the frame information of each of the sequentially acquired frames may include information related to a sync frame and information indicating an interval between a sync frame and a non-sync frame.

[0192] An image processing method of an electronic device (101) according to one embodiment may include an operation of decoding a sync frame adjacent to a first time point when it is determined that the time required for the calculated decoding exceeds a reference time.

[0193] The operation of calculating the time required for decoding according to one embodiment may include calculating the time required for decoding by further considering the fps of the video.

[0194] An image processing method of an electronic device (101) according to one embodiment may include an operation of acquiring reading speed information of an image while playing back an image. An image processing method of an electronic device (101) according to one embodiment may include an operation of calculating a second time required for decoding based on the acquired reading speed information of the image and the sizes of frames from a first time point to a second time point representing a previous sync frame. An image processing method of an electronic device (101) according to one embodiment may include an operation of decoding an image based on the maximum performance of a codec according to the resolution of the image and the frames from the first time point to the second time point if it is confirmed that the calculated second time required for decoding is within a reference time.

[0195] An image processing method of an electronic device (101) according to one embodiment may include an operation of checking weight information of frames of an image while playing back an image. An image processing method of an electronic device (101) according to one embodiment may include an operation of setting a second reference time related to decoding of frames of an image based on the checked weight information of frames of an image.

[0196] An image processing method of an electronic device (101) according to one embodiment may include an operation of decoding an image based on the maximum performance of a codec according to the resolution of the image and frames from a first point in time to a second point in time, if the time required for the calculated decoding is confirmed to be within a second reference time.

[0197] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0198] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0199] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0200] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0201] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0202] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101), display (231); Memory (220) for storing instructions; and Contains a processor (240), The above instructions, when executed by the processor (240), cause the electronic device (101) to: While playing a video, detect an input requesting playback of a first point in the video, Based on the table related to the frames of the above video, the number of frames from the first point in time to the second point in time representing the previous sync frame is determined, Based on the maximum performance of the codec according to the resolution of the above video and the number of frames confirmed above, the time required for decoding is calculated, and An electronic device that decodes the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if the time required for the above-described decoding is confirmed to be within the reference time.

2. In paragraph 1, The above reference time is a time for determining whether to perform decoding based on frames from the first time point to the second time point when an input requesting playback of the first time point is detected, and The above reference time is an electronic device in which the time is obtained from an application that plays the video, or is a time set as a default.

3. In paragraph 1 or 2, The above instructions, when executed by the processor (240), cause the electronic device (101) to: While playing the above video, check whether a table related to the frames of the above video exists, and An electronic device that obtains a table related to frames of the image, if a table related to frames of the image exists.

4. In paragraph 3, The above instructions, when executed by the processor (240), cause the electronic device (101) to: If a table related to the frames of the above video does not exist, a table related to the frames of the above video is created, Sequentially acquire frame information for each frame of the above video, and Based on the frame information of each of the sequentially acquired frames, the table related to the frames of the generated image is updated, An electronic device wherein each of the above frame information includes information related to a sync frame and information indicating an interval between the sync frame and a frame that is not a sync frame.

5. In any one of paragraphs 1 to 4, The above instructions, when executed by the processor (240), cause the electronic device (101) to: An electronic device that, if it is determined that the time required for the above-described decoding exceeds the above-described reference time, decodes a sync frame adjacent to the first time point.

6. In any one of paragraphs 1 to 5, The above instructions, when executed by the processor (240), cause the electronic device (101) to: An electronic device that calculates the time required for decoding by further considering the fps of the above video.

7. In any one of paragraphs 1 to 6, The above instructions, when executed by the processor (240), cause the electronic device (101) to: While playing the above video, the reading speed information of the above video is obtained, Based on the reading speed information of the acquired image and the sizes of frames from the first point in time to the second point in time representing the previous sync frame, the second time required for the decoding is calculated, and An electronic device that decodes the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if the second time required for the above-described decoding is confirmed to be within the above-described reference time.

8. In paragraph 1, The above instructions, when executed by the processor (240), cause the electronic device (101) to: While playing the above video, check the weight information of the frames of the above video, Based on the above-mentioned confirmed weight information, a second reference time related to decoding of frames of the video is set, An electronic device that decodes the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if the time required for the above-described decoding is confirmed to be within the second reference time.

9. In a method for processing images in an electronic device, An action for detecting, during playback of a video, an input requesting playback of a first point in time of said video; An operation of checking the number of frames from the first point in time to the second point in time representing the previous sync frame based on a table related to frames of the above video; An operation of calculating the time required for decoding based on the maximum performance of the codec according to the resolution of the above image and the number of the confirmed frames; and A method including an operation of decoding the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if the time required for the above-described decoding is confirmed to be within the reference time.

10. In paragraph 9, The above reference time is a time for determining whether to perform decoding based on frames from the first time point to the second time point when an input requesting playback of the first time point is detected, and The above reference time is a time obtained from the application that plays the video, or a time set as default.

11. In clause 9 or 10, An action to check whether a table related to frames of the video exists while playing the video; An operation of obtaining a table related to frames of the image, if a table related to frames of the image exists; An action of creating a table related to frames of the image, if a table related to frames of the image does not exist; An operation of sequentially acquiring frame information of each frame of the above video; and Further comprising an operation of updating a table related to frames of the generated image based on frame information of each of the sequentially acquired frames, A method wherein each of the above frame information includes information related to a sync frame and information indicating an interval between the sync frame and a frame that is not the sync frame.

12. In any one of paragraphs 9 to 11, A method further comprising an operation of decoding a sync frame adjacent to the first time point if it is determined that the time required for the above-described decoding exceeds the above-described reference time.

13. In any one of paragraphs 9 to 12, The operation to calculate the time required for the above decoding is: A method including calculating the time required for decoding by further considering the fps of the above image.

14. In any one of paragraphs 9 to 13, An operation of acquiring reading speed information of the video while playing the video; An operation of calculating a second time required for the decoding based on the reading speed information of the acquired image and the sizes of frames from the first time point to the second time point representing the previous sync frame; and A method further comprising an operation of decoding the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if the second time required for the above-described decoding is confirmed to be within the above-described reference time.

15. In paragraph 9, An action of checking weight information of frames of the video while playing the video; An operation of setting a second reference time related to decoding of frames of the video based on the above-mentioned confirmed weight information; and A method further comprising an operation of decoding the image based on the maximum performance of the codec according to the resolution of the image and the frames from the first point in time to the second point in time, if the time required for the above-described decoding is confirmed to be within the second reference time.

Images