Electronic device and method for outputting media by using configuration information about frame, and storage medium

Abstract

This electronic device may comprise: a memory for storing instructions; and at least one processor. When executed individually or collectively by the at least one processor, the instructions can cause the electronic device to: obtain a frame set including a plurality of frames of audio content to be played back; generate frame configuration information about the frame set including one or more frames from among the plurality of frames; and decode, by using the frame configuration information, the frame set including the one or more frames. The frame configuration information can include the number of the one or more frames and the size of each of the one or more frames.

Description

Electronic device, method, and storage medium for outputting media using configuration information for frames

[0001] The following descriptions relate to electronic devices, methods, and storage media that output media using configuration information for a frame.

[0002] An electronic device can obtain data about media. For example, the media may include audio or video. For example, the data may be referred to as a stream of the media. For example, the data may include a plurality of frames. The electronic device can output (or play) the media by processing the plurality of frames.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] An electronic device may include a memory that stores instructions. The electronic device may include at least one processor. The instructions, when the at least one processor is executed individually or collectively, may cause the electronic device to obtain a frame set including a plurality of frames of audio content to be played. The instructions, when the at least one processor is executed individually or collectively, may cause the electronic device to generate frame configuration information of the frame set including one or more of the plurality of frames. The frame configuration information may include a number of the one or more frames and a size of each of the one or more frames. The instructions, when the at least one processor is executed individually or collectively, may cause the electronic device to perform decoding of the frame set including the one or more frames using the frame configuration information.

[0005] An electronic device may include a first processor. The electronic device may include a second processor for decoding audio content. The first processor may be configured to obtain a frame set including a plurality of frames of the audio content. The first processor may be configured to generate frame configuration information of the frame set including one or more of the plurality of frames. The frame configuration information may include a number of the one or more frames and a size of each of the one or more frames. The first processor may be configured to generate one or more headers determined according to a codec type of the second processor using the frame configuration information. Each of the one or more headers may include the size of each of the one or more frames. The first processor may be configured to transmit the frame set including the one or more headers and the one or more frames to the second processor. The second processor may be configured to perform the decoding of the frame set.

[0006] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0007] FIG. 2 is a block diagram of an audio module according to various embodiments.

[0008] FIG. 3 illustrates an example of how an electronic device processes frames of media to play a frame indicated by a seek request, according to one embodiment.

[0009] FIG. 4 is a simplified block diagram of an exemplary electronic device, according to one embodiment.

[0010] FIG. 5 illustrates an example of an operational flow for a method in which an electronic device outputs a media stream using configuration information of a frame, according to one embodiment.

[0011] FIG. 6A illustrates an example of an operational flow for a method for an electronic device to generate configuration information of a frame, according to one embodiment.

[0012] FIG. 6b illustrates an example of a frame set including frames indicated in response to a search request, according to one embodiment.

[0013] FIG. 6c illustrates an example of a frame set including frame configuration information, according to one embodiment.

[0014] FIG. 7 illustrates an example of an operational flow for a method of determining a unit of media to be transmitted to a decoder, according to one embodiment.

[0015] FIG. 8A illustrates an example of an operational flow for a method of transmitting a frame set based on a codec type of a decoder, according to one embodiment.

[0016] FIG. 8b illustrates an example of a frame set according to a codec type of a decoder, according to one embodiment.

[0017] FIG. 9 illustrates an example of an operational flow for a method of transmitting a frame set based on capability information of a decoder, according to one embodiment.

[0018] FIGS. 10A through 10C illustrate examples of signal flow between a plurality of processors for a method of outputting media using frame configuration information, according to one embodiment.

[0019] FIG. 11 illustrates an example of an operational flow for a method in which an electronic device generates frame configuration information of a frame set of audio content and performs decoding using the generated frame configuration information, according to one embodiment.

[0020] The terms used in this disclosure are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this disclosure. Terms defined in general dictionaries among the terms used in this disclosure may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this disclosure. In some cases, even if a term is defined in this disclosure, it cannot be interpreted to exclude embodiments of the present disclosure.

[0021] The various embodiments of the present disclosure described below illustrate a hardware-based approach as an example. However, since the various embodiments of the present disclosure include techniques utilizing both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0022] In addition, in the present disclosure, expressions such as "more than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled. However, this is merely a description for expressing an example and does not exclude descriptions such as "more than" or "less than." Conditions described as "more than" may be replaced with "more than," conditions described as "less than," and conditions described as "more than and less than" may be replaced with "more than and less than." In addition, hereinafter, "A" to "B" mean at least one of the elements from A (including A) to B (including B).

[0023] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0024] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0025] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0026] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0027] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0028] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0029] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0030] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0031] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0032] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0033] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0034] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0035] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0037] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0038] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0039] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0041] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0042] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0043] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0044] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0045] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0046] FIG. 2 is a block diagram of an audio module according to various embodiments.

[0047] Referring to the block diagram (200) of FIG. 2, the audio module (170) may include, for example, an audio input interface (210), an audio input mixer (220), an analog to digital converter (ADC) (230), an audio signal processor (240), a digital to analog converter (DAC) (250), an audio output mixer (260), or an audio output interface (270).

[0048] The audio input interface (210) can receive an audio signal corresponding to a sound acquired from the outside of the electronic device (101) as part of the input module (150) or through a microphone (e.g., a dynamic microphone, a condenser microphone, or a piezo microphone) configured separately from the electronic device (101). For example, when the audio signal is acquired from an external electronic device (102) (e.g., a headset or a microphone), the audio input interface (210) can receive the audio signal by being directly connected to the external electronic device (102) through a connection terminal (178) or wirelessly (e.g., Bluetooth communication) through a wireless communication module (192). According to one embodiment, the audio input interface (210) can receive a control signal (e.g., a volume control signal received through an input button) related to the audio signal acquired from the external electronic device (102). The audio input interface (210) includes a plurality of audio input channels and can receive different audio signals for each corresponding audio input channel among the plurality of audio input channels. According to one embodiment, additionally or alternatively, the audio input interface (210) can receive audio signals from other components of the electronic device (101), such as the processor (120) or the memory (130).

[0049] The audio input mixer (220) can synthesize a plurality of input audio signals into at least one audio signal. For example, according to one embodiment, the audio input mixer (220) can synthesize a plurality of analog audio signals input through the audio input interface (210) into at least one analog audio signal.

[0050] The ADC (230) can convert an analog audio signal into a digital audio signal. For example, according to one embodiment, the ADC (230) can convert an analog audio signal received through the audio input interface (210) or, additionally or alternatively, an analog audio signal synthesized through the audio input mixer (220) into a digital audio signal.

[0051] The audio signal processor (240) may perform various processing on a digital audio signal input through the ADC (230) or a digital audio signal received from another component of the electronic device (101). For example, according to one embodiment, the audio signal processor (240) may change a sampling rate, apply one or more filters, perform interpolation processing, amplify or attenuate all or part of a frequency band, process noise (e.g., reduce noise or echo), change a channel (e.g., switch between mono and stereo), mix, or extract a specified signal on one or more digital audio signals. According to one embodiment, one or more functions of the audio signal processor (240) may be implemented in the form of an equalizer.

[0052] The DAC (250) can convert a digital audio signal into an analog audio signal. For example, according to one embodiment, the DAC (250) can convert a digital audio signal processed by an audio signal processor (240) or a digital audio signal obtained from another component of the electronic device (101) (e.g., a processor (120) or a memory (130)) into an analog audio signal.

[0053] The audio output mixer (260) can synthesize a plurality of audio signals to be output into at least one audio signal. For example, according to one embodiment, the audio output mixer (260) can synthesize an audio signal converted into analog through the DAC (250) and another analog audio signal (e.g., an analog audio signal received through the audio input interface (210)) into at least one analog audio signal.

[0054] The audio output interface (270) can output an analog audio signal converted by the DAC (250), or additionally or alternatively, an analog audio signal synthesized by the audio output mixer (260), to the outside of the electronic device (101) through the audio output module (155). The audio output module (155) can include, for example, a speaker such as a dynamic driver or a balanced armature driver, or a receiver. According to one embodiment, the audio output module (155) can include a plurality of speakers. In this case, the audio output interface (270) can output an audio signal having a plurality of different channels (e.g., stereo or 5.1 channels) through at least some of the speakers among the plurality of speakers. According to one embodiment, the audio output interface (270) can be directly connected to an external electronic device (102) (e.g., an external speaker or a headset) through a connection terminal (178) or wirelessly through a wireless communication module (192) to output an audio signal.

[0055] According to one embodiment, the audio module (170) can generate at least one digital audio signal by synthesizing a plurality of digital audio signals using at least one function of the audio signal processor (240) without separately having an audio input mixer (220) or an audio output mixer (260).

[0056] According to one embodiment, the audio module (170) may include an audio amplifier (not shown) (e.g., a speaker amplifier circuit) capable of amplifying an analog audio signal input through an audio input interface (210) or an audio signal to be output through an audio output interface (270). According to one embodiment, the audio amplifier may be configured as a separate module from the audio module (170).

[0057] FIG. 3 illustrates an example of how an electronic device processes frames of media to play a frame indicated by a seek request, according to one embodiment.

[0058] The electronic device (101) of FIG. 3 may be an example of the electronic device (101) of FIG. 1. For example, the display (160) of the electronic device (101) may be included in the display module (160) of FIG. 1.

[0059] FIG. 3 illustrates an example (300) of receiving a seek request from a user while an electronic device (101) outputs (or plays) media. For convenience of explanation, in example (300), the media is illustrated as music and the data of the media includes audio; however, the embodiments of the present disclosure are not limited thereto. For example, the data of the media may include the audio and a video output together with the audio. For example, the data of the media may be referred to as a media file, a media stream, media content, or media information.

[0060] Referring to example (300), the electronic device (101) can output (or play) the media. For example, the electronic device (101) can output a screen indicating that the media is being played through the display (160). In addition, for example, the electronic device (101) can output audio of the media through an output device (not shown) (e.g., a speaker). For example, the output device can include at least a part of the audio output module of FIG. 1. In example (300), the electronic device (101) can be outputting a data portion (e.g., a frame) of the media corresponding to timing (301) among the running time (e.g., 2 minutes and 23 seconds) of the media. For example, the running time can represent the total time length of the media.

[0061] Referring to example (300), the electronic device (101) may obtain a user input (303) for a seek request for the media. For example, the input (303) may include a touch input on an area of ​​the display (160) of the electronic device (101). However, the embodiments of the present disclosure are not limited thereto. For example, the input (303) may include a gesture or a voice. For example, the input (303) may be used to indicate a timing (305) after the timing (301) during the running time of the media. For example, the electronic device (101) may receive (or recognize, identify) the seek request upon obtaining the input (303). For example, the electronic device (101) may output a data portion (e.g., a frame) of the media corresponding to a timing (305) subsequent to the timing (301) upon receiving the search request according to the input (303). In the example (300) of FIG. 3, the timing (305) is illustrated as representing a time subsequent to the timing (301), but this is merely for convenience of explanation and the embodiments of the present disclosure are not limited thereto. For example, the timing (305) may also represent a time prior to the timing (301).

[0062] As in example (300), the electronic device (101) may receive an input (303) requesting a search for timing (305), and may stop playing a data portion of the media corresponding to timing (301) and start playing a data portion of the media corresponding to timing (305) according to the received input (303). At this time, in order to play the data portion of the media corresponding to timing (305), the electronic device (101) may perform processing on the data of the media. For example, the processing may include removing (or skipping, discarding) at least a portion of the composition of the media. In the example, the at least a portion may include a data portion (e.g., one or more frames) of the media between timing (301) and timing (305). For a specific example related to this, reference may be made to example (350) of FIG. 3.

[0063] Referring to example (350), the electronic device (101) may perform the removal (or skipping, discarding) of at least a portion of the media. For example, the at least a portion may include one or more frames of the media. For example, the media may be composed of frames. For example, if the media is audio, the frames may be referred to as audio frames. For example, if the media is video, the frames may be referred to as video frames. For example, if the media includes both video and audio, the frames may include video frames and audio frames. The frame may include information on the amplitude (or loudness) of a signal at a specific time interval (or timing). For example, if the frame is a frame of audio data, the signal may be an audio signal.

[0064] In the above example, the media is described as being composed of the frames, but the embodiments of the present disclosure are not limited thereto. For example, the media may be composed of frame sets. Each of the frame sets may be referred to as a single structure including a plurality of frames. For example, the frame set may be referred to as a chunk or chunk data. For example, in order to reduce power consumption when playing the media, the electronic device (101) may compose the frames into frame set units, read each frame set, or transmit each frame set to a decoder. For example, the decoder may represent a media decoder (or audio decoder) that performs decoding to play (or output) the media. In other words, in order to reduce the number of times data of the media is read, the electronic device (101) may process the media by frame set rather than by frame. If the above media is audio, the playback method described above may be referred to as a low-power audio playback method.

[0065] For example, the electronic device (101) can recognize a frame set (360) of the media. For example, the electronic device (101) can acquire frame sets of the media stored in the memory of the electronic device (101) (e.g., memory (130) of FIG. 1) or frame sets of the media acquired from an external electronic device (not shown) (or a server). For example, the stored frame sets and the acquired frame sets can represent data constituting the media. The electronic device (101) can recognize a frame set (360) among the frame sets constituting the media (e.g., the stored frame sets or the acquired frame sets) according to an input (303). For example, the electronic device (101) can recognize a frame set (360) including a frame (366) corresponding to a timing (305) at which a search is requested according to the input (303). For example, frame sets (360, 370) may be included in frame sets constituting the media (e.g., the stored frame sets or the acquired frame sets).

[0066] For example, the electronic device (101) may configure the media by frame set in order to reduce power consumption. When the media is configured by frame set, the electronic device (101) may not directly recognize the frame (366) but may recognize the frame set (360) including the frame (366) according to the input (303). For example, the electronic device (101) may identify the time value (383) (e.g., 0 ms) at which the first frame (361) of the frame set (360) starts, rather than the time value (381) (e.g., 100 ms) at which the frame (366) starts among the media. The reason the time value (383) is identified instead of the time value (381) may be because the media is configured by frame set. As mentioned above, as a result of the search request, a frame (e.g., the initial frame (361)) different from the frame indicated by the input (303) (e.g., frame (366)) may be incorrectly recognized.

[0067] In addition, when the media includes video and audio, even if the search result for the video indicates a time value (381), if the search result for the audio indicates a time value (383), the video may not be output and may wait until the audio indicates a time value (381). This may be because the video is rendered according to the output time of the audio in order to synchronize the video and the audio. For example, in the example (350) of FIG. 3, when the time value (383) (e.g., 0 ms) is recognized in response to a search request for the audio, and a portion of video content corresponding to the time value (381) (e.g., 100 ms) is recognized in response to a search request for the video, the output of the video may be performed after waiting for 100 ms. This may be because the video is rendered according to the output time of the audio.

[0068] In the example (350) of FIG. 3, each of the time values ​​(381, 383) is defined as an offset from the time at which the frame set (360) begins, but the embodiments of the present disclosure are not limited thereto. For example, each of the time values ​​(381, 383) may also be defined as an offset from the first frame (not shown) among the frames of the media.

[0069] For example, the electronic device (101) can perform decoding on the media. For example, if the media is configured by frame sets, the electronic device (101) can perform decoding on a frame set basis. Or, for example, if the decoding is performed based on an offloading method, the electronic device (101) can perform decoding on a frame set basis. The offloading method may be referred to as offload, offloading, offload decoding, or offload technique. For example, the offloading method may be performed in an external electronic device (e.g., the electronic device (102) of FIG. 1) different from the electronic device (101), or may include a case where another processor within the electronic device (101) different from the processor (e.g., the processor (120) of FIG. 1) within the electronic device (101) that performs processing (e.g., demuxing, reconfiguration) of the media includes a decoder (or media decoder, audio decoder). As described above, when the electronic device (101) performs decoding on a frame set-by-frame basis, it cannot reconfigure frames within the frame set to be decoded, and thus cannot perform a playback (or seek) operation from a desired frame. For example, the electronic device (101) cannot perform reconstructing of the frame set (360) during decoding (e.g., removing (or skipping, discarding) some frames). In other words, the electronic device (101) (or the processor within the electronic device (101)) cannot adjust the decoded audio data, cannot discard invalid decoded audio data, and cannot perform a seek operation to a frame at a desired location because it does not directly manage a decoder for audio when using an offloading method for audio.Accordingly, the electronic device (101) cannot perform decoding from the frame (366) within the search-requested frame set (360), and can perform decoding from the first frame (361) of the frame set (360).

[0070] For example, when an error in decoding occurs in a frame (366) while the electronic device (101) is performing decoding of a frame set (360), the electronic device (101) may perform decoding from a frame set (370) following the frame set (360) rather than from a frame (367) following the frame (366). Alternatively, when an error in decoding occurs in a frame (366), the electronic device (101) may not have a way to check a frame (367) or a frame set (370) following the frame (366), and thus decoding of the media may fail.

[0071] For example, when an electronic device (101) transmits, receives, and processes data by frame set while providing a streaming service through a network, a delay may occur depending on the size of the frame set.

[0072] Hereinafter, the device, method, and storage medium according to embodiments of the present disclosure can generate configuration information including the number of frames included in a frame set, the size of each frame, the time interval of each frame, and a timestamp indicating the first frame of the frame set. For example, the configuration information may be referred to as metadata or frame configuration information. The device, method, and storage medium according to embodiments of the present disclosure can accurately recognize frames according to a search request using the configuration information and remove unnecessary frames, thereby performing an accurate search operation, even if the media is configured by frame set. In addition, the device, method, and storage medium according to embodiments of the present disclosure can perform recovery when a decoding error occurs by transmitting a frame set including the configuration information to a decoder, even if decoding is performed (e.g., offloading) by frame set. In addition, the device, method, and storage medium according to embodiments of the present disclosure can prevent delay by adjusting the size of the frame set in advance when providing a streaming service through a network.

[0073] FIG. 4 is a simplified block diagram of an exemplary electronic device, according to one embodiment.

[0074] The electronic device (101) of FIG. 4 may be an example of the electronic device (101) of FIG. 1. For example, the electronic device (101) may include a processor (400) and / or a decoding processor (450). For example, the processor (400) may include at least a portion of the processor (120) of FIG. 1. For example, the decoding processor (450) may include at least a portion of the audio module (170) of FIG. 1. For example, the decoding processor (450) may include the ADC (230) and the DAC (250) of FIG. 2. However, the embodiments of the present disclosure are not limited thereto. In addition, for example, the decoding processor (450) may include at least a portion of the components of the auxiliary processor (123) of FIG. 1. For example, the decoding processor (450) may include hardware components such as a digital signal processor (DSP), an NPU, a GPU, or a CPU for low power consumption. However, the embodiments of the present disclosure are not limited thereto. For example, the decoding processor (450) may be referred to as a low-power processor, an offloading processor, a decoding device, a low-power decoder, or a low-power offloader.

[0075] For example, the processor (400) may include various processing circuits and / or multiple processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits including at least one processor, one or more of which may be configured to individually and / or collectively perform the various functions described below in a distributed manner. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform various functions, these terms encompass, for example, and without limitation, situations where one processor performs some of the recited functions and other processor(s) perform other parts of the recited functions, and also situations where one processor may perform all of the recited functions. Additionally, the at least one processor may include a combination of processors that perform the various functions enumerated / disclosed, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform the various functions.

[0076] According to one embodiment, the processor (400) may include a demuxer (405), a chunk configurator (410), a metadata generator (415), a playing manager (420), a transmission unit discriminator (425), and a frame header adder (430). For example, each of the demuxer (405), the chunk configurator (410), the metadata generator (415), the playing manager (420), the transmission unit discriminator (425), and the frame header adder (430) may be implemented as software within the processor (400). In the example of FIG. 4, the processor (400) is illustrated as including a demuxer (405), a chunk configurator (410), a metadata generator (415), a playback manager (420), a transmission unit discriminator (425), and a frame header inserter (430), but embodiments of the present disclosure are not limited thereto. For example, the processor (400) may be implemented with one or more components that implement functions (or operations) performed by each of the demuxer (405), the chunk configurator (410), the metadata generator (415), the playback manager (420), the transmission unit discriminator (425), and the frame header inserter (430).

[0077] According to one embodiment, the processor (400) may include a decoder (435). For example, the processor (400) may include a decoder (435) implemented in hardware, software, or a combination of hardware and software. However, the embodiments of the present disclosure are not limited thereto. For example, the processor (400) may not include a decoder (435). For example, if the electronic device (101) includes a decoding processor (450) that includes a decoder (455), the decoder (435) may not be included within the processor (400). Alternatively, even if the electronic device (101) includes a decoding processor (450), the processor (400) may further include a decoder (435) to improve decoding performance.

[0078] Although not illustrated in FIG. 4, the electronic device (101) may further include components for outputting media. For example, the electronic device (101) may further include a speaker (e.g., an audio output module (155) of FIG. 1) for outputting (or providing) auditory information such as audio. For example, the electronic device (101) may further include a display (e.g., a display module (160) of FIG. 1) for outputting (or providing) visual information such as video (or image). For example, the electronic device (101) may further include a motor (e.g., a haptic module (179) of FIG. 1) for providing haptic feedback based on vibration. For example, the electronic device (101) may further include a memory (e.g., a memory (130) of FIG. 1) for storing data of the media. For example, the electronic device (101) may further include a communication circuit (e.g., a communication module (190) of FIG. 1) for obtaining (or receiving, downloading) media data from an external electronic device (or server).

[0079] In one embodiment, the processor (400) may demux data of the media using the demuxer (405). For example, if the data includes both video and audio, the processor (400) may separate the video and the audio from the data using the demuxer (405). For example, the processor (400) may transmit a request to the demuxer (405) so that the metadata generator (415) and the chunk configurator (410) are executed as the data of the media is acquired. For example, the request may cause the metadata generator (415) and the chunk configurator (410) to be executed. However, the embodiments of the present disclosure are not limited thereto. For example, each of the metadata generator (415) and the chunk configurator (410) may be executed regardless of the request by the demuxer (405).

[0080] According to one embodiment, the processor (400) may recognize the composition (or composition unit) of the data of the media using the demuxer (405). For example, the processor (400) may determine, using the demuxer (405), whether the data is composed of a plurality of frames or a plurality of frame sets. For example, each of the plurality of frame sets may include one or more frames.

[0081] According to one embodiment, the processor (400) may recognize a frame request transmitted from the playback manager (420) to the demuxer (405). For example, the frame request may indicate a frame following a currently playing frame. Or, for example, the frame request may indicate a frame set following a currently playing frame set. Or, for example, the frame request may indicate a frame for which a search request has been made. Or, for example, the frame request may indicate a frame set including a frame for which a search request has been made. According to one embodiment, the frame request may include a size of a frame set that the decoder can support (or a supportable size, a maximum size). For example, the processor (400) may recognize the maximum size of a frame set that the decoder can support by checking capability information of the decoder (435) within the processor (400) or receiving capability information of the decoder (455) of the decoding processor (450). The processor (400) can include the maximum size in the frame request and transmit it to the demuxer (405).

[0082] According to one embodiment, the processor (400) may configure a frame set using the chunk configurator (410). For example, the processor (400) may reconfigure, using the chunk configurator (410), the frame set including (or composed of) a plurality of frames to include one or more frames among the plurality of frames. For example, when a search request is received and a frame indicated by the search request is a different frame from a first frame among the plurality of frames, the one or more frames may represent a remaining portion from which one or more other frames prior to a time value of the frame for which the search request is made are removed among the plurality of frames. In other words, the one or more other frames may represent a portion of the plurality of frames that is different from the one or more frames and is removed by the search request. The one or more other frames may be referred to as a removed frame or a skipped frame. Or, for example, when the maximum size is recognized, the frame set may be reconfigured such that the one or more frames correspond to the maximum size. A specific example of the operation of recognizing and reconstructing the above maximum size is described in FIG. 9 below.

[0083] According to one embodiment, the processor (400) may, when the media is structured by frames, merge (or, distinguish, organize, distribute) a specified number of frames into a single frame set. For example, even if the media is structured by frames, the processor (400) may organize the media into frame sets to reduce the number of decoding and reading operations. In this case, each of the frame sets may include the specified number of frames. For example, the specified number may be plural.

[0084] According to one embodiment, the processor (400) may generate frame configuration information for one or more frames within a frame set using the metadata generator (415). For example, the frame configuration information may include a time value (or timestamp) indicating an initial frame among the one or more frames within the media, a number of the one or more frames, a time interval of each of the one or more frames, and a size of each of the one or more frames. For example, the frame set may be a reconstructed frame set. For example, the frame set may include one or more frames among a plurality of frames. For example, the one or more frames may include a remaining portion from which one or more other frames are removed from the plurality of frames.

[0085] For example, the time value indicating the first frame may indicate a time value indicating a frame indicated by the search request, which is a first frame among the one or more frames when a search request is received. For example, the number of the one or more frames may indicate the number of frames included in the reconstructed frame set. For example, the time interval of each of the one or more frames may indicate the time length of each frame included in the reconstructed frame set. For example, the size of each of the one or more frames may include the number of bits (or bytes) of each frame included in the reconstructed frame set. For example, the time value and the time interval may be calculated based on at least one of a file format of the media (or a specification of the file format), a codec type in which the media is encoded, a bit rate of the media, or a sampling rate.

[0086] The above example describes an example of generating frame configuration information when a frame set including a plurality of frames is reconstructed to include one or more frames, but the embodiments of the present disclosure are not limited thereto. For example, the processor (400) may also generate frame configuration information for a plurality of frames included in an acquired frame set without reconstructing the frame set using the chunk configurator (410).

[0087] In the above example, it is exemplified that the frame configuration information includes a time value (or timestamp) indicating a first frame among the one or more frames in the media, the number of the one or more frames, a time interval of each of the one or more frames, and a size of each of the one or more frames, but the present disclosure is not limited thereto. According to one embodiment, the frame configuration information may include at least one of the information of the above example. For example, the frame configuration information may include the number of the one or more frames and the size of each of the one or more frames. When the time value indicating the first frame is obtained through other information, and the time intervals of each of the one or more frames are the same, the frame configuration information may include information indicating the number and the size among the time value, the number, the time interval, and the size. Alternatively, for example, the frame configuration information may include information indicating the size among the time value, the number, the time interval, and the size.

[0088] A specific example of a method for generating the above frame configuration information is described in FIG. 6a below. A specific example of the above frame configuration information is described in FIG. 6c below.

[0089] According to one embodiment, the processor (400) may determine whether a search request is received using the playback manager (420). For example, the processor (400) may recognize the search request based on an input to the electronic device (101) and transmit the recognized search request to the playback manager (420). When the search request is received, the processor (400) may transmit the frame request from the playback manager (420) to the demuxer (405). For example, the frame request may indicate a frame for which the search request is made (or a frame set including the frame for which the search request is made). Alternatively, for example, the processor (400) may transmit the frame request from the playback manager (420) to the demuxer (405). For example, the frame request may indicate a frame following a current frame (or a frame set following a current frame set).

[0090] For example, the processor (400) may transmit a request to the playback manager (420) to cause the transmission unit discriminator (425) and the frame header inserter (430) to be executed. For example, the request may cause the execution of the transmission unit discriminator (425) and the frame header inserter (430). However, the embodiments of the present disclosure are not limited thereto. For example, each of the transmission unit discriminator (425) and the frame header inserter (430) may be executed regardless of the request by the playback manager (420).

[0091] According to one embodiment, the processor (400) may use the transmission unit determiner (425) to determine a unit (hereinafter, a transmission unit) transmitted from the playback manager (420) to the decoder (435) or the decoder (455). For example, the processor (400) may determine the transmission unit determined according to the decoding method of the decoder (435) or the decoding method of the decoder (455). For example, the transmission unit may correspond to the decoding method of the decoder (435) or the decoding method of the decoder (455). For example, the decoding method may include frame-by-frame decoding or frame set-by-frame decoding. A specific example of a method for determining the transmission unit is described below in FIG. 7.

[0092] According to one embodiment, the processor (400) may insert (or add) a header for each frame of a frame set using a frame header inserter (430). For example, if the frame set is reconstructed, the frame set may include one or more frames, and the frame header inserter (430) may generate one or more headers corresponding to the one or more frames. For example, the number of the one or more headers may correspond to the number of the one or more frames. For example, the processor (400) may identify the number of the one or more frames using the frame configuration information of the frame set, and generate the one or more headers corresponding to the identified number. However, the embodiments of the present disclosure are not limited thereto. For example, if the frame set is not reconstructed, the frame set may include a plurality of frames, and the frame header inserter (430) may generate a plurality of headers corresponding to the plurality of frames.

[0093] According to one embodiment, the processor (400) may insert a header for each frame of the frame set, as required, according to a codec type used in the decoder (435) or the decoder (455). For example, the codec type may include AAC (advanced audio coding), Opus, or MP3 (mpeg layer 3). However, the embodiments of the present disclosure are not limited thereto. For example, the codec type may further include other codec types. For example, when the codec type is AAC, the header may be an adts (audio data transport stream) header. The adts header may be used to find the position of a frame following a frame in which an error has occurred within the frame set, taking into account a decoding error (e.g., frame corruption) when the transmission unit is a frame set. For example, the adts header may include a frame-by-frame size of the frame configuration information. For example, when the codec is Opus, the header may be an Opus header. The Opus header may be used to distinguish the positions of frames within the frame set when the transmission unit is the frame set. For example, the Opus header may include the frame-by-frame size of the frame configuration information. For example, when the codec is MP3, the header may not be inserted. For example, the Opus header may have a size of 4 bytes. In the case of MP3, since the position of a frame can be identified through a sync word, a separate header may not be added (or inserted).

[0094] For example, the processor (400) may reconstruct (or generate) the frame set by inserting a header for each frame in the frame set. For example, the processor (400) may recognize the positions (or times) of the frames in the frame set using the frame configuration information of the frame set, and reconstruct (or generate) the frame set including a header for each of the frames. For example, the header may be concatenated with a corresponding frame.

[0095] According to one embodiment, the processor (400) may generate information (hereinafter, “header indication information”) for indicating a header added according to the codec type using the frame header inserter (430). For example, the information may indicate information indicating that the frame set includes the adts header or the Opus header. For example, the frame set transmitted to the decoder (435) or decoder (455) may further include the information.

[0096] A specific example of how to create and insert the above header is described in FIG. 8a below. A specific example of a set of frames into which a header is inserted is described in FIG. 8b.

[0097] According to one embodiment, the processor (400) may decode the acquired frame set (or frame) using the decoder (435) within the processor (400). For example, the frame set may be reconstructed using the frame configuration information, or may represent a frame set in which a header is inserted using the frame configuration information. According to one embodiment, the processor (400) may output the decoded frame set. For example, the processor (400) may output (or play) the decoded frame set by controlling an output device (e.g., a speaker).

[0098] Alternatively, according to one embodiment, the processor (400) may cause a decoder (455) external to the processor (400) to decode the acquired frame set (or frames). For example, the frame set may be reconstructed using the frame configuration information, or may represent a frame set with a header inserted using the frame configuration information. For example, the processor (400) may transmit the frame set to the decoding processor (450). The decoding processor (450) may decode the received frame set. According to one embodiment, the decoding processor (450) may output the decoded frame set. For example, the decoding processor (450) may output (or reproduce) the decoded frame set by controlling an output device (e.g., a speaker).

[0099] In the example of FIG. 4, an example is shown in which the electronic device (101) includes a processor (400) and a decoding processor (450), but embodiments of the present disclosure are not limited thereto. For example, the electronic device (101) may include the processor (400), and an external electronic device (e.g., the electronic device (102) of FIG. 1) may include the decoding processor (450). Accordingly, the electronic device (101) may transmit a frame set reconstructed using the frame configuration information or in which a header is inserted using the frame configuration information to the external electronic device, and the external electronic device may decode the frame set. Thereafter, the external electronic device may output the decoded frame set using an audio output device of the external electronic device. For example, the external electronic device may include a TWS (true wireless stereo) or XR (extended reality) device (e.g., HMD). However, embodiments of the present disclosure are not limited thereto.

[0100] Furthermore, while the example of FIG. 4 illustrates an example in which the processor (400) generates the frame configuration information, reconfigures the frame set, and inserts a header, the embodiments of the present disclosure are not limited thereto. For example, multiple processors may perform the above operations separately. Specific examples related to this are described below in FIGS. 10A to 10C.

[0101] FIG. 5 illustrates an example of an operational flow for a method in which an electronic device outputs a media stream using configuration information of a frame, according to one embodiment.

[0102] The method of FIG. 5 may be performed by the electronic device (101) of FIG. 4. For example, the operations of the method may be controlled by the processor (400) of the electronic device (101). For example, at least some of the operations of the method may be controlled by the processor (400), and at least other some of the operations may be controlled by the decoding processor (450). In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0103] According to one embodiment, in operation (500), the electronic device (101) may obtain a media stream. For example, the electronic device (101) may obtain the media stream, which is data of media to be played. For example, the media may include audio. If the media includes the audio, the media stream may include audio content (or audio file, audio stream, audio data, audio information). Or, for example, the media may include audio and video. If the media includes the audio and the video, the media stream may include audio content and video content (or video file, video stream, video data, video information).

[0104] According to one embodiment, the electronic device (101) may read the media stream stored in the memory of the electronic device (101) (e.g., the memory (130) of FIG. 1)) or obtain (or receive) the media stream from an external electronic device (or server) through the communication circuit of the electronic device (101) (e.g., the communication module (190) of FIG. 1).

[0105] According to one embodiment, in operation (505), the electronic device (101) may perform demuxing of the media stream. For example, the electronic device (101) may perform the demuxing to distinguish (or split) data included in the media stream. For example, if the media stream includes audio content and video content, the audio content and the video content may be distinguished (or split) by performing the demuxing. Alternatively, if the media stream includes audio content, a plurality of components (e.g., streams) constituting the audio content may be distinguished (or split). Hereinafter, for convenience of explanation, a case in which the media stream includes audio content is exemplified, but embodiments of the present disclosure are not limited thereto.

[0106] According to one embodiment, the electronic device (101) may determine the composition unit of the audio content based on the demuxing. For example, the electronic device (101) may determine whether the audio content is composed in units of frames or units of frame sets. The fact that the audio content is composed in units of frames may indicate that the audio content is composed of a plurality of frames. The fact that the audio content is composed in units of frame sets may indicate that the audio content is composed of a plurality of frame sets, and each of the plurality of frame sets includes a plurality of frames.

[0107] According to one embodiment, in operation (510), the electronic device (101) may perform reconfiguration of a frame set. For example, the electronic device (101) may reconfigure the frame set including (or consisting of) a plurality of frames to include one or more frames among the plurality of frames.

[0108] For example, the electronic device (101) may perform reconfiguration of the frame set when the search request is received and the frame indicated by the search request is not the first frame among the plurality of frames. For example, when the search request is received, the electronic device (101) may obtain a time value of the frame indicated by the search request. When the frame having the time value is different from the first frame among the plurality of frames in the frame set, the electronic device (101) may remove one or more other frames prior to the time value of the frame among the plurality of frames. Accordingly, the electronic device (101) may identify one or more frames including a remaining portion from which the one or more other frames are removed among the plurality of frames. For example, the remaining portion may include the frame. The electronic device (101) may reconfigure the frame set to include the one or more frames, which are the remaining portion.

[0109] In the above example, it is described that the frame set is reconfigured to include one or more frames modified from the plurality of frames, but the embodiments of the present disclosure are not limited thereto. For example, the frame set may include one or more frames including the plurality of frames. In other words, if the search request is not received and a next frame set is requested, or if the search request is received but the frame indicated by the search request is the first frame in the frame set, the electronic device (101) may not reconfigure the frame set. Accordingly, the electronic device (101) may identify the one or more frames including (or corresponding to) the plurality of frames constituting the frame set.

[0110] According to one embodiment, in operation (515), the electronic device (101) may generate frame configuration information. For example, the electronic device (101) may generate frame configuration information of the frame set including the identified one or more frames. For example, the identified one or more frames may correspond to the plurality of frames or include the remaining portion among the plurality of frames. For example, the frame configuration information may be referred to as metadata or configuration information.

[0111] For example, the frame configuration information may include a time value (or timestamp) indicating an initial frame among the one or more frames in the audio content, the number of the one or more frames, a time interval of each of the one or more frames, and a size of each of the one or more frames. For example, the frame set may be a reconstructed frame set. For example, the frame set may include the one or more frames among a plurality of frames. In the above example, the frame configuration information includes a time value (or timestamp) indicating an initial frame among the one or more frames in the media, the number of the one or more frames, a time interval of each of the one or more frames, and a size of each of the one or more frames, but the present disclosure is not limited thereto. According to one embodiment, the frame configuration information may include at least one of the information of the examples. For example, the frame configuration information may include the number of the one or more frames and the size of each of the one or more frames. The time value indicating the first frame is obtained through other information, and when the time interval of each of the one or more frames is the same, the frame configuration information may include information indicating the number and the size among the time value, the number, the time interval, and the size.

[0112] For example, the time value indicating the first frame may indicate a time value indicating a frame indicated by the search request, which is a first frame among the one or more frames within the audio content, when a search request is received. For example, when the audio content has a running time of 10 minutes, the time value indicating the first frame of the frame configuration information may indicate a time value (e.g., 3 minutes) indicating the frame indicated by the search request from a time value (e.g., 0 ms) of the first frame among the frames constituting the audio content.

[0113] For example, the number of said one or more frames may indicate the number of frames included in the reconstructed frame set. For example, the time interval of each of said one or more frames may indicate the time length of each frame included in the reconstructed frame set. For example, the size of each of said one or more frames may include the number of bits (or bytes) of each frame included in the reconstructed frame set. For example, the time value and the time interval may be calculated based on at least one of a file format (or a specification of the file format) of the audio content, a codec type in which the audio content is encoded, a bit rate of the audio content, or a sampling rate.

[0114] In the above example, an example is described of generating frame configuration information when the frame set including the plurality of frames is reconstructed and includes one or more frames, but the embodiments of the present disclosure are not limited thereto. For example, the processor (400) may also generate frame configuration information for the plurality of frames included in the acquired frame set without reconstructing the frame set.

[0115] According to one embodiment, the electronic device (101) may insert the generated frame configuration information into the frame set. For example, if reconfiguration is performed, the electronic device (101) may insert the frame configuration information into the frame set including one or more frames. Alternatively, for example, if reconfiguration is not performed, the electronic device (101) may insert the frame configuration information into the frame set including the plurality of frames. In other words, the electronic device (101) may generate the frame set including the frame configuration information.

[0116] Specific details on the method for generating frame configuration information and inserting the frame configuration information into a frame set as described above are described in FIGS. 6a to 6c below.

[0117] In addition, in FIG. 5, an example of reconstructing the frame set and generating the frame configuration information is illustrated, but the embodiments of the present disclosure are not limited thereto. The electronic device (101) may generate frame configuration information of a frame set including a plurality of frames before being reconstructed, and may reconstruct the frame set to include one or more frames among the plurality of frames using the frame configuration information. For example, after generating the frame configuration information of the frame set including the plurality of frames, the electronic device (101) may determine whether a frame indicated by the search request is a first frame when a search request is received using the frame configuration information. The electronic device (101) may reconstruct the frame set to include one or more frames by removing some of the plurality of frames according to the result of the determination. For example, if the result of the determination is that the frame is not the first frame, the electronic device (101) may remove some of the plurality of frames including a frame having a time value prior to the frame.

[0118] According to one embodiment, the electronic device (101) can determine the transmission unit of the audio content to be transmitted to the decoder. For example, the decoder may be included in the processor (400) of the electronic device (101) or may be included in another processor external to the processor (400) of the electronic device (101) (e.g., the decoding processor (450) of FIG. 4). Alternatively, for example, the decoder may be a decoder of an external electronic device (e.g., TWS, HMD) connected to the electronic device (101) and configured to output audio.

[0119] For example, the electronic device (101) may determine the transmission unit in a frame set unit when the decoder (e.g., decoder (435) of FIG. 4) within the processor (400) of the electronic device (101) supports multi-frame decoding, or when the decoder (e.g., decoder (455) of FIG. 4) of the decoding processor (450) of the electronic device (101) is used (or an offloading technique is used), or when the decoder of the external electronic device is used (or an offloading technique is used). Alternatively, the electronic device (101) may determine the transmission unit in a frame unit when the offloading technique is not used and the decoder within the processor (400) of the electronic device (101) does not support multi-frame decoding. Specific details regarding the method for determining the transmission unit are described below in FIG. 7.

[0120] According to one embodiment, the electronic device (101) can recognize a codec type. For example, when the transmission unit is a frame set unit, the electronic device (101) can recognize the codec type to be used for decoding the frame set. For example, the electronic device (101) can recognize the codec type of the data of the acquired media. Alternatively, for example, the electronic device (101) can also recognize the codec type of a decoder (e.g., decoder (435) or decoder (455)) that will perform decoding on the data of the media.

[0121] For example, the electronic device (101) may determine whether a header is required based on the recognized codec type. For example, if the codec type is MP3, the position of each frame within a frame set can be identified through a sync word, and thus the addition of a header may not be necessary. Conversely, if the codec type is an ACC codec or Opus, the position of each frame within a frame set cannot be identified without additional information, and thus the addition of a header may be necessary.

[0122] According to one embodiment, in operation (520), the electronic device (101) may generate and add a header. For example, if the electronic device (101) determines that addition of the header is necessary, the electronic device (101) may generate a header according to the codec type using the frame configuration information.

[0123] For example, when the frame set is reconstructed in operation (510), the electronic device (101) may generate one or more headers corresponding to the one or more frames of the frame set using the frame configuration information. The electronic device (101) may generate the one or more headers having a number corresponding to the number of the one or more frames in the frame configuration information. For example, each of the one or more headers may include a size of each of the one or more frames in the frame configuration information. According to one embodiment, when the electronic device (101) determines that generation (or addition) of the one or more headers is necessary, the electronic device (101) may obtain audio information of the frame set. For example, the audio information may include a sampling rate, a number of channels, or a size of the frame set. For example, the audio information may be obtained from the demuxer (405) of FIG. 4.

[0124] Alternatively, for example, the electronic device (101) may generate a plurality of headers corresponding to the plurality of frames of the frame set using the frame configuration information when the frame set is not reconstructed. The electronic device (101) may generate the plurality of headers having a number corresponding to the number of the plurality of frames in the frame configuration information. For example, each of the plurality of headers may include a size of each of the plurality of frames in the frame configuration information.

[0125] In one embodiment, the electronic device (101) may, after generating a header, add the header to a frame set. For example, the electronic device (101) may add the header to a frame set so as to be concatenated to a frame corresponding to the header.

[0126] For example, if the electronic device (101) generates one or more headers corresponding to the one or more frames, the electronic device (101) may add each of the one or more headers to the frame set so as to be concatenated with each of the one or more corresponding frames.

[0127] Alternatively, for example, if the electronic device (101) generates a plurality of headers corresponding to the plurality of frames, the electronic device (101) may add each of the plurality of headers to the frame set so as to be connected to each of the plurality of corresponding frames.

[0128] According to one embodiment, the electronic device (101) may generate additional header indication information according to the codec type. For example, the header indication information may indicate that the frame set includes the adts header or the Opus header. For example, the frame set to be transmitted to the decoder may further include the header indication information together with the header.

[0129] As described above, if the codec type is MP3, the creation of the header is not necessary, and therefore, operation (520) may be omitted. Specific details regarding the recognition of the codec time and the creation and addition of a header according to the codec type are described below with reference to FIGS. 8a and 8b.

[0130] According to one embodiment, in operation (525), the electronic device (101) may perform decoding of a frame set.

[0131] For example, when the electronic device (101) uses a decoder (e.g., decoder (435) of FIG. 4) within the processor (400) of the electronic device (101) for the decoding, the electronic device (101) can transmit the frame set with a header added to each frame to the decoder (e.g., decoder (435) of FIG. 4) within the processor (400), and perform the decoding of the frame set through the decoder (e.g., decoder (435) of FIG. 4) within the processor (400).

[0132] Alternatively, for example, if the electronic device (101) uses a decoder external to the processor (400) (e.g., decoder (455) of FIG. 4), the electronic device (101) may transmit the frame set with a header added to each frame to the decoder within the decoding processor (450) (e.g., decoder (455) of FIG. 4), and perform the decoding of the frame set through the decoder within the decoding processor (450) (e.g., decoder (455) of FIG. 4).

[0133] Alternatively, according to one embodiment, when the electronic device (101) utilizes a decoder of the external electronic device, the electronic device (101) may transmit the frame set with a header added to each frame to the external electronic device. For example, the electronic device (101) may cause decoding of the frame set by transmitting the frame set with a header added to the external electronic device.

[0134] According to one embodiment, in operation (530), the electronic device (101) may output the audio content. For example, the electronic device (101) may output (or play) the audio content for the decoded frame set.

[0135] For example, when the electronic device (101) performs decoding using a decoder (e.g., decoder (435) of FIG. 4) within the processor (400), the electronic device (101) can output the audio content for the decoded frame set through an audio output device of the electronic device (101).

[0136] Alternatively, for example, if the electronic device (101) performs decoding using a decoder (e.g., decoder (455) of FIG. 4) within the decoding processor (450), the electronic device (101) may output the audio content for the decoded frame set through an audio output device of the electronic device (101).

[0137] Alternatively, for example, the electronic device (101) may cause the external electronic device (e.g., TWS, HMD) to output the audio content for the frame set by transmitting the decoded frame set to an external electronic device for audio output when decoding is performed within the electronic device (101).

[0138] Alternatively, according to one embodiment, if decoding is performed within the external electronic device, the audio content for the frame set may be output via an audio output device of the external electronic device.

[0139] In the example of FIG. 5, reconfiguration of one frame set, generation of frame configuration information of one frame set, generation and addition of a header for one frame set, performance of decoding for one frame set, and output of audio content for one frame set are illustrated, but embodiments of the present disclosure are not limited thereto. For example, the electronic device (101) may perform the operations of FIG. 5 for each of a plurality of frame sets constituting the audio content. At this time, at least some of the operations of FIG. 5 (e.g., decoding and output (or, playback) using the external electronic device) may be caused by the electronic device (101) and may also be performed by the external electronic device.

[0140] FIG. 6A illustrates an example of an operational flow for a method for an electronic device to generate frame configuration information, according to one embodiment. FIG. 6B illustrates an example of a frame set including frames indicated in response to a search request, according to one embodiment. FIG. 6C illustrates an example of a frame set including frame configuration information, according to one embodiment.

[0141] The method of FIG. 6A may be performed by the electronic device (101) of FIG. 4. For example, the operations of the method may be controlled by the processor (400) of the electronic device (101). Alternatively, for example, the operations of the method may be controlled by a plurality of processors that divide and perform the functions of the processor (400). In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. The method of FIG. 6A may represent specific examples of operations (500) to (515) of FIG. 5.

[0142] According to one embodiment, in operation (600), the electronic device (101) may obtain audio content. For example, the electronic device (101) may read the media stream stored in the memory of the electronic device (101) (e.g., the memory (130) of FIG. 1) or obtain (or receive) the media stream from an external electronic device (or server) through the communication circuit of the electronic device (101) (e.g., the communication module (190) of FIG. 1).

[0143] According to one embodiment, in operation (605), the electronic device (101) may determine whether the audio content is composed of frame sets. For example, the electronic device (101) may determine whether the audio content is composed of frame units or frame sets. The fact that the audio content is composed of frame units may indicate that the audio content is composed of a plurality of frames. The fact that the audio content is composed of frame sets may indicate that the audio content is composed of a plurality of frame sets, and each of the plurality of frame sets includes a plurality of frames.

[0144] In operation (605), if the audio content is configured in frame set units, the electronic device (101) can perform operation (620). Conversely, in operation (605), if the audio content is configured in frame units, the electronic device (101) can perform operation (610).

[0145] According to one embodiment, in operation (610), the electronic device (101) may determine whether to merge a plurality of frames into a frame set unit. For example, if the audio content is composed of frame units, the electronic device (101) may determine whether to merge (or, compose, distinguish, distribute) the plurality of frames constituting the audio content into a plurality of frame sets. For example, each of the plurality of frame sets may include a specified number of frames. For example, if the plurality of frames constituting the audio content are 1000 and the specified number is 8, the plurality of frames may be distributed into 125 frame sets. The above example is merely for convenience of explanation, and embodiments of the present disclosure are not limited thereto.

[0146] Referring to the above, the electronic device (101) can distinguish the audio content into frame set units in order to reduce processing complexity within the electronic device (101) and reduce power consumption by reducing the number of accesses (e.g., number of reads), even if the audio content is composed of frame units.

[0147] In operation (610), if the plurality of frames constituting the audio content are distinguished in frame set units, the electronic device (101) may perform operation (635). Conversely, in operation (610), if the plurality of frames constituting the audio content are not distinguished in frame set units, the electronic device (101) may perform operation (615).

[0148] According to one embodiment, in operation (615), the electronic device (101) may perform frame-by-frame processing. For example, if the electronic device (101) does not distinguish the plurality of frames constituting the audio content into frame sets, the electronic device (101) may perform processing (e.g., reading a frame, transmitting a frame, decoding a frame, outputting (or playing) a frame) for each of the plurality of frames. In other words, the electronic device (101) may refrain from (or skip) generating frame configuration information for the plurality of frames constituting the audio content.

[0149] According to one embodiment, in operation (620), the electronic device (101) may determine whether a search request is received. For example, the electronic device (101) may obtain a user input for a search request for the audio content. For example, the input may include a touch input, a gesture, or a voice on an area of ​​a display of the electronic device (101). For example, the electronic device (101) may receive (or recognize, identify) the search request upon obtaining the input.

[0150] In operation (620), if the search request is received, the electronic device (101) may perform operation (625). Conversely, in operation (620), if the search request is not received, the electronic device (101) may perform operation (635).

[0151] According to one embodiment, in operation (625), the electronic device (101) may determine whether a reconfiguration of a frame set is required. For example, when the search request is received, the electronic device (101) may obtain a time value of a frame indicated by the search request. The frame indicated by the search request may be included in the frame set composed of a plurality of frames. The number of the plurality of frames in the frame set may correspond to the specified number. For example, the electronic device (101) may determine whether the time value of the frame is a first frame among the plurality of frames in the frame set. A method for comparing the time value of the frame and the first frame may be referred to FIG. 6B.

[0152] Referring to FIG. 6B, the frame set (650) may be composed of the specified number (e.g., 8) of frames (e.g., frame #1, frame #2, frame #3, frame #4, frame #5, frame #6, frame #7, frame #8). For example, each frame in the frame set (650) may have a time duration of 20 ms. For example, when the frame (656) is indicated by the search request, the electronic device (101) may obtain the time value (660) of the frame (656). For example, the electronic device (101) may compare the time value (660) with the time value (650a) of the first frame (651) in the frame set (650). At this time, the time value (650a) of the first frame (651) may also be referenced as the time value of the frame set (650). As in the example of FIG. 6b, if the time value (660) is different from the time value (650a), the electronic device (101) may determine that the frame set (650) needs to be reconfigured. Unlike the example of FIG. 6b, if the time value (660) is the same as (or corresponds to) the time value (650a), the electronic device (101) may determine that the frame set (650) does not need to be reconfigured.

[0153] Referring back to FIG. 6A, in operation (625), if reconfiguration of the frame set is required, the electronic device (101) may perform operation (630). Alternatively, in operation (625), if reconfiguration of the frame set is not required, the electronic device (101) may perform operation (635).

[0154] According to one embodiment, in operation (630), the electronic device (101) may perform reconfiguration of the frame set. For example, the electronic device (101) may perform reconfiguration of the frame set if the frame indicated by the search request is different from the first frame among the plurality of frames constituting the frame set. For example, the electronic device (101) may reconfigure the frame set to include one or more frames among the plurality of frames. For example, the one or more frames may include a remaining portion from which one or more other frames prior to a time value of the frame for which the search request is made among the plurality of frames are removed. The one or more other frames may represent a portion from which the search request is removed among the plurality of frames. A method for reconfiguring the frame set to include the one or more frames may refer again to FIG. 6B.

[0155] Referring to FIG. 6B, when the electronic device (101) determines that the frame set (650) needs to be reconfigured, the electronic device (101) may reconfigure the frame set (650) to include one or more frames including the frame (656) indicated by the search request. For example, the one or more frames may include frames (e.g., frame #6, frame #7, frame #8) after the time value (660) of the frame (656). For example, the electronic device (101) may remove (or skip) a portion of the frame set (650) to reconfigure the frame set (650). For example, the portion may remove one or more other frames (e.g., frame #1, frame #2, frame #3, frame #4, frame #5) before the time value (660) among the multiple frames within the frame set (650). For example, the one or more other frames may represent frames of time intervals (665) prior to the time value (660) or between the time value (660) and the time value (650a).

[0156] Referring back to FIG. 6A, according to one embodiment, in operation (635), the electronic device (101) may generate frame configuration information of the frame set. For example, the frame configuration information of the frame set may include a time value (or timestamp) indicating an initial frame among one or more frames of the frame set within the audio content, a number of the one or more frames, a time interval of each of the one or more frames, and a size of each of the one or more frames. The frame configuration information of the frame set may be referenced as frame configuration information for the one or more frames within the frame set.

[0157] In the above example, it is exemplified that the frame configuration information includes a time value (or timestamp) indicating a first frame among the one or more frames in the media, the number of the one or more frames, a time interval of each of the one or more frames, and a size of each of the one or more frames, but the present disclosure is not limited thereto. According to one embodiment, the frame configuration information may include at least one of the information of the above example. For example, the frame configuration information may include the number of the one or more frames and the size of each of the one or more frames. When the time value indicating the first frame is obtained through other information, and the time intervals of each of the one or more frames are the same, the frame configuration information may include information indicating the number and the size among the time value, the number, the time interval, and the size.

[0158] For example, the time value indicating the first frame may indicate a time value indicating a frame indicated by the search request, which is a first frame among the one or more frames when a search request is received. For example, the number of the one or more frames may indicate the number of frames included in the reconstructed frame set. For example, the time interval of each of the one or more frames may indicate the time length of each frame included in the reconstructed frame set. For example, the size of each of the one or more frames may include the number of bits (or bytes) of each frame included in the reconstructed frame set. For example, the time value and the time interval may be calculated based on at least one of a file format of the media (or a specification of the file format), a codec type in which the media is encoded, a bit rate of the media, or a sampling rate.

[0159] According to one embodiment, the one or more frames in the frame set may include the plurality of frames having the specified number constituting the frame set. For example, when the plurality of frames constituting the audio content are distinguished by frame set units in operation (610), the electronic device (101) may identify the one or more frames including the plurality of frames having the specified number. Or, for example, when the search request is not received in operation (620), the electronic device (101) may identify the one or more frames including the plurality of frames having the specified number. Or, for example, when the electronic device (101) determines that reconfiguration of the frame set is not necessary in operation (625), the electronic device (101) may identify the one or more frames including the plurality of frames having the specified number.

[0160] According to one embodiment, the one or more frames in the frame set may include a portion of the plurality of frames having the specified number constituting the frame set. For example, when the electronic device (101) determines in operation (625) that the frame set needs to be reconfigured, it may remove one or more other frames from among the plurality of frames having the specified number. The one or more frames may represent a remaining portion of the plurality of frames from which the one or more other frames have been removed. For example, a first frame from among the one or more frames may be a frame indicated by the search request.

[0161] An example of the frame configuration information for the frame set including one or more of the above frames may be referred to in FIG. 6c.

[0162] Referring to FIG. 6C, a frame set (670) may represent a frame set reconstructed from a frame set (650) of FIG. 6B. For example, the frame set (670) may include one or more frames (e.g., frame#6, frame#7, frame#8) that include a frame (656) indicated by a search request. For example, the frame set (670) may represent a frame set from which one or more other frames (e.g., frame#1, frame#2, frame#3, frame#4, frame#5, frame#6, frame#7, frame#8) having a specified number (e.g., 8) of the frame set (650) are removed before the frame (656).

[0163] For example, the frame set (670) may include frame configuration information (680) (or metadata). For example, the frame configuration information (680) of the frame set (670) may include information about one or more frames. For example, the frame configuration information (680) may include a time value (681) (timestamp) indicating a first frame (e.g., frame (656)) among the one or more frames. In the example of FIG. 6C, the time value (681) may be 100 ms. For example, the frame configuration information (680) may include a number (683) (frame_count) of the one or more frames. In the example of FIG. 6C, the number (683) may be 3. For example, the frame configuration information (680) may include a time interval (685) (frame_duration[frame_count]) of each of the one or more frames. For example, the time interval (685) may be configured in the form of an array according to the number (683). In the example of FIG. 6c, the time interval (685) may be [20ms, 20ms, 20ms]. In the above example, the case where the time intervals for each frame are the same is exemplified, but the embodiment of the present disclosure is not limited thereto. For example, the time intervals for each frame may be different. In addition, for example, the time interval (685) may include one time interval value. For example, the time interval (685) may include one time interval value for one or more frames when the time intervals of each of the one or more frames according to the number (683) are the same (or have only one time interval value). When the time interval (685) has one time interval value for a plurality of the numbers (683), it can be understood that each of the one or more frames according to the number (683) have the same time interval value.In other words, the time section (685) may be configured as either an array form or a single value in which the maximum size of the array corresponds to the number (683). For example, the frame configuration information (680) may include the size (687) (frame_size[frame_count]) of each of the one or more frames. For example, the size (687) may be configured as an array form according to the number (683). For example, the size (687) may indicate the bit size (or byte size) of each frame.

[0164] For example, the time value (681) may be included in the frame configuration information (680) in the data format of int64. For example, the number (683) may be included in the frame configuration information (680) in the data format of uint32. For example, the time interval (685) may be included in the frame configuration information (680) in the data format of int64. For example, the size (687) may be included in the frame configuration information (680) in the data format of uint32.

[0165] In the example of FIG. 6c, the frame configuration information (680) is exemplified as including a time value (681) indicating a first frame among the one or more frames within the media, the number (683) of the one or more frames, a time interval (685) of each of the one or more frames, and a size (687) of each of the one or more frames, but the present disclosure is not limited thereto. According to one embodiment, the frame configuration information (680) may include at least one of the information of the examples. For example, the frame configuration information (680) may include the number (683) of the one or more frames and the size (687) of each of the one or more frames. The time value (681) indicating the first frame is obtained through other information, and when the time intervals (685) of each of the one or more frames are the same, the frame configuration information (680) may include information indicating the number (683) and the size (687) among the time value (681), the number (683), the time interval (685), and the size (687). Alternatively, for example, the frame configuration information (680) may include information indicating the size (687) among the time value (681), the number (683), the time interval (685), and the size (687).

[0166] Also, in FIG. 6A, an example of reconstructing a frame set and generating frame configuration information is illustrated, but the embodiments of the present disclosure are not limited thereto. The electronic device (101) may generate frame configuration information of a frame set including a plurality of frames having a specified number before being reconstructed, and may reconstruct the frame set to include one or more frames among the plurality of frames using the frame configuration information. For example, after generating the frame configuration information of the frame set including the plurality of frames, the electronic device (101) may determine whether a frame indicated by a search request is an initial frame when a search request is received using the frame configuration information. The electronic device (101) may also reconstruct the frame set to include one or more frames by removing one or more other frames among the plurality of frames based on the result of the determination.

[0167] With reference to FIGS. 6A to 6C, it is assumed that the electronic device (101) is performing playback for a first frame set among a plurality of frame sets constituting the audio content.

[0168] For example, if the search request is not received, the electronic device (101) may recognize a second frame set following the first frame set and generate frame configuration information of the second frame set. The second frame set may be a frame set that is continuous with the first frame set.

[0169] Alternatively, when the search request is received, the electronic device (101) may recognize a third frame set that is different from the first frame set and determine whether the third frame set requires reconfiguration. For example, the third frame set may include a frame indicated by the search request. Thereafter, if the third frame set requires reconfiguration, the electronic device (101) may perform the reconfiguration and generate frame configuration information of the reconfigured third frame set. Alternatively, if the third frame set does not require reconfiguration, the electronic device (101) may generate frame configuration information of the third frame set. The reconfigured third frame set may include a smaller number of frames than the third frame set before (or after) reconfiguration.

[0170] The above example describes a case where the third frame set and the first frame set are different, but the embodiments of the present disclosure are not limited thereto. For example, if the frame indicated by the search request is included in the first frame set, the electronic device (101) can identify the first frame set, determine whether to reconfigure the first frame set, and generate frame configuration information.

[0171] In the above example, it is assumed that the electronic device (101) is playing the first frame set, but the embodiments of the present disclosure are not limited thereto. For example, the embodiments of the present disclosure can also be applied to a case where the electronic device (101) is not playing the media content.

[0172] FIG. 7 illustrates an example of an operational flow for a method of determining a unit of media to be transmitted to a decoder, according to one embodiment.

[0173] The method of FIG. 7 may be performed by the electronic device (101) of FIG. 4. For example, the operations of the method may be controlled by the processor (400) of the electronic device (101). Alternatively, for example, the operations of the method may be controlled by a plurality of processors that divide and perform the functions of the processor (400). In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. The method of FIG. 7 may be performed between operations (515) and (520) of FIG. 5.

[0174] According to one embodiment, in operation (700), the electronic device (101) may recognize a set of frames. For example, the electronic device (101) may read the set of frames including frame configuration information. For example, the set of frames may include one or more frames and the frame configuration information for the one or more frames. For example, the one or more frames may include a plurality of frames having a specified number, or may include a remaining portion from which one or more other frames among the plurality of frames are removed according to a reconfiguration of the set of frames.

[0175] According to one embodiment, in operation (705), the electronic device (101) may determine whether decoding based on offloading is performed. For example, the electronic device (101) may determine whether decoding based on offloading is performed for the recognized frame set. For example, the decoding based on offloading may be performed by a decoding processor (450) different from the processor (400) of the electronic device (101), or may be performed by an external electronic device (e.g., TWS, HMD) connected to the electronic device (101).

[0176] In operation (705), if the decoding based on the offloading is performed, the electronic device (101) may perform operation (715). Alternatively, in operation (705), if the decoding based on the offloading is not performed, the electronic device (101) may perform operation (710). For example, if the decoding based on the offloading is not performed, this may indicate a case where the decoder (435) of the processor (400) within the electronic device (101) is used.

[0177] According to one embodiment, in operation (710), the electronic device (101) may determine whether multi-frame decoding is supported. For example, the electronic device (101) may determine whether the decoder (435) within the processor (400) supports the multi-frame decoding. The multi-frame decoding may refer to a function of performing decoding on multiple frames simultaneously. For example, the multi-frame decoding may be referred to as frame set decoding or chunk decoding.

[0178] In operation (710), if the multi-frame decoding is supported, the electronic device (101) can perform operation (715). Conversely, in operation (710), if the multi-frame decoding is not supported, the electronic device (101) can perform operation (720).

[0179] According to one embodiment, in operation (715), the electronic device (101) may determine frame set unit transmission. For example, the electronic device (101) may determine to transmit data to be decoded in frame set units to a decoder that will perform decoding for the frame set.

[0180] According to one embodiment, in operation (720), the electronic device (101) may determine frame-by-frame transmission. For example, the electronic device (101) may determine to transmit data to be decoded on a frame-by-frame basis to a decoder that will perform decoding on the frame set. For example, the electronic device (101) may individually transmit each of the one or more frames within the frame set.

[0181] FIG. 8A illustrates an example of an operational flow for a method of transmitting a frame set based on a codec type of a decoder, according to one embodiment. FIG. 8B illustrates an example of a frame set based on a codec type of a decoder, according to one embodiment.

[0182] The method of FIG. 8A may be performed by the electronic device (101) of FIG. 4. For example, the operations of the method may be controlled by the processor (400) of the electronic device (101). Alternatively, for example, the operations of the method may be controlled by a plurality of processors that divide and perform the functions of the processor (400). In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. The method of FIG. 8A may represent specific examples of operations (520) to (530) of FIG. 5.

[0183] Although not illustrated in FIG. 8A, the electronic device (101) may determine to transmit audio content data in frame set units to a decoder that will perform decoding of the audio content. For convenience of explanation, it is assumed below that one frame set among the data of the audio content is transmitted to the decoder, but the embodiments of the present disclosure are not limited thereto. In this case, the frame set may include one or more frames.

[0184] According to one embodiment, in operation (800), the electronic device (101) can recognize a codec type. For example, the codec type can include AAC (advanced audio coding), Opus, or MP3 (mpeg layer 3). However, the embodiments of the present disclosure are not limited thereto. For example, the codec type can further include other codec types. For example, the electronic device (101) can recognize the codec type of the data of the acquired audio content. Alternatively, for example, the electronic device (101) can also recognize the codec type of the decoder (e.g., decoder (435) or decoder (455)) that performs decoding on the data of the audio content.

[0185] According to one embodiment, in operation (805), the electronic device (101) may determine whether addition of a header is required based on the codec type. For example, the electronic device (101) may determine whether additional information (e.g., the header) is required to identify the position (or time value) of each of the one or more frames within the frame set. For example, if the codec type is MP3, the position of each frame within the frame set can be identified through a sync word, and thus addition of a header may not be required. Conversely, if the codec type is an ACC codec or Opus, the position of each frame within the frame set cannot be identified without additional information, and thus addition of a header may be required.

[0186] In operation (805), if addition of the header is required, the electronic device (101) may perform operation (810). Conversely, in operation (805), if addition of the header is not required, the electronic device (101) may perform operation (820).

[0187] According to one embodiment, in operation (810), the electronic device (101) can recognize the size of a frame using frame configuration information. For example, the electronic device (101) can recognize the size of each of the one or more frames within the frame set using the frame configuration information of the frame set. For example, using information indicating the size of each of the one or more frames of the frame configuration information, the electronic device (101) can confirm the size of each of the one or more frames within the frame set.

[0188] According to one embodiment, in operation (815), the electronic device (101) may generate and add a header according to the codec type. For example, the electronic device (101) may generate the header using the frame configuration information. For example, the electronic device (101) may generate one or more headers using information indicating the number of the one or more frames of the frame configuration information. For example, the number of the one or more headers may correspond to the number of the one or more frames. In this case, the frame configuration information used for generating and adding the one or more headers may be removed from the frame set.

[0189] For example, when the codec type is AAC, the header may be an adts (audio data transport stream) header. The adts header may be used to find the position of a frame following a frame in which an error has occurred within a frame set, taking into account a decoding error (e.g., frame corruption) when the transmission unit is a frame set. For example, the adts header may include information indicating the size of each of the one or more frames of the frame configuration information.

[0190] For example, if the codec is Opus, the header may be an Opus header. The Opus header may be used to distinguish the positions of frames within the frame set when the transmission unit is the frame set. For example, the Opus header may include information indicating the size of each of the one or more frames of the frame configuration information.

[0191] In one embodiment, the electronic device (101) may add (or insert) the generated header into the frame set. For example, the electronic device (101) may add the header into the frame set so as to be concatenated to a frame corresponding to the header.

[0192] According to one embodiment, the electronic device (101) may generate additional header indication information according to the codec type. For example, the header indication information may indicate that the frame set includes the adts header or the Opus header. For example, the frame set to be transmitted to the decoder may further include the header indication information together with the header.

[0193] For a specific example of how to perform the creation and addition of a header as described above, reference may be made to FIG. 8b.

[0194] Referring to FIG. 8B, a frame set (850) is illustrated containing a specified number of frames (e.g., five), but embodiments of the present disclosure are not limited thereto. For example, the specified number may be changed, or the frame set (850) may be reconfigured.

[0195] For example, a frame set (850) may include five frames (851, 852, 853, 854, 855) and frame configuration information (857). For example, the frame configuration information (857) may include a time value of frame (851), which is the first frame among frames (851, 852, 853, 854, 855), the number of frames (851, 852, 853, 854, 855), a time interval of each of frames (851, 852, 853, 854, 855), and a size of each of frames (851, 852, 853, 854, 855).

[0196] For example, the electronic device (101) can determine whether the addition of a header is necessary. For example, the electronic device (101) can determine that the addition of the header is necessary if the codec type of the decoder is an AAC codec or Opus. For example, if the addition of the header is necessary, the electronic device (101) can generate the header and add the header to the frame set. At this time, the frame configuration information used for the generation and addition of the header can be removed from the frame set.

[0197] For example, the frame set (860) may represent a frame set when the codec type of the decoder is the AAC codec. For example, the electronic device (101) may generate five adts headers (861, 862, 863, 864, 865) using information indicating the number of frame configuration information (857). For example, the adts header (861) may be concatenated with the frame (851) and may include information indicating the size of the frame (851). For example, the adts header (862) may be concatenated with the frame (852) and may include information indicating the size of the frame (852). For example, the adts header (863) may be concatenated with the frame (853) and may include information indicating the size of the frame (853). For example, the adts header (864) may be concatenated with the frame (854) and may include information indicating the size of the frame (854). For example, the adts header (865) may be concatenated with the frame (855) and may include information indicating the size of the frame (855).

[0198] For example, the frame set (870) may represent a frame set when the codec type of the decoder is the Opus codec. For example, the electronic device (101) may generate five Opus headers (871, 872, 873, 874, 875) using information indicating the number of frame configuration information (857). For example, the Opus header (871) may be concatenated with the frame (851) and may include information indicating the size of the frame (851). For example, the Opus header (872) may be concatenated with the frame (852) and may include information indicating the size of the frame (852). For example, the Opus header (873) may be concatenated with the frame (853) and may include information indicating the size of the frame (853). For example, the Opus header (874) may be concatenated with the frame (854) and may include information indicating the size of the frame (854). For example, the Opus header (875) may be concatenated with the frame (855) and may include information indicating the size of the frame (855). For example, each of the Opus headers (871, 872, 873, 874, 875) may have a size of 4 bytes.

[0199] For example, the frame set (880) may represent a frame set when the codec type of the decoder is MP3. For example, the electronic device (101) may generate a frame set (880) including frames (851, 852, 853, 854, 855) when the codec type is MP3, since addition of the header is not necessary.

[0200] Referring again to FIG. 8A, in one embodiment, at operation 820, the electronic device (101) may transmit a set of frames. For example, the electronic device (101) may transmit the set of frames to the decoder. For example, the set of frames may include one or more frames and one or more headers for the one or more frames, if addition of the header is required. Alternatively, for example, the set of frames may include one or more frames, if addition of the header is not required.

[0201] FIG. 9 illustrates an example of an operational flow for a method of transmitting a frame set based on capability information of a decoder, according to one embodiment.

[0202] The method of FIG. 9 may be performed by the electronic device (101) of FIG. 4. For example, the operations of the method may be controlled by the processor (400) of the electronic device (101). Alternatively, for example, the operations of the method may be controlled by a plurality of processors that divide and perform the functions of the processor (400). In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0203] According to one embodiment, in operation (900), the electronic device (101) can recognize a frame set. For example, the electronic device (101) can read the frame set including frame configuration information. For example, the electronic device (101) can read the frame set including the frame configuration information from the demuxer (405) of FIG. 4. For example, the frame set can include one or more frames and the frame configuration information for the one or more frames. For example, the one or more frames can include a plurality of frames having a specified number, or can include a remaining portion from which one or more other frames among the plurality of frames are removed according to the reconfiguration of the frame set.

[0204] According to one embodiment, in operation (905), the electronic device (101) can recognize the size of the frame set that the decoder can support (hereinafter, referred to as the supportable size). For example, the electronic device (101) can obtain capability information of the decoder (or a processor including the decoder (e.g., decoding processor (450)) and recognize the supportable size of the frame set based on the capability information. For example, the supportable size can be determined based on the size of a frame buffer of the decoder or the allowable delay time of a service (e.g., real-time streaming). For example, the supportable size can be referred to as the maximum size of the frame set that the decoder can support.

[0205] According to one embodiment, in operation (910), the electronic device (101) may determine whether the size of the frame set exceeds the supportable size. For example, the electronic device (101) may compare the size of the frame set with the supportable size. For example, as illustrated in FIG. 5, if the frame set is reconstructed before generating frame configuration information of the frame set, the electronic device (101) may compare the size of the frame set with the maximum size when reconstructing the frame set. Alternatively, for example, if the frame configuration information of the frame set is generated, the electronic device (101) may compare the size of the frame set recognized using the frame configuration information with the maximum size.

[0206] In operation (910), if the size exceeds the maximum size, the electronic device (101) may perform operation (920). Conversely, in operation (910), if the size is less than or equal to the maximum size, the electronic device (101) may perform operation (915).

[0207] According to one embodiment, in operation (915), the electronic device (101) may transmit the frame set to the decoder. For example, the electronic device (101) may perform processing on the frame set (e.g., adding frame configuration information, or generating and adding a header) without performing reconfiguration to reduce the size of the frame set, and transmit the processed frame set to the decoder.

[0208] According to one embodiment, in operation (920), the electronic device (101) may reconfigure the frame set. For example, if the size exceeds the maximum size, the electronic device (101) may reconfigure the size of the frame set to correspond to the maximum size. For example, the number of one or more frames included in the frame set may be changed to correspond to the maximum size.

[0209] According to one embodiment, in operation (925), the electronic device (101) may transmit the reconstructed frame set to the decoder. For example, the electronic device (101) may perform the reconstructing so that the size of the frame set corresponds to the maximum size, and then perform processing (e.g., adding frame configuration information or generating and adding a header) on the reconstructed frame set, and transmit the processed frame set to the decoder.

[0210] FIGS. 10A through 10C illustrate examples of signal flow between a plurality of processors for a method of outputting media using frame configuration information, according to one embodiment.

[0211] FIGS. 10A through 10C illustrate examples of the method performed by a plurality of processors (1001, 1002). In FIGS. 10A through 10C, the plurality of processors (1001, 1002) are illustrated as including a first processor (1001) and a second processor (1002), but embodiments of the present disclosure are not limited thereto. For example, the plurality of processors (1001, 1002) may include three or more processors.

[0212] According to one embodiment, the first processor (1001) and the second processor (1002) may be included in one processor (e.g., the processor (400) of FIG. 4). For example, each of the first processor (1001) and the second processor (1002) may perform functions of the processor (400) differently. When the first processor (1001) and the second processor (1002) are included in one processor (400), the first processor (1001) and the second processor (1002) may be included in one electronic device (e.g., the electronic device (101)).

[0213] Alternatively, according to one embodiment, the first processor (1001) and the second processor (1002) may be distinct from each other. For example, the first processor (1001) may be a processor (400) and the second processor (1002) may be a decoding processor (450). When the first processor (1001) is a processor (400) and the second processor (1002) is a decoding processor (450), the first processor (1001) and the second processor (1002) may be included in one electronic device (e.g., the electronic device (101)).

[0214] Alternatively, according to one embodiment, the first processor (1001) and the second processor (1002) may each be included in different electronic devices. For example, the first processor (1001) may be included in the electronic device (101), and the second processor (1002) may be included in an external electronic device connected to the electronic device (101). The external electronic device may include a TWS or HMD for outputting sound.

[0215] Referring to FIG. 10A, in operation (1005), the first processor (1001) may perform reconstruction of a frame set. For example, the first processor (1001) may perform reconstruction of a requested frame set among the frame sets constituting the audio content. For example, the frame set may include a frame indicated by a search request, or may be a frame set following the currently playing frame set.

[0216] In operation (1010), the first processor (1001) may generate frame configuration information. For example, the first processor (1001) may generate the frame configuration information of the frame set.

[0217] In FIG. 10A, an example is shown in which the frame configuration information is generated after reconstructing the frame set, but embodiments of the present disclosure are not limited thereto. For example, after generating the frame configuration information, reconstructing the frame set may be performed using the frame configuration information according to a received search request. For example, the reconstructing of the frame set may be performed by the first processor (1001) or the second processor (1002). According to one embodiment, the reconstructing of the frame set may be performed based on capability information of a decoder that will process the frame set (e.g., a size or a maximum size that the decoder can support).

[0218] In operation (1015), the first processor (1001) may generate and add a header. For example, the first processor (1001) may generate the header using the frame configuration information and add the header to the frame set. In operation (1020), the first processor (1001) may transmit the frame set with the header added to the second processor (1002).

[0219] In operation (1025), the second processor (1002) may perform decoding of the frame set. For example, the second processor (1002) may perform decoding of the frame set received from the first processor (1001). Thereafter, the second processor (1002) may output (or play) the decoded frame set.

[0220] Referring to FIG. 10b, in operation (1035), the first processor (1001) may perform reconstruction of a frame set. For example, the first processor (1001) may perform reconstruction of a requested frame set among the frame sets constituting the audio content. For example, the frame set may include a frame indicated by a search request, or may be a frame set following the currently playing frame set.

[0221] In operation (1040), the first processor (1001) may generate frame configuration information. For example, the first processor (1001) may generate the frame configuration information of the frame set.

[0222] In FIG. 10b, an example is shown in which the frame configuration information is generated after reconstructing the frame set, but the embodiments of the present disclosure are not limited thereto. For example, after generating the frame configuration information, reconstructing the frame set may be performed using the frame configuration information according to a received search request. For example, the reconstructing of the frame set may be performed by the first processor (1001) or the second processor (1002). According to one embodiment, the reconstructing of the frame set may be performed based on capability information of a decoder that will process the frame set (e.g., a size or maximum size that the decoder can support).

[0223] In operation (1045), the first processor (1001) may transmit the frame set including the frame configuration information to the second processor (1002).

[0224] In operation (1050), the second processor (1002) may generate and add a header. For example, the second processor (1002) may generate the header using the frame configuration information of the received frame set and add the header within the frame set.

[0225] In operation (1055), the second processor (1002) may perform decoding of the frame set. For example, the second processor (1002) may perform decoding of the frame set received from the first processor (1001). Thereafter, the second processor (1002) may output (or play) the decoded frame set.

[0226] Referring to FIG. 10c, in operation (1065), the first processor (1001) may perform reconstruction of a frame set. For example, the first processor (1001) may perform reconstruction of a requested frame set among the frame sets constituting the audio content. For example, the frame set may include a frame indicated by a search request, or may be a frame set following the currently playing frame set.

[0227] In operation (1070), the first processor (1001) may transmit the reconstructed frame set to the second processor (1002).

[0228] In operation (1075), the second processor (1002) may generate frame configuration information of the received frame set. For example, the second processor (1002) may generate the frame configuration information of the frame set.

[0229] In operation (1080), the second processor (1002) may generate and add a header. For example, the second processor (1002) may generate the header using the frame configuration information of the received frame set and add the header within the frame set.

[0230] In operation (1085), the second processor (1002) may perform decoding of the frame set. For example, the second processor (1002) may perform decoding of the frame set received from the first processor (1001). Thereafter, the second processor (1002) may output (or play) the decoded frame set.

[0231] In Fig. 10c, an example is shown in which the frame configuration information is generated after reconstructing the frame set, but the embodiments of the present disclosure are not limited thereto. For example, the first processor (1001) may transmit the demuxed frame set to the second processor (1002), and the second processor (1002) may generate the frame configuration information of the frame set, and then reconstruct the frame set using the frame configuration information according to the received search request. For example, the reconstructing of the frame set may be performed by the first processor (1001) or the second processor (1002). According to one embodiment, the reconstructing of the frame set may be performed based on capability information of a decoder that will process the frame set (e.g., a size or a maximum size that the decoder can support).

[0232] According to one embodiment, the electronic device (101) may use at least one of the methods of FIGS. 10A to 10C while outputting the frame sets constituting the audio content. For example, the electronic device (101) may use the method of FIG. 10A for a first frame set among the frame sets constituting the audio content, and may use the method of FIG. 10B for a second frame set following the first frame set among the frame sets.

[0233] According to one embodiment, the electronic device (101) may select one of the methods of FIGS. 10A to 10C to be used while outputting the frame sets constituting the audio content by performing communication with the external electronic device connected to the electronic device (101). For example, the first processor (1001) of the electronic device (101) and the second processor (1002) of the external electronic device may process a first frame set among the frame sets constituting the audio content based on the method of FIG. 10A. Thereafter, the external electronic device may request the electronic device (101) to perform processing on a second frame set following the first frame set among the frame sets constituting the audio content based on the method of FIG. 10C. For example, the external electronic device may transmit the request when the battery of the external electronic device is sufficient (e.g., when the battery of the external electronic device is being charged), when the communication performance between the external electronic device and the electronic device (101) is poor, or when the processing performance of the second processor (1002) of the external electronic device is good. The above examples only illustrate cases where the external electronic device can perform relatively many operations for processing a frame set, and the embodiments of the present disclosure are not limited thereto. Although the above examples illustrate cases where the external electronic device can perform relatively many operations, the embodiments of the present disclosure may also be applied to cases where the external electronic device can perform relatively few operations. For example, after the external electronic device performs processing on the second frame set constituting the audio content based on the method of FIG. 10c, the external electronic device may request processing on a third frame set following the second frame set among the frame sets constituting the audio content based on the method of FIG. 10a.

[0234] FIG. 11 illustrates an example of an operational flow for a method in which an electronic device generates frame configuration information of a frame set of audio content and performs decoding using the generated frame configuration information, according to one embodiment.

[0235] The method of FIG. 11 may be performed by the electronic device (101) of FIG. 4. For example, the operations of the method may be controlled by at least one processor of the electronic device (101). For example, the operations may be controlled as the at least one processor is executed individually or collectively. For example, the at least one processor may include the processor (400) or the decoding processor (450) of FIG. 4. In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0236] According to one embodiment, in operation (1110), the electronic device (101) may obtain a frame set including a plurality of frames of audio content to be played. For example, the unit of the audio content may be a frame set. The plurality of frames included in the frame set may correspond to a specified number of frames constituting the frame set.

[0237] According to one embodiment, in operation (1120), the electronic device (101) may generate frame configuration information of the frame set including one or more frames among the plurality of frames.

[0238] For example, the frame configuration information may include a time value (or timestamp) indicating an initial frame among one or more frames of the frame set within the audio content, the number of the one or more frames, a time interval of each of the one or more frames, and a size of each of the one or more frames. The frame configuration information of the frame set may be referenced as frame configuration information for the one or more frames within the frame set.

[0239] In the above example, it is exemplified that the frame configuration information includes a time value (or timestamp) indicating a first frame among the one or more frames in the media, the number of the one or more frames, a time interval of each of the one or more frames, and a size of each of the one or more frames, but the present disclosure is not limited thereto. According to one embodiment, the frame configuration information may include at least one of the information of the above example. For example, the frame configuration information may include the number of the one or more frames and the size of each of the one or more frames. When the time value indicating the first frame is obtained through other information, and the time intervals of each of the one or more frames are the same, the frame configuration information may include information indicating the number and the size among the time value, the number, the time interval, and the size. Alternatively, for example, the frame configuration information may include information indicating the size among the time value, the number, the time interval, and the size.

[0240] For example, the time value indicating the first frame may indicate a time value indicating a frame indicated by the search request, which is a first frame among the one or more frames when a search request is received. For example, the number of the one or more frames may indicate the number of frames included in the reconstructed frame set. For example, the time interval of each of the one or more frames may indicate the time length of each frame included in the reconstructed frame set. For example, the size of each of the one or more frames may include the number of bits (or bytes) of each frame included in the reconstructed frame set. For example, the time value and the time interval may be calculated based on at least one of a file format of the media (or a specification of the file format), a codec type in which the media is encoded, a bit rate of the media, or a sampling rate.

[0241] According to one embodiment, the one or more frames in the frame set may include the plurality of frames having the specified number constituting the frame set. For example, the electronic device (101) may identify the one or more frames including the plurality of frames having the specified number when no search request is received. Alternatively, for example, the electronic device (101) may identify the one or more frames including the plurality of frames having the specified number when it is determined that reconfiguration of the frame set is not necessary in response to the search request.

[0242] According to one embodiment, the one or more frames in the frame set may include a portion of the plurality of frames having the specified number constituting the frame set. For example, when the electronic device (101) determines that the frame set needs to be reconfigured in response to the search request, it may remove one or more other frames from among the plurality of frames having the specified number. The one or more frames may represent a remaining portion of the plurality of frames from which the one or more other frames have been removed. For example, a first frame from among the one or more frames may be a frame indicated by the search request.

[0243] Although not shown in Fig. 11, specific details regarding the method for generating the frame configuration information may be referenced to Figs. 6a to 6c.

[0244] According to one embodiment, in operation (1130), the electronic device (101) may perform decoding of the frame set including the one or more frames using the frame configuration information.

[0245] For example, the electronic device (101) can recognize (or read) the frame set including the one or more frames and the frame configuration information for the one or more frames. For example, the electronic device (101) can determine the transmission unit of data to be transmitted to a decoder that will perform decoding of the recognized frame set. For example, specific details on a method for determining the transmission unit can be referenced to FIG. 7.

[0246] For example, when the electronic device (101) determines the transmission unit as the frame set, it can recognize the codec type and determine whether to add a header based on the codec type. For example, when addition of the header is necessary, the electronic device (101) can generate a header according to the codec type using the frame configuration information and add the generated header to the frame set. At this time, the frame configuration information can be removed from the frame set. For example, when addition of the header is not necessary, the electronic device (101) can remove the frame configuration information from the frame set. The electronic device (101) can transmit the frame set with the header added or the frame set with the header not added to the decoder, depending on the codec type. For specific details on a method of determining whether to add the header based on the codec type and transmitting the frame set to the decoder, reference may be made to FIGS. 8A and 8B.

[0247] According to one embodiment, the set of frames transmitted to the decoder may be decoded. For example, the decoder may be included within the at least one processor. The decoded set of frames may be output (or played) through an audio output device of the electronic device (101) or an audio output device of an external electronic device connected to the electronic device (101).

[0248] Although not illustrated in FIG. 11, according to one embodiment, the electronic device (101) may recognize the maximum size of a frame set that the decoder can support and reconfigure the frame set to correspond to the maximum size. For specific details on a method for recognizing the maximum size and reconfiguring the frame set, reference may be made to FIG. 9.

[0249] As described above, the device, method, and storage medium according to embodiments of the present disclosure can generate frame configuration information including the number of frames included in a frame set, the size of each frame, the time interval of each frame, and a timestamp indicating the first frame of the frame set. The device, method, and storage medium according to embodiments of the present disclosure can accurately recognize a frame according to a search request using the configuration information and remove unnecessary frames, thereby performing an accurate search operation, even if the media is configured by frame set. In addition, the device, method, and storage medium according to embodiments of the present disclosure can perform recovery when an error in decoding occurs by transmitting a frame set including the configuration information to a decoder, even if decoding is performed (e.g., offloading) by frame set. In addition, the device, method, and storage medium according to embodiments of the present disclosure can prevent delay by adjusting the size of a frame set in advance when providing a streaming service through a network.

[0250] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0251] As described above, the electronic device (101) may include a memory (130) that stores instructions. The electronic device (101) may include at least one processor (400, 450). The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to obtain a frame set including a plurality of frames of audio content to be played. The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to generate frame configuration information of the frame set including one or more frames among the plurality of frames. The frame configuration information may include the number of the one or more frames and the size of each of the one or more frames. The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to perform decoding of the frame set including the one or more frames using the frame configuration information.

[0252] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to determine whether a seek request for the audio content is received. The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to identify the one or more frames including the plurality of frames if the seek request is not received. The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to obtain a time value of a frame indicated by the seek request if the seek request is received.

[0253] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to determine whether the frame having the time value is a first frame among the plurality of frames. The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to identify one or more frames including the plurality of frames if the frame is the first frame among the plurality of frames. The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to remove one or more other frames from among the plurality of frames prior to the time value of the frame when the frame is different from the first frame from among the plurality of frames, and to reconstruct the frame set composed of the plurality of frames to include one or more frames including the frame that is different from the one or more other frames from among the plurality of frames.

[0254] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to recognize a codec type for the decoding. The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to generate one or more headers determined according to the codec type using the frame configuration information. Each of the one or more headers may include the size of each of the one or more frames among the frame configuration information. The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to insert the one or more headers into the frame set including the one or more frames. The above decoding may be performed on the frame set into which one or more headers are inserted.

[0255] According to one embodiment, the codec type may include AAC (advanced audio coding) or Opus.

[0256] According to one embodiment, the frame configuration information may further include a time value indicating an initial frame among the one or more frames within the audio content, and a time interval (duration) of each of the one or more frames. The number of the one or more headers may correspond to the number of the one or more frames. Each of the one or more headers may be concatenated for each of the one or more frames using the time value of the frame configuration information, the number of the one or more frames, and the time interval of each of the one or more frames.

[0257] According to one embodiment, each of the time value and the time interval of the frame configuration information may be calculated based on at least one of a format of the audio content, a bit rate of the audio content, or a sampling rate of the audio content.

[0258] According to one embodiment, the audio content may be output through the electronic device (101) or through an external electronic device connected to the electronic device (101), based on the decoded frame set.

[0259] According to one embodiment, the number of said one or more frames may correspond to a maximum size of said set of frames that can be supported by a decoder for said decoding.

[0260] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to generate the frame configuration information for the one or more frames including the plurality of frames, and then receive a seek request for the audio content. The instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to identify, using the frame configuration information, one or more other frames after the time value of the frame among the one or more frames, when a time value of a frame indicated by the seek request is different from a time value indicating a first frame among the one or more frames of the frame configuration information. The above instructions, when individually or collectively executed by the at least one processor (400, 450), may cause the electronic device (101) to reconfigure the frame set composed of the one or more frames to include the one or more other frames.

[0261] In a method performed by an electronic device (101) as described above, the method may include an operation of obtaining a frame set including a plurality of frames of audio content to be played. The method may include an operation of generating frame configuration information of the frame set including one or more frames among the plurality of frames. The frame configuration information may include a number of the one or more frames and a size of each of the one or more frames. The method may include an operation of performing decoding of the frame set including the one or more frames using the frame configuration information.

[0262] The non-transitory computer-readable storage medium as described above may store one or more programs including instructions that cause, when individually or collectively executed by at least one processor (400, 450) of the electronic device (101), to obtain a frame set including a plurality of frames of audio content to be played back. The non-transitory computer-readable storage medium may store one or more programs including instructions that cause, when individually or collectively executed by the at least one processor (400, 450) of the electronic device (101), to generate frame configuration information of the frame set including one or more frames of the plurality of frames. The frame configuration information may include the number of the one or more frames and the size of each of the one or more frames. The non-transitory computer-readable storage medium may store one or more programs including instructions that cause the at least one processor (400, 450) of the electronic device (101) to perform decoding of the frame set including the one or more frames using the frame configuration information when executed individually or collectively.

[0263] As described above, the electronic device (101) may include a first processor (400). The electronic device (101) may include a second processor (450) for decoding audio content. The first processor (400) may be configured to obtain a frame set including a plurality of frames of the audio content. The first processor (400) may be configured to generate frame configuration information of the frame set including one or more frames among the plurality of frames. The frame configuration information may include the number of the one or more frames and the size of each of the one or more frames. The first processor (400) may be configured to generate one or more headers determined according to a codec type using the frame configuration information. Each of the one or more headers may include the size of each of the one or more frames. The first processor (400) may be configured to transmit the frame set including the one or more headers and the one or more frames to the second processor (450). The second processor (450) may be configured to perform the decoding of the frame set.

[0264] According to one embodiment, the first processor (400) may be configured to determine whether a seek request for the audio content is received. If the seek request is not received, the first processor (400) may be configured to identify one or more frames including the plurality of frames. If the seek request is received, the first processor (400) may be configured to obtain a time value of a frame indicated by the seek request.

[0265] According to one embodiment, the first processor (400) may be configured to determine whether the frame having the time value is a first frame among the plurality of frames. The first processor (400) may be configured to identify one or more frames including the plurality of frames when the frame is the first frame among the plurality of frames. The first processor (400) may be configured to remove one or more other frames prior to the time value of the frame among the plurality of frames when the frame is different from the first frame among the plurality of frames, and to reconstruct the frame set composed of the plurality of frames to include one or more frames including the frame that is different from the one or more other frames among the plurality of frames.

[0266] According to one embodiment, the first processor (400) may be configured to recognize the codec type. The first processor (400) may be configured to insert the one or more headers determined according to the recognized codec type into the frame set including the one or more frames.

[0267] According to one embodiment, the first processor (400) may be configured to transmit, to the second processor (450), information indicating that the frame set includes the one or more headers and the one or more frames, together with the frame set including the one or more headers.

[0268] According to one embodiment, the codec type may include AAC (advanced audio coding) or Opus.

[0269] According to one embodiment, the frame configuration information may further include a time value indicating an initial frame among the one or more frames within the audio content, and a time interval (duration) of each of the one or more frames. The number of the one or more headers may correspond to the number of the one or more frames. Each of the one or more headers may be concatenated for each of the one or more frames using the time value of the frame configuration information, the number of the one or more frames, and the time interval of each of the one or more frames.

[0270] According to one embodiment, each of the time value and the time interval of the frame configuration information may be calculated based on at least one of a format of the audio content, a bit rate of the audio content, or a sampling rate of the audio content.

[0271] According to one embodiment, the second processor (450) may be configured to transmit the decoded frame set to an audio output device of the electronic device (101) or an external electronic device connected to the electronic device (101). The audio content may be output based on the decoded frame set.

[0272] According to one embodiment, the second processor (450) may be configured to transmit capability information indicating a maximum size of the frame set that can be supported by the second processor (450) to the first processor (400). The first processor (400) may be configured to identify one or more frames corresponding to the maximum size based on the capability information.

[0273] As described above, the method performed by the electronic device (101) may include an operation of obtaining a frame set including a plurality of frames of audio content. The method may include an operation of generating frame configuration information of the frame set including one or more of the plurality of frames. The frame configuration information may include a number of the one or more frames and a size of each of the one or more frames. The method may include an operation of generating one or more headers determined according to a codec type of a decoder using the frame configuration information. Each of the one or more headers may include the size of each of the one or more frames. The method may include an operation of transmitting the frame set including the one or more headers and the one or more frames to the decoder. The method may include an operation of performing the decoding of the frame set.

[0274] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0275] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0276] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0277] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0278] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0279] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101), A memory (130) storing instructions and including one or more storage media; and At least one processor (400, 450) comprising a processing circuit, The above instructions, when individually or collectively executed by the at least one processor (400, 450), cause the electronic device (101) to: Obtaining a frame set including a plurality of frames of audio content to be played; Generating frame configuration information of the frame set including one or more frames among the plurality of frames, wherein the frame configuration information includes the number of the one or more frames and the size of each of the one or more frames; and Using the above frame configuration information, causing decoding of the frame set including the one or more frames to be performed, Electronic devices (101).

2. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (400, 450), cause the electronic device (101) to: Determining whether a seek request for said audio content is received; If the above search request is not received, identifying one or more frames including the plurality of frames; and When the above search request is received, causing the time value of the frame indicated by the above search request to be obtained. Electronic devices (101).

3. In claim 2, The above instructions, when individually or collectively executed by the at least one processor (400, 450), cause the electronic device (101) to: Determine whether the frame having the above time value is the first frame among the plurality of frames; If the above frame is the first frame among the plurality of frames, identifying one or more frames including the plurality of frames; and If the above frame is different from the first frame among the plurality of frames: removing one or more other frames prior to the time value of said frame among said plurality of frames; and causing said frame set composed of said plurality of frames to be reconstructed to include said one or more frames including said frame that is different from said one or more other frames among said plurality of frames; Electronic devices (101).

4. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (400, 450), cause the electronic device (101) to: Recognize the codec type for the above decoding; Using the above frame configuration information, one or more headers are generated that are determined according to the codec type, and each of the one or more headers includes the size of each of the one or more frames among the frame configuration information; and causing said one or more headers to be inserted into said frame set containing said one or more frames, The above decoding is performed on the frame set into which one or more headers are inserted. Electronic devices (101).

5. In claim 4, The above codec type includes AAC (advanced audio coding) or Opus. Electronic devices (101).

6. In claim 4, The above frame configuration information further includes a time value indicating an initial frame among the one or more frames within the audio content, and a time interval (duration) of each of the one or more frames, The number of said one or more headers corresponds to the number of said one or more frames, and Each of said one or more headers is concatenated for each of said one or more frames using said time value of said frame configuration information, said number of said one or more frames, and said time interval of each of said one or more frames. Electronic devices (101).

7. In claim 6, Each of the time value and the time section of the frame configuration information is calculated based on at least one of the format of the audio content, the bit rate of the audio content, or the sampling rate of the audio content. Electronic devices (101).

8. In claim 1, The above audio content is output through the electronic device (101) or through an external electronic device connected to the electronic device (101), based on the decoded frame set. Electronic devices (101).

9. In claim 1, The number of said one or more frames corresponds to the maximum size of said frame set that can be supported by the decoder for said decoding. Electronic devices (101).

10. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (400, 450), cause the electronic device (101) to: After generating the frame configuration information for the one or more frames including the plurality of frames, receiving a seek request for the audio content; If the time value of the frame indicated by the search request is different from the time value indicating the first frame among the one or more frames of the frame configuration information, using the frame configuration information, identify one or more other frames after the time value of the frame among the one or more frames; and causing said set of frames, comprising said one or more frames, to be reconstructed to include said one or more other frames; Electronic devices (101).

11. In a method performed by an electronic device (101), An operation of obtaining a frame set including a plurality of frames of audio content to be played; An operation of generating frame configuration information of the frame set including one or more frames among the plurality of frames, the frame configuration information including the number of the one or more frames and the size of each of the one or more frames; and An operation of performing decoding of the frame set including the one or more frames using the frame configuration information, method.

12. In claim 11, The above method: An action for determining whether a seek request for said audio content is received; If the above search request is not received, the operation of identifying one or more frames including the plurality of frames; and When the above search request is received, an operation for obtaining a time value of a frame indicated by the above search request is included. method.

13. In claim 12, The above method: An operation for determining whether the frame having the above time value is the first frame among the plurality of frames; An operation of identifying one or more frames including the plurality of frames, when the frame is the first frame among the plurality of frames; and If the above frame is different from the first frame among the plurality of frames: An operation of removing one or more other frames prior to the time value of the frame among the plurality of frames; and An operation of reconstructing the frame set composed of the plurality of frames to include one or more frames including the frame, which frame is different from one or more other frames among the plurality of frames. method.

14. In claim 11, The above method: An action for recognizing the codec type for the above decoding; An operation of generating one or more headers determined according to the codec type using the above frame configuration information, each of the one or more headers including the size of each of the one or more frames among the frame configuration information; and comprising the action of inserting said one or more headers into said frame set including said one or more frames; The above decoding is performed on the frame set into which one or more headers are inserted. method.

15. In a non-transitory computer-readable storage medium, when individually or collectively executed by at least one processor (400, 450) of an electronic device (101), the electronic device (101) : Obtaining a frame set including a plurality of frames of audio content to be played; Generating frame configuration information of the frame set including one or more frames among the plurality of frames, wherein the frame configuration information includes the number of the one or more frames and the size of each of the one or more frames; and storing one or more programs including instructions causing decoding of the frame set including the one or more frames using the frame configuration information; A non-transitory computer-readable storage medium.

Images