Electronic device for outputting audio and operating method thereof

Abstract

An electronic device includes a memory storing instructions and a processor. The instructions may cause an electronic device to: obtain, from first media data, a first block of audio data corresponding to an end point of the first media data; identifying a last playback position of the first audio data block based on the end point; acquiring a second audio data block to be used as a search target from second media data based on a start point of the second media data corresponding to the end point; searching audio data corresponding to the final playback position in the second audio data block; determining a playback starting position of the second audio data block based on the searched audio data; and outputting audio data after the playback start position among the second media data after completion of audio playback until the final playback position.

Description

Technical Field

[0001] Embodiments of this disclosure relate to an electronic device for outputting audio data and a method for operating the electronic device. Background Technology

[0002] With the development of electronic communication technology, various functions have been integrated into communication devices or electronic devices. These electronic devices are also designed to perform interoperability functions by communicating and cooperating with other electronic devices. For example, portable electronic devices (e.g., mobile terminals, tablet terminals, or wearable electronic devices) include sound source playback functions in addition to communication functions, and can play and output sound from various sound sources relevant to the application. They can output sound not only from sound sources stored at the time of manufacture, but also by installing additional applications. Summary of the Invention

[0003] The electronic device and its operating method according to embodiments of the present disclosure can output audio data obtained by demultiplexing and decoding.

[0004] The electronic device and its operating method according to embodiments of the present disclosure can seamlessly play back audio from individual media files.

[0005] The electronic device and its operating method according to embodiments of the present disclosure can determine the playback start position of another media file by detecting the same audio data in another media data based on audio data corresponding to the last playback position of one media data.

[0006] According to embodiments of this disclosure, an electronic device may include a memory storing instructions, a speaker, and at least one processor operatively connected to the memory and the speaker. When executed by at least one processor, the instructions may cause the electronic device to obtain a first audio data block from first media data corresponding to the end time of the first media data. When executed by at least one processor, the instructions may cause the electronic device to identify the last playback position of the first audio data block based on the end time. When executed by at least one processor, the instructions may cause the electronic device to obtain a second audio data block from second media data, corresponding to the end time, to be used as a search target, based on the start time of second media data. When executed by at least one processor, the instructions may cause the electronic device to search for audio data in the second audio data block corresponding to the last playback position. When executed by at least one processor, the instructions may cause the electronic device to determine the playback start position of the second audio data block based on detected audio data. When executed by at least one processor, the instructions may cause the electronic device to output audio data to the speaker starting from the playback start position of the second media data after completing audio playback of the first media data up to the last playback position of the first audio data block.

[0007] According to embodiments of this disclosure, a method for operating an electronic device may include: obtaining a first audio data block corresponding to an end time of the first media data from first media data. The method may include: identifying a last playback position of the first audio data block based on the end time. The method may include: obtaining a second audio data block to be used as a search target from the second media data, corresponding to the end time, based on a start time of the second media data. The method may include: searching for audio data in the second audio data block corresponding to the last playback position. The method may include: determining a playback start position of the second audio data block based on detected audio data. The method may include: after completing audio playback of the first media data up to the last playback position of the first audio data block, outputting audio data starting from the playback start position of the second media data to a speaker.

[0008] According to embodiments of this disclosure, in a non-transitory computer-readable storage medium storing one or more programs, the one or more programs may include instructions that, when executed by at least one processor, cause an electronic device to obtain a first audio data block corresponding to an end time of the first media data from first media data, identify a last playback position of the first audio data block based on the end time, obtain a second audio data block to be used as a search target from the second media data based on a start time of second media data, search for audio data corresponding to the last playback position in the second audio data block, determine a playback start position of the second audio data block based on the detected audio data, and, after completing audio playback of the first media data up to the last playback position of the first audio data block, output audio data starting from the playback start position of the second media data to a speaker. Attached Figure Description

[0009] The above and other aspects, features and advantages of specific embodiments of this disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0010] Figure 1 This is a block diagram illustrating an electronic device in a network environment according to various embodiments.

[0011] Figure 2 This is a block diagram illustrating the configuration of an electronic device for outputting audio according to an embodiment of the present disclosure.

[0012] Figure 3 This is a diagram illustrating split playback of a media file according to an embodiment of the present disclosure.

[0013] Figure 4 This is a diagram illustrating a mismatch between audio data due to the absence of a reference frame, according to an embodiment of the present disclosure.

[0014] Figure 5 This is a diagram illustrating an audio interruption caused by an inaccurate playback start position according to an embodiment of the present disclosure.

[0015] Figure 6 This is a diagram illustrating the playback of a media file including overlapping recording periods according to an embodiment of the present disclosure.

[0016] Figure 7 This is a block diagram illustrating a configuration for continuous audio playback according to an embodiment of the present disclosure.

[0017] Figure 8 This is a flowchart illustrating a method for continuous audio playback according to an embodiment of the present disclosure.

[0018] Figure 9 This is a flowchart illustrating a process for determining the last playback position according to an embodiment of the present disclosure.

[0019] Figure 10 This is a flowchart illustrating a process for determining the last playback position of an audio data block according to an embodiment of the present disclosure.

[0020] Figure 11 This is a diagram illustrating an example of determining the last playback position of media data with a set end time according to an embodiment of the present disclosure.

[0021] Figure 12 This is a diagram illustrating an example of determining the last playback position of media data for which no end time is set, according to an embodiment of the present disclosure.

[0022] Figure 13 This is a flowchart illustrating a process for determining target audio data for searching according to an embodiment of the present disclosure.

[0023] Figure 14 This is a diagram illustrating the setting of the demultiplexing start position according to an embodiment of the present disclosure.

[0024] Figure 15 This is a diagram illustrating a demultiplexing operation from the start position of demultiplexing according to an embodiment of the present disclosure.

[0025] Figure 16 This is a diagram illustrating the determination of a target audio data block for searching according to an embodiment of the present disclosure.

[0026] Figure 17 This is a flowchart illustrating a process for detecting audio data according to an embodiment of the present disclosure.

[0027] Figure 18 This is a diagram illustrating the determination of a search target channel according to an embodiment of the present disclosure.

[0028] Figure 19 This is a diagram illustrating the determination of search reference data based on the last playback position according to an embodiment of the present disclosure.

[0029] Figure 20 This is a diagram illustrating the determination of search reference data based on the last position according to an embodiment of the present disclosure.

[0030] Figure 21 This is a diagram illustrating an audio data search according to an embodiment of the present disclosure.

[0031] Figure 22 This is a diagram illustrating the determination of the playback start position based on search results according to an embodiment of the present disclosure.

[0032] Figure 23 This is a diagram illustrating the determination of the playback start position based on search results according to an embodiment of the present disclosure.

[0033] Figure 24 This is a diagram illustrating operations in response to the failure to detect audio data, according to an embodiment of the present disclosure.

[0034] Figure 25 This is a diagram illustrating the operation of outputting video frames based on the timestamp of changing audio data according to an embodiment of the present disclosure. Detailed Implementation

[0035] Figure 1 This is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments.

[0036] Reference Figure 1 In network environment 100, electronic device 101 can communicate with electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or with at least one of electronic device 104 or server 108 via a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 101 can communicate with electronic device 104 via server 108. According to an embodiment, electronic device 101 may include a processor 120, memory 130, input module 150, sound output module 155, display module 160, audio module 170, sensor module 176, interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, user identification module (SIM) 196, or antenna module 197. In some embodiments, at least one of the above components (e.g., connection terminal 178) may be omitted from electronic device 101, or one or more other components may be added to electronic device 101. In some embodiments, some of the components described above (e.g., sensor module 176, camera module 180, or antenna module 197) may be implemented as a single integrated component (e.g., display module 160).

[0037] Processor 120 may run software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of electronic device 101 connected to processor 120, and may perform various data processing or calculations. According to an embodiment, as at least part of the data processing or calculation, processor 120 may store commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132, process the commands or data stored in volatile memory 132, and store the result data in non-volatile memory 134. According to an embodiment, processor 120 may include a main processor 121 (e.g., central processing unit (CPU) or application processor (AP)) or an auxiliary processor 123 (e.g., graphics processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor central processor, or communication processor (CP)) that is operationally independent of or combined with the main processor 121. For example, when electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or may be implemented as part of the main processor 121.

[0038] When the main processor 121 is inactive (e.g., in sleep) state, the auxiliary processor 123 (rather than the main processor 121) can control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190), or when the main processor 121 is active (e.g., running an application), the auxiliary processor 123 can work with the main processor 121 to control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190). According to embodiments, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) functionally associated with the auxiliary processor 123. According to embodiments, the auxiliary processor 123 (e.g., a neural processing unit) may include hardware architecture dedicated to artificial intelligence model processing. Artificial intelligence models can be generated through machine learning. For example, such learning can be performed via electronic device 101 where artificial intelligence is performed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple layers of artificial neural networks. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network, or a combination of two or more thereof, but is not limited thereto. Additionally or optionally, the artificial intelligence model may include software structures in addition to hardware structures.

[0039] Memory 130 may store various data used by at least one component of electronic device 101 (e.g., processor 120 or sensor module 176). The various data may include, for example, software (e.g., program 140) and input or output data for commands associated with it. Memory 130 may include volatile memory 132 or non-volatile memory 134.

[0040] The program 140 may be stored as software in the memory 130, and the program 140 may include, for example, an operating system (OS) 142, middleware 144, or application 146.

[0041] The input module 150 can receive commands or data from outside the electronic device 101 (e.g., a user) that will be used by other components of the electronic device 101 (e.g., processor 120). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (e.g., buttons), or digital pen (e.g., stylus).

[0042] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. According to an embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0043] Display module 160 can visually provide information to the outside of electronic device 101 (e.g., to a user). Display device 160 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. According to an embodiment, display module 160 may include a touch sensor adapted to detect touch or a pressure sensor adapted to measure the intensity of the force caused by touch.

[0044] The audio module 170 can convert sound into electrical signals and vice versa. According to an embodiment, the audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or headphones of an external electronic device (e.g., electronic device 102) that is directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0045] Sensor module 176 can detect the operating state of electronic device 101 (e.g., power or temperature) or the environmental state outside electronic device 101 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. According to embodiments, sensor module 176 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0046] Interface 177 may support one or more specific protocols used to enable electronic device 101 to connect directly (e.g., wired) or wirelessly to external electronic devices (e.g., electronic device 102). According to embodiments, interface 177 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0047] Connection 178 may include a connector, through which electronic device 101 may be physically connected to an external electronic device (e.g., electronic device 102). According to embodiments, connection 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0048] The haptic module 179 can convert electrical signals into mechanical stimuli (e.g., vibration or motion) or electrical stimuli that can be recognized by a user through his touch or kinesthesia. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0049] Camera module 180 can capture still or moving images. According to an embodiment, camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0050] The power management module 188 manages the power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0051] Battery 189 can power at least one component of electronic device 101. According to an embodiment, battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable rechargeable battery, or a fuel cell.

[0052] Communication module 190 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 101 and external electronic devices (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. Communication module 190 may include one or more communication processors capable of operating independently of processor 120 (e.g., application processor (AP)) and supporting direct (e.g., wired) or wireless communication. According to embodiments, communication module 190 may include wireless communication module 192 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 194 (e.g., local area network (LAN) communication module or power line communication (PLC) module). One of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, 5G network, next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components separate from each other (e.g., multiple chips). The wireless communication module 192 can identify and verify the electronic device 101 in the communication network (such as the first network 198 or the second network 199) using user information (e.g., the International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0053] Wireless communication module 192 can support 5G networks following 4G networks and next-generation communication technologies (such as new radio (NR) access technologies). NR access technologies can support enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), or ultra-reliable low-latency communication (URLLC). Wireless communication module 192 can support high-frequency bands (e.g., millimeter-wave bands) to achieve, for example, high data transmission rates. Wireless communication module 192 can support various technologies used to ensure performance in high-frequency bands, such as, for example, beamforming, massive MIMO, full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. Wireless communication module 192 can support various requirements specified in electronic device 101, external electronic devices (e.g., electronic device 104), or network systems (e.g., second network 199). According to an embodiment, the wireless communication module 192 may support peak data rates (e.g., 20 Gbps or greater) for implementing eMBB, lost coverage (e.g., 164 dB or less) for implementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip) for implementing URLLC.

[0054] Antenna module 197 can transmit or receive signals or power to or from the exterior of electronic device 101 (e.g., external electronic device). According to an embodiment, antenna module 197 may include an antenna comprising a radiating element formed of a conductive material or conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, antenna module 197 may include multiple antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 198 or a second network 199) can be selected from the multiple antennas by, for example, communication module 190 (e.g., wireless communication module 192). Signals or power can then be transmitted or received between communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, additional components besides the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally incorporated into antenna module 197.

[0055] According to various embodiments, antenna module 197 may form a millimeter-wave antenna module. According to embodiments, the millimeter-wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and multiple antennas (e.g., an array antenna), wherein the RFIC is disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and is capable of supporting a specified high-frequency band (e.g., a millimeter-wave band), and the multiple antennas are disposed on or adjacent to a second surface (e.g., a top surface or a side surface) of the printed circuit board and are capable of transmitting or receiving signals in the specified high-frequency band.

[0056] At least some of the aforementioned components can be interconnected and communicate signals (e.g., commands or data) between them via an inter-peripheral communication scheme (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0057] According to an embodiment, commands or data can be sent or received between electronic device 101 and external electronic device 104 via server 108 connected to a second network 199. Each of electronic device 102 or electronic device 104 can be a device of the same type as electronic device 101, or a device of a different type. According to an embodiment, all or some operations to be performed on electronic device 101 can be performed on one or more of external electronic devices 102, external electronic devices 104, or server 108. For example, if electronic device 101 is required to automatically perform a function or service, or is required to perform a function or service in response to a request from a user or another device, electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service instead of running the function or service, or electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service in addition to running the function or service. Upon receiving the request, the one or more external electronic devices may perform the requested at least portion of the function or service, or perform additional functions or services related to the request, and transmit the result of the execution to electronic device 101. Electronic device 101 may provide the result as at least a partial response to the request, with or without further processing of the result. For this purpose, technologies such as cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing may be used. Electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and / or neural networks. According to embodiments, external electronic device 104 or server 108 may be included in a second network 199. Electronic device 101 can be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology or IoT-related technologies.

[0058] Figure 2 This is a block diagram illustrating the configuration of an electronic device for outputting audio according to an embodiment of the present disclosure.

[0059] refer to Figure 2 The electronic device (e.g., electronic device 101) may include at least one processor 200 (e.g., Figure 1 The processor 120) and the speaker 250 (e.g., Figure 1The audio output module 155). The processor 200 may include at least one of a controller 210, a demultiplexer 220, or a decoder 230. At least one of the controller 210, demultiplexer 220, or decoder 230 may be implemented as software running by the processor 200 or as a hardware module. In embodiments, at least one of the memory 215 or audio buffer 240 may be included in the processor 200, or may be implemented as a separate memory (e.g., ...). Figure 1 (Memory 130).

[0060] Demultiplexer 220 can receive media data (e.g., at least one media file including audio and video frames) and demultiplex the media data into audio and video frames. The audio frames can be encoded (e.g., compressed) audio frames. The audio frames can be sent to decoder 230. Decoder 230 can generate audio data including Pulse Code Modulation (PCM) data by decoding (e.g., decompressing) the audio frames according to a specified codec (e.g., Moving Picture Experts Group (MPEG)). Audio buffer 240 can store the audio data until it is output to speaker 250. Audio buffer 240 can store the audio data generated by decoder 230 in blocks of audio data of a specified size.

[0061] The controller 210 can control the operation of the demultiplexer 220, the decoder 230, and / or the audio buffer 240. According to the disclosed embodiments, the controller 210 can determine the frames input to the demultiplexer 220, manage timestamps indicating the playback start time of PCM data output from the decoder 230, back up (e.g., copy and store) at least a portion of the PCM data stored in the audio buffer 240 to the memory 215, and determine the PCM data to be output from the audio buffer 240 based on the playback start position (e.g., data position or time position) controlled according to embodiments of this disclosure.

[0062] Before outputting the PCM data stored in the audio buffer 240 to the speaker 250, the controller 210 may perform audio rendering (not shown), such as volume adjustment or resampling of the sound to be output to the speaker 250. In embodiments of this disclosure, outputting audio data (e.g., PCM) to the speaker 250 may include performing audio rendering (such as volume adjustment or resampling) on ​​the audio data.

[0063] The memory 215 may, under the control of the controller 210, store at least one of the following: metadata (e.g., at least one of sample rate, channel count, or sample byte size) associated with at least one media data to be played back; the start or end time of the media data; audio data read from the audio buffer 240 (e.g., at least one audio data block); or the timestamp of the audio data.

[0064] Electronic device 101 (e.g., Figure 2 The processor 200 can be configured to continuously play back one or more media data (e.g., first media data and second media data). The electronic device 101 (e.g., Figure 2 The processor 200 can access memory (e.g., Figure 1 Read one or more media data from the memory 130, or from an external electronic device (e.g., via the communication module 190) through the communication module 190. Figure 1 Electronic device 102, Figure 1 Electronic device 104 or Figure 1 The server 108) receives one or more media data. In this embodiment, one or more media data may be received via an audio streaming service.

[0065] Electronic device 101 (e.g., Figure 2 The processor 200 can obtain audio data (e.g., PCM data) of at least one audio frame of the first media data via demultiplexer 220 and decoder 230, store the audio data in audio buffer 240, and output audio data up to a specified end time of the first media data and audio data up to the end time of the first media data to speaker 250. For subsequent playback of the second media data, the electronic device 101 (e.g., Figure 2 The processor 200 can request the demultiplexer 220 to output at least one audio frame corresponding to a specified start time of the second media data. The electronic device 101 (e.g., processor 200) can obtain audio data (e.g., PCM data) by decoding the at least one audio frame output from the demultiplexer 220 via the decoder 230, and output the audio data corresponding to the start time of the second media data and subsequent audio data to the speaker 250.

[0066] Figure 3 This is a diagram illustrating the segmented playback of a media file according to an embodiment of the present disclosure.

[0067] Reference Figure 3 According to an embodiment, electronic device 101 (e.g., Figure 2The processor 200 can divide a media file 310 (e.g., media.mp4) with a length of 10000ms into first media data 312 and second media data 314, and process the first media data 312 and the second media data 314 respectively through different demultiplexing and decoding processes (e.g., demultiplexer 220 and decoder 230).

[0068] According to an embodiment, electronic device 101 (e.g., processor 200) can play back media file 310 by dividing it into first media data 312 corresponding to a time period from 0ms to 5000ms and second media data 314 corresponding to a time period from 5000ms to 10000ms based on a specified time point (e.g., 5000ms). Therefore, the specified time point (e.g., 5000ms) can be set as end_time indicating the end time of the first media data 312 and start_time indicating the start time of the second media data 314, respectively. According to an embodiment, electronic device 101 (e.g., processor 200) can control demultiplexer 220 and decoder 230 such that the second media data 314 can be played back continuously after the first media data 312.

[0069] According to an embodiment, since the first media data 312 and the second media data 314 are configured as frames by the demultiplexer 220, audio frames following the first media data 312 may at least partially overlap with audio frames preceding the second media data 314. However, depending on the decoding method of the audio frames, the audio data at the end time of the first media data 312 and the audio data at the start time of the second media data 314 may be discontinuous, which may cause audio interruptions in the audio output to the speaker 250.

[0070] For example, decoder 230 can improve decoding performance by referencing one or more previous audio frames used for decoding each audio frame. At least one of the previous audio frames of the second media data 314 can be decoded without a reference audio frame (e.g., a previous audio frame), which may reduce continuity with the first media data 312 and cause audio interruptions in the audio output to speaker 250.

[0071] Figure 4 This is a diagram illustrating a mismatch between audio data due to the absence of a reference frame, according to an embodiment of the present disclosure.

[0072] Reference Figure 4 Audio buffer (e.g., Figure 2The audio buffer 240 can store audio data (e.g., PCM data) generated by decoding a specified number (e.g., one or more) of audio frames of the first media data 312 in units of audio data blocks. The first media data (e.g., PCM data) stored in the audio buffer 240 after demultiplexing and decoding... Figure 3 The last audio data block 402 of the second media data 312 may have a size of, for example, TS = 4991ms and 4096 bytes. The last audio data block 402 may include audio data generated as a result of decoding at least one audio frame having, for example, TS = 4991ms, and PCM data corresponding to each audio frame can be generated by referring to the decoding of previous audio frames (not shown) within the first media data 312. Electronic device 101 (e.g., Figure 2 The processor 200 can send signals to the speaker (e.g., Figure 2 The speaker 250 outputs audio data 404 until it matches the first media data in the last audio data block 402 (e.g., Figure 3 The last playback position 406 corresponds to the end_time (e.g., 5000ms) of the first media data 312.

[0073] According to an embodiment, in order to continuously play back the second media data following the first media data 312 (e.g., Figure 3 The second media data 314), electronic device 101 (e.g., processor 200) can operate a demultiplexer for the second media data 314 (e.g., Figure 2 The demultiplexer 220) and decoder (e.g., Figure 2 Demultiplexer 220 can start demultiplexing from the audio frame with TS = 4991ms that is closest to the start_time (e.g., 5000ms) of the second media data 314, and output the demultiplexed audio frame to decoder 230. Through demultiplexer 220 and decoder 230, the first audio data block 412 of the second media data 314 corresponding to the start_time (e.g., 5000ms) of the second media data 314 can be stored in audio buffer 240. Therefore, similar to the last audio data block 402 of the first media data 312, the first audio data block 412 of the second media data 314 can include audio data generated from at least one audio frame with TS = 4991ms.

[0074] The first audio data block 412 of the second media data 314 may include the initial decoding result of the second media data 314 through a separate decoding process compared to the first media data 312. This is because the decoder (e.g., Figure 2If the decoder 230 decodes the preceding audio frames of the second media data 314 without any previous audio frames available for reference, the PCM data of the first audio data block 412 of the second media data 314 may not match the PCM data of the last audio data block 402 of the first media data 312 420. Due to the PCM data mismatch 420, when the electronic device 101 (e.g., processor 200) plays back the audio data 414 after the playback start position 416 corresponding to the start time 416 of the first audio data block 412 412 414 404 404 404 404 404 404 406 406 406 406 406 406 4000 ms ...

[0075] Figure 5 This is a diagram illustrating an audio interruption caused by an inaccurate playback start position according to an embodiment of the present disclosure.

[0076] Reference Figure 5 Electronic device 101 (e.g., Figure 2 The processor 200) can process the first media data (e.g., Figure 3 The audio data 506 prior to the end_time of the first media data 312 is output to the speaker (e.g., Figure 2 The speaker 250 is used for audio playback of the first media data 312. This is achieved through a demultiplexer (e.g., Figure 2 The demultiplexer 220 and the decoder (e.g., Figure 2 The decoder 230 is then stored in the audio buffer (e.g., Figure 2 The last audio data block 502 of the first media data 312 in the audio buffer 240 may have a size of, for example, TS = 4991ms and 4096 bytes. The electronic device 101 (e.g., processor 200) may output audio data 506 to the speaker 250 until the last playback position P1 504 corresponding to the end_time (e.g., 5000ms) in the last audio data block 502 of the first media data 312.

[0077] In order to continuously replay the second media data (e.g., Figure 3 The electronic device 101 (e.g., processor 200) can operate the demultiplexer 220 and decoder 230 to output audio data corresponding to the start_time of the second media data 314. The electronic device 101 (e.g., processor 200) can control the demultiplexer 220 to output at least one audio frame corresponding to the start_time (=5000ms) in the media file 310.

[0078] In the example of (a), due to inaccuracies in determining the location of the data to be demultiplexed, demultiplexer 220 may not accurately detect at least one audio frame corresponding to the position closest to the start_time of 4991ms in the second media data 314, and may output at least one audio frame starting from an incorrect position (e.g., 4996ms). As a result, the first audio data block 512 of the second media data 314 may include audio data (e.g., PCM data) with TS = 4996ms.

[0079] Electronic device 101 (e.g., processor 200) may assume that the first audio data block 512 includes audio data with TS = 4991ms, and outputs audio data to speaker 250 from the actual playback position P3 516 based on the last playback position 504 of the first media data 312. However, the expected playback position P2 514, which actually includes audio data corresponding to the last playback position 504 of the first media data 312, is earlier than the actual playback position P3 516; therefore, audio data in the time interval from P2 to P3 may be lost.

[0080] In the example of (b), due to inaccuracies in determining the location of the data to be demultiplexed, the demultiplexer 220 may not accurately detect the audio frame corresponding to the audio data at the location closest to the start_time of 4991ms in the second media data 314, and may output at least one audio frame starting from an incorrect location (e.g., 4986ms). As a result, the first audio data block 522 of the second media data 314 may include audio data (e.g., PCM data) with TS = 4986ms.

[0081] Electronic device 101 (e.g., processor 200) may assume that the first audio data block 522 includes audio data with TS = 4991ms, and outputs audio data to speaker 250 from the actual playback position P4 526 based on the last playback position 504 of the first media data 312. However, the expected playback position P5 524, which actually includes the audio data corresponding to the last playback position 504 of the first media data 312, is later than the actual playback position P4 526. Therefore, audio data during the time period from P4 to P5 may be redundantly output.

[0082] As described above, due to the absence of a reference frame or inaccuracy in determining the location to be demultiplexed during decoding, the audio data of the second media data 314 may overlap with some audio data of the first media data 312, or some audio data of the second media data 314 may be omitted during audio playback, which may lead to audio interruption.

[0083] Figure 6 This is a diagram illustrating the playback of a media file including overlapping recording periods according to an embodiment of the present disclosure.

[0084] refer to Figure 6 The first media file 602 and the second media file 604 can be recorded separately and include at least partially overlapping data 610, for example, through a motion picture function. When the electronic device 101 (e.g., Figure 2 When the processor 200 plays back the first media file 602 and the second media file 604, which include overlapping data 610, the user watching the video can determine that the audio playback quality is poor because the audio and video of the overlapping data 610 are redundantly output.

[0085] As mentioned above, due to audio interruption or in the first media data that may include overlapping data (e.g., Figure 3 First Media Data 312 or Figure 6 First media file 602) and second media data (e.g., Figure 3 Second media data 314 or Figure 6 During continuous playback of the second media file (604), audio overlap may occur, which may cause unpleasant sensations for the user, such as ticking sounds, which may cause discomfort.

[0086] In embodiments of this disclosure, when first media data (e.g., first media data 312 or first media file 602) and second media data (e.g., second media data 314 or second media file 604) are played back consecutively, electronic device 101 (e.g., processor 200) can obtain the last audio data block corresponding to the end time of the first media data and mark the last playback position within the last audio data block.

[0087] In embodiments of this disclosure, media data may include audio data (e.g., compressed audio frames), video data (e.g., compressed video frames), and metadata, and a media file (e.g., media file 310) may be defined as a collection of media data of a specified time or a specified size, stored in memory using a single extension. In embodiments of this disclosure, first media data or second media data may be defined as a unit of media data that serves as input to demultiplexing and decoding processes for audio playback. In embodiments of this disclosure, first media data or second media data may be defined as a single media file or at least a portion of a single media file.

[0088] In embodiments of this disclosure, an audio data block (e.g., a first audio data block or a second audio data block) can be defined as a single data unit comprising PCM data of a specified size generated by decoding. In embodiments, an audio data block can be defined as a unit of data generated by a decoder (e.g., Figure 2 The decoder 230) is used to demultiplexer (e.g., Figure 2 The PCM dataset is generated by decoding a specified number (e.g., one or more) of audio frames output by the demultiplexer 220. In embodiments of this disclosure, the first audio data block or the second audio data block may be the smallest unit of segmentation processing for audio playback.

[0089] In embodiments of this disclosure, electronic device 101 (e.g., processor 200) can obtain a first audio data block corresponding to the start time of the second media data as a search target, and accurately determine the playback start position of the second media data by detecting audio data that is the same as the audio data at the last playback position in the second audio data block.

[0090] When a single media file (e.g., media file 310) is divided and played back continuously, embodiments of this disclosure can prevent audio interruption, and when media files (e.g., media files 602 and 604) that include overlapping recording periods are played back continuously, embodiments of this disclosure can remove overlapping data and play back audio and video continuously without interruption.

[0091] Figure 7 This is a block diagram illustrating a configuration for continuous audio playback according to an embodiment of the present disclosure.

[0092] Reference Figure 7 Electronic device 101 (e.g., Figure 2The processor 200 may include a last audio data determiner 712, a last playback position determiner 714, a search target determiner 716, an audio data detector 718, and a playback start position determiner 720. In an embodiment, at least one of the audio data determiner 712, the last playback position determiner 714, the search target determiner 716, the audio data detector 718, or the playback start position determiner 720 may be implemented as a software module executed by the processor 200. According to an embodiment, at least one of the audio data determiner 712, the last playback position determiner 714, the search target determiner 716, the audio data detector 718, or the playback start position determiner 720 may be implemented as a separate processor.

[0093] According to an embodiment, the final audio data determiner 712 may receive a first audio data block 702 from the audio buffer 240. The first audio data block 702 includes PCM data generated by decoding at least one audio frame of the first media data (e.g., first media data 312 or first media file 602), and identifies that the first audio data block 702 includes PCM data corresponding to the end_time of the first media data 702. The final audio data determiner 712 may read the first audio data block 702 from the audio buffer 240 and back it up (e.g., copy and store) in another memory space. In an embodiment, the first audio data block 702 may be stored in memory 215.

[0094] According to an embodiment, the last playback position determiner 714 can determine the last playback position 722 of the first audio data block 702. In an embodiment, the last playback position 722 may include a byte offset from the start time of the first audio data block 702. For example, the last playback position 722 may indicate the last byte of audio data (e.g., audio data 404 or audio data 506) used for audio playback in the first audio data block 702.

[0095] According to an embodiment, the search target determiner 716 can control the demultiplexer 220 to output at least one audio frame among the audio frames forming the second media data (e.g., the second media data 314 or the second media file 604) as the search target, and receive a second audio data block 704 from the audio buffer 240, the second audio data block 704 including PCM data generated by decoding at least one audio frame by the decoder 230.

[0096] According to an embodiment, the audio data detector 718 can determine search reference audio data in the first audio data block 702 based on the last playback position 722 determined by the last playback position determiner 714. The audio data detector 718 can use the search reference audio data to search for a second audio data block, determine whether the second audio data block includes the same audio data as the search reference audio data, and determine the location of the audio data that is the same as the search reference audio data.

[0097] According to an embodiment, the playback start position determiner 720 can determine the position detected by the audio data detector 718 as the playback start position 724 of the second media data.

[0098] Figure 8 This is a flowchart illustrating a method for continuous audio playback according to embodiments of the present disclosure. In the embodiments, at least one of the operations described below can be performed by electronic device 101 (e.g., Figure 2 The processor 200 executes the operation. According to an embodiment, at least one of the operations described below may be omitted, modified, or reordered. For example, at least one of the operations described below may be executed in parallel with another operation, or may be executed regardless of the order shown.

[0099] Reference Figure 8 In operation 810, electronic device 101 (e.g., Figure 2 The processor 200 can obtain information related to the first media data (e.g., ...) from the first media data. Figure 3 First Media Data 312 or Figure 6 The first audio data block corresponding to the specified end time (e.g., end_time) of the first media file 602. In an embodiment, the electronic device 101 (e.g., processor 200) can use a demultiplexer (e.g., Figure 2 The demultiplexer 220 obtains at least one subsequent audio frame from a plurality of audio frames of the first media data, and can read a first audio data block from an audio buffer (e.g., audio buffer 240), the first audio data block comprising audio frames obtained by a decoder (e.g., Figure 2 The decoder 230) obtains PCM data by decoding at least one audio frame. In an embodiment, operation 810 may include... Figure 9 At least one of operation 902, operation 904 or operation 906.

[0100] In operation 820, electronic device 101 (e.g., processor 200) can identify, based on the end time, a first data position associated with the last audio data in the first audio data block to be used for audio playback (e.g., Figure 7The last playback position 722. In an embodiment, the last playback position may indicate the position of the last audio data (e.g., one byte) used for audio playback in the first audio data block, or the position of the next data (e.g., one byte) following the last audio data used for audio playback in the first audio data block. For example, the last playback position may include a byte offset from the start time of the first audio data block. For example, the last playback position may indicate which byte in the first audio data block the last audio data or the next data is located in.

[0101] In an embodiment, operation 820 may include Figure 9 Operation 908. In an embodiment, electronic device 101 (e.g., processor 200) may determine the last playback position based on at least one of the timestamp and size of the first audio data block or the end time of the first media data.

[0102] In operation 830, electronic device 101 (e.g., processor 200) may continue playing (e.g., outputting to speaker 250) audio data until the last playback position of the first audio data block. Although operation 830 is shown as being performed after operations 810 and 820, electronic device 101 (e.g., processor 200) may perform at least one of operations 810, 820, 840, 850, or 860 while playing back the audio data of the first media data until the last playback position. In embodiments, operation 830 may include... Figure 9 Operation 910. In an embodiment, the electronic device 101 (e.g., processor 200) can complete the audio playback of the first media data after playing back the audio data up to the last playback position. According to an embodiment, the electronic device 101 (e.g., processor 200) can complete the audio playback of the first media data after playing back the audio data up to the last playback position and the audio data at the last playback position.

[0103] In operation 840, electronic device 101 (e.g., processor 200) may acquire a second audio data block corresponding to a specified start time of the second media data (e.g., second media data 314 or second media file 604). In an embodiment, electronic device 101 (e.g., processor 200) may acquire one or more preceding audio frames of a plurality of audio frames of the second media data via demultiplexer 220 and read the second audio data block from audio buffer 240, the second audio data block comprising PCM data obtained by decoding the one or more audio frames by decoder 230. In an embodiment, operation 840 may include... Figure 13 The process.

[0104] In operation 850, electronic device 101 (e.g., processor 200) may search for audio data corresponding to the last playback position in a second audio data block. In an embodiment, electronic device 101 (e.g., processor 200) may determine search reference audio data in a first audio data block 702 based on the last playback position identified in operation 820, and search for the same audio data as the search reference audio data in the second audio data block. In an embodiment, operation 850 may include... Figure 17 The process.

[0105] In operation 860, electronic device 101 (e.g., processor 200) may determine, based on the detected audio data, a data position (e.g., playback start position 724) within a second audio data block that is associated with the first audio data to be used for audio playback. In embodiments, the playback start position may indicate the start position (e.g., byte position) of the detected audio data, or a position preceding the detected audio data. For example, the playback start position may include a byte offset from the start time of the second audio data block.

[0106] In operation 870, after completing audio playback of the first media data based on the last playback position, the electronic device 101 (e.g., processor 200) can output audio data (e.g., PCM data) of the second media data from the playback start position of the second audio data block to the speaker 250. According to an embodiment, the electronic device 101 (e.g., processor 200) can complete audio playback of the first media data by outputting audio data up to the last playback position of the first audio data block to the speaker 250. Then, the electronic device 101 (e.g., processor 200) can output audio data at the playback start position and audio data after the playback start position to the speaker 250.

[0107] Figure 9 This is a flowchart illustrating a process for determining the last playback position according to an embodiment of the present disclosure. In the embodiment, at least one of the operations described below may be performed by electronic device 101 (e.g., processor 200). According to the embodiment, at least one of the operations described below may be omitted, modified, or reordered. For example, at least one of the operations described below may be performed in parallel with another operation, or may be performed independently of the order shown. In the embodiment, Figure 9 The process can correspond to Figure 8 Operations 810, 820, and 830.

[0108] Reference Figure 9 In operation 902, electronic device 101 (e.g., Figure 2 The processor 200) can process the first media data (e.g., Figure 3First Media Data 312 or Figure 6 The first media file 602) is demultiplexed and decoded to obtain a first audio data block (e.g., ) including PCM data corresponding to at least one audio frame. Figure 7 The first audio data block 702). In an embodiment, electronic device 101 (e.g., processor 200) can access audio data from an audio buffer (e.g., Figure 2 The audio buffer 240 reads the first audio data block.

[0109] In operation 904, electronic device 101 (e.g., processor 200) can determine whether the first audio data block is the last audio data block of the first media data.

[0110] In an embodiment, when an end_time indicating the end time of the first media data is set (e.g., set by processor 200 to identify the playback end time of the first media data 312 in media file 310), electronic device 101 (e.g., processor 200) can determine whether the first audio data block includes audio data corresponding to end_time based on the timestamp and size of the first audio data block. Electronic device 101 (e.g., processor 200) can identify the timestamp of the audio data block from at least one audio frame corresponding to the audio data block before demultiplexing and decoding.

[0111] For example, electronic device 101 (e.g., processor 200) can calculate a playback time length (e.g., in ms or µs) corresponding to the size of the first audio data block (e.g., in bytes) based on at least one of the sampling rate, channel count, or sample byte size applied to the first media data. When the playback end time of the first audio data block, which is the sum of the timestamp and the playback time length calculated as the first audio data block, is after end_time, electronic device 101 (e.g., processor 200) can determine that the first audio data block is the last audio data block of the first media data. When the playback end time of the first audio data block is before end_time, electronic device 101 (e.g., processor 200) can determine that the first audio data block is not the last audio data block of the first media data.

[0112] In an embodiment, when no end_time indicating the end time of the first media data is set (e.g., when the first media data includes the first media file 602), the electronic device 101 (e.g., processor 200) can determine whether the first audio data block includes the last decoded result containing audio data. When no subsequent decoded result exists, the electronic device 101 (e.g., processor 200) can determine that the first audio data block is the last audio data block of the first media data. When a subsequent decoded result exists, the electronic device 101 (e.g., processor 200) can determine that the first audio data block is not the last audio data block of the first media data.

[0113] According to an embodiment, when it is determined that the first audio data block is the last audio data block of the first media data, the electronic device 101 (e.g., processor 200) can proceed to operation 906. Conversely, when it is determined that the first audio data block is not the last audio data block of the first media data, the electronic device 101 (e.g., processor 200) can proceed to operation 912.

[0114] In operation 912, electronic device 101 (e.g., processor 200) can output audio data of the first audio data block to speaker 250.

[0115] In operation 906, electronic device 101 (e.g., processor 200) can back up (e.g., copy and store) the first audio data block in a separate memory area (e.g., memory 215) so that it can be used for subsequent audio searches (e.g., operation 850).

[0116] In operation 908, electronic device 101 (e.g., processor 200) can identify the last playback position corresponding to the end_time of the first media data from the first audio data block. The last playback position can be calculated based on metadata (e.g., at least one of the sample rate, channel count, or sample byte size of the first media data), the timestamp of the first audio data block 1100, or the end_time set for at least one of the first media data. See later for further details. Figure 10 An embodiment of operation 908 is described.

[0117] In operation 910, electronic device 101 (e.g., processor 200) can output audio data up to the last playback position of the first audio data block (e.g., audio data 1102 to be used) to speaker 250. Remaining audio data at and after the last playback position (e.g., audio data 1104 to be skipped) can be skipped and not output to speaker 250. For example, the remaining audio data can be deleted immediately or after a specified time.

[0118] Figure 10This is a flowchart illustrating a process for determining the last playback position of an audio data block according to an embodiment of the present disclosure. In the embodiment, at least one of the operations described below can be performed by electronic device 101 (e.g., Figure 2 The processor 200 executes the operation. According to an embodiment, at least one of the operations described below may be omitted, modified, or reordered. For example, at least one of the operations described below may be executed in parallel with another operation, or may be executed regardless of the order shown. In an embodiment, Figure 10 The process can correspond to Figure 9 Operation 908.

[0119] refer to Figure 10 In operation 1002, electronic device 101 (e.g., Figure 2 The processor 200 can identify the timestamp (TS) and size of the first audio data block. The electronic device 101 (e.g., processor 200) can store the data in memory (e.g., ...). Figure 2 The memory 215) pre-stores or reads data from the memory with the first media (e.g., Figure 3 First Media Data 312 or Figure 6 Metadata related to the first media file (602) (e.g., at least one of the sampling rate, channel count, or sample byte size).

[0120] In operation 1004, electronic device 101 (e.g., processor 200) can determine whether an end_time indicating a set playback end time exists for the first media data. For example, for first media data identified as being used for split playback (e.g., first media data 312), an end_time corresponding to a split position (e.g., 5000ms) specified by electronic device 101 (e.g., processor 200) can be set. For example, separately generated first media data (e.g., first media file 602) may not have an end_time. When an end_time exists, electronic device 101 (e.g., processor 200) can proceed to operation 1006. When an end_time does not exist, electronic device 101 (e.g., processor 200) can proceed to operation 1012.

[0121] In operation 1006, electronic device 101 (e.g., processor 200) can calculate the playback time length of the first audio data block based on the size of the first audio data block.

[0122] For example, the parameter values ​​used to calculate the last playback position are as follows. Although milliseconds (ms) are used as the time unit to describe the following examples, other examples using microseconds (µs) are also available.

[0123] - Sampling rate: 48000

[0124] - Channel count (channel_count): 2 (e.g., left channel and right channel)

[0125] - Sample size in bytes (sample_size_byte): 2

[0126] - Timestamp of the first audio data block (TS): 4991ms

[0127] - Size of the first audio data block: pcm_byte: 4096 bytes

[0128] -End time: 5000ms

[0129] The electronic device 101 (e.g., processor 200) can calculate the playback time length (e.g., pcmByteToTimeMs) of the first audio data block based on the size of the first audio data block (e.g., pcm_byte), as follows:

[0130] pcmByteToTimeMs = pcm_byte×1000000 / sample_rate / channel_count / sample_size_byte

[0131] 21.333ms =4096×1000000 / 48000 / 2 / 2

[0132] As mentioned above, the playback time length corresponding to the first audio data block is 21.333ms.

[0133] In operation 1008, electronic device 101 (e.g., processor 200) can calculate the size of the audio to be skipped in the first audio data block.

[0134] In the example above, the sum of the timestamp and playback time of the first audio data block is 4991 + 21.333 = 5012.333 ms. To play up to the end_time of 5000 ms, the audio data corresponding to the last 12.333 ms (= 5012.333 ms - 5000 ms) of the first audio data block can be skipped (e.g., audio data 1104 to be skipped).

[0135] Let drop_time1 be 12.333ms (=12,333µs), and the electronic device 101 (e.g., processor 200) can calculate the size of the audio data to be skipped in the first audio data block (e.g., skipAudioSize1), as follows:

[0136] pre_skipAudioSize1 = sampling_rate × channel_count × sample_size_byte × drop_time1 / 1,000,000

[0137] skipAudioSize1= FrameSize roundup pre_skipAudioSize1

[0138] 2,367 = 48000 × 2 × 2 × 12,333 / 1000000

[0139] 2368 = FrameSize roundup 2368

[0140] Electronic device 101 (e.g., processor 200) can use the FrameSize roundup function to align pre_skipAudioSize1, calculated from drop_time1, to the frame size (e.g., FrameSize = channel_count × sample_size_byte). Since FrameSize = 2 × 2 = 4, the final size of the audio data to be skipped can be calculated as 2368 bytes, which is a multiple of FrameSize.

[0141] In operation 1010, electronic device 101 (e.g., processor 200) can determine the last playback position of the first audio data block based on the size of the audio to be skipped.

[0142] In the example above, since the size of the audio data to be skipped is 2368 bytes, the electronic device 101 (e.g., processor 200) can calculate the size of the audio data to be used as 1728 (= pcm_byte - skipAudioSize1 = 4096 - 2368). The last playback position can then be determined as 1728, indicating the byte offset from the start time of the first audio data block to the last byte of the audio data to be used. For first media data with a set end_time, the last playback position can indicate the first audio data in the last audio data block of the first media data (e.g., the first audio data block) that was not output for audio playback.

[0143] Please refer to later Figure 11 Examples of operations 1006, 1008, and 1010 are described.

[0144] In operation 1012, since no end_time is set for the first media data, the electronic device 101 (e.g., processor 200) can identify that the first audio data block is the last audio block of the first media data generated by demultiplexing and decoding, and determine the last position of the first audio data block as the last playback position.

[0145] Figure 11 This is a diagram illustrating an example of determining the last playback position of media data with a set end time according to an embodiment of the present disclosure.

[0146] refer to Figure 11 , with first media data (e.g., Figure 3 The first audio data block 1100 corresponding to the end_time of the first media data 312 can have a size of 4096 bytes and a TS of 4991ms. Electronic device 101 (e.g., Figure 2 The processor 200 can calculate the size of the audio data 1104 to be skipped, which is 2368 bytes, based on the playback time length (e.g., 21.333ms) and TS of the first audio data block 1100.

[0147] The final playback position 1110 can be calculated as 4096-2368=1728. The electronic device 101 (e.g., processor 200) can determine the portion before the final playback position 1110 as audio data 1102 to be used for audio playback, and the portion after the final playback position 1110 as audio data 1104 to be skipped for audio playback.

[0148] Figure 12 This is a diagram illustrating an example of determining the last playback position of media data for which no end time is set, according to an embodiment of the present disclosure.

[0149] refer to Figure 12 , with first media data (e.g., Figure 6 The first audio data block 1200 corresponding to the last audio data of the first media file 602 can have a size of 4096 bytes and a TS of 4991ms. Electronic device 101 (e.g., Figure 2 The processor 200 can determine 4096, which indicates the last position of the first audio data block 1200, as the last playback position 1210. For first media data without an end_time set, the last playback position 1210 can indicate the last audio data to be output for audio playback.

[0150] Figure 13This is a flowchart illustrating a process for determining target audio data for searching according to embodiments of the present disclosure. In embodiments, at least one of the operations described below can be performed by electronic device 101 (e.g., Figure 2 The processor 200 executes the operation. According to an embodiment, at least one of the operations described below may be omitted, modified, or reordered. For example, at least one of the operations described below may be executed in parallel with another operation, or may be executed regardless of the order shown. In an embodiment, Figure 13 The process can correspond to Figure 8 Operation 840.

[0151] Reference Figure 13 In operation 1302, electronic device 101 (e.g., Figure 2 The processor 200 can identify the size and start time (e.g., start_time) of the second media data. For example, it indicates the second media data identified for segmented playback (e.g., Figure 3 The start_time of the playback start time of the second media data (314) can be set to a segmentation position (e.g., 5000ms) specified by the electronic device 101 (e.g., processor 200). For example, the start_time of separately generated second media data (e.g., second media file 604) can be 0.

[0152] In operation 1304, electronic device 101 (e.g., processor 200) can determine whether start_time is greater than 0. When start_time is greater than 0, electronic device 101 (e.g., processor 200) can proceed to operation 1306. When start_time is not greater than 0 (e.g., it is 0), electronic device 101 (e.g., processor 200) can proceed to operation 1308.

[0153] In operation 1306, electronic device 101 (e.g., processor 200) may set the demultiplexing start position to an audio frame that is earlier than start_time by a specified value (e.g., X = 2). To ensure that the previous audio frame used for reference is used for decoding the audio frame corresponding to start_time, electronic device 101 (e.g., processor 200) may determine the audio frame that is X (e.g., 2 frames) earlier than the audio frame with the timestamp closest to start_time as the demultiplexing start position of the second media data.

[0154] In operation 1308, electronic device 101 (e.g., processor 200) can set the demultiplexing start position to 0. In operation 1310, electronic device 101 (e.g., processor 200) can use a demultiplexer (e.g., Figure 2The demultiplexer 220) is used at the start of demultiplexing (e.g., Figure 14 The audio frames at and following the demultiplexing start position (1406) are demultiplexed. At least one audio frame before start_time can be referenced by decoder 230 for decoding the audio frame corresponding to start_time. Since decoder 230 decodes the audio frame corresponding to start_time by referencing at least one previous audio frame, the decoding quality can be improved, thereby obtaining the same decoding result as the PCM data corresponding to end_time in the first media data.

[0155] In operation 1312, electronic device 101 (e.g., processor 200) can decode audio frames output from demultiplexer 220 via decoder 230. Decoder 230 can store the PCM data generated by decoding the audio frames in audio buffer 250.

[0156] In operation 1314, electronic device 101 (e.g., processor 200) can determine whether the size or playback time of the audio data (e.g., PCM data) output from decoder 230 and stored in audio buffer 250 is greater than a specified threshold (e.g., TH1). When the size or playback time of the decoded audio data is not greater than TH1, electronic device 101 (e.g., processor 200) can return to operation 1312 to decode the next audio frame. When the size or playback time of the decoded audio data is greater than TH1, electronic device 101 (e.g., processor 200) can proceed to operation 1316.

[0157] In operation 1316, electronic device 101 (e.g., processor 200) can identify audio data stored in audio buffer 250 with a size greater than TH1 as the second audio data block to be searched. See later. Figure 16 Examples of operations 1312, 1314 and 1316 are described.

[0158] Figure 14 This is a diagram illustrating the setting of the demultiplexing start position according to an embodiment of the present disclosure.

[0159] Reference Figure 14 The demultiplexed audio frame table 1400 can store inputs to the demultiplexer (e.g., Figure 2 The TS value of the audio frame of the demultiplexer 220. For example, when the start_time of the second media data used for demultiplexing for playback is set to 5000ms, the TS closest to the start_time is 4991ms. Electronic device 101 (e.g., Figure 2The processor 200 can determine the demultiplexing start position 1406 as an audio frame 1404 (e.g., an audio frame with a TS of 4947.33ms) that is 2 values ​​earlier than the audio frame 1402 with a TS of 4991ms that is closest to the start_time.

[0160] In an embodiment, when there is no audio frame two audio frames earlier than the audio frame corresponding to start_time (e.g., audio frame 1402), the electronic device 101 (e.g., processor 200) can determine the audio frame one audio frame earlier (e.g., an audio frame with a TS of 4968.67ms) as the demultiplexing start position. In an embodiment, when there is no audio frame one audio frame earlier than the audio frame corresponding to start_time (e.g., audio frame 1402), the electronic device 101 (e.g., processor 200) can determine the audio frame corresponding to start_time (e.g., audio frame 1402) as the demultiplexing start position.

[0161] In an embodiment, electronic device 101 (e.g., processor 200) may determine an audio frame (e.g., an audio frame with a TS of 4935ms (not shown)) corresponding to a time position that is 70ms earlier than start_time (e.g., 4930ms) as the demultiplexing start position.

[0162] Figure 15 This is a diagram illustrating a demultiplexing operation from the start position of demultiplexing according to an embodiment of the present disclosure.

[0163] refer to Figure 15 In operation 1502, the demultiplexer (e.g., Figure 2 The demultiplexer 220 can begin demultiplexing from a demultiplexing start position (e.g., timestamp 4948.33ms) determined, for example, in operation 1306 or operation 1308. In operation 1504, the demultiplexer 220 can demultiplex the second media data and output a first audio frame (e.g., audio frame 1514 with a TS of 4948.33ms) designated as the demultiplexing start position. In operation 1506, the demultiplexer 220 can output a second audio frame (e.g., audio frame 1516 with a TS of 4969.67ms) from the demultiplexing start position. In operation 1508, the demultiplexer 220 can output a third audio frame (e.g., audio frame 1518 with a TS of 4991ms) from the demultiplexing start position.

[0164] According to an embodiment, an audio frame 1518 with a TS of 4991ms corresponds to the start_time of the second media data, and the decoder (e.g., Figure 2The decoder 230 can decode the audio frame 1518 with a TS of 4991ms by referring to the audio frame 1514 with a TS of 4948.33ms and the audio frame 1516 with a TS of 4969.67ms, and output the same decoding result (e.g., PCM data) as the audio frame with a TS of 4991ms in the last audio data block (e.g., the first audio data block) of the first media data.

[0165] In an embodiment, electronic device 101 (e.g., Figure 2 The processor 200 can send at least one of encoder padding information or encoder delay information to the decoder 230, which refers to the mute audio data segment obtained by the demultiplexer 220. The decoder 230 can discard the mute audio data in the mute audio data segment based on at least one of the encoder padding information or encoder delay information, and not include it in the second audio data block.

[0166] Figure 16 This is a diagram illustrating the determination of a target audio data block for searching according to an embodiment of the present disclosure.

[0167] refer to Figure 16 Electronic device 101 (e.g., Figure 2 The processor 200 can control the demultiplexer (e.g., Figure 2 The demultiplexer 220) and decoder (e.g., Figure 2 The decoder 230 repeatedly performs demultiplexing and decoding until it is ensured that there is audio data of a size greater than or equal to a specified threshold (e.g., TH1) or playback time length. For example, the following are examples of operations (e.g., operations 1312, 1314, and 1316) for determining the second audio data block when TH1 = 19200 bytes or 100ms.

[0168] In operation 1602, decoder 230 can decode an audio frame from demultiplexer 220 with a TS of 4948.33ms (which is set to the demultiplexing start position) to generate PCM data 1612 with a size of 4096 bytes, and store it in an audio buffer (e.g., Figure 2 The audio data 1622 stored in the audio buffer 240 is 4096 bytes, as PCM data 1612 is the first decoding result of the second media data.

[0169] In operation 1604, decoder 230 can decode an audio frame with a TS of 4969.67ms from demultiplexer 220 to generate PCM data 1614 with a size of 4096 bytes, and store it in audio buffer 240. The size of the decoded audio data 1624 stored in audio buffer 240 is 8192 bytes.

[0170] In operation 1606, decoder 230 can decode an audio frame with a TS of 4991ms from demultiplexer 220 to generate PCM data 1614 with a size of 4096 bytes, and store it in audio buffer 240. The size of the decoded audio data 1624 stored in audio buffer 240 is 12288 bytes.

[0171] In operation 1608, decoder 230 can decode an audio frame with a TS of 5012.33ms from demultiplexer 220 to generate PCM data 1618 with a size of 4096 bytes, and store it in audio buffer 240. The size of the decoded audio data 1628 stored in audio buffer 240 is 16384 bytes.

[0172] In operation 1610, decoder 230 can decode an audio frame with a TS of 5033.63ms from demultiplexer 220 to generate PCM data 1620 with a size of 4096 bytes, and store it in audio buffer 240. The size of the decoded audio data 1630 stored in audio buffer 240 is 20480 bytes.

[0173] Electronic device 101 (e.g., processor 200) can recognize that the decoded audio data 1630 of the second media data stored in audio buffer 240 is larger than the size corresponding to 100ms (e.g., 19200 bytes), and determine the decoded audio data 1630 stored in audio buffer 240 as the second audio data block to be searched.

[0174] Figure 17 This is a flowchart illustrating a process for detecting audio data according to an embodiment of the present disclosure. In the embodiment, at least one of the operations described below can be performed by electronic device 101 (e.g., Figure 2 The processor 200 executes the operation. According to an embodiment, at least one of the operations described below may be omitted, modified, or reordered. For example, at least one of the operations described below may be executed in parallel with another operation, or may be executed regardless of the order shown. In an embodiment, Figure 17 The process can correspond to Figure 8 Operation 850.

[0175] Reference Figure 17 In operation 1702, electronic device 101 (e.g., Figure 2 The processor 200 can determine the target channel for the search. In an embodiment, the channel counts of the first media data and the second media data can have a value of 2 or greater, and the electronic device 101 (e.g., processor 200) can determine at least one channel (e.g., L channel and R channel) among a plurality of channels to be used as the search target.

[0176] According to an embodiment, when each audio frame of the first media data and the second media data includes two or more audio channels (e.g., a left audio channel and a right audio channel), the electronic device 101 (e.g., a processor 200) can detect the location of matching audio data in the first media data and the second media data more quickly and efficiently by identifying any one of the audio channels as the search target.

[0177] According to an embodiment, when the channel count is 1, all audio data can be the comparison target. When the channel count is greater than or equal to 2, the electronic device 101 (e.g., processor 200) can identify any one audio channel (e.g., the left channel) as the search target. In an embodiment, for more accurate audio searching, the electronic device 101 (e.g., processor 200) can identify one or more audio channels (e.g., the left and right channels) as search targets. Reference will be made later. Figure 18 An embodiment of operation 1702 is described.

[0178] In operation 1704, electronic device 101 (e.g., processor 200) can calculate the search reference data size (e.g., TH2 = FrameSize × NofFrames = 4 × 8 = 32 bytes) based on the frame size (e.g., FrameSize = 4) and a specified number of search reference frames (e.g., NofFrames = 8). Electronic device 101 (e.g., processor 200) can set the number of search reference frames according to any standard.

[0179] In operation 1706, electronic device 101 (e.g., processor 200) can determine whether the size (e.g., skipAudioSize1) of the audio data to be skipped based on the last playback position of the first audio data block is greater than or equal to the search reference data size. The size of the audio data to be skipped can be greater than or equal to the search reference data size when decoding skipped audio data not used for audio playback from multiple audio frames that are greater than or equal to the number of search reference data frames used for audio search. When the size of the audio data to be skipped is greater than or equal to the search reference data size, electronic device 101 (e.g., processor 200) can proceed to operation 1708. When the size of the audio data to be skipped is less than the search reference data size, electronic device 101 (e.g., processor 200) can proceed to operation 1710.

[0180] In operation 1708, electronic device 101 (e.g., processor 200) can determine audio data at the last playback position and subsequent audio data in the first audio data block as search reference data. In an embodiment, the search reference data may include audio data (e.g., PCM data) of the size of the search reference data from audio data in the first audio data block not used for audio playback (e.g., audio data 1104 to be skipped). In an embodiment, the search reference data may include PCM data from at least one selected channel (e.g., L channel) of PCM data from the last playback position and subsequent PCM data in the first audio data block. (See later...) Figure 19 Examples of operations 1704, 1706, and 1708 are described.

[0181] In operation 1710, electronic device 101 (e.g., processor 200) can determine audio data up to the last playback position of the first audio data block as search reference data. In an embodiment, the search reference data may include audio data (e.g., PCM data) of the size of the search reference data from the first audio data block used for audio playback (e.g., audio data 1104 to be skipped). In an embodiment, the search reference data may include PCM data from at least one selected channel (e.g., L channel) of PCM data from the last playback position and preceding the last playback position in the first audio data block. When the search reference data includes audio data for audio playback, electronic device 101 (e.g., processor 200) can determine audio data following the same audio data as the search reference data in the second media data as the playback start position.

[0182] In this embodiment, when no end_time is set for the first media data, the last playback position 1210 can be determined as the last position of the first audio data block 1200, as shown in the reference. Figure 12The electronic device 101 (e.g., processor 200) can determine the audio data preceding the last position of the first audio data block 1200 as search reference data. (See below for further details.) Figure 20 An embodiment of operation 1710 is described.

[0183] In operation 1712, electronic device 101 (e.g., processor 200) can determine whether the start_time set for the second media data is greater than 0. When start_time is greater than 0, electronic device 101 (e.g., processor 200) can proceed to operation 1714. When start_time is not greater than 0 (e.g., it is 0), electronic device 101 (e.g., processor 200) can proceed to operation 1716.

[0184] In operation 1714, electronic device 101 (e.g., processor 200) can search for a second audio data block from the data position corresponding to start_time. In operation 1716, electronic device 101 (e.g., processor 200) can search for a second audio data block from its start position. In operations 1714 and 1716, electronic device 101 (e.g., processor 200) can search for the same audio data in the second audio block as the search reference data.

[0185] In an embodiment, electronic device 101 (e.g., processor 200) can set the search start position of the second audio data block based on the TS of the first audio data block. For example, when the TS of the first audio data block is 4991ms, electronic device 101 (e.g., processor 200) can determine the position of the second audio data block corresponding to 4991ms, and search for audio data that is the same as the search reference data from the determined position.

[0186] In operation 1718, electronic device 101 (e.g., processor 200) can determine whether audio data identical to the search reference data exists in the second audio data block. When audio data identical to the search reference data exists, electronic device 101 (e.g., processor 200) can proceed to operation 1720. On the other hand, when audio data identical to the search reference data does not exist, electronic device 101 (e.g., processor 200) can proceed to operation 1722.

[0187] In operation 1720, electronic device 101 (e.g., processor 200) may determine the playback start position based on the position of the detected audio data within the second audio data block. In embodiments, the playback start position may be set to the position of the detected audio data or to the next byte of the detected audio data. In operation 1722, electronic device 101 (e.g., processor 200) may determine the start position of the second audio data block (e.g., byte 0) as the playback start position.

[0188] Please refer to later Figure 21 , Figure 22 and Figure 23 Examples of operations 1712, 1714, 1718, and 1720 are described below. Reference will be made later. Figure 24 An embodiment of operation 1722 is described.

[0189] Figure 18 This is a diagram illustrating the determination of a search target channel according to an embodiment of the present disclosure.

[0190] refer to Figure 18 This shows a specified number of 1800 (e.g., 8) search reference frames in the first audio data block. When the channel count is 2, each audio frame (e.g., audio frame 1802) defined by a specified frame size (e.g., FrameSize) may contain audio data for the L channel and audio data for the R channel.

[0191] The audio data of the L channel and the audio data of the R channel can be repeatedly arranged in each audio frame (e.g., audio frame 1802). When the sample byte size (sample_size_byte) is 2, the electronic device 101 (e.g., Figure 2 The processor 200 can calculate the frame size based on the channel count and the sample byte size as follows.

[0192] FrameSize = channel_count × sample_size_byte

[0193] Electronic device 101 (e.g., Figure 2 The processor 200 can identify the audio data (e.g., PCM data) of the L channel at the start time of the audio data and move FrameSize = 4 bytes to identify the audio data (e.g., PCM data) of the next L channel.

[0194] Figure 19 This is a diagram illustrating the determination of search reference data based on the last playback position according to an embodiment of the present disclosure.

[0195] refer to Figure 19According to an embodiment, the first audio data block 1900 corresponding to the end_time of the first media data (e.g., first media data 312) may have a size of 4096 bytes and a TS of 4991ms. According to an embodiment, relative to the last playback position 1906, the first audio data block 1900 may be divided into 1728 bytes of audio data 1902 to be used and 2368 bytes of audio data 1904 to be skipped.

[0196] Electronic device 101 (e.g., Figure 2 The processor 200 can determine a search reference data 1908 of a specified size (e.g., 32 bytes) starting from the last playback position 1906 (e.g., 1728 bytes) in the audio data 1904 to be skipped. The search reference data 1908 may include PCM data specifying the search reference data size (e.g., 32 bytes). In an embodiment, the search reference data 1908 may include PCM data from at least one selected channel (e.g., L channel) of PCM data from the last playback position in the first audio data block and thereafter.

[0197] Electronic device 101 (e.g., processor 200) may determine search reference data 1908 (e.g., 1728 bytes) starting from the last playback position 1906 based on the last playback position 1906, which indicates the location of data not used for audio playback.

[0198] Figure 20 This is a diagram illustrating the determination of search reference data based on the last position according to an embodiment of the present disclosure.

[0199] refer to Figure 20 The first audio data block 2000 may include the last audio data 2002 of the first media data without an end_time set, and all audio data 2002 of the first audio data block 2000 may be used for audio playback. Electronic device 101 (e.g., Figure 2 The processor 200 can determine audio data of a specified size (e.g., 32 bytes) preceding the last playback position 2004, which indicates the last position of the first audio data block 2000, as search reference data 2006.

[0200] Electronic device 101 (e.g., processor 200) can determine audio data (e.g., PCM data) of a specified search reference data size (e.g., 32 bytes) as search reference data 2006 from audio data 2002 used for audio playback in the first audio data block 2000. In an embodiment, search reference data 2000 may include PCM data from at least one selected channel (e.g., L channel) of PCM data from the last playback position 2004 of the first audio data block 2000 and preceding PCM data.

[0201] Figure 21 This is a diagram illustrating an audio data search according to an embodiment of the present disclosure.

[0202] refer to Figure 21 , with first media data (e.g., Figure 3 The first audio data block 2100 corresponding to the end_time of the first media data 312 in the first audio data block 2100 can have a size of 4096 bytes and a TS of 4991ms. Relative to the last playback position 2106, the first audio data block 2100 can be divided, for example, into 1728 bytes of audio data 2102 to be used and 2368 bytes of audio data 2104 to be skipped. Electronic device 101 (e.g., Figure 2 The processor 200 in the audio data 2104 can determine a search reference data 2108 of a specified size (e.g., 32 bytes) starting from the last playback position 2106 (e.g., 1728 bytes) in the audio data 2104 to be skipped.

[0203] For channel count 2, electronic device 101 (e.g., processor 200) may perform audio data search using only the L channel. Search reference data 2108 may include L channel audio data (e.g., PCM data) corresponding to a specified number of audio frames (e.g., 8 frames), and electronic device 101 (e.g., processor 200) may search a second audio data block 2110 to determine whether it includes the same audio data as search reference data 2108.

[0204] Electronic device 101 (e.g., processor 200) can set a search start position 2116 (e.g., 9920 bytes) for the first audio data block 2110, corresponding to a start_time (e.g., 5000ms) set for the second media data, and begin searching for audio data identical to the search reference data 2108 from the search start position 2116. In an embodiment, the second audio data block 2110 may include demultiplexed and decoded audio data that does not start from an audio frame (e.g., audio frame 1402) at 4991ms corresponding to the 5000ms start_time, but rather from an audio frame (e.g., audio frame 1404) at 4948.33ms. Therefore, the search start position 2116 corresponding to the start_time of the second audio data block 2110 can be set to a non-zero data position (e.g., 9920 bytes).

[0205] The start_time of the second media data is 5000ms, and 5000ms - 4948.33ms = 51.667ms (= 51667). When drop_time2 is set to 57,667 The size of the audio data to be skipped in the second audio data block 2110 (e.g., skipAudioSize2) can be calculated as follows. In the following equation, pre_skipAudioSize2 can be the input to the roundup function in integer form.

[0206] pre_skipAudioSize2 = sampling_rate × channel_count × sample_size_byte × drop_time2 / 1000000

[0207] skipAudioSize2 = FrameSize roundup pre_skipAudioSize2

[0208] 9920.064 = 48000 × 2 × 2 × 51667 / 1000000

[0209] 9920 = FrameSize roundup 9920

[0210] According to the calculation equation, electronic device 101 (e.g., processor 200) can calculate that the size of the audio data to be skipped during the audio data search in the second audio data block 2110 is 9920 bytes. This calculated size of the audio data to be skipped becomes the search start position 2116 of the second audio data block 2110.

[0211] Electronic device 101 (e.g., processor 200) can search for audio data that is the same as the reference search data 2108 from the search start position 2116. When no matching data is detected in the audio data 2114 from the search start position 2116 to the end of the second audio data block 2110, electronic device 101 (e.g., processor 200) can search for matching data again in the audio data 2112 from the start position of the second audio data block 2110 to a position before the search start position 2116 (e.g., 9916 bytes).

[0212] Following the above sequence, electronic device 101 (e.g., processor 200) can search for audio data that is the same as the L-channel audio data of search reference data 2108, starting from the search start position 2116 of the second audio data block 2110.

[0213] Figure 22 This is a diagram illustrating the determination of the playback start position based on search results according to an embodiment of the present disclosure.

[0214] refer to Figure 22, with first media data (e.g., Figure 3 The first audio data block 2200 corresponding to the end_time of the first media data 312 in the first audio data block 2200 can have a size of 4096 bytes and a TS of 4991ms. Relative to the last playback position 2206, the first audio data block 2200 can be divided, for example, into 1728 bytes of audio data 2102 to be used and 2368 bytes of audio data 2204 to be skipped. Electronic device 101 (e.g., Figure 2 The processor 200 can determine a search reference data 2208 of a specified size (e.g., 32 bytes) starting from the last playback position 2206 (e.g., 1728 bytes) in the audio data 2204 to be skipped. In an embodiment, the search reference data 2208 may include audio data corresponding to at least one selected channel (e.g., L-channel audio data).

[0215] The second audio data block 2210 to be used as a search target may have a size of, for example, 20,480 bytes and a time interval (TS) of 4,948.33 ms. Electronic device 101 (e.g., processor 200) may search the second audio data block 2210 to determine whether it includes the same audio data as search reference data 2208. In an embodiment, electronic device 101 (e.g., processor 200) may search the second audio data block 2210 using only audio data (e.g., 16 bytes) corresponding to at least one selected channel in search reference data 2208.

[0216] In an embodiment, electronic device 101 (e.g., processor 200) may detect audio data 2216a that is identical to search reference data 2208 at a position corresponding to 4968.33ms in the second audio data block 2110. The detected position may be determined as playback start position 2216. Electronic device 101 (e.g., processor 200) may determine to skip 3840 bytes of audio data 2212 prior to playback start position 2216 and perform audio playback using 16640 bytes of audio data 2214 from playback start position 2216 onwards.

[0217] In one embodiment, using the sampling rate, channel count, and sample byte size, the playback time corresponding to 3840 bytes can be calculated as 20ms (=3840×1000000 / 48000 / 2 / 2). In another embodiment, the electronic device 101 (e.g., processor 200) can change the TS of the second audio data block 2210 to 5000ms. In another embodiment, the electronic device 101 (e.g., processor 200) can change the TS of the second audio data block 2210 to 4968.33ms (=4948.33ms + 20ms). The electronic device 101 (e.g., processor 200) can output video frames synchronized with the changed TS (e.g., 4968.33ms).

[0218] Figure 23 This is a diagram illustrating the determination of the playback start position based on search results according to an embodiment of the present disclosure.

[0219] Reference Figure 23 The first audio data block 2300 may have a size of 4096 bytes and a TS of 4991 ms. The first audio data block 2300 may include the last audio data 2302 of the first media data without an end_time set, and all audio data 2302 of the first audio data block 2300 (e.g., 4096 bytes) may be used for audio playback. Electronic device 101 (e.g., Figure 2 The processor 200 can determine a specified size (e.g., 32 bytes) of audio data preceding the last playback position 2304 that indicates the end of the first audio data block 2300 as search reference data 2306.

[0220] Electronic device 101 (e.g., processor 200) can determine audio data (e.g., PCM data) of a specified search reference data size in audio data 2302 for audio playback of the first audio data block 2300 as search reference data 2306. In an embodiment, search reference data 2306 may include PCM data of at least one selected channel (e.g., L channel) in the PCM data at and before the last playback position 2304 of the first audio data block 2300.

[0221] The second audio data block 2310 to be used as a search target may have a size of, for example, 20480 bytes and a TS of 0 ms. Electronic device 101 (e.g., processor 200) may search the second audio data block 2310 to determine whether it includes the same audio data as search reference data 2306. In an embodiment, electronic device 101 (e.g., processor 200) may search the second audio data block 2310 using only audio data (e.g., 16 bytes) corresponding to at least one selected channel in search reference data 2306.

[0222] In an embodiment, electronic device 101 (e.g., processor 200) may detect audio data 2316a that is identical to search reference data 2306 at a position corresponding to 16,600 bytes of the second audio data block 2310. Playback start position 2318 may be determined as a position (e.g., 16,632 bytes) after the size of the search reference data (e.g., 32 bytes) from the detected position 2316. Electronic device 101 (e.g., processor 200) may determine to skip 16,600 bytes of audio data 2312 before playback start position 2318 and 32 bytes of search reference data 2316a after the detected position 2316, and perform audio playback using 3,848 bytes of audio data 2314 at and after playback start position 2318.

[0223] In an embodiment, electronic device 101 (e.g., processor 200) can change the TS of the second audio data block 2310 to 86.625ms. Electronic device 101 (e.g., processor 200) can output video frames synchronized with the changed TS (=86.625ms).

[0224] Figure 24 This is a diagram illustrating the operation in response to a failed audio data search according to an embodiment of the present disclosure.

[0225] Reference Figure 24 The second audio data block 2410, to be used as the search target, may have a size of 20480 bytes and a TS of 4948.33 ms. The electronic device 101 (e.g., processor 200) may fail to detect the same audio data in the second audio data block 2410 as the search reference data (not shown). In this case, the playback start position 2416 of the second audio data block 2410 may be determined to correspond to the start_time (e.g., 5000 ms) set for the second media data.

[0226] In this embodiment, since the second audio data block 2410 includes audio data corresponding to a TS that is 4948.33ms earlier than the start_time, the electronic device 101 (e.g., processor 200) can calculate the size of the audio data 2412 to be skipped in the second audio data block 2410 during audio playback. Since the start_time of the second media data is 5000ms, the time to be skipped in the second audio data block 2410 is 5000ms - 4948.33ms = 51.667ms = 51667ms. Here, the value can be calculated as follows with 57667. The size of the corresponding audio data.

[0227] AudioSize = sampling_rate × channel_count × sample_size_byte ×drop_time3 / 1,000,000

[0228] 9920.064 = 48000 × 2 × 2 × 51667 / 1000000

[0229] According to the calculation equation, the electronic device 101 (e.g., processor 200) can determine the size of the audio data to be skipped during audio playback in the second audio data block 2410 as 9920 bytes. The size of the audio data to be skipped becomes the playback start position 2416 of the second audio data block 2410 corresponding to start_time = 5000ms. The electronic device 101 (e.g., processor 200) can skip the audio data 2412 before the playback start position 2416 and continuously output the audio data 2414 from the playback start position 2416 onwards to the speaker 250 after the playback of the first media data.

[0230] In an embodiment, electronic device 101 (e.g., processor 200) may determine the TS of the video frame to be output based on the TS of the audio data output according to the playback start position, and decode the video frame of the determined TS to output it to display module 160 so that it is synchronized with the audio data.

[0231] Figure 25 This is a diagram illustrating the operation of a TS output video frame based on changes in audio data according to an embodiment of the present disclosure.

[0232] Reference Figure 25 The first audio data block 2500 may have a size of 4096 bytes and a TS of 4991 ms. The first audio data block 2500 may include the last audio data 2502 of the first media data without an end_time set, and all audio data 2502 of the first audio data block 2500 (e.g., 4096 bytes) may be used for audio playback. Electronic device 101 (e.g., Figure 2 The processor 200 can determine a specified size (e.g., 32 bytes) of audio data preceding the last playback position 2504 that indicates the end of the first audio data block 2500 as search reference data 2506.

[0233] Electronic device 101 (e.g., processor 200) can determine audio data (e.g., PCM data) of a specified search reference data size in audio data 2502 for audio playback of the first audio data block 2500 as search reference data 2506. In an embodiment, search reference data 2506 may include PCM data from at least one selected channel (e.g., L channel) from the last playback position 2504 of the first audio data block 2500 and previous PCM data.

[0234] The second audio data block 2510, to be used as a search target, may have a size of 20480 bytes and a TS of 0 ms. Electronic device 101 (e.g., processor 200) may search the second audio data block 2510 to determine whether it includes the same audio data as search reference data 2506. In an embodiment, electronic device 101 (e.g., processor 200) may search the second audio data block 2510 using only audio data (e.g., 16 bytes) corresponding to at least one selected channel in search reference data 2506.

[0235] In an embodiment, electronic device 101 (e.g., processor 200) may detect audio data 2516a that is identical to search reference data 2506 at a position corresponding to 16600 bytes of the second audio data block 2510. Playback start position 2518 may be determined as the position following the search reference data size (e.g., 16632 bytes) from the detected position 2516. Electronic device 101 (e.g., processor 200) may determine to skip 16600+32 bytes of audio data 2512 prior to playback start position 2518 and perform audio playback using 3848 bytes of audio data 2514 at and after playback start position 2518.

[0236] In this embodiment, the TS of the playback start position 2518 of the second media data can be changed to 86.625 ms. The electronic device 101 (e.g., processor 200) can use a demultiplexer (e.g., Figure 2The demultiplexer 220 demultiplexes the second media data (e.g., the second media file 604) to obtain video frames 2520, 2522, 2524, and 2526, and decodes the video frames 2520, 2522, 2524, and 2526 using a video decoder (not shown). When outputting the decoded video frames (e.g., video rendering), the electronic device 101 (e.g., the processor 200) may skip video frames 2520, 2522, and 2524 at 0ms, 33.33ms, and 66.67ms earlier than the TS (e.g., 86.625ms) of the playback start position 2518, and not output them (e.g., not perform video rendering). The electronic device 101 (e.g., the processor 200) may output the video frame 2526 at 100ms and subsequent video frames (not shown) to the display module 160 via video rendering.

[0237] When continuously playing back first media data (e.g., first media file 602) and second media data 604, which include overlapping recording periods (e.g., overlapping data 610), the electronic device 101 (e.g., processor 200) can remove the overlapping audio and video periods and output them without interruption through the above operations.

[0238] When media file 310 is divided into first media data 312 and second media data 314, which may include overlapping data, and played back continuously, the electronic device 101 and its operation method according to embodiments of the present disclosure can prevent audio interruption by detecting the accurate last playback position of the first media data 312 and determining the playback start position of the second media data 314 based on the last playback position.

[0239] When continuously playing back a first media file 602 and a second media file 604 that include overlapping recording periods, the electronic device 101 and its operation method according to embodiments of the present disclosure can prevent audio interruption by determining the last playback position of the first media file 602, using the last playback position to detect overlapping data in the second media file 604, and outputting audio after excluding overlapping data.

[0240] The electronic device 101 and its operation method according to embodiments of the present disclosure can skip overlapping data periods based on a changed audio TS and output video frames synchronized with the audio frames, providing users with a seamless playback experience of both video and audio.

[0241] An electronic device 101 according to an embodiment may include a memory 215 storing instructions, a speaker 250, and at least one processor 200 operatively connected to the memory and the speaker. When executed by the at least one processor, the instructions may cause the electronic device to obtain a first audio data block corresponding to the end time of the first media data from first media data. When executed by the at least one processor, the instructions may cause the electronic device to identify the last playback position of the first audio data block based on the end time. When executed by the at least one processor, the instructions may cause the electronic device to obtain a second audio data block to be used as a search target from the second media data based on the start time of the second media data. When executed by the at least one processor, the instructions may cause the electronic device to search for audio data in the second audio data block corresponding to the last playback position. When executed by the at least one processor, the instructions may cause the electronic device to determine the playback start position of the second audio data block based on detected audio data. When executed by the at least one processor, the instructions may cause the electronic device to output audio data starting from the playback start position of the second media data to the speaker after completing audio playback of the first media data up to the last playback position of the first audio data block.

[0242] In one embodiment, the first audio data block may include first PCM (Pulse Code Modulation) data obtained by decoding one or more audio frames of the first media data. In another embodiment, the second audio data block may include second PCM data obtained by decoding one or more audio frames of the second media data.

[0243] In an embodiment, the instructions may cause the electronic device to: generate decoded audio data by decoding audio frames preceding and following a specified number of audio frames starting from an audio frame corresponding to the start time in the second media data; and obtain a second audio data block of a specified size including the decoded audio data.

[0244] In an embodiment, the instructions may cause the electronic device to: determine search reference data, including audio data of a specified size, in the first audio data block based on the last playback position; search for audio data identical to the search reference data in the second audio data block; and determine the playback start position based on the position of the detected audio data identical to the search reference data.

[0245] In an embodiment, the instructions may cause the electronic device to: select at least one search target channel based on the channel count of the first media data; and compare the audio data of the selected search target channel in the search reference data with the second audio data block.

[0246] In an embodiment, the instructions may cause the electronic device to: calculate the size of audio data to be skipped without audio playback in the first audio data block based on the end time of the first media data; determine whether the size of the audio data to be skipped is greater than the size of the search reference data; if the size of the audio data to be skipped is greater than the size of the search reference data, determine that the search reference data includes audio data after the last playback position; and if the size of the audio data to be skipped is not greater than the size of the search reference data, determine that the search reference data includes audio data before the last playback position.

[0247] In an embodiment, the instruction may cause the electronic device to change the timestamp of the second audio data block based on the playback start position.

[0248] In an embodiment, the last playback position can be determined based on at least one of the size of the first audio data block, the sampling rate, the channel count, the sample byte size, or the timestamp indicating the start time of playback.

[0249] In an embodiment, the instructions may cause the electronic device to: output audio data to the speaker before and at the last playback position in the first audio data block; and skip audio data after the last playback position in the first audio data block.

[0250] In an embodiment, the instructions may cause the electronic device to: skip audio data in the second audio data block prior to the playback start position; and output audio data in the second audio data block at and after the playback start position to the speaker.

[0251] A method for operating an electronic device 101 according to an embodiment may include: obtaining (810) a first audio data block corresponding to an end time of the first media data from first media data. The method may include: identifying (820) a last playback position of the first audio data block based on the end time. The method may include: obtaining (840) a second audio data block to be used as a search target from the second media data based on a start time of the second media data. The method may include: searching (850) for audio data corresponding to the last playback position in the second audio data block. The method may include: determining (860) a playback start position of the second audio data block based on detected audio data. The method may include: after completing audio playback of the first media data up to the last playback position of the first audio data block, outputting (870) audio data starting from the playback start position of the second media data to the speaker.

[0252] In one embodiment, the first audio data block may include first PCM (Pulse Code Modulation) data obtained by decoding one or more audio frames of the first media data. In another embodiment, the second audio data block may include second PCM data obtained by decoding one or more audio frames of the second media data.

[0253] In an embodiment, obtaining the second audio block may include: generating decoded audio data by decoding audio frames preceding and following a specified number of audio frames starting from the audio frame corresponding to the start time in the second media data; and obtaining a second audio data block of a specified size including the decoded audio data.

[0254] In one embodiment, searching for audio data corresponding to the last playback position in the second audio data block may include: determining search reference data in the first audio data block, which includes audio data of a specified size, based on the last playback position; and searching for audio data in the second audio data block that is the same as the search reference data.

[0255] In one embodiment, searching for audio data corresponding to the last playback position in the second audio data block may include: selecting at least one target search channel based on the channel count of the first media data; and comparing the audio data of the selected target search channel in the search reference data with the second audio data block.

[0256] In an embodiment, determining the search reference data may include: calculating the size of audio data to be skipped without playback in the first audio data block based on the end time of the first media data; determining whether the size of the audio data to be skipped is greater than the size of the search reference data; if the size of the audio data to be skipped is greater than the size of the search reference data, determining that the search reference data includes audio data after the last playback position; and if the size of the audio data to be skipped is not greater than the size of the search reference data, determining that the search reference data includes audio data before the last playback position.

[0257] In an embodiment, the method may further include: changing the timestamp of the second audio data block based on the playback start position.

[0258] In an embodiment, the last playback position can be determined based on at least one of the size of the first audio data block, the sampling rate, the channel count, the sample byte size, or the timestamp indicating the start time of playback.

[0259] In an embodiment, the method may further include: outputting audio data to the speaker before and at the last playback position in the first audio data block; and skipping audio data after the last playback position in the first audio data block.

[0260] In an embodiment, outputting audio data starting from the playback start position of the second media data may include: skipping audio data in the second audio data block before the playback start position; and outputting audio data in the second audio data block at and after the playback start position to the speaker.

[0261] The electronic device according to various embodiments of this disclosure can be one of a variety of types of electronic devices. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. According to embodiments of this disclosure, the electronic device is not limited to those described above.

[0262] It should be understood that the various embodiments of this disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the specific embodiments, but rather to include various changes, equivalents, or substitutions to the respective embodiments. In the description of the drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that nouns in the singular form corresponding to terms may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include any one or all possible combinations of the items enumerated together with the corresponding phrase among the plurality of phrases. As used herein, terms such as “first” and “second” or “first” and “second” may be used to simply distinguish the respective component from another component and do not limit the components in other respects (e.g., importance or order). It will be understood that, whether the terms “operably” or “communically” are used or not, if an element (e.g., a first element) is referred to as “combined with another element (e.g., a second element),” “combined to another element (e.g., a second element),” “connected to another element (e.g., a second element),” or “connected to another element (e.g., a second element)”, it means that the element can be directly (e.g., wiredly) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0263] As used in connection with this disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as "logic," "logic block," "part," or "circuit." A module may be a single integrated component adapted to perform one or more functions, or the smallest unit or part of such a single integrated component. For example, according to embodiments, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0264] The embodiments described herein can be implemented as software (e.g., program 140) including one or more instructions readable by a machine (e.g., electronic device 101) stored in a storage medium (e.g., internal memory 136 or external memory 138). For example, under the control of a processor, the processor (e.g., processor 120) of the machine (e.g., electronic device 101) can invoke and execute at least one of the one or more instructions stored in the storage medium, with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code that can be run by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. The term "non-transitory" means only that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between data being stored semi-permanently in the storage medium and data being temporarily stored in the storage medium.

[0265] According to embodiments, methods according to various embodiments of this disclosure may be included and provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an app store (e.g., the Play Store™), or may be distributed directly between two user devices (e.g., smartphones) (e.g., downloaded or uploaded). If distributed online, at least a portion of the computer program product may be temporarily generated, or at least a portion of the computer program product may be stored at least temporarily in a machine-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a forwarding server).

[0266] According to various embodiments, each of the above-described components (e.g., a module or program) may include a single entity or multiple entities, and some of the multiple entities may be separately disposed in different components. According to various embodiments, one or more of the above-described components may be omitted, or one or more other components may be added. Optionally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. According to various embodiments, the integrated component may still perform the one or more functions of each of the multiple components in the same or similar manner as a corresponding component of the multiple components performed one or more functions prior to integration. According to various embodiments, the operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be run in a different order or omitted, or one or more other operations may be added.

Claims

1. An electronic device (101), comprising: Memory for storing instructions (215); Speaker (250); as well as At least one processor (200) is operatively connected to the memory and the speaker. Wherein, the instructions, when executed by the at least one processor, cause the electronic device to: Obtain the first audio data block corresponding to the end time of the first media data from the first media data; The last playback position of the first audio data block is identified based on the end time; Based on the start time of the second media data, obtain a second audio data block corresponding to the end time from the second media data to be used as a search target; Search for audio data in the second audio data block that corresponds to the last playback position; The playback start position of the second audio data block is determined based on the detected audio data; and After the audio playback of the first media data up to the last playback position of the first audio data block is completed, audio data starting from the playback start position of the second media data is output to the speaker.

2. The electronic device according to claim 1, wherein, The first audio data block includes first PCM (Pulse Code Modulation) data obtained by decoding one or more audio frames of the first media data, and The second audio data block includes second PCM data obtained by decoding one or more audio frames of the second media data.

3. The electronic device according to claim 1 or 2, wherein, The instruction causes the electronic device to: Decoded audio data is generated by decoding the audio frames preceding and following a specified number of audio frames starting from the audio frame corresponding to the start time in the second media data. as well as Obtain a second audio data block of a specified size that includes the decoded audio data.

4. The electronic device according to any one of claims 1 to 3, wherein, The instruction causes the electronic device to: Based on the last playback position, search reference data including audio data of a specified size is determined in the first audio data block; Search for audio data in the second audio data block that is identical to the search reference data; as well as The playback start position is determined based on the position of the detected audio data that is the same as the search reference data.

5. The electronic device according to claim 4, wherein, The instruction causes the electronic device to: Select at least one target search channel based on the channel count of the first media data; as well as The audio data of the selected search target channel in the search reference data is compared with the second audio data block.

6. The electronic device according to claim 4 or 5, wherein, The instruction causes the electronic device to: Based on the end time of the first media data, calculate the size of the audio data to be skipped without audio playback in the first audio data block; Determine whether the size of the audio data to be skipped is greater than the size of the search reference data; If the size of the audio data to be skipped is greater than the size of the search reference data, it is determined that the search reference data includes audio data after the last playback position; as well as If the size of the audio data to be skipped is not greater than the size of the search reference data, the search reference data is determined to include audio data prior to the last playback position.

7. The electronic device according to any one of claims 1 to 6, wherein, The instruction causes the electronic device to change the timestamp of the second audio data block based on the playback start position.

8. The electronic device according to any one of claims 1 to 7, wherein, The final playback position is determined based on at least one of the size of the first audio data block, the sampling rate, the channel count, the sample byte size, or the timestamp indicating the start time of playback.

9. The electronic device according to any one of claims 1 to 8, wherein, The instruction causes the electronic device to: The speaker outputs audio data before and at the last playback position in the first audio data block; as well as Skip the audio data after the last playback position in the first audio data block.

10. The electronic device according to any one of claims 1 to 9, wherein, The instruction causes the electronic device to: Skip the audio data in the second audio data block prior to the playback start position; as well as The audio data at the playback start position and after the playback start position in the second audio data block are output to the speaker.

11. A method for operating an electronic device (101), comprising: Obtain (810) a first audio data block corresponding to the end time of the first media data from the first media data; The last playback position of the first audio data block is identified (820) based on the end time; Based on the start time of the second media data, obtain (840) a second audio data block corresponding to the end time from the second media data to be used as a search target; Search (850) for audio data corresponding to the last playback position in the second audio data block; The playback start position of the second audio data block is determined (860) based on the detected audio data; as well as After the audio playback of the first media data is completed based on the last playback position, the audio data of the second media data after the playback start position is output to the speaker (870).

12. The method according to claim 11, wherein, The first audio data block includes first PCM (Pulse Code Modulation) data obtained by decoding one or more audio frames of the first media data, and The second audio data block includes second PCM data obtained by decoding one or more audio frames of the second media data.

13. The method according to claim 11 or 12, wherein, Obtaining the second audio block includes: Decoded audio data is generated by decoding audio frames preceding a specified number of audio frames starting from the audio frame corresponding to the start time, and one or more audio frames following the start time in the second media data; and Obtain a second audio data block of a specified size that includes the decoded audio data.

14. The method according to any one of claims 11 to 13, wherein, Searching for audio data corresponding to the last playback position in the second audio data block includes: Based on the last playback position, search reference data including audio data of a specified size is determined in the first audio data block; and Search for audio data in the second audio data block that is the same as the search reference data.

15. The method according to claim 14, wherein, Searching for audio data corresponding to the last playback position in the second audio data block includes: Select at least one target search channel based on the channel count of the first media data; and The audio data of the selected search target channel in the search reference data is compared with the second audio data block.

Images