Electronic device and display system

Abstract

This electronic device comprises: a communication interface; a display; at least one processor; and a memory comprising at least one storage medium for storing instructions. The instructions, when executed individually or collectively by the at least one processor, cause the electronic device to: obtain, via the communication interface, a series of encoded source frames, a respective source frame duration for each source frame, and first reference point timing information for a first reference point defined on a first source frame among the series of source frames; generate playback frames for playback via the display by decoding the source frames; determine a playback delay value on the basis of the first reference point timing information and the respective source frame durations of source frames corresponding to playback frames that have been played back prior to the acquisition of the first reference point timing information; and perform delay control on the basis of the playback delay value.

Description

Electronic devices and display systems

[0001] The present disclosure relates to a method for controlling playback delay in multimedia data streaming and an electronic device providing the same.

[0002] In widely used low-latency streaming technology, multimedia data transmitted from a transmitter over a network is received and played simultaneously on a display device. The transmitter can encode original multimedia data (initial multimedia data or first multimedia data) to create a stream of source frames for transmission and transmit the encoded multimedia data to a display device.

[0003] A display device can receive and decode a stream of source frames and play a series of playback frames from the decoded stream of source frames. To maintain a high level of playback quality, the playback speed of the playback frames in the display device must match the speed of the original multimedia data in the transmitter, and it is desirable that the delay of the playback frames relative to the source frames be maintained at a target level. However, due to various causes such as jitter between the transmitter's source clock and the display device's playback clock, variations in transmission delay time due to changes in network conditions, and variations in the processor and system processing speeds of the transmitter and display device, respectively, the delay state of the playback frames generated by the display device can fluctuate significantly over time.

[0004] In order to prevent degradation of the playback quality of multimedia data, it may be necessary to continuously monitor the delay state, that is, the variation in the amount of delay, present in the playback frames on the display device, and to take appropriate control measures so that the amount of delay can be maintained at a target level according to the monitoring results. In particular, a method may be required to finely monitor and control the variation in the amount of playback delay when source frames of a frame rate determined variably according to the Variable Refresh Rate (VRR) are received and played, and / or when source frames of variable length are received and played according to the Variable Bit Rate (VBR) method.

[0005] The present disclosure provides a playback delay control method and an apparatus supporting such a control method, wherein, in relation to the streaming of encoded multimedia data source frames, a receiver can monitor the playback delay amount in fine time units at the subframe level and perform an operation to maintain the playback delay amount at a target level in a timely manner.

[0006] According to one aspect of the present disclosure, an electronic device comprises: a communication interface; a display; at least one processor; and a memory comprising at least one storage medium for storing instructions. When the instructions are executed individually or collectively by the at least one processor, the electronic device is configured to: obtain, through the communication interface, a series of encoded source frames, a source frame time length for each source frame, and first reference point timing information for a first reference point defined on a first source frame among the series of source frames; generate playback frames for playback through the display by decoding the source frames; determine a playback delay value based on the first reference point timing information and the source frame time lengths of source frames corresponding to playback frames completed prior to the acquisition of the first reference point timing information; and perform delay control based on the playback delay value.

[0007] According to one aspect of the present disclosure, a display system comprises: an encoder configured to encode image information into a series of source frames; a time length generating circuit configured to generate source frame time length information for each of the source frames; a reference point timing information generating circuit configured to generate first reference point timing information corresponding to a first reference point determined on a first source frame among the series of source frames; and a video transmission device comprising a first communication interface configured to transmit stream data including the series of source frames, source frame time length information for each of the source frames, and the first reference point timing information, and a second communication interface configured to receive the stream data; a decoder configured to generate a series of playback frames from the source frames of the stream data; a display configured to play the series of playback frames; and a delay amount determining circuit configured to determine a playback delay value based on the first reference point timing information and the time length of each source frame of the source frames corresponding to the playback frames that have been played by the display prior to the acquisition of the first reference point timing information. and includes a video receiving device comprising a delay control circuit configured to perform delay control based on the above playback delay value.

[0008] The effects obtainable from the exemplary embodiments of the present disclosure are not limited to those mentioned above, and other unmentioned effects can be clearly derived and understood by those skilled in the art to which the exemplary embodiments of the present disclosure belong from the description below. That is, unintended effects resulting from the implementation of the exemplary embodiments of the present disclosure can also be derived by those skilled in the art from the exemplary embodiments of the present disclosure.

[0009] The aforementioned aspects and other aspects, features, and advantages of the embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings.

[0010] FIG. 1 illustrates the configuration of a multimedia data transmitter according to one embodiment of the present disclosure.

[0011] FIG. 2 illustrates the configuration of a display device according to one embodiment of the present disclosure.

[0012] FIG. 3 illustrates the function of a transmitter controller according to one embodiment of the present disclosure.

[0013] FIG. 4 is a flowchart of data processing for transmission of a transmitter according to one embodiment of the present disclosure.

[0014] FIG. 5 illustrates the function of a receiver controller according to one embodiment of the present disclosure.

[0015] FIG. 6 shows an exemplary case of calculating the delay amount of playback frames for source frames, performed at a receiver according to one embodiment of the present disclosure.

[0016] FIG. 7 shows an exemplary case of calculating the delay amount of playback frames for source frames, performed at a receiver according to one embodiment of the present disclosure.

[0017] FIG. 8 shows an exemplary case of calculating the delay amount of playback frames for source frames, performed at a receiver according to one embodiment of the present disclosure.

[0018] FIG. 9 shows an exemplary case of calculating the delay amount of playback frames for source frames, performed at a receiver according to one embodiment of the present disclosure.

[0019] FIGS. 10a and FIGS. 10b are flowcharts of data processing of a receiver according to one embodiment of the present disclosure.

[0020] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0021] FIG. 1 illustrates the configuration of a multimedia data transmitter according to one embodiment of the present disclosure.

[0022] According to one embodiment, the transmitter (100) may include an original multimedia supply unit (110), a first controller (120), a first memory (130), and a first communication interface (140). The transmitter (100) may include additional components in addition to the illustrated components, or at least one of the illustrated components may be omitted.

[0023] According to one embodiment, the original multimedia supply unit (110) can receive various multimedia content via wired or wireless methods from other devices or external servers. According to one embodiment, the original multimedia supply unit (110) can receive television broadcast content transmitted via wired or wireless methods, video content received from a video service server, game content received from a game service server or game device, and / or video content received from various other devices. According to one embodiment, the original multimedia supply unit (110) can process the received multimedia content to generate an original multimedia data stream (initial multimedia data or first multimedia data) and supply the generated original multimedia data stream to the first controller (120).

[0024] According to one embodiment, the first controller (120) may be configured to perform overall control operations of the transmitter (100). The first controller (120) may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes digital signals. However, it is not limited thereto, and may include or be defined by one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics processing unit (GPU), a communication processor (CP), or an ARM processor. Additionally, the first controller (120) may be implemented as a system on chip (SoC) or large scale integration (LSI) with a built-in processing algorithm, or may be implemented in the form of a field programmable gate array (FPGA). Furthermore, the first controller (120) can perform various functions by executing computer executable instructions stored in the first memory (130). The first controller (120) is the original multimedia It can be electrically connected to the supply unit (110), the first memory (130), and the first communication interface (140).

[0025] According to one embodiment, the first memory (130) may be implemented as an internal memory such as ROM (e.g., EEPROM (electrically erasable programmable read-only memory)) or RAM included in the first controller (120), or as a memory separate from the first controller (120). In this case, the first memory (130) may be implemented as a memory within the transmitter (100) or as a memory that can be attached to and detached from the transmitter (100), depending on the purpose of data storage. For example, data for driving the transmitter (100) may be stored in the memory within the transmitter (100), and data for the expansion function of the transmitter (100) may be stored in the memory that can be attached to and detached from the transmitter (100).

[0026] In the case of memory within the transmitter (100), it may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one-time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash, etc.), hard drive, or solid state drive (SSD). In the case of memory that is detachable from the transmitter (100), it may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (e.g., USB memory).

[0027] According to one embodiment, the first controller (120) can receive an original multimedia data stream from the original multimedia supply unit (110). The first controller (120) can encode the received original multimedia data stream to generate a series of source frames for transmission. The first controller (120) can generate a series of source frames encoded to have a delay and a number of bits determined according to encoding options based on the characteristics of the original multimedia data, network environment and / or device specifications, etc. The first controller (120) can generate source clock information for each source frame based on the source clock of the transmitter (100). The first controller (120) can transmit the generated source frames to the first communication interface (140) along with the corresponding source clock information.

[0028] According to one embodiment, the first communication interface (140) can transmit and receive various data, including multimedia data and various control data, between itself and the display device (200). According to one embodiment, the first communication interface (140) can transmit source frames and corresponding source clock information to the outside. According to one embodiment, the first communication interface (140) can also receive information coming from the outside and transmit the received information to the first controller (120). The first communication interface (140) can be implemented as at least one wired communication circuit or wireless communication circuit, and each communication circuit can support a predetermined bandwidth. In some embodiments, the first communication interface (140) can transmit and receive data with the display device (200) using various protocols.

[0029] FIG. 2 illustrates the configuration of a display device according to one embodiment of the present disclosure.

[0030] According to one embodiment, the display device (200) may include a second controller (210), a second memory (220), a second communication interface (230), and a display (240). The display device (200) may include additional components in addition to the illustrated components, or at least one of the illustrated components may be omitted.

[0031] According to one embodiment, the second communication interface (230) can receive source frames and source clock information transmitted from the transmitter (100). According to one embodiment, the second communication interface (230) can transmit feedback control data generated in the display device (200) to the transmitter (100). The second communication interface (230) may be implemented as at least one wired communication circuit or wireless communication circuit, and each communication circuit may support a predetermined bandwidth. In some embodiments, the second communication interface (230) may transmit and receive data with the transmitter (100) using various protocols.

[0032] According to one embodiment, the second controller (210) can perform overall control operations of the display device (200). The second controller (210) may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes digital signals. However, it is not limited thereto, and may include or be defined by one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics processing unit (GPU), a communication processor (CP), or an ARM processor. Additionally, the second controller (210) may be implemented as a system on chip (SoC) or large scale integration (LSI) with a built-in processing algorithm, or may be implemented in the form of a field programmable gate array (FPGA). Furthermore, the second controller (210) can perform various functions by executing computer executable instructions stored in the second memory (220). The second controller (210) is the second memory (220), It can be electrically connected to the second communication interface (230) and the display (240).

[0033] According to one embodiment, the second memory (220) may be implemented as an internal memory such as ROM (e.g., EEPROM (electrically erasable programmable read-only memory)) or RAM included in the second controller (210), or as a memory separate from the second controller (210). In this case, the second memory (220) may be implemented as a memory within the display device (200) or as a memory that can be attached to and detached from the display device (200), depending on the purpose of data storage. For example, data for driving the display device (200) may be stored in the memory of the display device (200), and data for the expansion function of the display device (200) may be stored in a memory that can be attached to and detached from the display device (200).

[0034] The memory of the display device (200) may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one-time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD). In the case of memory that is detachable from the display device (200), it may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (e.g., USB memory).

[0035] According to one embodiment, the second controller (210) can acquire a series of source frames transmitted from the transmitter (100) and corresponding source clock information through the second communication interface (230). The second controller (210) can decode the received source frames to generate a series of playback frames. The second controller (210) can monitor the delay state of the playback frames for the source frames, for example, the amount of delay in the decoding and / or playback processing for the source frames. The second controller (210) can monitor the delay state of the decoding and / or playback processing currently taking place within the display device (200) based on the source clock information regarding the source frames acquired through the second communication interface (230). The second controller (210) can perform a necessary delay control operation according to the result of monitoring the delay state.

[0036] According to one embodiment, the display (240) may be configured to display images corresponding to playback frames generated by the second controller (210). The display (240) may include at least one display panel of various types, such as a liquid crystal display panel (LCD), a light-emitting diode panel (LED), an organic light-emitting diode panel (OLED), and a plasma display panel (PDP), and may include at least one panel driving unit for driving the display panel.

[0037] FIG. 3 illustrates the function of a transmitter controller according to one embodiment of the present disclosure. According to one embodiment, the transmitter controller (300) may be implemented as at least part of the first controller (120) of the transmitter (100) of FIG. 1.

[0038] According to one embodiment, the transmitter controller (300) can receive an original multimedia data stream. According to one embodiment, the received original multimedia data stream may include video data. According to one embodiment, the received original multimedia data stream may have a predetermined original speed (e.g., the time interval between each frame input of the original multimedia data stream). According to one embodiment, the transmitter controller (300) can acquire source clock information (SC). According to one embodiment, the transmitter controller (300) can perform various operations described below based on the acquired source clock information (SC).

[0039] According to one embodiment, the transmitter controller (300) may include an encoder (310). According to one embodiment, the encoder (310) may receive an original multimedia data stream. According to one embodiment, the encoder (310) may encode the original multimedia data stream into a series of source frames at the original speed of the original multimedia data stream. According to one embodiment, the encoder (310) may encode the original multimedia data stream into a series of source frames based on various information including characteristics of the original multimedia data stream, network conditions, and / or specifications of related devices.

[0040] According to one embodiment, the transmitter controller (300) may include a source frame time length generating unit (source frame time length generating circuit) (320). According to one embodiment, the source frame time length generating unit (320) may generate information indicating the source frame time length corresponding to each source frame generated by the encoder (310) described above, such as the total number of source clock cycles (v_total_cycle_s) allocated to each source frame. According to one embodiment, the source frame time length (v_total_cycle_s) of a source frame may be the total number of source clock cycles that occurred during the interval from the input of the source frame to the input of the next source frame.

[0041] According to one embodiment, the transmitter controller (300) may include a reference point timing information generation unit (reference point timing information generation circuit) (330). According to one embodiment, the reference point timing information generation unit (330) may determine one or more time reference points to be placed in the middle of source frames generated by the aforementioned encoder (310) according to a predetermined standard. According to one embodiment, each time reference point placed in the middle of a series of source frames may be any point in time in the middle of the duration of any one source frame. According to one embodiment, the reference point timing information generation unit (330) may generate each reference point timing information (enc_cycle) corresponding to each of the one or more time reference points placed in the middle of the source frames.

[0042] According to one embodiment, reference point timing information for a given time reference point may represent the total accumulated source clock cycles from the beginning of a series of source frames to a given time reference point, for example, the reference point timing. According to one embodiment, reference point timing information ("enc_cycle") for a given time reference point may represent the accumulated source clock cycles from the time reference point immediately preceding the given time reference point (if the given reference point is the first reference point of the source frame stream, the immediately preceding reference point may be the beginning of the source frame stream) to the given time reference point. According to one embodiment, reference point timing information for a given time reference point may represent the number of source clock cycles from the beginning of the source frame to which the given reference point belongs to the given reference point to the given reference point. According to various embodiments, reference point timing information may be determined in various ways to specify the temporal position of the given reference point within the source frames based on various factors such as the characteristics of the source frames, the device characteristics of the transceiver, and the characteristics of the network conditions.

[0043] According to one embodiment, the transmitter controller (300) may include a transmission stream buffer (340). According to one embodiment, each source frame generated by the aforementioned encoder (310), the source frame time length (hereinafter "v_total_cycle_s") assigned to each source frame generated by the aforementioned source frame time length generator (320), and timing information (enc_cycle) regarding each time reference point generated by the aforementioned reference point timing information generator (330) may be stored in the transmission stream buffer (340). According to one embodiment, the source frames and various time information stored in the transmission stream buffer (340) may be appropriately integrated and processed in accordance with the network protocol, and then transmitted to the first communication interface (140) of FIG. 1 to be transmitted externally. According to one embodiment, the transmission stream buffer (340) may process input data in a first-in, first-out manner and transmit the processed input data to the first communication interface (140), but the embodiments of the present disclosure are not limited thereto.

[0044] FIG. 4 is a flowchart of transmission data processing of a transmitter according to one embodiment of the present disclosure. According to one embodiment, the transmission data processing disclosed in FIG. 4 may be at least partially implemented by the first controller (120) of the transmitter (100) of FIG. 1 and / or the transmitter controller (300) of FIG. 3.

[0045] According to one embodiment, in operation (402), an original multimedia data stream (i.e., an initial multimedia data stream or a first multimedia data stream) may be obtained. In one embodiment, the original multimedia data may include a stream of data frames generated from various video content received from outside the electronic device. In one example, the original multimedia data stream may have an original rate (e.g., the input rate of each data frame of the original multimedia data stream).

[0046] In operation (404), whether there is a frame skip request can be determined, for example, by at least one processor of an electronic device. According to one embodiment, the frame skip request may be information transmitted as feedback information from a display device (200) for playback delay control as described below. According to one embodiment, the frame skip request may be information obtained through the first communication interface (140) of FIG. 1.

[0047] In operation (404), if it is determined that there is a frame skip request, for example by at least one processor of an electronic device, the procedure proceeds to operation (406) to perform processing according to the frame skip request, for example, selecting frames to skip and skip processing. According to one embodiment, when there is a frame skip request, the transmitter analyzes the characteristics of multimedia data to determine a skippable section, and accordingly extracts and skips the frames to skip. In one embodiment, the transmitter analyzes the characteristics of multimedia data to determine, for example, scene transition sections, MUTE sections, sections with excessively low image quality, etc., as skippable sections, and the embodiments of the present disclosure are not limited thereto. In various embodiments, the determination of whether there is a frame skip request and the processing according to the skip request may be performed at any various point in time during the operation of the transmitter.

[0048] In operation (408), encoding of the original multimedia data stream may be performed. According to one embodiment, the original multimedia data stream may be encoded into a series of source frames at the original speed of the original multimedia data stream. According to one embodiment, the original multimedia data stream may be encoded into a series of source frames based on various information including the characteristics of the original multimedia data stream, network conditions, and / or specifications of related devices. According to one embodiment, various video compression algorithms, such as MPEG (Moving Picture Experts Group)-4 and HEVC (High Efficiency Video Coding), may be used for encoding the original multimedia data stream.

[0049] In operation (410), it may be determined whether the creation of one source frame is completed, for example, by at least one processor of an electronic device. In operation (410), if it is determined that the creation of one source frame is completed, for example by at least one processor of an electronic device, the procedure may proceed to operation (412). In operation (412), a source frame time length for the created source frame may be generated. In one embodiment, for example, the total number of source clock cycles (v_total_cycle_s) allocated to the created source frame may be measured and provided as the source frame time length of the corresponding frame.

[0050] In operation (414), for example, at least one processor of an electronic device may determine whether it is time for a time reference point to be inserted. Whether it is time for a time reference point to be inserted may be determined according to a predetermined criterion. According to one embodiment, the time reference point may be at any point in the middle of the duration of any source frame among a series of source frames, that is, at any point in the middle of the interval between the source frame and the next source frame. According to one embodiment, the time reference point may be inserted periodically or non-periodically according to a predetermined criterion to check the delay state occurring when decoding and playing the source frames. According to one embodiment, there may be one or more time reference points within each source frame. According to one embodiment, there may be one time reference point for every few source frames. According to one embodiment, the criteria for inserting the time reference point may be dynamically determined adaptively according to various situations, such as frame rate, delay management policy, network conditions, original multimedia characteristics, etc., and the present disclosure is not limited to specific embodiments.

[0051] In operation (414), if it is determined that a time reference point needs to be inserted, for example by at least one processor of an electronic device, the procedure proceeds to operation (416) to determine reference point timing information (enc_cycle) corresponding to the time reference point. In one embodiment, the reference point timing information (enc_cycle) corresponding to the time reference point may represent the number of accumulated source clock cycles from the time reference point immediately preceding the time reference point to the time reference point. In one embodiment, the reference point timing information of any one time reference point on a series of source frames may represent the total number of accumulated source clock cycles from the start of the series of source frames to the given time reference point.

[0052] According to one embodiment, reference point timing information (enc_cycle) for any one time reference point may represent the cumulative number of source clock cycles from the time reference point immediately preceding that time reference point to that time reference point. As described above, reference point timing information for any one time reference point may be determined in any manner determined according to various considerations. In some embodiments, the determination of whether insertion of a time reference point is necessary and the determination of reference point timing information may be performed at any various point in time during the operation of the transmitter.

[0053] In operation (418), the source frames generated earlier and control data regarding the source frames, such as source frame time length corresponding to each source frame and reference point timing information (enc_cycle) corresponding to each time reference point, can be integrated and processed based on (according to) a predetermined protocol. According to one embodiment, the integrated and processed final information can be transmitted externally.

[0054] FIG. 5 illustrates the function of a receiver controller according to one embodiment of the present disclosure. According to one embodiment, the receiver controller (500) may be implemented as at least part of a second controller (210) of the display device (200) of FIG. 1.

[0055] According to one embodiment, the receiver controller (500) may be configured to receive data transmitted from a transmitter via a network. According to one embodiment, the receiver controller (500) may receive a stream of source frames and various control data. According to one embodiment, the receiver controller (500) may receive the source frame time length of each source frame transmitted from the transmitter and reference point timing information of each time reference point. According to one embodiment, the receiver controller (500) may obtain replay clock information (RC). According to one embodiment, the receiver controller (500) may perform various operations described below based on the obtained replay clock information (RC).

[0056] According to one embodiment, the receiver controller (500) may include a receiving stream buffer (510). According to one embodiment, the stream of source frames and control data (e.g., source frame time length and reference point timing information for each time reference point) acquired by the receiver controller (500) may be acquired and stored by the receiving stream buffer (510).

[0057] According to one embodiment, the receiver controller (500) may include a decoder (520). According to one embodiment, the decoder (520) may obtain source frames and control data (e.g., source frame time length and reference point timing information for each time reference point) from the receiving stream buffer (510). According to one embodiment, the decoder (520) may decode the obtained source frames to generate playback frames to be displayed through a display.

[0058] According to one embodiment, the decoder (520) can generate a playback frame based on the source frame time length (v_total_cycle_s) corresponding to each source frame. According to one embodiment, a playback frame generated by a decoder (520) may have a playback time length (v_total_cycle_d) determined according to the source frame time length ((v_total_cycle_s)) of a corresponding source frame. According to one embodiment, the playback time length (v_total_cycle_d) may be the total number of playback clock cycles representing the output interval of playback frames generated by the decoder (520). According to one embodiment, a playback frame generated by the decoder (520) may have a playback time length (v_total_cycle_d) equal to the total number of source clock cycles of a corresponding source frame. According to one embodiment, a playback frame generated by the decoder (520) may have a playback time length (v_total_cycle_d) equal to the total number of source clock cycles of a corresponding source frame. According to one embodiment, a playback frame generated by the decoder (520) may have a playback time length (v_total_cycle_d) equal to the total number of source clock cycles of a corresponding source frame, and the embodiments of the present disclosure are not limited to specific examples. According to one embodiment, The decoder (520) can generate final playback frame duration information (dec_cycle) indicating the number of playback clock cycles elapsed from the start of playback of the most recently decoded playback frame until the time of obtaining reference point timing information. According to one embodiment, the playback frame generated by the decoder (520) can be transmitted to a display for display.

[0059] According to one embodiment, the receiver controller (500) may include a delay amount determining unit (delay amount determining circuit) (530). According to one embodiment, the delay amount determining unit (530) may obtain the total number of source frames that have been decoded by the decoder (520) and completed playback via the receiving stream buffer (510), and the source frame time length (v_total_cycle_s) of each of these source frames. According to one embodiment, the delay amount determining unit (530) may obtain reference point timing information (enc_cycle) of each time reference point existing among the series of source frames. According to one embodiment, the delay amount determining unit (530) may obtain final playback frame duration information (dec_cycle) generated by the decoder (520).

[0060] According to one embodiment, the delay amount determining unit (530) can determine a delay amount (delay_cycle) regarding decoding and playback in the receiver controller (500) whenever reference point timing information (enc_cycle) of a time reference point is acquired. According to one embodiment, the delay amount determining unit (530) can calculate (or determine) the delay amount by the following mathematical formula 1 (wherein L is the number of frames decoded and played back through the decoder (520), and frames that have been played back are not included therein).

[0061]

[0062] In the above Equation 1, the reference point time position of the reference point may be the number of source clock cycles accumulated from the start of the source frame stream to the corresponding time reference point. According to one embodiment, if the reference point timing information (enc_cycle) transmitted from the transmitter along with the source frame directly represents the number of accumulated source clock cycles from the start point of the series of source frames to a given time reference point, the reference point timing information (enc_cycle) may be used directly as the reference point time position. Alternatively, according to one embodiment, if the reference point timing information (enc_cycle) transmitted from the transmitter along with the source frame represents the number of accumulated source clock cycles from the reference point immediately preceding the reference point (if the reference point is the first reference point of the source frame stream, the reference point immediately preceding may be the start point of the source frame stream) to the reference point, the reference point time position may be obtained by Equation 2 (where M is the number of reference point timing information obtained up to the reference point).

[0063]

[0064] According to one embodiment, if reference point timing information (enc_cycle) transmitted from a transmitter along with a source frame indicates the number of source clock cycles from the beginning of the source frame to which the reference point belongs to the reference point to the reference point, the reference point time position can be obtained by Equation 3 (wherein N is the number of source frames up to the source frame preceding the source frame to which the reference point belongs, and skipped frames are not included therein).

[0065]

[0066] According to one embodiment, the receiver controller (500) may include a delay management unit (delay management circuit) (540). According to one embodiment, the delay management unit (540) receives a delay amount calculated (or determined) by the delay amount determination unit (530) and can compare the delay amount with one or more predetermined parameters. According to one embodiment, the delay management unit (540) determines whether the current delay state is excessively high or low based on the result of the comparison, and, based on the determination result, can generate a delay management request necessary to maintain such a delay state at an appropriate level.

[0067] According to one embodiment, a delay management request that may be generated by the delay management unit (540) may include, for example, a request to increase / decrease the playback time length of a playback frame relative to the source frame time length of a source frame by a predetermined amount or to adjust it upward / downward by a predetermined ratio, a frame skip request, a frame repeat request, etc., and the embodiments of the present disclosure are not limited thereto. According to one embodiment, a delay management request generated by the delay management unit (540), such as a request to increase / decrease the playback time length of a playback frame relative to the source frame time length of a source frame by a predetermined amount or to adjust it upward / downward by a predetermined ratio, a frame repeat request, etc., may be transmitted to the delay control unit (550) described later. According to one embodiment, a delay management request generated by the delay management unit (540), such as a frame skip request, may be transmitted to a transmitter through a communication interface and a network.

[0068] According to one embodiment, the receiver controller (500) may include a delay control unit (550). According to one embodiment, the delay control unit (550) may receive a delay management request from the delay management unit (540). According to one embodiment, the delay control unit (550) may perform an operation according to the received delay management request. According to one embodiment, when the delay control unit (550) receives a request to increase / decrease the playback time length of a playback frame relative to the source frame time length of a source frame by a predetermined amount or to adjust it upward / downward by a predetermined ratio, for example, the delay control unit (550) may appropriately increase or decrease the playback time length (v_total_cycle_d) of the playback frames generated by the decoder (520). According to one embodiment, when the playback time length of a playback frame relative to the source frame time length is increased or decreased by a predetermined amount or adjusted upward or downward by a predetermined ratio for delay control, such amount of increase or decrease or ratio of upward or downward adjustment may be a value determined according to receiver device characteristics (e.g., display specifications), delay state, etc., and is not limited to a specific example. According to one embodiment, the delay control unit (550) may, for example, when a frame repeat request is obtained, cause the display of a selected playback frame among the playback frames generated from the decoder (520) and transmitted to and played on the display to be repeated. The determination of which playback frame's display to repeat may be performed according to various criteria and methods.

[0069] FIG. 6 illustrates an exemplary case of calculating the delay amount of playback frames for source frames, performed in a receiver according to one embodiment of the present disclosure. According to one embodiment, the delay amount calculation illustrated in FIG. 6 may be implemented at least partially by a second controller (210) of the display device (200) of FIG. 2 and / or a receiver controller (500) of FIG. 5. The embodiments of the drawings are merely examples, and the present disclosure is not limited to exemplary embodiments.

[0070] As illustrated, the receiver first receives a source frame (611) and, after a delay time DELAY1 has elapsed, generates and plays a corresponding playback frame (621). As illustrated, the source frame (611) has a total source clock cycle count (v_total_cycle_s) of 1,000. Following the reception of the source frame (611), the receiver receives source frames (612, 613), each having a total source clock cycle count (v_total_cycle_s) of 1,000, while generating and playing a playback frame (622) following the playback frame (621). As illustrated, a reference point (631) is placed in the middle of the source frame (613). As described, the reference point timing information (enc_cycle) of the reference point (631) of the source frame (613) is 2,500, which can represent the total number of source clock cycles from the previous reference point (here, the starting point of the source frame (611)) to the reference point (631). The frame that has been decoded and played back up to the reference point (631) is one playback frame (621), and the duration from the start of playback of the most recently decoded playback frame (622) to the reference point (631), i.e., the final playback frame duration information (dec_cycle), is 300. Here, if the delay amount (delay_cycle1) (641) based on the reference point (631) is calculated using Equations 1 and 2, it is as follows.

[0071] Delay Amount(delay_cycle1)(641) = 2,500 - (1,000 + 300) = 1,200

[0072] The receiver also receives source frames (614 to 617) while generating and replaying frames (623 to 625). As illustrated, source frames (614 to 617) each have a total source clock cycle count (v_total_cycle_s) of 1,000, and a reference point (632) is placed in the middle of source frame (617). As illustrated, the reference point timing information of the reference point (632) of the source frame (617) is 4,000. According to one embodiment, the reference point timing information 4,000 may represent the total source clock cycle count from the previous reference point (631) to the current reference point (632). The frames decoded and replayed up to the reference point (632) are four replay frames (621 to 624), and the final replay frame duration information is 900. Here, if we calculate the delay amount (delay_cycle2) (642) based on the reference point (632) using mathematical formulas 1 and 2, it is as follows.

[0073] Delay Amount(delay_cycle2)(642) = (2,500 + 4,000) - (4,000 + 900) = 1,600

[0074] In this example, it can be seen that, for instance, by at least one processor of the electronic device, the delay amount (delay_cycle1) (641) at reference point (631) was 1,200, while the delay amount (delay_cycle2) (642) at reference point (632) was increased to 1,600. This can show that the decoding and playback delay at the receiver has increased more than before.

[0075] FIG. 7 illustrates an exemplary case of calculating the delay amount of playback frames for source frames, performed in a receiver according to one embodiment of the present disclosure. According to one embodiment, the delay amount calculation illustrated in FIG. 7 may be implemented at least partially by a second controller (210) of the display device (200) of FIG. 2 and / or a receiver controller (500) of FIG. 5. The embodiments of the drawings are merely examples, and the present disclosure is not limited to exemplary embodiments.

[0076] As illustrated, the receiver receives source frames (711 to 713) and, after a delay time DELAY2 has elapsed, generates and plays a playback frame (721) corresponding to the received frame (711). As illustrated, each of the source frames (711 to 713) has a total source clock cycle count (v_total_cycle_s) of 1,000. Following the reception of the source frames (711 to 713), the receiver receives source frames (714, 715) each having a total source clock cycle count (v_total_cycle_s) of 1,000, and generates and plays playback frames (722, 723) following the playback frame (721). As illustrated, a reference point (731) is placed in the middle of the source frame (715). As described, the reference point timing information (enc_cycle) of the reference point (731) of the source frame (715) is 4,500. In one embodiment, the reference point timing information (enc_cycle) may represent the total number of source clock cycles from the start point of the source frame stream (the start point of the source frame (711)) to the reference point (731). The frames decoded and played back up to the reference point (731) are the played frames (721, 722), and the duration from the start of playback of the most recently decoded played frame (723) to the reference point (731), i.e., the final played frame duration information (dec_cycle), is 300. Here, if the delay amount (delay_cycle3) (741) based on the reference point (731) is calculated using Equation 1, it is as follows.

[0077] Delay Amount(delay_cycle3)(741) = 4,500 - (2,000 + 300) = 2,200

[0078] In one embodiment, the receiver may determine that a frame skip is necessary because the delay amount of 2,200 is an excessively large value. In one embodiment, the receiver may transmit a frame skip request to the transmitter as feedback data. According to one embodiment, the transmitter that receives the frame skip request may determine to skip the source frame (716) that follows the source frame (715). According to one embodiment, as illustrated, the source frame (716) is skipped and subsequent source frames (717, 718) are transmitted and received by the receiver. The receiver also receives the source frames (717, 718) while generating and regenerating the regenerated frames (724 to 726). As illustrated, the source frames (717, 718) each have a total source clock cycle count (v_total_cycle_s) of 1,000, and a reference point (732) is placed in the middle of the source frame (718). As described, the reference point timing information of the reference point (732) of the source frame (718) is 6,700. In one embodiment, the reference point timing information (enc_cycle) may represent the total number of source clock cycles from the start point of the source frame stream (the start point of the source frame (711)) to the reference point (732). The frames decoded and played back up to the reference point (732) are five playback frames (721 to 725), and the duration from the start of playback of the most recently decoded playback frame (726) to the reference point (732), i.e., the final playback frame duration information, is 400. Here, if the delay amount (delay_cycle4) (742) based on the reference point (732) is calculated using Equation 1, it is as follows.

[0079] Delay Amount(delay_cycle4)(742) = 6,700 - (5,000 + 400) = 1,300

[0080] In this example, it can be seen that, by at least one processor of the electronic device, the delay amount (delay_cycle3) (741) at reference point (731) was 2,200, whereas the delay amount (delay_cycle4) (742) at reference point (732) after frame skipping at the transmitter was reduced to 1,300. This can show that the decoding and playback delay at the receiver is reduced compared to before.

[0081] FIG. 8 illustrates an exemplary case of calculating the delay amount of playback frames for source frames, performed in a receiver according to one embodiment of the present disclosure. According to one embodiment, the delay amount calculation illustrated in FIG. 8 may be implemented at least partially by a second controller (210) of the display device (200) of FIG. 2 and / or a receiver controller (500) of FIG. 5. The embodiments of the drawings are merely examples, and the present disclosure is not limited to exemplary embodiments.

[0082] As illustrated, the receiver first receives a source frame (811) and, after a delay time DELAY3 has elapsed, generates and plays a playback frame (821) corresponding to the received frame (811). As illustrated, the source frame (811) has a total source clock cycle count (v_total_cycle_s) of 1,000. Following the reception of the source frame (811), the receiver receives a source frame (812) having a total source clock cycle count (v_total_cycle_s) of 1,000 and, following the playback frame (821), generates and plays a playback frame (822). As illustrated, a reference point (831) is placed in the middle of the source frame (812). As illustrated, the reference point timing information (enc_cycle) of the reference point (831) of the source frame (812) is 1,500. In one embodiment, reference point timing information (enc_cycle) may represent the total number of source clock cycles from the start point of the source frame stream (the start point of the source frame (811)) to the reference point (831). The frame that has been decoded and played back up to the reference point (831) is one playback frame (821), and the number of playback clock cycles (dec_cycle) elapsed from the start of playback of the most recently decoded playback frame (822) to the reference point (831) is 450. Here, if the delay amount (delay_cycle5) (841) based on the reference point (831) is calculated using Equation 1, it is as follows.

[0083] Delay Amount(delay_cycle5)(841) = 1,500 - (1,000 + 450) = 50

[0084] In one embodiment, the receiver may determine that a frame repeat is required because the delay amount 50 is an excessively small value. In one embodiment, the receiver may generate a frame repeat request. According to one embodiment, based on the frame repeat request, the receiver decoder may repeat the same playback frame (823) after the playback of the playback frame (822) is completed.

[0085] In one embodiment, the receiver performs frame repeat as described above, while receiving source frames (813 to 815) and generating and replaying frames (824, 825). As illustrated, the source frames (813 to 815) each have a total source clock cycle count (v_total_cycle_s) of 1,000, and a reference point (832) is placed in the middle of the source frame (815). As illustrated, the reference point timing information of the reference point (832) of the source frame (815) is 4,800. According to one embodiment, the reference point timing information 4,800 may represent the total source clock cycle count from the start point of the source frame stream (the start point of the source frame (811)) to the reference point (832). The frames decoded and played back up to the reference point (832) are three playback frames (821, 822, 824) (excluding the playback repeated frame (823)), and the number of playback clock cycles elapsed from the start of playback of the most recently decoded playback frame (825) to the reference point (832) is 700. Here, if we calculate the delay amount (delay_cycle6) (842) based on the reference point (832) using Equation 1, it is as follows.

[0086] Delay Amount(delay_cycle6)(842) = 4,800 - (3,000 + 700) = 1,100

[0087] In this example, for instance, by at least one processor of the electronic device, it can be seen that the delay amount (delay_cycle5) (841) at the reference point (831) was 50, whereas the delay amount (delay_cycle6) (842) at the reference point (832) after the frame repeat is performed has increased to 1,100. This may show that the decoding and playback delay at the receiver has increased more than before.

[0088] FIG. 9 illustrates an exemplary case of calculating the delay amount of playback frames for source frames, performed in a receiver according to one embodiment of the present disclosure. According to one embodiment, the delay amount calculation illustrated in FIG. 9 may be implemented at least partially by a second controller (210) of the display device (200) of FIG. 2 and / or a receiver controller (500) of FIG. 5. The embodiments of the drawings are merely examples, and the present disclosure is not limited to exemplary embodiments.

[0089] As illustrated, the receiver first receives source frames (911, 912), and then generates and plays a corresponding playback frame (921) after a delay time DELAY4 has elapsed. According to one embodiment, the source frames received by the receiver may be frames generated according to a variable frame rate (VRR) method. As illustrated, the source frame (911) has a total source clock cycle count (v_total_cycle_s) of 1,000, and the source frame (912) has a total source clock cycle count (v_total_cycle_s) of 1,500. The receiver generates and plays a playback frame (921) corresponding to the source frame (911) after DELAY4 has elapsed from the reception of the first source frame (911). The receiver receives a source frame (913) having a total source clock cycle count (v_total_cycle_s) of 1,200 following the reception of source frames (911, 912), and generates and plays a playback frame (922) following the playback frame (921). As illustrated, a reference point (931) is placed in the middle of the source frame (913). As illustrated, the reference point timing information (enc_cycle) of the reference point (931) of the source frame (913) is 3,000. In one embodiment, the reference point timing information (enc_cycle) may represent the total source clock cycle count from the start point of the source frame stream (the start point of the source frame (911)) to the reference point (931). The frame that has been decoded and played back up to the reference point (931) is one playback frame (921), and the number of playback clock cycles (dec_cycle) elapsed from the start of playback of the most recently decoded playback frame (922) to the reference point (931) is 200. Here, if we calculate the delay amount (delay_cycle7) (941) based on the reference point (931) using Equation 1, it is as follows.

[0090] Delay Amount(delay_cycle7)(941) = 3,000 - (1,000 + 200) = 1,800

[0091] The receiver also receives source frames (914 to 917) while generating and reproducing playback frames (923 to 925). As illustrated, source frame (914) has a total source clock cycle count (v_total_cycle_s) of 1,000, source frame (915) has a total source clock cycle count of 1,300, source frame (916) has a total source clock cycle count of 1,000, and source frame (917) has a total source clock cycle count of 1,000. A reference point (932) is placed in the middle of source frame (917). As illustrated, the reference point timing information of the reference point (932) of source frame (917) is 7,300. According to one embodiment, the reference point timing information 7,300 may represent the total source clock cycle count from the start point of the source frame stream (the start point of source frame (911)) to the reference point (932). The frames decoded and played back up to the reference point (932) are four playback frames (921 to 924), and the number of playback clock cycles elapsed from the start of playback of the most recently decoded playback frame (925) to the reference point (932) is 1,200. Here, if we calculate the delay amount (delay_cycle8) (942) based on the reference point (932) using Equation 1, it is as follows.

[0092] Delay Amount(delay_cycle8)(942) = 7,300 - ((1,000 + 1,500 + 1,200 + 1000) + 1,200) = 1,400

[0093] In this example, it can be seen that, for instance, by at least one processor of the electronic device, the delay amount (delay_cycle7) (941) at reference point (931) was 1,800, while the delay amount (delay_cycle2) (942) at reference point (932) was reduced to 1,400. This can show that the decoding and playback delay at the receiver has been reduced further compared to before.

[0094] FIGS. 10a and FIGS. 10b are flowcharts of data processing of a receiver according to one embodiment of the present disclosure. According to one embodiment, the data processing disclosed in FIGS. 10a and FIGS. 10b may be at least partially implemented by a second controller (210) of the display device (200) of FIG. 2 and / or a receiver controller (500) of FIG. 5.

[0095] According to one embodiment, in the operation (1002), source frames transmitted through a network and related control data may be received. In one embodiment, the received control data may include information such as, for example, a source frame time length corresponding to each source frame and reference point timing information (enc_cycle) corresponding to each time reference point.

[0096] In operation (1004), it may be determined whether reference point timing information (enc_cycle) corresponding to a time reference point placed in any source frame among the source frames has been received, for example by at least one processor of an electronic device. In operation (1004), if it is determined that reference point timing information (enc_cycle) has been received, for example by at least one processor of an electronic device, the procedure may proceed to operation (1006) to determine the final playback frame duration information (dec_cycle). Then, in operation (1008), a decoding and playback delay value (CYC_CUR) corresponding to the reference point may be calculated (or determined). According to one embodiment, the decoding and playback delay value (CYC_CUR) may be calculated (or determined) according to, for example, Equation 1 and Equation 2 described above.

[0097] According to one embodiment, in operation (1010), for example, by at least one processor of an electronic device, it may be determined whether the calculated (or determined) decoding and playback delay value (CYC_CUR) is greater than a first parameter (CYC_T_MAX). In one example, the first parameter (CYC_T_MAX) may be a predetermined target maximum delay time value. In operation (1010), for example, if it is determined by at least one processor of an electronic device that the delay value (CYC_CUR) is greater than the first parameter (CYC_T_MAX) (i.e., if the delay value exceeds a predetermined target maximum delay time value), the procedure proceeds to operation (1012) to generate a frame skip request, and the generated frame skip request may be transmitted to a transmitter via a network as feedback data. In operation (1014), the playback time length of the playback frame relative to the source frame time length to be used when decoding the source frame may be reduced by a predetermined amount or adjusted downward by a predetermined ratio. As described above, the amount to which the playback time length is reduced may be a predetermined value based on receiver device characteristics (e.g., display specifications), delay state, etc., and is not limited to a specific example. In one embodiment, if it is determined that the delay value (CYC_CUR) is greater than the first parameter (CYC_T_MAX), for example, at least one processor of the electronic device may perform two-stage delay control by considering the delay amount to be excessively excessive and reducing the playback time length of the playback frame along with a frame skip request. After the playback time length of the playback frame is reduced, decoding of the source frame may be performed in operation (1016) according to the adjustment.

[0098] In operation (1010), if it is determined by, for example, at least one processor of the electronic device that the delay value (CYC_CUR) is less than or equal to the first parameter (CYC_T_MAX), the procedure proceeds to operation (1018) to determine whether the delay value (CYC_CUR) is greater than the second parameter (CYC_T_HIGH). In one example, the second parameter (CYC_T_HIGH) may be a predetermined target upper delay time value. In operation (1018), if it is determined by, for example, at least one processor of the electronic device that the delay value (CYC_CUR) is greater than the second parameter (CYC_T_HIGH), the procedure proceeds to operation (1014) to reduce the playback time length of the playback frame relative to the source frame time length used during decoding of the source frame by a predetermined amount or adjust it downward by a predetermined ratio. In one embodiment, for example, if a delay value (CYC_CUR) is determined by at least one processor of an electronic device to be greater than a first parameter (CYC_T_MAX) but greater than a second parameter (CYC_T_HIGH), a first-stage delay control can be performed to lower the playback time length of a playback frame, considering that the delay amount is not excessively high but is still high. After the playback time length of the playback frame is lowered, decoding of the source frame can be performed in operation (1016) based on the adjustment result.

[0099] In operation (1018), if, for example, at least one processor of the electronic device determines that the delay value (CYC_CUR) is less than or equal to the second parameter (CYC_T_HIGH), the procedure proceeds to operation (1020) to determine whether the calculated (or determined) decoding / playback delay value (delay_cycle) is less than the third parameter (CYC_T_MIN). In one example, the third parameter (CYC_T_MIN) may be a predetermined target minimum delay time value. In operation (1020), if, for example, at least one processor of the electronic device determines that the delay value (CYC_CUR) is less than the third parameter (CYC_T_MIN), the procedure proceeds to operation (1022) and a frame repeat request may be generated. In one embodiment, if it is determined that the delay value (CYC_CUR) is smaller than the third parameter (CYC_T_MIN), for example, at least one processor of the electronic device may perform delay control of frame repeat request generation, considering the delay amount to be insufficient. In one embodiment, the insufficient delay amount may be resolved while frame playback repetition is performed according to the frame repeat request. In this embodiment, in the case of such insufficient delay, one-stage delay control of frame repeat request generation is performed, but the embodiments of the present disclosure are not limited thereto. In another embodiment, if it is determined that the delay is excessively insufficient, two-stage delay control may be performed to increase the playback time length of the playback frame relative to the time length of the source frame along with the frame repeat request.

[0100] In operation (1020), if it is determined by, for example, at least one processor of the electronic device that the delay value (CYC_CUR) is not smaller than the third parameter (CYC_T_MIN), the procedure proceeds to operation (1024) to determine whether the delay value (CYC_CUR) is smaller than the fourth parameter (CYC_T_LOW). In one example, the fourth parameter (CYC_T_LOW) may be a predetermined target lower delay time value. In operation (1024), if it is determined by, for example, at least one processor of the electronic device that the delay value (CYC_CUR) is smaller than the fourth parameter (CYC_T_LOW), the procedure may proceed to operation (1026). In operation (1026), the playback time length of the playback frame relative to the source frame time length to be used when decoding the source frame may be increased by a predetermined amount or adjusted upward by a predetermined ratio. In one embodiment, if it is determined that the delay value (CYC_CUR) is not lower than the third parameter (CYC_T_MIN) but lower than the fourth parameter (CYC_T_LOW), delay control can be performed, for example by at least one processor of an electronic device, to increase the playback time length of the playback frame, assuming that the delay amount is somewhat low. After the time length of the playback frame is increased, the procedure proceeds to operation (1016), and decoding of the source frame can be performed based on the adjustment result.

[0101] According to one aspect of the present disclosure, an electronic device comprises: a communication interface; a display; at least one processor; and a memory comprising at least one storage medium for storing instructions. When the instructions are executed individually or collectively by the at least one processor, the electronic device is configured to: obtain, through the communication interface, a series of encoded source frames, a source frame time length for each source frame, and first reference point timing information for a first reference point defined on a first source frame among the series of source frames; generate playback frames for playback through the display by decoding the source frames; determine a playback delay value based on the first reference point timing information and the source frame time lengths of source frames corresponding to playback frames completed prior to the acquisition of the first reference point timing information; and perform delay control based on the playback delay value.

[0102] According to one aspect of the present disclosure, each source frame time length is the total number of clock cycles allocated to the corresponding source frame during encoding, and the first reference point timing information indicates the temporal position of the first reference point on the series of source frames.

[0103] According to one aspect of the present disclosure, the first reference point timing information represents the total number of clock cycles from the beginning of the first source frame among the series of source frames to the first reference point, and the instructions, when executed individually or collectively by the at least one processor, cause the electronic device to determine the total number of clock cycles during encoding.

[0104] According to one aspect of the present disclosure, the first reference point timing information represents the total number of clock cycles from another reference point existing immediately prior to the first reference point to the first reference point, and the instructions, when executed individually or collectively by the at least one processor, cause the electronic device to determine the total number of clock cycles during encoding.

[0105] According to one aspect of the present disclosure, when the instructions are executed individually or collectively by the at least one processor, the electronic device determines the playback delay value based further on time information elapsed from the start of playback of the most recently decoded playback frame to the time of acquisition of the first reference point timing information based on the time of acquisition of the first reference point timing information.

[0106] According to one aspect of the present disclosure, when the instructions are executed individually or collectively by the at least one processor, the electronic device is configured to: acquire a first value representing the temporal position of the first reference point on the series of source frames based on the first reference point timing information; acquire a second value comprising the sum of the time lengths of each source frame of source frames corresponding to playback frames completed prior to the acquisition of the first reference point timing information, and the time elapsed from the start of playback of the most recently decoded playback frame to the acquisition time of the first reference point timing information based on the acquisition time of the first reference point timing information; and determine the playback delay value as a result of subtracting the second value from the first value.

[0107] According to one aspect of the present disclosure, when the instructions are executed individually or collectively by the at least one processor, the electronic device is configured to periodically define one or more reference points on the series of source frames based on a predetermined reference.

[0108] According to one aspect of the present disclosure, the delay control may include at least one of adjusting the playback time for a playback frame, generating a skip request for one or more source frames, or repeating playback of one or more playback frames.

[0109] According to one aspect of the present disclosure, the memory stores one or more parameters, and when the instructions are executed individually or collectively by the at least one processor, the electronic device performs the delay control based on a comparison of the playback delay value and the one or more parameters.

[0110] According to one aspect of the present disclosure, the one or more parameters correspond to a target maximum delay time, a target upper delay time, a target lower delay time, and a target minimum delay time, and based on the playback delay value being greater than the target maximum delay time, the delay control may include at least one of down-adjusting the playback time for a playback frame and generating a skip request for one or more source frames.

[0111] According to one aspect of the present disclosure, when the playback delay value is a value between the target maximum delay time and the target upper delay time, the delay control may include downward adjustment of the playback time for the playback frame.

[0112] According to one aspect of the present disclosure, based on the fact that the playback delay value is smaller than the target minimum delay time, the delay control may include the repeated playback of one or more playback frames.

[0113] According to one aspect of the present disclosure, based on the fact that the playback delay value is a value between the target minimum delay time and the target lower delay time, the delay control may include an upward adjustment of the playback time for a playback frame.

[0114] According to one aspect of the present disclosure, a display system comprises: an encoder configured to encode image information into a series of source frames; a time length generating circuit configured to generate source frame time length information for each of the source frames; a reference point timing information generating circuit configured to generate first reference point timing information corresponding to a first reference point determined on a first source frame among the series of source frames; and a video transmission device comprising a first communication interface configured to transmit stream data including the series of source frames, source frame time length information for each of the source frames, and the first reference point timing information, and a second communication interface configured to receive the stream data; a decoder configured to generate a series of playback frames from the source frames of the stream data; a display configured to play the series of playback frames; and a delay amount determining circuit configured to determine a playback delay value based on the first reference point timing information and the time length of each source frame of the source frames corresponding to the playback frames that have been played by the display prior to the acquisition of the first reference point timing information. and includes a video receiving device comprising a delay control circuit configured to perform delay control based on the above playback delay value.

[0115] According to one aspect of the present disclosure, the delay amount determining circuit is further configured to determine the playback delay value based on time information elapsed from the start of playback of the most recently generated playback frame by the decoder to the time of acquisition of the first reference point timing information, based on the time of acquisition of the first reference point timing information.

[0116] According to one aspect of the present disclosure, the delay amount determining circuit obtains a first value representing the temporal position of the first reference point on the series of source frames based on the first reference point timing information, obtains a second value by summing the total sum of the time lengths of each source frame of the source frames corresponding to the playback frames that have been completed by the display, and the time elapsed from the start of playback of the most recently generated playback frame by the decoder to the time of obtaining the reference point timing information based on the time of obtaining the first reference point timing information, and calculates the playback delay value as a result of subtracting the second value from the first value.

[0117] According to one aspect of the present disclosure, the image transmission device is configured to operate based on a source clock, and the first reference point timing information represents the total number of source clock cycles from the start of the first source frame among the series of source frames to the first reference point.

[0118] According to one embodiment of the present disclosure, the image transmission device is configured to operate based on a source clock, and the first reference point timing information represents the total number of source clock cycles from another reference point existing immediately prior to the first reference point to the first reference point.

[0119] According to one aspect of the present disclosure, the image receiving device further includes a memory storing one or more parameters, and the delay control circuit is configured to perform the delay control based on a comparison of the playback delay value and the one or more parameters.

[0120] According to one aspect of the present disclosure, the delay control by the delay control circuit comprises at least one of upward adjustment of the playback time for a playback frame, downward adjustment of the playback time for a playback frame, generation of a skip request for one or more source frames and transmission to the video transmission device, or repeated playback of one or more playback frames.

[0121] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a display device, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0122] One or more embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, but include various modifications, equivalents, or substitutions of said embodiments. For example, a component expressed in the singular should be understood as a concept including a plural component unless the context clearly implies only the singular. It should be understood that the term "and / or" as used in this document encompasses any possible combination of one or more of the items listed. Terms such as "comprising," "having," and "consisting of" used in this disclosure are intended merely to indicate the existence of the features, components, parts, or combinations thereof described in this disclosure, and the use of such terms is not intended to exclude the existence or addition of one or more other features, components, parts, or combinations thereof. In this document, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order).

[0123] The terms “part” or “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. The “part” or “module” may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, the “part” or “module” may be implemented in the form of an application-specific integrated circuit (ASIC).

[0124] In one or more embodiments of this document, the term “in the case of” as used may be interpreted, depending on the context, to mean “when,” or “at the time of,” or “in response to a decision,” or “in response to a detection.” Similarly, “in the case where it is determined,” or “in the case where it is detected,” may be interpreted, depending on the context, to mean “at the time of determination,” or “in response to a decision,” or “at the time of detection,” or “in response to a detection.”

[0125] Programs executed by the display systems and electronic devices described in this document may be implemented by hardware components, software components, and / or combinations of hardware and software components. Programs may be executed by any system capable of executing computer-readable instructions.

[0126] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software may be implemented as a computer program containing instructions stored on computer-readable storage media. Examples of computer-readable storage media include magnetic storage media (e.g., ROM (Read-Only Memory), RAM (Random-Access Memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROMs, DVDs (Digital Versatile Discs)). Computer-readable storage media may be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. Computer programs may be distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0127] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. As an electronic device, Communication interface; display; At least one processor; and It includes memory comprising at least one storage medium for storing instructions, and When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Through the communication interface above, a series of encoded source frames, a time length for each source frame for each source frame, and first reference point timing information for a first reference point determined on a first source frame among the series of source frames are obtained, and By decoding the above source frames, playback frames for playback through the display are generated, and Based on the first reference point timing information and the time length of each source frame of the source frames corresponding to the playback frames that were completed playback prior to the acquisition of the first reference point timing information, a playback delay value is determined, and An electronic device that causes delay control to be performed based on the above playback delay value.

2. In Paragraph 1, The time length of each source frame is the total number of clock cycles allocated to the corresponding source frame during encoding, and The above first reference point timing information is an electronic device that indicates the temporal position of the above first reference point on the series of source frames.

3. In Paragraph 1, The first reference point timing information above indicates the total number of clock cycles from the start of the first source frame among the series of source frames to the first reference point, and The above instructions, when executed individually or collectively by the at least one processor, cause the electronic device to determine the total number of clock cycles during encoding.

4. In Paragraph 1, The first reference point timing information represents the total number of clock cycles from another reference point existing immediately prior to the first reference point to the first reference point, and An electronic device that, when the above instructions are executed individually or collectively by the at least one processor, causes the electronic device to determine the total number of clock cycles during encoding.

5. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that determines the playback delay value based further on time information elapsed from the start of playback of the most recently decoded playback frame to the time of acquisition of the first reference point timing information, based on the time of acquisition of the first reference point timing information.

6. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Based on the first reference point timing information, a first value representing the temporal position of the first reference point on the series of source frames is obtained, and A second value is obtained by summing the total sum of the time lengths of each source frame corresponding to the playback frames completed prior to the acquisition of the first reference point timing information, and the time elapsed from the start of playback of a playback frame—the playback frame being the most recently decoded based on the acquisition time of the first reference point timing information—to the acquisition time of the first reference point timing information. An electronic device that determines the playback delay value as a result of subtracting the second value from the first value.

7. In Paragraph 1, An electronic device in which, when the above instructions are executed individually or collectively by the at least one processor, the electronic device periodically defines one or more reference points on the series of source frames based on predetermined criteria.

8. In Paragraph 1, The above delay control comprises at least one of adjusting the playback time for a playback frame, generating a skip request for one or more source frames, or repeating playback of one or more playback frames, in an electronic device.

9. In Paragraph 8, The above memory stores one or more additional parameters, and When the above instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that performs delay control based on a comparison of the above playback delay value and one or more parameters.

10. In Paragraph 9, The above one or more parameters correspond to a target maximum delay time, a target upper delay time, a target lower delay time, and a target minimum delay time, and An electronic device, wherein, based on the playback delay value being greater than the target maximum delay time, the delay control comprises at least one of downward adjustment of the playback time for a playback frame and generating a skip request for one or more source frames.

11. In Paragraph 10, An electronic device in which, when the above playback delay value is a value between the above target maximum delay time and the above target upper delay time, the delay control includes downward adjustment of the playback time for a playback frame.

12. In Paragraph 10, An electronic device in which, based on the fact that the above playback delay value is smaller than the above target minimum delay time, the delay control includes the repeated playback of one or more playback frames.

13. In Paragraph 10, An electronic device, wherein the delay control includes an upward adjustment of the playback time for a playback frame, based on the fact that the above playback delay value is a value between the above target minimum delay time and the above target lower delay time.

14. As a display system, An encoder configured to encode video information into a series of source frames; A time length generation circuit configured to generate source frame time length information for each of the above source frames; A reference point timing information generation circuit configured to generate first reference point timing information corresponding to a first reference point determined on a first source frame among the above series of source frames; and A video transmission device comprising a first communication interface configured to transmit stream data including the above series of source frames, each source frame time length information regarding each of the source frames, and the first reference point timing information, and A second communication interface configured to receive the above stream data; A decoder configured to generate a series of playback frames from the source frames of the stream data; A display configured to play the above series of playback frames; A delay amount determining circuit configured to determine a playback delay value based on the first reference point timing information and the time length of each source frame of source frames corresponding to playback frames that have been completed by the display prior to the acquisition of the first reference point timing information; and A delay control circuit configured to perform delay control based on the above playback delay value. Image receiving device including A display system including 15. In Paragraph 14, The above delay amount determination circuit is further configured to determine the playback delay value based on time information elapsed from the start of playback of a playback frame—which is the most recently generated by the decoder based on the acquisition time of the first reference point timing information—to the acquisition time of the first reference point timing information, or A display system further configured such that the above delay amount determination circuit acquires a first value representing the temporal position of the first reference point on the series of source frames based on the first reference point timing information, acquires a second value by summing the total sum of the time lengths of each source frame corresponding to the playback frames that have been completed by the display, and the time elapsed from the start of playback of a playback frame—the playback frame being the most recently generated by the decoder based on the time of acquisition of the first reference point timing information—to the time of acquisition of the first reference point timing information, and calculates the playback delay value as a result of subtracting the second value from the first value.

Images