Electronic device and control method therefor

Abstract

This electronic device comprises: an input / output interface; a memory for storing at least one instruction; and at least one processor for executing the at least one instruction, wherein the at least one processor acquires a first image having a variable refresh rate through the input / output interface, and when a second image having a fixed refresh rate is acquired, generates a multi-view screen in which the first image and the second image are displayed together by varying a screen refresh rate of the second image on the basis of a current refresh rate corresponding to the variable refresh rate.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device capable of processing an image having a variable refresh rate and an image having a fixed refresh rate into a multi-view screen, and a method for controlling the same.

[0002] Thanks to the advancement of electronic technology, various types of electronic devices are being developed and distributed. In particular, the development and distribution of display devices such as TVs and mobile devices are actively underway.

[0003] For example, it was possible to provide VRR (Variable Refresh Rate) video signals from servers, game consoles, etc., to display devices, etc.

[0004] The embodiments of the disclosure may solve at least one of the previously described problems and / or disadvantages and provide the advantages described below. Accordingly, the embodiments of the present disclosure provide an electronic device and a method for controlling the same that can process images having a variable refresh rate and images having a fixed refresh rate into a multi-view screen.

[0005] Additional embodiments will be presented in the detailed description below, some of which are obvious from the detailed description, and other embodiments can also be presented through learning from the presented embodiments.

[0006] An electronic device according to an embodiment of the present disclosure includes an input / output interface, a memory for storing at least one instruction, and at least one processor configured to execute at least one instruction to acquire a first image having a variable refresh rate through the input / output interface, acquire a second image having a fixed refresh rate, and change the fixed refresh rate of the second image to the current refresh rate of the first image to generate a multi-view screen in which the first image and the second image are displayed together.

[0007] A control method for an electronic device according to an embodiment of the present disclosure includes the steps of acquiring a first image having a variable refresh rate, acquiring a second image having a fixed refresh rate, and changing the fixed refresh rate of the second image to the current refresh rate of the first image to generate a multi-view screen in which the first image and the second image are displayed together.

[0008] An electronic device according to an embodiment of the present disclosure is disclosed. The electronic device comprises an input / output interface, a memory for storing at least one instruction, and at least one processor for executing said at least one instruction. The at least one processor acquires a first image having a variable refresh rate through the input / output interface and, upon acquiring a second image having a fixed refresh rate, generates a multi-view screen in which the first image and the second image are displayed together by varying the screen refresh rate of the second image based on a current refresh rate corresponding to the variable refresh rate.

[0009] If the screen refresh rate of the second image is greater than the minimum refresh rate of the variable refresh rate, the above at least one processor can change the minimum refresh rate of the first image's variable refresh rate.

[0010] The above at least one processor can generate the multiview screen such that the screen refresh rate of the multiview screen corresponds to the screen refresh rate of the first image.

[0011] The above at least one processor can synchronize the screen refresh rate by repeatedly arranging frames of the second image so that the screen refresh rate of the second image corresponds to the current screen refresh rate of the first image.

[0012] The above at least one processor can adjust the Vfront (Vertical Front Porch) of the second image to synchronize the Vsync of the first image with the Vsync start time of the second image.

[0013] The above at least one processor determines the resolution of the multi-view screen and can scale at least one of the resolutions of the first image and the second image in correspondence with the determined resolution of the multi-view.

[0014] The electronic device further includes a display that displays an image using a single timing controller, and the at least one processor can control the timing controller to display the multi-view screen.

[0015] The electronic device further includes a display that displays an image using a plurality of timing controllers, and the at least one processor can vary the screen playback frequency of the second image when at least one of the plurality of timing controllers processes the first image and the second image together.

[0016] The above at least one processor can generate a multi-view in the form of a Picture in Picture (PIP) of the first image and the second image.

[0017] The above at least one processor can obtain the current playback frequency of the first image based on playback frequency information within the VTEM (Video Timing Extended Metadata) packet obtained through the input / output interface.

[0018] A control method for an electronic device according to one embodiment of the present disclosure includes the steps of acquiring a first image having a variable refresh rate, acquiring a second image having a fixed refresh rate, and generating a multi-view screen in which the first image and the second image are displayed together by varying the refresh rate of the second image based on a refresh rate corresponding to the variable refresh rate.

[0019] The control method may further include the step of changing the minimum playback frequency of the variable playback frequency of the first image if the screen playback frequency of the second image is greater than the minimum playback frequency of the variable playback frequency.

[0020] The step of generating the multi-view screen above can generate the multi-view screen such that the screen refresh rate of the multi-view screen corresponds to the screen refresh rate of the first video.

[0021] The step of generating the multi-view screen above can synchronize the screen refresh rates by repeatedly arranging frames of the second image so that the screen refresh rate of the second image corresponds to the current screen refresh rate of the first image.

[0022] The step of generating the multi-view screen above can synchronize the Vsync of the first image with the Vsync start time of the second image by adjusting the Vfront (Vertical Front Porch) of the second image.

[0023] The control method may further include the step of determining the resolution of the multi-view screen and scaling at least one of the resolutions of the first image and the second image in correspondence with the determined resolution of the multi-view.

[0024] The control method may further include the step of outputting the multi-view screen to a display that displays an image using a single timing controller.

[0025] The control method further includes the step of outputting a multi-view screen to a display that displays an image using a plurality of timing controllers, and the step of generating the multi-view screen may vary the screen playback frequency of the second image when at least one of the plurality of timing controllers processes the first image and the second image together.

[0026] The step of generating the multi-view screen above can generate a multi-view in the form of a Picture in Picture (PIP) with the first and second images.

[0027] A non-transient computer-readable recording medium storing a program for executing a control method for an electronic device according to one embodiment of the present disclosure, wherein the control method comprises the steps of acquiring a first image having a variable refresh rate, acquiring a second image having a fixed refresh rate, and generating a multi-view screen in which the first image and the second image are displayed together by varying the refresh rate of the second image based on a refresh rate corresponding to the variable refresh rate.

[0028] The above-described or other aspects, features, and benefits of embodiments of the present disclosure will become more apparent from the following description with reference to the accompanying drawings. In the accompanying drawings:

[0029] FIG. 1 is a drawing for explaining a multi-view display operation according to one embodiment of the present disclosure,

[0030] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure,

[0031] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure,

[0032] FIG. 4 is a flowchart for explaining a multi-view setting operation using an image having a variable refresh rate according to one embodiment of the present disclosure,

[0033] Figure 5 is a diagram illustrating the structure of a VTEM (Video Timing Extended Metadata) packet.

[0034] FIG. 6 is a diagram for explaining the difference between an image with a variable refresh rate and an image with a fixed refresh rate according to one embodiment of the present disclosure,

[0035] FIG. 7 is a drawing for explaining a method for synchronizing playback frequency according to one embodiment of the present disclosure,

[0036] FIG. 8 is a drawing for explaining a multi-view generation operation in terms of resolution according to an embodiment of the present disclosure,

[0037] FIG. 9 is a drawing for explaining a multi-view generation operation in terms of resolution according to an embodiment of the present disclosure,

[0038] FIG. 10 is a drawing for explaining the arrangement structure of a multiview according to one embodiment of the present disclosure,

[0039] FIG. 11 is a drawing for explaining the arrangement structure of a multiview according to one embodiment of the present disclosure,

[0040] FIG. 12 is a drawing for explaining a multi-view display operation when a plurality of TCONs are provided according to an embodiment of the present disclosure,

[0041] FIG. 13 is a drawing for explaining a multi-view display operation when a plurality of TCONs are provided according to one embodiment of the present disclosure, and,

[0042] FIG. 14 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0043] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0044] In describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description is omitted.

[0045] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0046] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0047] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components, etc.) and do not exclude the presence of additional features.

[0048] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0049] Expressions such as “first,” “second,” “first,” or “second” used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0050] Where it is stated that a component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., a third component).

[0051] On the other hand, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between said certain component and said other component.

[0052] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0053] Instead, in some situations, the expression “device configured to do something” may mean that the device is “capable of doing something” together with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing the said operation (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing the said operation by executing one or more software programs stored in a memory device.

[0054] In the embodiments, a 'module' or 'part' performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'parts' may be integrated into at least one module and implemented by at least one processor, except for the 'module' or 'part' that needs to be implemented in specific hardware.

[0055] Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0056] Meanwhile, the various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0057] Meanwhile, an electronic device according to various embodiments of the present disclosure may include, for example, at least one of a terminal device, a tablet PC, a desktop PC, a laptop PC, a server, or a wearable device. The wearable device may include at least one of an accessory type (e.g., a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or head-mounted device (HMD)), a fabric or clothing integrated type (e.g., electronic clothing), a body-attached type (e.g., a skin pad or tattoo), or a bio-implantable circuit.

[0058] In some embodiments, the electronic device is, for example, a television, a DVD (digital video disk) player, audio, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync). TM , Apple TV TM , or Google TV TM ), game console (e.g., Xbox) TM PlayStation TMIt may include at least one of an electronic dictionary, an electronic key, a camcorder, or an electronic photo frame. Meanwhile, among the electronic devices described above, a device equipped with a display may be referred to as a display device. Meanwhile, even if the electronic device of the present disclosure does not have a display, it may be a set-top box, a PC, or a game console that provides images to a display device.

[0059] Hereinafter, embodiments according to the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them.

[0060] FIG. 1 is a drawing for explaining a multi-view display operation according to one embodiment of the present disclosure.

[0061] Referring to FIG. 1, the electronic device (100) can display a screen corresponding to an image. Here, the screen includes an image displayed on the display of the electronic device (100). The image may be referred to by terms such as a frame. Various types of objects, such as icons, text, photos, videos, widgets, etc., may be displayed on the screen.

[0062] Meanwhile, in the illustrated example, for ease of explanation, it is assumed that the electronic device (100) directly displays an image, but in the implementation, the electronic device may not have a display and may be a device that outputs an image to another device.

[0063] And the electronic device (100) can display a multi-view screen in which multiple images (101, 103) are displayed at once. Here, a multi-view screen is configured to allow viewing of two or more types of content simultaneously. Such a multi-view screen may be referred to as a multiple screen, a dual screen, etc.

[0064] And the content consists of images, audio, or a combination thereof and includes information that can be provided to a user visually, audibly, or audiovisually through an electronic device. For example, the content may include voice content, music content, image content, video content, widget content, web pages, etc.

[0065] And each content may be obtained through the same source (e.g., HDMI, internet network, broadcast, etc.) or through a different source. For example, the first video may be obtained through an HDMI terminal, and the second video may be obtained through a broadcast signal or a USB 3.1 terminal.

[0066] In the illustrated example, a multi-view screen is shown in which two images are arranged in a Picture-in-Picture (PIP) format. However, in implementation, the size of each image may be equal, and three or more images may be used. Here, PIP is a layout of a multi-view in which a sub-image is additionally placed within a main image.

[0067] Meanwhile, the source device (10) recently supports VRR (Variable Refresh Rate). This VRR is referred to as variable refresh rate, variable refresh rate, etc., and refers to a dynamic display method that can automatically and continuously change the refresh rate smoothly without the user changing the refresh rate.

[0068] These VRRs support a typical refresh rate range (e.g., 30Hz to 144Hz), and these refresh rates are referred to as the VRR range (or refresh rate range).

[0069] Refresh rate refers to the speed at which a display shows data for a single screen, and may also be called frame rate, refresh frequency, screen refresh rate, frame rate, frame rate, or refresh rate. For example, frames per second refers to the number of screens displayed per second, and the unit fps (or frames / second) can be used.

[0070] Therefore, a fixed refresh rate means that the playback frequency is constant during the video display process, while a variable refresh rate means that the playback frequency changes during the video display process.

[0071] VRR can change the refresh rate of the display in response to cases where the frame rate varies due to the complexity of rendering during the display process of game content. Accordingly, for video provided in the VRR method, the electronic device (100) matches the screen refresh rate with the frame provided by the source source, thereby enabling smooth screen output without tearing or stuttering.

[0072] Previously, multi-view operation could not be performed while this VRR function was in progress. This is because the other videos to be displayed together have different refresh rates, and it is difficult for electronic devices to process videos at two different refresh rates simultaneously.

[0073] Specifically, if an electronic device controls a display using a single timing controller, it was not possible to process two videos with different refresh rates together, given that the single timing controller must operate at a single refresh rate.

[0074] For example, in a display with a single timing controller, the refresh rate (or refresh rate) and resolution of each source cannot be adjusted independently even if an HDMI VRR signal is input. Consequently, conventional electronic devices (or display devices) have restricted the use of HDMI VRR signals in multi-view situations.

[0075] Meanwhile, in the case of a display using multiple timing controllers, each image had to correspond to the number of timing controllers, and the display area had to be limited to the area of ​​the timing controller.

[0076] That is, even if two timing controllers are provided, it could not be applied when two images (101, 103) need to be displayed in a PIP format as in FIG. 1. In addition, it could not be operated when the number of timing controllers is less than the number of images displayed together. For example, it could not be operated even when there are three images but two timing controllers.

[0077] To solve such problems, the electronic device (100) according to the present disclosure supports a method for displaying a video having a variable refresh rate and a video having a fixed refresh rate in a multi-view.

[0078] For example, the electronic device (100) obtains the current playback frequency of a first image (101) having a variable playback frequency, sets the playback frequency of a second image (103) to match the current playback frequency of the first image (101), and merges the two images to have the same playback frequency to create a multi-view screen. The specific configuration and operation of the electronic device (100) will be described later with reference to FIG. 2.

[0079] Meanwhile, although the illustrated example illustrates and explains the case of merging two screens, three or more images can be merged into a multi-view during implementation. In addition, the layout form being merged can be of various types, not just the PIP form.

[0080] As described above, the electronic device according to the present disclosure can generate a multi-view screen for an image having a variable refresh rate and an image having a fixed refresh rate, thereby improving the user experience.

[0081]

[0082] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0083] Referring to FIG. 2, the electronic device (100) may include an input / output interface (110), memory (120), and a processor (130).

[0084] The input / output interface (110) may be any one of the following interfaces: HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), and DVI (Digital Visual Interface).

[0085] The input / output interface (110) can input and output at least one of audio and video signals. Depending on the implementation example, the input / output interface (110) may include separate ports for inputting and outputting only audio signals and for inputting and outputting only video signals, or it may be implemented as a single port for inputting and outputting both audio and video signals.

[0086] The input / output interface (110) may include at least one port and may include multiple ports. If multiple ports are included, the input / output interface may receive different images from different ports.

[0087] And the input / output interface (110) can provide a video signal corresponding to a screen generated by the electronic device (100) or an audio signal together with the video signal to an external device (e.g., a display device, an STB, etc.). The screen generated here may be a multi-view screen.

[0088] For example, video can be input through one port and a multi-view screen can be output through another port. For example, the input / output interface (110) can receive a first video having a variable refresh rate through the first port, and through the second port, the first video and the second video are merged and a multi-view screen having a variable refresh rate can be output. At this time, some of the multiple ports may support the same interface method, and at least two ports may support different interface methods.

[0089] To this end, at least one port of the input / output interface (110) may support an interface method that supports a variable refresh rate. For example, an interface method that supports a variable refresh rate may be HDMI 2.1 or DisplayPort.

[0090] The memory (120) may store instructions, programs, or data for controlling the electronic device (100). Here, the instructions may include instructions for transmitting and receiving a source image with a variable refresh rate, instructions for creating a multi-view screen, instructions for varying the refresh rate of an image with a fixed refresh rate, etc.

[0091] For example, the memory (120) may be implemented as internal memory such as ROM (e.g., EEPROM (electrically erasable programmable read-only memory)) or RAM included in the processor (130), or as memory separate from the processor (130). In this case, the memory (120) may be implemented as memory embedded in the electronic device (100) or as memory that can be attached to the electronic device (100), depending on the purpose of data storage. For example, data for operating the electronic device (100) may be stored in memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in memory that can be attached to the electronic device (100).Meanwhile, the memory embedded in the electronic device (100) is implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD), and the memory that can be attached to the electronic device (100) can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (e.g., USB memory).

[0092] The memory (120) may temporarily store a first image obtained through the input / output interface (110) or store a multi-view screen generated in the process described later. Additionally, the memory (120) may store content corresponding to the second image or the second image.

[0093] And the memory (120) can store various contents (e.g., broadcast content, applications, etc.) received through the communication unit (150) described later.

[0094] Meanwhile, although the illustrated example shows the electronic device (100) being composed of a single memory, when distinguishing between volatile memory and non-volatile memory, the electronic device (100) may be described as including multiple memories.

[0095] The processor (130) controls the overall operation of the electronic device (100). Specifically, the processor (130) is connected to the configuration of the electronic device including memory (120) and can control the overall operation of the electronic device by executing at least one instruction stored in the memory (120) as described above. In particular, the processor (130) can be implemented as a single processor (130) as well as as a plurality of processors (130).

[0096] The processor (130) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing operations. The processor (130) may include at least one electrical circuit and may process instructions (or programs, data, etc.) stored in memory individually or collectively in a distributed manner.

[0097] The processor (130) may include a processor assembly comprising one or more processing circuits. The processor (130) may include any processing circuit that is operative to control the performance and operation of one or more components of an electronic device (e.g., memory and / or driving device (sensor)). For example, the processor (130) (e.g., AP) may be implemented as a system on chip (SoC) (e.g., a single chip or a chipset). For example, the processor (130) may be implemented as multiple cores (or at least one core circuit), multiple chips, or multiple chipsets.

[0098] For example, the processor (130) may include one or more processing circuits. The processor (130) may include one or more processing circuits configured to perform various functions of the present disclosure individually and / or collectively. As an example without limitation, at least a portion of the processor (130) may be included in a first chip of the electronic device (100), and at least another portion of the processor (130) may be included in a second chip of an electronic device different from the first chip of the electronic device (100).

[0099] For example, the processor (130) may include a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a display controller, a memory controller, a storage controller, a communication processor (CP), and / or a sensor interface. These components of the processor (130) are merely exemplary. The processor (130) may include additional components other than those described above. Additionally, some components of the processor (130) may be omitted. Furthermore, some components of the processor (130) may be included as separate components of the electronic device (100) outside of the processor (130). For example, some components of the processor (130) (e.g., a memory controller) may be included within other components (e.g., at least a portion of memory, an interface (e.g., available for connection to at least one component of the electronic device (100)), a display).

[0100] The processor (130) can cause other components of the electronic device (100) to perform various operations by executing instructions stored in memory.

[0101] The processor (130) processes setting values, function commands, etc. according to a stored control program or control data, and can output control signals related to functions that the electronic device can perform or communication signals for communicating with an external electronic device.

[0102] The processor (130) can acquire a first image using an input / output interface (110). The first image acquired at this time may be an image having a variable refresh rate. If a multi-view display command is received during the display or processing of such first image, the source of a second image to be included in the multi-view is selected, and the second image can be acquired based on the selection.

[0103] For example, the second image may be an image obtained through another port within the same input / output interface (110), content previously stored in memory (120), or an image obtained through the communication unit (150) described later.

[0104] Meanwhile, in the implementation, after receiving a multi-view display command, a first image (or source source) may be selected, and the processor (130) may determine whether the resolution image has a variable refresh rate or a fixed refresh rate based on the selected first image.

[0105] When a multi-view display command is input in this manner, the processor (130) can determine source sources belonging to the multi-view and determine a display layout for each source source. For example, if two screens need to be displayed, the processor (130) can determine whether to display the two screens equally or in a PIP format. In addition, even when a layout is determined, the processor (130) can determine whether to place the first image or the second image in which of the two areas.

[0106] Such a decision may be determined by the user's choice and may be performed based on the user's prior selection results or the previous usage pattern, etc. For example, if the user used the previous multi-view as PIP and selected the variable refresh rate video first, the first selected video can be used as the main video. Then, the video selected thereafter can be used as the sub-video. If such a selection process exists, the processor (130) may decide to place the first video on the background side of the PIP video and the second video on the picture area side.

[0107] And the processor (130) can check whether there is a video with a variable refresh rate among the sources constituting the multiview. For example, if at least one of the sources is obtained through an input / output interface, the processor (130) can check whether it has a variable refresh rate by using the enable signal (VRR_EN) in the VTEM (Video Timing Extended Metadata) packet.

[0108] If each source has a fixed playback frequency, the processor (130) can generate a multi-view screen through operations such as changing the resolution without a separate operation of synchronizing (or matching) the playback frequency.

[0109] Here, resolution is information about how many pixels a display screen contains, and can be expressed based on the number of horizontal and vertical pixels of the display. For example, the resolution can be the number of pixels (pixels) simultaneously displayed on the screen, calculated by multiplying the number of horizontal pixels and the number of vertical pixels.

[0110] For example, resolutions may include SD (Standard Definition), HD (High Definition), FHD (Full High Definition), QHD (Quad High Definition), 4K UHD (Ultra High Definition), 8K UHD (Ultra High Definition), or higher resolutions. For example, if the screen resolution is "1920×1080", there are 1,920 pixels horizontally and 1,080 pixels vertically, so a total of 2,073,600 pixels may be included on the screen.

[0111] And the processor (130) can check the frequency range (or range) of the video having a variable refresh rate and the refresh rate of the video having a fixed refresh rate. If the screen refresh rate of the second video is greater than the minimum refresh rate of the variable refresh rate, the processor (130) can change the minimum refresh rate of the first video's variable refresh rate. Conversely, if the minimum refresh rate is higher than the refresh rate of the second video, the change of the minimum refresh rate may not be performed.

[0112] If one of the multiple sources has a variable refresh rate, the processor (130) can perform an operation to change the refresh rate of the video having a fixed refresh rate. For example, the processor (130) can generate a multi-view screen in which the first video and the second video are displayed together by varying the screen refresh rate of the second video based on the current refresh rate corresponding to the variable refresh rate. Accordingly, the generated multi-view screen has the same refresh rate as the first video.

[0113] For such operation, the processor (130) can identify the current playback frequency of the first video of variable playback frequency. Specifically, the processor (130) can obtain the current playback frequency of the first video based on playback frequency information within a VTEM (Video Timing Extended Metadata) packet obtained through the input / output interface.

[0114] When the current playback frequency of the first video is confirmed, the processor (130) can synchronize the screen playback frequency of the second video by repeating the frames of the second video so that the screen playback frequency of the second video corresponds to the current screen playback frequency of the first video. Accordingly, the playback frequency of the multi-view screen may be the same as the playback frequency of the first video. Here, the synchronization of the playback frequency includes not only making the number of frames per second (or refresh rate) of the two videos the same, but also matching the Vsync in each frame.

[0115] Meanwhile, although it was described above that the frames of the second image are simply repeated, instead of simply repeating them, frames generated by performing frame interpolation may also be used.

[0116] And when each frame of the second video becomes equal to the number of frames of the first video as in this way, the processor (130) can adjust the Vfront (Vertical Front Porch) of the second video to synchronize the Vsync of the first video with the Vsync start time of the second video. That is, even if the number of frames of the two videos is the same, flicker may occur if the start times (Vsync) of the two videos are different. Therefore, the processor (130) can synchronize the two frames so that they have Vsync at the same time.

[0117] Here, Vsync stands for Vertical Synchronization, a signal indicating the start of a frame. Vfront is the interval between the end of the frame and the start of the Vertical Synchronization (Vsync) signal.

[0118] And the processor (130) determines the resolution of the multi-view screen and can scale at least one of the resolutions of the first image and the second image in correspondence with the determined multi-view resolution. Such operations will be described later with reference to FIG. 8 or FIG. 9.

[0119] Here, scaling may include an operation that transforms the resolution of an image. Upscaling is a method of increasing resolution by inserting new pixels between existing ones, and can be used to convert a low-resolution source into a high-resolution image. Such upscaling can be performed by taking into account the aspect ratio of the input image and the aspect ratio of the display to be shown.

[0120] Downscaling, conversely to upscaling, may include techniques that convert a high-resolution source to a lower resolution. For example, scaling may include AI scaling, which utilizes artificial intelligence (AI) technology, and non-AI scaling. Non-AI scaling may include Nearest Neighbor, Bilinear, Bicubic, Lanczos, Spline, NGU, etc.

[0121] Such scaling can be performed even when the resolutions of the first image, the second image, and the multi-view surface are the same. For example, since the size of the image displayed on the screen may change as the multi-view progresses, a scaling (or image editing) operation accordingly may be applied.

[0122] The processor (130) can perform various image processing. For example, it can perform tasks such as correcting the color tone of two images to be similar or correcting the brightness to be similar. Such tasks may be performed individually for each image, or they may be performed after the multi-view screen is created.

[0123] When a multi-view screen is generated, the processor (130) can control the input / output interface (110) to output the multi-view screen. Additionally, the processor (130) may generate a VTEM packet for the screen and transmit it to a display device. Meanwhile, since the playback frequency in the present disclosure is the same as the playback frequency of the first video, the VTEM of the first video may be transmitted to the source device as is.

[0124] Meanwhile, if the electronic device (100) is equipped with a display, it can display a multi-view screen. Meanwhile, when implementing, it is also possible to transmit the video to an external device and display it simultaneously on an internal display.

[0125] As described above, the electronic device according to the present disclosure can generate a multi-view screen for an image having a variable refresh rate and an image having a fixed refresh rate, thereby improving the user experience.

[0126] Meanwhile, although the operation of the electronic device (100) has been described above assuming that only one of the multiple source sources has a variable refresh rate, the operation of the present disclosure may also be performed when two images have a variable refresh rate. This will be described later with reference to FIG. 13.

[0127] Meanwhile, although only a simple configuration constituting the electronic device (100) has been illustrated and described above, various additional configurations may be provided during implementation. This will be explained below with reference to FIG. 3.

[0128]

[0129] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0130] Referring to FIG. 3, the electronic device (100) may include an input / output interface (110), memory (120), processor (130), display (140), communication unit (150), microphone (160) and speaker (170).

[0131] The configuration of the input / output interface (110), memory (120), and processor (130) was previously described in FIG. 2, and only the operation different from FIG. 2 will be described below.

[0132] The display (140) can be implemented as various types of displays such as LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diodes) display, PDP (Plasma Display Panel), Micro LED, etc. The display (140) may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as a-si TFT, LTPS (low temperature poly silicon) TFT, OTFT (organic TFT), etc. Meanwhile, the display (140) can be implemented as a touchscreen combined with a touch sensor, a flexible display, a 3D display, etc.

[0133] The display (140) can display various images. For example, the display (140) can display a first image, a second image, a multi-view, etc. generated by the processor (130).

[0134] Such a display (140) may include a timing controller for controlling multiple pixels. For example, the display (140) may include one timing controller, or it may include multiple timing controllers. The operation when the display (140) includes multiple timing controllers will be described later in FIGS. 12 and FIGS. 13.

[0135] The communication unit (150) is configured to communicate with various types of external devices according to various types of communication methods. The communication unit (150) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, and a wireless communication module, etc. Here, each communication module may include at least one hardware chip or hardware circuit.

[0136] Wi-Fi modules and Bluetooth modules can perform communication via Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information, such as SSID and session key, is transmitted and received first; after establishing a communication connection using this information, various information can be transmitted and received.

[0137] The infrared communication module performs communication according to infrared communication (IrDA, infrared Data Association) technology, which transmits data wirelessly over short distances using infrared rays located between visible light and millimeter waves.

[0138] In addition to the communication method described above, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

[0139] In addition, the communication unit (150) may include at least one wired communication module that performs communication using a LAN (Local Area Network) module, an Ethernet module, a pair cable, a coaxial cable, a fiber optic cable, or a UWB (Ultra Wide-Band) module.

[0140] According to one example, the communication unit (150) may use the same communication module (e.g., Wi-Fi module) to communicate with external devices such as a remote control and an external server.

[0141] According to other examples, the communication unit (150) may use different communication modules (e.g., Wi-Fi modules) to communicate with external devices such as a remote control and external servers. For example, the communication unit (150) may use at least one of an Ethernet module or a Wi-Fi module to communicate with an external server, and may use a BT module to communicate with an external device such as a remote control. However, this is merely one example, and the communication unit (150) may use at least one of various communication modules when communicating with multiple external devices or external servers.

[0142] The communication unit (150) can receive content or receive a video corresponding to the content. Such content can be various, such as movies, music videos, dramas, short videos, etc. And while it is assumed that the content is video, it may also be an image or be referred to as video.

[0143] Meanwhile, such a video may be a fixed refresh rate video or a variable refresh rate video. That is, although FIG. 2 describes receiving the first video, i.e., a video having a variable refresh rate, through the input / output interface (110), the variable refresh rate may be received through communication during implementation. For example, given that technology for wirelessly transmitting and receiving signals such as HDMI is applied recently, the first video may be acquired by the communication unit (150) when HDMI is transmitted wirelessly.

[0144] Meanwhile, the communication unit (150) can receive a broadcast signal and obtain a broadcast image based on the received broadcast signal. Here, the broadcast signal may be a signal according to the terrestrial digital multimedia broadcasting standard (DMB) and may be a signal according to the terrestrial ultra-high definition broadcasting standard.

[0145] And the communication unit (150) can receive information, etc., necessary for providing various applications and services of the electronic device (100) from an external device.

[0146] The microphone (160) can receive the user's voice when active. For example, the microphone (160) may be formed integrally on the upper side, front side, or side side of the electronic device (100').

[0147] The microphone (160) may include various configurations such as a microphone for collecting analog user voice, an amplifier circuit for amplifying the collected user voice, an A / D conversion circuit for sampling the amplified user voice and converting it into a digital signal, and a filter circuit for removing noise components from the converted digital signal.

[0148] When a user's voice is input through such a microphone (160), the processor (130) can check the content of the user's voice and perform an action corresponding to the content of the voice. For example, the content of the voice here may be a multi-view progress command or a selection of a source to be used for the multi-view.

[0149] Additionally, when implementing, the layout of the multiview may be selected by the user, and the processor (130) may change the display form based on the layout selected by the user. For example, while the first image is displayed on the left side of the multiview and the second image is displayed on the right side of the multiview, in response to a command from the user, the processor (130) may display the first image on the right side of the multiview and display the first image on the left side of the multiview.

[0150] Alternatively, while the first image is displayed on the left side of the multiview and the second image is displayed on the right side of the multiview, in response to a user's command, the processor (130) may use the first image as the main image and display the second image as a PIP sub-screen.

[0151] Meanwhile, although it has been described above that user voice is input through the microphone (160), the microphone may be provided in a remote control for controlling the electronic device (100'), and user voice input through the microphone provided in the remote control may be input to the electronic device (100') and processed through the communication unit (150) described above.

[0152] The electronic device (100') can operate not only based on the configuration or remote control provided in the electronic device (100'), but also according to the control command of the terminal device. For example, if the electronic device is a TV or a set-top box, recently, manufacturers provide applications for controlling the TV or set-top box. Such applications can provide a function that allows the terminal device to be used as a remote control for the electronic device.

[0153] Accordingly, when a user executes an application to control a TV or set-top box using a terminal device and inputs a voice command through the terminal device, the electronic device (100') can perform a voice recognition operation and a corresponding voice recognition result using the voice signal input through the terminal device.

[0154] The speaker (170) can output sound. Specifically, the speaker (170) may be a component that outputs various audio data processed at the input / output interface, as well as various notification sounds or voice messages.

[0155] The processor (130) can select a video among multiple videos to output sound and control the speaker (170) so that sound corresponding to the video is output. For example, when a multiview is executed, the processor (130) can control the speaker (170) so that sound corresponding to the main video among multiple videos is output.

[0156] And based on the user's sound switching command, the processor (130) can control the speaker (170) to output sound corresponding to the sub-video.

[0157] And the speaker (170) may also output result information corresponding to the voice recognition operation described later.

[0158] Meanwhile, although the electronic device (100) in FIG. 3 is illustrated and described as including a display (140), if the electronic device (100') is a device such as a set-top box that does not include a display, the display configuration may be omitted. Also, depending on the implementation form, the speaker and microphone described above may also be omitted. Additionally, although not illustrated in FIG. 3, other components (e.g., a camera) may be included.

[0159]

[0160] FIG. 4 is a flowchart illustrating a multi-view setting operation using an image having a variable refresh rate according to one embodiment of the present disclosure.

[0161] Referring to FIG. 4, when the user first selects the multi-view setting, the electronic device (100) can check whether one of the sources is HDMI (S410). Meanwhile, in the illustrated example, not only is it HDMI, but if the source supports a variable refresh rate, DisplayPort or other interface methods can also be applied in addition to HDMI.

[0162] And the electronic device (100) can check whether the HDMI source is set as the main source (S420). Meanwhile, in implementation, it can operate even if the HDMI source is set as a sub-source. Therefore, in implementation, checking whether the HDMI source is the main source may be omitted.

[0163] And the electronic device (100) can check whether the HDMI source has a variable refresh rate (S430). Specifically, the electronic device (100) can check whether it is operating at a variable refresh rate by checking VRR_En among the VTEM packets of the HDMI source. The VTEM packets will be described later in FIG. 5.

[0164] If the HDMI source has a variable refresh rate, the electronic device (100) can perform an operation to adjust the refresh rate of the sub-video as described above. If it does not have a variable refresh rate, a multi-view operation in the general case can be performed.

[0165] Meanwhile, if the HDMI source has a variable refresh rate, the electronic device (100) can check whether the refresh rate of the sub-source is greater than the minimum refresh rate of the main source (S440). Specifically, the electronic device (100) can obtain the maximum refresh rate and minimum refresh rate of the main source and check the minimum refresh rate and the refresh rate of the sub-source. Meanwhile, if there are multiple sub-sources, the electronic device (100) can compare the refresh rate with the highest among the multiple sub-sources with the minimum refresh rate.

[0166] If the playback frequency of the sub-source is greater than the minimum playback frequency (S440_Y), the electronic device (100) can update the minimum playback frequency of the main source (S460). For example, the electronic device (100) can communicate with the source source providing the main source to change the minimum playback frequency of the main source to the playback frequency of the sub-source. Meanwhile, if there are multiple sub-sources, the electronic device (100) can change the minimum playback frequency to the playback frequency of the one with the largest playback frequency among the multiple sub-sources.

[0167] If the playback frequency of the sub-source is equal to or smaller than the minimum playback frequency, the electronic device (100) can use the existing minimum playback frequency as is.

[0168] Meanwhile, although the process of updating the minimum playback frequency of the main source based on the playback frequency of the sub source has been described above, the playback frequency of the sub source may be adjusted during implementation. For example, if the playback frequency of the sub source is faster than the minimum playback frequency of the main source, the electronic device (100) may request the sub source to provide video at the minimum playback frequency of the main source, thereby receiving the sub source at the changed playback frequency.

[0169] Once such a setting is completed, the electronic device (100) can perform the operation of generating a multi-view based on the current playback frequency of the variable playback frequency.

[0170] Specifically, the electronic device can determine the current main video refresh rate by analyzing the VTEM packet (S470). Here, the VTEM packet may include information regarding the minimum refresh rate (or minimum refresh rate), the maximum refresh rate (or maximum refresh rate), and the current refresh rate (or current refresh rate). An example of a VTEM packet is described later in FIG. 5.

[0171] When the current playback frequency of the main video is confirmed, the electronic device (100) can repeat the frames of the sub-video to make them the same as the playback frequency of the main video (S480). In addition, the electronic device (100) can also perform synchronization with respect to the resolution of the video. Synchronization of the playback frequency is described in detail in FIG. 7, and synchronization of the resolution is described in detail in FIG. 8 and FIG. 9.

[0172] When a multi-view screen is created through such an operation, the created multi-view screen can be displayed or transmitted to another device (S490).

[0173]

[0174] Figure 5 is a diagram illustrating the structure of a VTEM (Video Timing Extended Metadata) packet.

[0175] VTEM is a feature introduced in the HDMI (High-Definition Multimedia Interface) 2.1 standard. Variable refresh rate can be controlled using this VTEM.

[0176] Referring to Fig. 5, VRR_En in the packet is bit information that distinguishes whether it is operating with a variable refresh rate. For example, if the value is 1, the variable refresh rate is applied, and if it is 0, it is not applied.

[0177] And Base_Vfront is the Vertical Front Porch (VFP), which is a blank period before the Vsync signal starts; during this period, no new image data is transmitted to the screen, and the display can use this time to prepare the next frame.

[0178] And Base_Refresh_Rate is the default refresh rate before starting the variable refresh rate, which can be used when the display sets the optimal refresh rate.

[0179]

[0180] FIG. 6 is a diagram illustrating the difference between an image with a variable playback frequency and an image with a fixed playback frequency according to one embodiment of the present disclosure.

[0181] Referring to FIG. 6, a first image and a second image are shown. Here, the first image is an image with a variable refresh rate, and in the illustrated example, the first image has a refresh rate range of 144Hz to 60Hz. This first image can be received through an HDMI port. For example, it can be seen that the first section has a refresh rate of 144Hz, then has a refresh rate of 120Hz, and then changes to a refresh rate of 60Hz.

[0182] The second image is an image with a fixed refresh rate, and in the illustrated example, the second image has a refresh rate of 60 Hz. Unlike the first image, it can be confirmed that an image with a fixed refresh rate has the same refresh rate across the entire area.

[0183] This second video may be received through an HDMI port different from the HDMI port receiving the first video, received through a USB 3.1 port, or obtained through an internet or broadcast signal.

[0184] In the illustrated example, the playback frequency of the second image is not greater than the minimum playback frequency of the first image; that is, the playback frequency of the second image and the minimum playback frequency are the same. Therefore, an operation to change the minimum playback frequency of the first image may not be performed. If the playback frequency of the second image is greater than the minimum playback frequency of the first image, the modification of the minimum playback frequency of the first image may be performed prior to it.

[0185] When the first video and the second video are input to the electronic device (100) in this manner, the playback frequency of the second video is 60Hz in the 144Hz and 120Hz intervals of the first video, so the two sources are not synchronized.

[0186] Accordingly, the electronic device (100) can create a multi-view screen having a playback frequency of 144Hz by changing the playback frequency of the second image from 60Hz to 144Hz in the 144Hz interval of the first image.

[0187] And the electronic device (100) can change the playback frequency of the second image from 60Hz to 120Hz in the 120Hz interval of the first image to create a multiview screen having a playback frequency of 120Hz (612).

[0188] Additionally, the electronic device (100) can create a multi-view screen with a 60Hz refresh rate without changing the refresh rate of the second image, since the refresh rates of the two images are the same in the 60Hz interval of the first image (611).

[0189] Below, the operation of generating a multi-view screen in the 120Hz interval of the first image will be explained in detail with reference to FIG. 7.

[0190]

[0191] FIG. 7 is a drawing for explaining a method of synchronizing playback frequency according to one embodiment of the present disclosure.

[0192] Referring to FIG. 7, a first image (710) and a second image (720) are displayed. Here, the first image is currently running at 120Hz, and the second image is running at a playback frequency slower than 120Hz.

[0193] In this way, when the first video and the second video have different playback frequencies, the electronic device (100) can repeat specific frames of the second video so that the second video (730) is synchronized with the first video (710). If the first video is currently 120Hz and the second video is operating at 60Hz, the electronic device can arrange each frame of the second video twice to have 120Hz.

[0194] Meanwhile, although Figure 7 illustrates and describes synchronization as simply repeating the second image, the above-described repetition may not be repeated, and frame interpolation may be applied to the frame placed in the middle.

[0195] Frame interpolation is the process of increasing the number of frames in a video or real-time rendered image. This significantly reduces phenomena such as screen shaking and allows for a more natural-looking image. For example, frame doubling, frame generation, and motion interpolation techniques can be utilized during frame interpolation.

[0196] And the electronic device can synchronize by varying the Vfront (Vertical Front Porch) value of the sub-source, that is, by making the start time of the main source's Vsync and the sub-source's Vsync the same.

[0197] Through such operation, the first image (710) and the second image (730) with a changed playback frequency can be synchronized at the time of operation, as shown in the right area of ​​FIG. 7.

[0198] The following describes the operation of changing the resolution of two images. Specifically, in a multi-view screen, instead of displaying each image on the screen as is, the size of one of the two images must be adjusted. This operation will be described later with reference to FIG. 8.

[0199] Meanwhile, although the above description illustrates and explains that the timing of the operation is adjusted and then the resolution is adjusted, in implementation, the operation of synchronizing the refresh rate may proceed after the resolution adjustment has been performed. Additionally, in implementation, it may be expressed that the resolution synchronization and the refresh rate (or refresh rate) synchronization proceed simultaneously during the multi-view creation process described above.

[0200]

[0201] FIG. 8 is a drawing for explaining the operation of generating a multiview in terms of resolution according to one embodiment of the present disclosure.

[0202] Referring to FIG. 8, the electronic device (100) includes an image processing device (830) and receives a first image (810) and a second image (820).

[0203] Here, the first video (810) may be a video having a current refresh rate of 120Hz and a 4K resolution. And the second video (820) may be a video having a refresh rate of 60Hz and a 2K resolution.

[0204] When such first video (810) and second video (820) are input, the video processing device (830) can generate a multi-view screen having 4K resolution and a refresh rate of 120Hz. In this way, the refresh rate of the multi-view screen can be maintained the same as the refresh rate of the first video, but the resolution of the multi-view may use the higher or lower resolution of the two videos, or a completely different resolution (e.g., a resolution selected by the user) may be used.

[0205] To this end, the image processing device (830) can repeat the second image to have a refresh rate of 120Hz as described in FIG. 7, and in the process, upscale the second image from 2K to 4K to correspond to the resolution of the multiview. In this way, if the first image and the second image have a refresh rate of 120Hz, they can be matched to a 4K resolution.

[0206] For example, when an HDMI VRR is input as a main source, the video processing device (830) can synchronize the resolution of the sub-source through HDMI infoframe information and check the current refresh rate of the HDMI VTEM packet to synchronize the refresh rate of the sub-source.

[0207] At this time, video and image quality processing can be performed by extending and decreasing the Vfront section based on the reference Hz of the main source.

[0208] Such a multi-view screen can be transmitted to the timing controller (145) of the display (140).

[0209] The timing controller (145) can provide a multi-view screen to each pixel at 120Hz. Specifically, the timing controller (145) can provide the received multi-view screen to the display panel using a VRR setting configured to match the characteristics of the display panel.

[0210] Accordingly, the multi-view screen (860) can display the first image (840) on the left and the second image (850) on the right.

[0211] Meanwhile, although the resolutions of the two screens were matched to the higher one in the above description, they may be matched to the lower one during implementation. This will be explained below with reference to FIG. 9.

[0212]

[0213]

[0214] FIG. 9 is a drawing for explaining the operation of generating a multiview in terms of resolution according to one embodiment of the present disclosure.

[0215] Referring to FIG. 9, the electronic device (100) includes an image processing device (930) and receives a first image (910) and a second image (920).

[0216] Here, the first video (910) may be a video having a current refresh rate of 120Hz and a 4K resolution. And the second video (920) may be a video having a refresh rate of 60Hz and a 2K resolution.

[0217] When such first image (910) and second image (920) are input, the image processing device (930) can generate a multi-view screen having a 2K resolution and a 120Hz refresh rate. In implementation, the resolution of the multi-view screen may not be 2K, but a resolution selected by the user may be used.

[0218] To this end, the image processing device (930) can repeat the second image to have a refresh rate of 120 Hz as described in FIG. 7, and also downscale the first image from 4K to 2K during the process. In this way, if the first image and the second image have a refresh rate of 120 Hz, they can be matched to a 2K resolution.

[0219] Such a multi-view screen can be transmitted to the timing controller (145) of the display (140). The timing controller (145) can provide the multi-view screen to each pixel at 120Hz. Accordingly, the multi-view screen (960) can display the first image (940) on the left and the second image (950) on the right.

[0220] As such, the multi-view screen generated in this disclosure has the same resolution and the same refresh rate, so a multi-view screen with a variable refresh rate can be displayed even when using a single TCON (Timing Controller). Furthermore, in a display device having multiple TCONs, a multi-view screen can be displayed even when the display area of ​​each TCON and the display area of ​​each screen in the multi-view are different, thereby improving the user experience.

[0221] Meanwhile, although an example of a multi-view screen configuration in which one of the two images is displayed on the left and the other on the right has been illustrated above, it is not limited to this when implemented. Other configuration examples will be described later with reference to FIGS. 10 and FIGS. 11.

[0222]

[0223] FIG. 10 is a drawing for explaining the arrangement structure of a multiview according to one embodiment of the present disclosure.

[0224] Referring to FIG. 10, the multi-view screen (1030) may include a main area (1010) and a sub-area (1020). This multi-view screen (1030) may have a PIP form as illustrated.

[0225] The main area (1010) is basically an area that uses the entire screen. For example, a first video having a variable refresh rate can be displayed in the main area (1010).

[0226] The sub-area (1020) is positioned in a portion of the lower right side of the main area, and the sub-image can be scaled to fit the size of the area. For example, a second image with a fixed playback frequency can be displayed in the sub-area (1020). In implementation, the sub-area (1020) may be positioned in the upper right, upper left, or lower left side, rather than the lower right side.

[0227] For example, when a user plays a game and films themselves for broadcasting purposes, they can receive video with a variable refresh rate from a PC or game console through an HDMI port. At this time, the video of themselves filming can be received through a USB 3.1 port and displayed in a sub-area (1020). When implemented, a broadcast screen transmitted from a broadcasting station may be displayed in the sub-area (1020), and a streaming video may also be displayed.

[0228] Meanwhile, during implementation, the main area may not only display the first image, but may also display the second image depending on the user's selection. For example, even if the first image is placed in the main area and the second image is placed in the sub area as in FIG. 10, the second image may be placed in the main area and the first image may be placed in the sub area according to the user's layout switching command. Additionally, based on the user's layout switching command, the layout may be changed from a PIP format to a layout where the two images are placed left and right as in FIG. 8 or FIG. 9.

[0229]

[0230] FIG. 11 is a drawing for explaining the arrangement structure of a multiview according to one embodiment of the present disclosure.

[0231] Referring to Fig. 11, the multiview screen can be divided into four areas (1110, 1120, 1130, 1140).

[0232] Among these four areas, the first area (1110) is the main area, and the second area (1120), third area (1130), and fourth area (1140) can be sub-areas. Accordingly, four images can be displayed simultaneously in FIG. 11. Accordingly, the first image displayed in the first area may be an image with a variable refresh rate, and the second to fourth images displayed in the second to fourth areas may be images with a fixed refresh rate.

[0233] In this way, when there are two or more sub-videos, the electronic device (100) can compare the minimum playback frequency of the first video with the playback frequency of the sub-videos to determine whether the playback frequency of at least one sub-video is greater than the minimum playback frequency. If the playback frequency of at least one sub-video is greater than the minimum playback frequency, the electronic device (100) can update the minimum playback frequency of the first video to the highest playback frequency among the sub-videos.

[0234] And the electronic device (100) can change the playback frequency of each video to match the current playback frequency of the first video. Meanwhile, if the playback frequencies of the sub-videos are all the same, each sub-video can be synchronized in the same pattern. If the playback frequencies of each sub-video are different, the synchronization method may differ for each sub-video, but the finally synchronized sub-videos can have the same playback frequency as the video having a variable playback frequency.

[0235] Meanwhile, in the illustrated example, the first area (1110) is referred to as the main area, but in implementation, other areas (1120, 1130, 1140) may be the main area. Also, in implementation, it may be divided into only three areas, or it may be divided into five or more areas.

[0236]

[0237] FIG. 12 is a drawing for explaining a multi-view display operation when a plurality of TCONs are provided according to one embodiment of the present disclosure.

[0238] Referring to FIG. 12, the display (140) may include a plurality of timing controllers (145-1, 145-2).

[0239] For example, the first timing controller (145-1) may be a timing controller that controls the left area of ​​the display, and the second timing controller (145-2) may be a timing controller that controls the right area of ​​the display.

[0240] In this case, since multiple timing controllers are provided, each timing controller can operate individually, the first image can be displayed at a first playback frequency using the first timing controller (145-1), and the second image can be displayed using the second timing controller (145-2). In this case, the two timing controllers can operate at the same playback frequency or at different playback frequencies. Therefore, the multi-view screen (1210) can be configured individually without modifying the playback frequency of each image.

[0241] However, this method of controlling each area of ​​the multi-view screen with individual timing controllers has limitations. In particular, since the control area of ​​the timing controller is physically fixed, it is not possible to add timing controllers or change the control area later.

[0242] Accordingly, as illustrated in FIG. 10, when the first image is to be displayed as the main image and the second image is to be displayed as a PIP, the first timing controller (145-1) can operate in accordance with the operating frequency of the first image, but the second timing controller (145-2) cannot operate in that it must display two images of different operating frequencies.

[0243] In addition, even when displaying three images or four images as in Fig. 11, operation is impossible in that at least one timing controller must display images operating at different frequencies.

[0244] Even when multiple timing controllers are provided in this manner, there are limitations on the configuration of the multi-view screen; however, as disclosed in the present disclosure, when the entire area of ​​the multi-view screen is synchronized with the same refresh rate and the same resolution, the multi-view screen can be displayed with various layouts and numbers regardless of the area of ​​the timing controller.

[0245] For example, if the first video has an operating range of 144 to 60 Hz and the second video has 60 Hz, and the current playback frequency of the first video is 120 Hz, the multi-view screen can have 120 Hz. Therefore, the first timing controller (145-1) can display the left area of ​​the multi-view screen at 120 Hz. And the second timing controller (145-2) can display the right area of ​​the multi-view screen at 120 Hz.

[0246]

[0247] FIG. 13 is a drawing for explaining a multi-view display operation when a plurality of TCONs are provided according to one embodiment of the present disclosure.

[0248] Referring to FIG. 13, the display (140) may include a plurality of timing controllers (145-1, 145-2).

[0249] For example, the first timing controller (145-1) may be a timing controller that controls the image for the left area of ​​the display, and the second timing controller (145-2) may be a timing controller that controls the image for the right area of ​​the display.

[0250] In such an arrangement, as shown in FIG. 10, a case is assumed where a first image is displayed as the main (1320) and a second image is displayed as a sub (1330) in a PIP-style multi-view screen (1310).

[0251] If the present disclosure is not applied, the first timing controller (145-1) displays only a portion of the first image and can operate in accordance with the playback frequency of the first image. However, the second timing controller (145-5) must display a portion of the first image and the second image together, but cannot operate because the playback frequencies of the two images are different.

[0252] However, as disclosed in the present disclosure, when the playback frequency and resolution of the two images are synchronized, the part of the first image to be displayed by the second timing controller (145-5) and the second image have the same playback frequency and the same resolution, so operation is possible.

[0253] For example, if the first video has an operating range of 144 to 60 Hz and the second video has 60 Hz, and the current playback frequency of the first video is 120 Hz, the multi-view screen can have 120 Hz. Therefore, the first timing controller (145-1) can display the left area of ​​the multi-view screen at 120 Hz. And the second timing controller (145-2) can display the right area of ​​the multi-view screen at 120 Hz.

[0254] Meanwhile, FIGS. 12 and 13 describe the operation in the case where multiple timing controllers are used, one having a variable refresh rate and the other having a fixed refresh rate. However, when implementing multiple timing controllers, it is possible to apply this to videos with two or more variable refresh rates.

[0255] For example, when displaying a screen such as Fig. 12, a first image with a variable refresh rate can be displayed using a first timing controller (145-1), and a second image with a variable refresh rate can be displayed using a second timing controller (145-2).

[0256] At this time, when a third video is added, the electronic device (100) can synchronize the third video with the playback frequency of either the first video or the second video. Accordingly, when the third video is synchronized with the first video, the first timing controller (145-1) displays a multi-view screen of the first video and the third video at the first playback frequency, and the second timing controller (145-2) displays the second video at the second playback frequency.

[0257] Additionally, when a fourth video is added, the fourth video is synchronized with the second video in terms of playback frequency, so that a multiview of the second video and the fourth video may be displayed at the second playback frequency in the second timing controller (145-2).

[0258]

[0259] FIG. 14 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0260] Referring to FIG. 14, when a multi-view display command is received, a first image having a variable refresh rate is obtained (1410). For example, during the display of an image having a variable refresh rate, a multi-view display command may be received, and another second image to be displayed together may be additionally selected.

[0261] Alternatively, conversely, during the display of a second video having a fixed refresh rate, a multi-view display command may be received from the user, and an additional video having a variable refresh rate may be selected. This first video may also be acquired through an interface method such as HDMI.

[0262] Then, a second video having a fixed playback frequency is obtained (1420). Here, the second video may be a video obtained through an HDMI port different from the port from which the first video described above was obtained, a video obtained through a USB 3.1 terminal, a streaming video obtained through an internet network, or a video stored in the electronic device or a video obtained through a broadcast network (i.e., a video obtained through a tuner).

[0263] When a multiview command is received for a first video having a variable refresh rate and a second video having a fixed refresh rate, the electronic device can check the refresh rate range of the first video. Here, the refresh rate range may be the maximum refresh rate and the minimum refresh rate. Such a refresh rate range can be determined through frame frequency information within the VTEM (Video Timing Extended Metadata) packet.

[0264] Meanwhile, when a first video is input to the electronic device through an HDMI terminal, the device may check in advance whether the first video is a variable refresh rate or not using VRR_en within the VTEM packet described above. That is, the operation of the present disclosure may be applied when it is confirmed that the first video operates at a variable refresh rate.

[0265] If the refresh rate of the second video is higher than the minimum value of the variable refresh rate confirmed through this process, a setting to adjust the variable range of the HDMI terminal can be performed. For example, if information that the first video operates in the range of 144Hz to 50Hz is obtained through a VTEM packet, and the second video operates at 60Hz, the electronic device can adjust the variable range to 144Hz to 60Hz through communication with the source source providing the first video.

[0266] When such adjustment is performed, the variable range of the first image can be changed to 144Hz to 60Hz.

[0267] In this way, when the minimum playback frequency is higher than that of the second video, the electronic device (100) can generate a multi-view screen using the first video and the second video. The generation of such a multi-view screen can be performed by changing the playback frequency and resolution.

[0268] First, the electronic device can determine the current playback frequency of the first image based on the frequency information within the VTEM packet described above, and can modify the frequency of the second image to correspond to the determined frequency. That is, a multi-view screen can be generated such that the playback frequency of the generated multi-view screen corresponds to the playback frequency of the first image.

[0269] For example, if the refresh rate of the first video changes as 144Hz -> 120Hz -> 60Hz, the refresh rate of the multi-view screen also changes as 144Hz -> 120Hz -> 60Hz, and the refresh rate of the second video can be adjusted to correspond to the current refresh rate of the first video by repeating the frames of the second video.

[0270] At this time, the electronic device can adjust the Vfront (Vertical Front Porch) of the second image to synchronize the start time of the Vsync of the first image and the Vsync of the second image. Through this process, flicker can be prevented from occurring in the multi-view screen.

[0271] Meanwhile, in the multi-view screen generation process described above, the electronic device may perform synchronization regarding resolution as well as frequency. Specifically, the electronic device determines the resolution of the multi-view screen and can scale at least one of the resolutions of the first image and the second image in correspondence with the determined multi-view resolution.

[0272] Meanwhile, if the electronic device includes a display, the multi-view screen generated on the display can be provided to display the multi-view screen. In this case, the display may include only one timing controller. Even if images operating at different frequencies are displayed as a multi-view in this manner, the electronic device processes the two images to have a single frequency during the process, so multi-view display is possible even with a display device equipped with a single timing controller.

[0273] In addition, the display described above may be equipped with a plurality of timing controllers. When a plurality of timing controllers are provided in this manner, one timing controller may display a first image, and another timing controller may display a second image. However, in this case, the images can only be displayed at the same ratio, and this cannot be applied to displaying in a PIP mode or in a multi-view mode that displays three or more screens simultaneously.

[0274] However, when matching the operating frequencies of the two images in the manner described above, normal operation is possible even when the two images are displayed in a PIP format or when three or more images are displayed at once, that is, when the two images are arranged in a vertical direction and a single timing controller must use both the first image and the second image.

[0275] Meanwhile, if the electronic device does not include a display, the multi-view screen may be output to a port other than the port receiving the first image, or the first image may be received via a wired method and the multi-view screen provided to a receiver device or a display device via a wireless method.

[0276] As described above, the control method according to the present disclosure enables normal operation even when an image corresponding to at least one screen constituting a multiview has a variable refresh rate while displaying a multiview that displays a plurality of images.

[0277] Meanwhile, methods according to at least some of the various embodiments of the present disclosure described above may be implemented in the form of an application that can be installed on an existing electronic device.

[0278] In addition, methods according to at least some of the various embodiments of the present disclosure described above may be implemented by software upgrades or hardware upgrades alone for existing electronic devices.

[0279] In addition, methods according to at least some of the various embodiments of the present disclosure described above may also be performed through an embedded server equipped in an electronic device, or through at least one external server among the electronic devices.

[0280] Meanwhile, according to one embodiment of the present disclosure, the various embodiments described above may be implemented as software containing instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory storage medium" simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium. For example, A 'non-transient storage medium' may include a buffer in which data is temporarily stored. According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store).TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., terminal devices). For online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0281] Various embodiments of the present disclosure may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., an electronic device (100)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions.

[0282] When the above-described instruction is executed by a processor, the processor may perform the function corresponding to the above-described instruction directly or by using other components under the control of the above-described processor. The instruction may include code generated or executed by a compiler or an interpreter.

[0283] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, Input / Output Interface; Memory for storing at least one instruction; and An electronic device comprising at least one processor configured to execute at least one instruction to acquire a first image having a variable refresh rate through the input / output interface, acquire a second image having a fixed refresh rate, and change the fixed refresh rate of the second image to the current refresh rate of the first image to generate a multiview screen in which the first image and the second image are displayed together.

2. In Paragraph 1, The above at least one processor is, An electronic device configured to execute at least one instruction to change the minimum playback frequency of the variable playback frequency of the first image when the fixed playback frequency of the second image is greater than the minimum playback frequency of the variable playback frequency of the first image.

3. In Paragraph 1, The above at least one processor is, An electronic device configured to execute at least one instruction to generate the multiview screen such that the playback frequency of the multiview screen is changed to the current playback frequency of the first video.

4. In Paragraph 1, The above at least one processor is, An electronic device configured to execute at least one instruction to synchronize the fixed playback frequency of the second image by repeatedly arranging frames of the second image so that the fixed playback frequency of the second image changes to the current screen playback frequency of the first image.

5. In Paragraph 1, The above at least one processor is, An electrical device configured to execute at least one instruction that synchronizes the Vsync of the first image with the Vsync start time of the second image by adjusting the Vfront (Vertical Front Porch) of the second image.

6. In Paragraph 1, The above at least one processor is, An electronic device configured to determine the resolution of the multi-view screen and to execute at least one instruction to scale at least one of the resolution of the first image and the resolution of the second image in correspondence with the determined resolution of the multi-view.

7. In Paragraph 1, It further includes a display that displays video using a single timing controller, The above at least one processor is, An electronic device configured to execute at least one instruction that controls the timing controller to display the multi-view screen.

8. In Paragraph 1, A display that displays images using multiple timing controllers; further comprising The above at least one processor is, An electronic device configured to execute at least one instruction that varies the screen refresh rate of the second image when at least one of the plurality of timing controllers processes the first image and the second image together.

9. In Paragraph 8, The above at least one processor is, An electronic device configured to execute at least one instruction that generates a multi-view in the form of a Picture in Picture (PIP) of the first and second images.

10. In Paragraph 1, The above at least one processor is, An electronic device configured to execute at least one instruction for obtaining the current playback frequency of the first image based on playback frequency information within a VTEM (Video Timing Extended Metadata) packet obtained through the input / output interface.

11. In a method for controlling an electronic device, A step of acquiring a first image having a variable refresh rate; A step of acquiring a second image having a fixed playback frequency; and A control method comprising the step of generating a multi-view screen in which the first image and the second image are displayed together by changing the fixed playback frequency of the second image to the current playback frequency of the first image.

12. In Paragraph 11, The step of generating the above multi-view screen is, A control method for generating a multiview screen such that the playback frequency of the multiview screen is changed to the current playback frequency of the first video.

13. In Paragraph 11, The step of generating the above multi-view screen is, A control method for synchronizing the fixed playback frequency of the second image by repeatedly arranging frames of the second image so that the screen playback frequency of the second image changes to the current screen playback frequency of the first image.

14. In Paragraph 11, The step of generating the above multi-view screen is, A control method for synchronizing the Vsync of the first image with the Vsync start time of the second image by adjusting the Vfront (Vertical Front Porch) of the second image.

15. A non-transient computer-readable recording medium storing a program for executing a method of controlling an electronic device, The above control method is in the electronic device, A step of acquiring a first image having a variable refresh rate; A step of acquiring a second image having a fixed playback frequency; and A computer-readable recording medium comprising the step of generating a multi-view screen in which the first image and the second image are displayed together by changing the fixed playback frequency of the second image to the current playback frequency of the first image.

Images