Electronic device for casting portion of content to adjacent display device and control method therefor

Abstract

An electronic device is disclosed. The electronic device may comprise: a memory for storing one or more instructions; a communication interface; a display; and one or more processors connected to the memory, the communication interface, and the display so as to control the electronic device, wherein the one or more processors execute the one or more instructions so as to: control the display to display multiple pieces of content streamed from multiple other electronic devices respectively through the communication interface; when a first user command for changing a display device of one piece of content among the multiple pieces of content is received, control the display to display a screen including information on multiple display devices; when a second user command for selecting one display device among the multiple display devices is received, control the communication interface to transmit, to the one display device, information on another electronic device that provides the one piece of content; and control the display to display content remaining after excluding the one piece of content among the multiple pieces of content.

Description

Electronic device for casting some content to an adjacent display device and method for controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device and a method for controlling the same for casting some content to an adjacent display device.

[0002] Advances in electronic technology have led to the development of a diverse range of electronic devices. In particular, recent developments have seen the development of various devices equipped with displays, leading to the deployment of numerous display devices in homes and offices.

[0003] Ways to utilize multiple displays include screen mirroring, casting, and multi-monitor.

[0004] Screen mirroring and casting are intended for watching videos on a large screen such as a TV rather than the small screen of the main device, while multi-monitoring is intended for expanding the screen by connecting multiple screens to one high-performance device.

[0005] However, screen mirroring is a method of copying the screen of the main device as it is, so it is not very useful during video calls and there may be a delay in the process of playing the screen received from the main device.

[0006] Casting casts the video being played on the main device to the target display device, and the main device is limited to a simple controller role. Therefore, in terms of video processing, the system resource utilization of the main device is very low, and it has limitations in that it is used for viewing a single video on the target display device rather than as an extension of the screen.

[0007] Multi-monitoring is a method of using two or more displays by connecting them to a single device, but there is a problem that the device's performance deteriorates when two or more displays are used with a low-spec device.

[0008] According to one embodiment of the present disclosure for achieving the above object, an electronic device includes a memory storing one or more commands, a communication interface, a display, and one or more processors connected to the memory, the communication interface, and the display to control the electronic device, wherein the one or more processors control the display to display a plurality of contents each streamed from a plurality of other electronic devices through the communication interface by executing the one or more commands, and when a first user command for changing a display device of one of the plurality of contents is received, control the display to display a screen including information about the plurality of display devices, and when a second user command for selecting one of the plurality of display devices is received, control the communication interface to transmit information about the other electronic device providing the one content to the one display device, and control the display to display the remaining contents excluding the one content among the plurality of contents.

[0009] In addition, when the first user command is received, the processor can identify whether a plurality of candidate display devices registered to the same account as the electronic device are turned on, control the communication interface to request performance information from the plurality of display devices that are turned on among the plurality of candidate display devices, and when performance information is received from each of the plurality of display devices through the communication interface, control the display to display a screen including information about the plurality of display devices based on the performance information.

[0010] And, the second user command includes a user command for selecting a first display device and a second display device among the plurality of display devices, and when the second user command is received, the processor can control the communication interface to transmit information about another electronic device that provides the one content to a high-performance display device among the first display device and the second display device based on the performance information.

[0011] Additionally, the processor can control the display to display a screen listing the plurality of display devices in order of performance based on the performance information.

[0012] In addition, the performance information may include at least one of whether each of the plurality of display devices supports a hardware codec, CPU (central processing unit) performance, RAM capacity, or GPU (graphics processing unit) support.

[0013] In addition, the processor controls the communication interface to transmit information about another electronic device providing the one content to the one display device, and when the other electronic device providing the one content streams the one content to the one display device, the processor can control the display to display the remaining content excluding the one content among the plurality of contents when the other electronic device providing the one content does not stream the one content to the electronic device.

[0014] And, the processor can control the display to change the display area of ​​the remaining contents excluding the one content among the plurality of contents.

[0015] In addition, the processor may control the display to downscale at least one content among the plurality of contents, to display the plurality of contents with the at least one content downscaled, to control the display to display the screen when the first user command is received, to control the communication interface to transmit information about another electronic device providing the one content to the one display device when the second user command is received, and to control the display to display the remaining contents excluding the one content among the plurality of contents without the downscaling.

[0016] And, when the electronic device is moved from the first location to the second location, the processor can control the display to update the plurality of display devices by adding a fourth display device to the plurality of display devices, excluding a third display device from the plurality of display devices based on the second location, and display a screen including information about the updated plurality of display devices.

[0017] Additionally, the processor can control the display to display the screen based on at least one of the hardware specifications or resources of the electronic device.

[0018] Meanwhile, according to one embodiment of the present disclosure, a method for controlling an electronic device may include a step of displaying a plurality of contents each streamed from a plurality of other electronic devices, a step of displaying a screen including information on a plurality of display devices when a first user command for changing a display device of one of the plurality of contents is received, a step of transmitting information on another electronic device providing the one content to the one display device when a second user command for selecting the one display device of the plurality of contents is received, and a step of displaying the remaining content excluding the one content among the plurality of contents.

[0019] In addition, the step of displaying the screen may include, when the first user command is received, identifying whether a plurality of candidate display devices registered to the same account as the electronic device are turned on, requesting performance information from the plurality of turned-on display devices among the plurality of candidate display devices, and, when performance information is received from each of the plurality of display devices, displaying a screen including information about the plurality of display devices based on the performance information.

[0020] And, the second user command includes a user command for selecting a first display device and a second display device among the plurality of display devices, and the step of transmitting to the one display device may, when the second user command is received, transmit information about another electronic device that provides the one content based on the performance information to a high-performance display device among the first display device and the second display device.

[0021] Additionally, the step of displaying the screen may display a screen listing the plurality of display devices in order of performance based on the performance information.

[0022] In addition, the performance information may include at least one of whether each of the plurality of display devices supports a hardware codec, CPU (central processing unit) performance, RAM capacity, or GPU (graphics processing unit) support.

[0023] In addition, the step of displaying the remaining content may be such that, when another electronic device providing the one content streams the one content to the one display device, and the other electronic device providing the one content does not stream the one content to the electronic device, the remaining content excluding the one content among the plurality of contents may be displayed.

[0024] And, the step of displaying the remaining content may change the display area of ​​the remaining content excluding the one content among the plurality of contents.

[0025] In addition, the step of displaying the plurality of contents may include downscaling at least one content among the plurality of contents, displaying the plurality of contents with the at least one content downscaled, and the step of displaying the remaining contents may display the remaining contents excluding the one content among the plurality of contents without the downscaling.

[0026] And, when the electronic device is moved from the first location to the second location, the method may further include a step of updating the plurality of display devices by adding a fourth display device to the plurality of display devices, excluding a third display device among the plurality of display devices based on the second location, and a step of displaying a screen including information about the updated plurality of display devices.

[0027] Additionally, the step of displaying the screen may display the screen based on at least one of the hardware specifications or resources of the electronic device.

[0028] FIG. 1 is a block diagram showing the configuration of an electronic system according to one embodiment of the present disclosure.

[0029] FIG. 2 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure.

[0030] FIG. 3 is a block diagram showing a detailed configuration of an electronic device according to an embodiment of the present disclosure.

[0031] FIG. 4 is a drawing for explaining screen expansion to multiple display devices in one embodiment of the present disclosure.

[0032] FIG. 5 is a drawing for explaining the operation of an electronic device during a screen expansion process in one embodiment of the present disclosure.

[0033] FIG. 6 and FIG. 7 are drawings for explaining the display of the UI provided in the screen expansion process and the result thereof in one embodiment of the present disclosure.

[0034] FIG. 8 is a flowchart illustrating priorities among multiple display devices in one embodiment of the present disclosure.

[0035] FIG. 9 is a flowchart illustrating the operation of an electronic device for screen expansion in one embodiment of the present disclosure.

[0036] FIG. 10 is a flowchart illustrating a method for controlling an electronic device according to an embodiment of the present disclosure.

[0037] The purpose of the present disclosure is to provide an electronic device and a control method thereof that secures resources of an electronic device through screen expansion and distributes images while taking into account the performance of a display device.

[0038] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0039] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0040] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0041] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0042] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0043] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0044] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0045] Various embodiments of the present disclosure will be described in more detail with reference to the attached drawings below.

[0046] FIG. 1 is a block diagram illustrating a configuration of an electronic system (1000) according to an embodiment of the present disclosure. As illustrated in FIG. 1, the electronic system (1000) includes an electronic device (100), a plurality of other electronic devices (200), and a plurality of display devices (300).

[0047] The electronic device (100) may be a device that displays multiple contents. For example, the electronic device (100) may be a device equipped with a display, such as a smartphone, tablet PC, digital camera, TV, desktop PC, or laptop, and display multiple contents through the display. Alternatively, the electronic device (100) may be a device that does not have a display, such as a set-top box (STB), but provides screen information including multiple contents through a device equipped with a display. In this case, the electronic device (100) may decode multiple contents and provide screen information including the decoded multiple contents to a device equipped with a display. However, the present invention is not limited thereto, and the electronic device (100) may be any device capable of directly or indirectly displaying multiple contents. For convenience of explanation, the electronic device (100) will be described below as having a display and directly displaying multiple contents through the display.

[0048] An electronic device (100) can receive a plurality of streams of content from a plurality of other electronic devices (200) and display the plurality of content through a display. The electronic device (100) can control at least one display device among the plurality of display devices (300) to provide at least one of the displayed content.

[0049] Each of the plurality of other electronic devices (200) may be a device that streams content to the electronic device (100). For example, each of the plurality of other electronic devices (200) may be a smartphone, a tablet PC, a digital camera, a TV, a desktop PC, a laptop, a server, etc. However, the present invention is not limited thereto, and each of the plurality of other electronic devices (200) may be any device that can stream content to the electronic device (100).

[0050] Each of the plurality of display devices (300) may be a device equipped with a display, such as a smartphone, tablet PC, digital camera, TV, desktop PC, or laptop, and capable of displaying content through the display. However, the present invention is not limited thereto, and each of the plurality of display devices (300) may be any device capable of displaying content.

[0051] The plurality of display devices (300) may be devices connected to the same network as the electronic device (100). Alternatively, the plurality of display devices (300) may be devices registered on the server based on the same ID and password as the electronic device (100). Alternatively, the plurality of display devices (300) may be devices positioned within a preset distance from the electronic device (100). Alternatively, the plurality of display devices (300) may be devices discovered according to the broadcasting operation of an advertising packet of the electronic device (100).

[0052] At least one of the plurality of display devices (300) may be a device that displays one of the plurality of contents being displayed by the electronic device (100) in response to a signal received from the electronic device (100). For example, a first display device among the plurality of display devices (300) may receive content streamed from another electronic device that provides one of the plurality of contents being displayed by the electronic device (100) in response to a signal received from the electronic device (100), and display the streamed content.

[0053] FIG. 2 is a block diagram showing the configuration of an electronic device (100) according to one embodiment of the present disclosure.

[0054] According to FIG. 2, the electronic device (100) may include a memory (110), a communication interface (120), a display (130), and a processor (140). However, the present invention is not limited thereto, and the electronic device (100) may be implemented in a form in which some components are excluded.

[0055] Memory (110) may refer to hardware that stores information such as data in an electrical or magnetic form so that a processor (140) or the like can access it. To this end, memory (110) may be implemented as at least one piece of hardware from among non-volatile memory, volatile memory, flash memory, hard disk drive (HDD), solid state drive (SSD), RAM, ROM, etc.

[0056] At least one instruction required for the operation of the electronic device (100) or processor (140) may be stored in the memory (110). Here, the instruction is a code unit that instructs the operation of the electronic device (100) or processor (140), and may be written in machine language, which is a language that a computer can understand.

[0057] The memory (110) may store data in the form of bits or bytes that can represent characters, numbers, images, etc. For example, information about multiple display devices (300) may be stored in the memory (110).

[0058] The memory (110) is accessed by the processor (140), and reading / writing / modifying / deleting / updating instructions, instruction sets, or data can be performed by the processor (140).

[0059] The communication interface (120) is a configuration that performs communication with various types of external devices according to various types of communication methods. For example, the electronic device (100) can perform communication with multiple other electronic devices (200), multiple display devices (300), access points, servers, etc., through the communication interface (120).

[0060] The communication interface (120) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, etc. Here, each communication module may be implemented in the form of at least one hardware chip.

[0061] Wi-Fi and Bluetooth modules communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, connection information, such as the SSID and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received. Infrared communication modules use infrared data association (IrDA) technology, which wirelessly transmits data over short distances using infrared light, which lies between visible light and millimeter waves.

[0062] In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0063] Alternatively, the communication interface (120) may include a wired communication interface such as HDMI, DP, Thunderbolt, USB, RGB, D-SUB, DVI, etc.

[0064] In addition, the communication interface (120) may include at least one of a LAN (Local Area Network) module, an Ethernet module, or a wired communication module that performs communication using a pair cable, a coaxial cable, or an optical fiber cable.

[0065] The display (130) is a component that displays content and can be implemented as a variety of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, a PDP (Plasma Display Panel), etc. The display (130) may also include a driving circuit, a backlight unit, etc. that can be implemented as a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (130) may be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, etc.

[0066] The processor (140) controls the overall operation of the electronic device (100). Specifically, the processor (140) is connected to each component of the electronic device (100) and can control the overall operation of the electronic device (100). For example, the processor (140) is connected to components such as a memory (110), a communication interface (120), a display (130), and the like and can control the operation of the electronic device (100).

[0067] The processor (140) may be implemented with one or more processors. In this case, the one or more processors may include one or more of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), a DSP (Digital Signal Processor), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of other components of the electronic device (100) and perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in the memory (110). For example, the one or more processors may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in the memory (110).

[0068] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an AI-specific processor). For example, a process of quantizing a neural network model according to an embodiment of the present disclosure may be performed by a general-purpose processor, and a process of learning or inferring the quantized neural network model may be performed by an AI-specific processor.

[0069] One or more processors may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When one or more processors are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory, such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0070] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0071] In embodiments of the present disclosure, one or more processors may refer to a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. However, for convenience of explanation, the operation of the electronic device (100) is described below using the expression processor (140).

[0072] The processor (140) can control the display (130) to display multiple contents streamed from multiple other electronic devices (200) via the communication interface (120). For example, the processor (140) can control the display (130) to display four contents streamed from four other electronic devices (200) via the communication interface (120). Here, the processor (140) can control the display (130) to divide the screen into four parts and display four contents in each of the four areas. However, the present invention is not limited thereto, and the processor (140) can display multiple contents in any number of different ways.

[0073] When a first user command for changing the display device of one of the plurality of contents is received, the processor (140) can control the display (130) to display a screen including the plurality of display devices (300). For example, when a first user command for changing the display device of the first content is received while displaying four contents, the processor (140) can control the display (130) to display a screen for selecting the plurality of display devices (300). For example, the screen can include identification information for the plurality of display devices (300) and a UI for selecting at least one of the plurality of display devices (300). Here, information for the plurality of display devices (300) can be pre-stored in the memory (110).

[0074] Alternatively, the memory (110) may have information on a plurality of candidate display devices registered to the same account as the electronic device (100) pre-stored. In this case, when a first user command is received, the processor (140) identifies whether a plurality of candidate display devices registered to the same account as the electronic device (100) are turned on, controls the communication interface (120) to request performance information from a plurality of display devices (300) that are turned on among the plurality of candidate display devices, and when performance information is received from each of the plurality of display devices (300) through the communication interface (120), controls the display (130) to display a screen including the plurality of display devices (300) based on the performance information.

[0075] However, the present invention is not limited thereto, and the memory (110) may have information on a plurality of candidate display devices connected to the same network as the electronic device (100) pre-stored. Alternatively, the memory (110) may have information on a plurality of candidate display devices positioned within a preset distance from the electronic device (100) pre-stored.

[0076] In addition, the processor (140) further identifies whether a plurality of candidate display devices are turned on, as well as whether a preset application is installed, whether a preset application is supported, etc., and controls the communication interface (120) to request performance information from a plurality of display devices (300) that are turned on among the plurality of candidate display devices and have a preset application installed and supported. When performance information is received from each of the plurality of display devices (300) through the communication interface (120), the processor (140) may control the display (130) to display a screen including the plurality of display devices (300) based on the performance information.

[0077] The processor (140) can control the display (130) to display a screen listing a plurality of display devices (300) in order of performance based on performance information. Here, the performance information can include at least one of whether each of the plurality of display devices (300) supports a hardware codec, CPU (central processing unit) performance, RAM capacity, or GPU (graphics processing unit) support.

[0078] However, the present invention is not limited thereto, and the processor (140) may also control the display (130) to display a screen listing multiple display devices (300) based on at least one of the positions, screen sizes, or user preferences of the multiple display devices (300).

[0079] When a second user command for selecting one display device among a plurality of display devices (300) is received, the processor (140) controls the communication interface (120) to transmit information about another electronic device providing one content to one display device, and controls the display (130) to display the remaining content except for one content among the plurality of contents.

[0080] For example, the processor (140) controls the communication interface (120) to transmit information about another electronic device providing one content to one display device, and when the other electronic device providing one content streams one content to one display device, and when the other electronic device providing one content does not stream one content to the electronic device (100), the processor (140) can control the display (130) to display the remaining content except one content among the plurality of contents. That is, one display device requests streaming of one content to the other electronic device providing one content based on the information about the other electronic device providing one content, and the other electronic device providing one content can stream the content that was being streamed to the electronic device (100) to one display device.

[0081] The processor (140) can control the display (130) to display a screen based on at least one of the hardware specifications or resources of the electronic device (100).

[0082] For example, the processor (140) may control the display (130) to downscale at least one content among a plurality of contents, to display the plurality of contents with at least one content downscaled, to control the display (130) to display a screen when a first user command is received, to control the communication interface (120) to transmit information about another electronic device providing one content to one display device when a second user command is received, and to control the display (130) to display the remaining content except for one content among the plurality of contents without downscaling.

[0083] However, it is not limited thereto, and even when displaying the remaining contents, the processor (140) may downscale at least one of the remaining contents and then display it based on at least one of the hardware specifications or resources.

[0084] Additionally, the processor (140) may control the display (130) to display a screen based on at least one of the hardware specifications or resources of the electronic device (100) even if the first user command is not received. For example, the processor (140) may control the display (130) to display a screen when the available resources of the electronic device (100) fall below a threshold value even if the first user command is not received. Alternatively, the processor (140) may control the display (130) to display a screen based on the number of multiple contents even if the first user command is not received.

[0085] The processor (140) can control the display (130) to change the display area of ​​the remaining content except for one of the plurality of contents. For example, when the processor (140) displays four contents but then changes to display two contents, the processor (140) can control the display (130) to divide the screen into two parts and display two contents in each of the two areas.

[0086] However, the present invention is not limited thereto, and the processor (140) may divide the screen into a plurality of sub-areas based on the remaining content, and control the display (130) to display the remaining content in each of the plurality of sub-areas. For example, the processor (140) may divide the screen into a first sub-area and a second sub-area having a different size from the first sub-area based on the resolution of the remaining content, and display the remaining content in the first sub-area and the second sub-area.

[0087] Meanwhile, although it has been described above that one display device among the plurality of display devices (300) is selected according to the second user command, it is not limited thereto. For example, the second user command includes a user command for selecting the first display device and the second display device among the plurality of display devices (300), and when the second user command is received, the processor (140) may control the communication interface (120) to transmit information about another electronic device that provides one content based on performance information to a high-performance display device among the first display device and the second display device.

[0088] Alternatively, the first user command includes a user command for changing display devices of the first content and the second content among a plurality of contents, and the second user command includes a user command for selecting the first display device and the second display device among the plurality of display devices (300), and the processor (140) may control the communication interface (120) to transmit information about another electronic device providing content with a relatively larger capacity to a high-performance display device among the first display device and the second display device based on capacity information of the first content and the second content and performance information of the first display device and the second display device, and to transmit information about another electronic device providing content with a relatively smaller capacity to a low-performance display device among the first display device and the second display device when the first user command and the second user command are received.

[0089] Meanwhile, when the electronic device (100) is moved from the first position to the second position, the processor (140) can control the display (130) to update the plurality of display devices (300) by adding a fourth display device to the plurality of display devices (300) excluding the third display device among the plurality of display devices (300) based on the second position, and to display a screen including the updated plurality of display devices (300).

[0090] For example, when the electronic device (100) is moved from a first location to a second location, the processor (140) may exclude a third display device that is located farthest from the electronic device (100) among the plurality of display devices (300) based on the second location. In addition, the processor (140) may control the display (130) to update the plurality of display devices (300) by adding a fourth display device that is located within a preset distance from the second location to the plurality of display devices (300), and to display a screen including the updated plurality of display devices (300).

[0091] FIG. 3 is a block diagram showing a detailed configuration of an electronic device (100) according to one embodiment of the present disclosure.

[0092] The electronic device (100) may include a memory (110), a communication interface (120), a display (130), and a processor (140). In addition, according to FIG. 3, the electronic device (100) may further include a user interface (150), a camera (160), a microphone (170), and a speaker (180). Among the components illustrated in FIG. 3, a detailed description of the overlapping parts with the components illustrated in FIG. 2 will be omitted.

[0093] The user interface (150) may be implemented with buttons, a touch pad, a mouse, a keyboard, etc., or may be implemented with a touch screen capable of performing both display and operation input functions. Here, the buttons may be various types of buttons, such as mechanical buttons, touch pads, wheels, etc., formed on any area of ​​the front, side, or back of the main body of the electronic device (100).

[0094] The camera (160) is configured to capture still images or moving images. The camera (160) can capture still images at a specific point in time, but can also capture still images continuously.

[0095] The camera (160) includes a lens, a shutter, an aperture, a solid-state image sensor, an AFE (Analog Front End), and a TG (Timing Generator). In addition, the camera (160) may further include a cover glass that is in contact with the lens. The shutter controls the time at which light reflected from a subject enters the camera (160), and the aperture controls the amount of light incident on the lens by mechanically increasing or decreasing the size of the opening through which light enters. The solid-state image sensor outputs an image by the photocharge as an electrical signal when light reflected from a subject is accumulated as a photocharge. The TG outputs a timing signal for reading out pixel data of the solid-state image sensor, and the AFE samples and digitizes the electric signal output from the solid-state image sensor.

[0096] The microphone (170) is configured to receive sound and convert it into an audio signal. The microphone (170) is electrically connected to the processor (140) and can receive sound under the control of the processor (140).

[0097] For example, the microphone (170) may be formed as an integrated unit integrated into the upper side, front side, side side, etc. of the electronic device (100). Alternatively, the microphone (170) may be provided in a remote control, etc., separate from the electronic device (100). In this case, the remote control may receive sound through the microphone (170) and provide the received sound to the electronic device (100).

[0098] The microphone (170) may include various configurations such as a microphone that collects sound in analog form, an amplifier circuit that amplifies the collected sound, an A / D conversion circuit that samples the amplified sound and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0099] Meanwhile, the microphone (170) may be implemented in the form of a sound sensor, and any method may be used as long as it has a configuration capable of collecting sound.

[0100] The speaker (190) is a component that outputs various audio data processed by the processor (140) as well as various notification sounds and voice messages.

[0101] As described above, the electronic device (100) can secure resources of the electronic device (100) by reducing the number of contents that the electronic device (100) must display by displaying some of the plurality of contents on the display device through screen expansion.

[0102] Additionally, the electronic device (100) can achieve optimal screen expansion by distributing content while taking into account the performance of the display device.

[0103] Hereinafter, the operation of the electronic device (100) will be described in more detail with reference to FIGS. 4 to 9. For convenience of explanation, individual embodiments are described in FIGS. 4 to 9. However, the individual embodiments of FIGS. 4 to 9 may be implemented in any combination.

[0104] FIG. 4 is a drawing for explaining screen expansion to multiple display devices (300) in one embodiment of the present disclosure.

[0105] The electronic device (100) can cast at least one of the plurality of contents being displayed by the electronic device (100) to a plurality of display devices (300).

[0106] For example, as illustrated in FIG. 4, the processor (140) can stream multiple contents from multiple electronic devices (200) and control the display (130) to display the multiple contents. For example, the processor (140) can control the first display device (A's TV #1) to display the first content at the upper left and the second content at the upper right among the plurality of contents, control the second display device (A's TV #2) to display the second content, control the third display device (A's Tablet) to display the third content at the lower left, control the fourth display device (A's Refrigerator) to display the fourth content at the lower right, and control the fifth display device (A's Phone #2) and the sixth display device (A's Phone #3) to display the third content and the fourth content, respectively.

[0107] That is, the processor (140) can control a single display device to display a single content. Furthermore, the processor (140) can control a single display device to display multiple contents. Furthermore, the processor (140) can control multiple display devices to display a single content. In this case, each of the multiple display devices can receive a single content, decode the content, and then display only the corresponding area.

[0108] This operation reduces the number of contents to be displayed by the electronic device (100), thereby securing the resources of the electronic device (100). Furthermore, the user can display important contents on the main device and less important contents on the peripheral devices, thereby enhancing immersion and usability. For example, the user can display video call contents on the main device and the remaining contents on the peripheral devices, thereby enhancing immersion.

[0109] FIG. 5 is a drawing for explaining the operation of an electronic device (100) during a screen expansion process in one embodiment of the present disclosure.

[0110] The electronic device (100) can control the display device to display at least one content among a plurality of contents through the display device, thereby reducing the load on the electronic device (100). This is because the electronic device (100) does not receive the content and provide the received content to the display device.

[0111] For example, according to conventional Partial Mirroring as illustrated in FIG. 5, User A Main Screen (electronic device, 540) can receive Media Streams from User C (510) and User B (520) via Media Server (530), directly display the Media Stream received from User B (520), and transmit the Media Stream received from User C (510) to User A Secondary Screen (550). Through this operation, User A Main Screen (540) may incur a load of having to process two streams.

[0112] As illustrated in FIG. 5, according to Partial Casting, User A Main Screen (electronic device, 540) receives a Media Stream from User B (520) via Media Server (530) and directly displays the Media Stream received from User B (520), and User A Secondary Screen (550) receives a Media Stream from User C (510) via Media Server (530) and directly displays the Media Stream received from User C (510). Through this operation, User A Main Screen (540) only has to process one stream, so the load can be reduced compared to the conventional method.

[0113] FIG. 6 and FIG. 7 are drawings for explaining the display of the UI provided in the screen expansion process and the result thereof in one embodiment of the present disclosure.

[0114] As illustrated in FIG. 6, the processor (140) can receive multiple contents from multiple electronic devices (200-1, 200-2, 200-3, 200-4) and display a screen including the multiple contents.

[0115] Here, the processor (140) can downscale at least one content among a plurality of contents based on at least one of the hardware specifications or resources of the electronic device (100), and control the display (130) to display the plurality of contents in which the at least one content is downscaled.

[0116] When a first user command for changing the display device of at least one content among a plurality of contents is received, the processor (140) can control the display (130) to display a screen (610) including a plurality of display devices (300). In FIG. 6, for convenience of explanation, it is described that the first user command is a command for changing the display device of the content received from a second electronic device (200-2) and a third electronic device (200-3).

[0117] When a first user command is received, the processor (140) controls the communication interface (120) to request performance information from a plurality of display devices (300), and when performance information is received from each of the plurality of display devices (300) through the communication interface (120), the processor (140) can control the display (120) to display a screen (610) including the plurality of display devices (300) based on the performance information. For example, the plurality of display devices (300) included in the screen (610) can be sorted in order of performance.

[0118] When a second user command for selecting at least one display device among a plurality of display devices (300) is received, the processor (140) may control the communication interface (130) to transmit information about a second electronic device (200-2) and a third electronic device (200-3) to at least one display device. For example, when a second user command for selecting a first display device (TV #1) and a third display device (TV #3) as shown in screen 620 is received, the processor (140) may control the communication interface (130) to transmit information about the second electronic device (200-2) and the third electronic device (200-3) to the first display device (TV #1) and the third display device (TV #3). Here, the processor (140) may further consider the performance of the content and the display devices. For example, the processor (140) can control the communication interface (130) to transmit information about a third electronic device (200-3) that provides relatively high-resolution content to a first display device (TV #1) that has relatively high performance, and to transmit information about a second electronic device (200-2) that provides relatively low-resolution content to a third display device (TV #3) that has relatively low performance.

[0119] Through this operation, the processor (140) controls the display (130) to display contents received from the first electronic device (200-1) and the fourth electronic device (200-4), as illustrated in FIG. 7, and the first display device (TV #1) can display contents received from the third electronic device (200-3), and the third display device (TV #3) can display contents received from the second electronic device (200-2). Here, since the load of the electronic device (100) has been reduced, the processor (140) can control the display (130) to display contents received from the first electronic device (200-1) and the fourth electronic device (200-4) at the original resolution without downscaling. In addition, since each of the first display device (TV #1) and the third display device (TV #3) displays only one content, the content can be displayed at the original resolution without downscaling.

[0120] Meanwhile, although the above description describes that each display device provides performance information to the electronic device (100), this is not limited thereto. For example, each display device may provide identification information of each display device to the electronic device (100), and the electronic device (100) may obtain performance information of each display device based on the identification information. For example, the electronic device (100) may transmit identification information to a server and receive performance information corresponding to the identification information from the server.

[0121] In addition, although the above description describes that the screen (610) is displayed according to the first user command, it is not limited thereto. For example, the processor (140) may control the display (130) to display a screen (610) including multiple display devices (300) based on at least one of the hardware specifications or resources of the electronic device (100).

[0122] Alternatively, the processor (140) may perform a screen expansion operation without displaying the screen (610) based on at least one of the hardware specifications or resources of the electronic device (100).

[0123] For example, the processor (140) may provide information about another electronic device providing content with the highest resolution among a plurality of contents to the display device (300) with the best performance based on at least one of the hardware specifications or resources of the electronic device (100). The processor (140) may control the display (130) to display the remaining contents except for the content with the highest resolution, and may identify the load at this time. If the load is less than a preset load, the processor (140) may not perform additional operations. Alternatively, if the load is greater than or equal to the preset load, the processor (140) may provide information about another electronic device providing content with the highest resolution among the remaining contents to the display device with the best performance among the remaining display devices. By repeating this operation, the processor (140) may maintain the load below the preset load.

[0124] FIG. 8 is a flowchart for explaining priorities among multiple display devices (300) in one embodiment of the present disclosure.

[0125] The processor (140) can identify priorities among the plurality of display devices (300) based on performance information. The processor (140) can control the display (130) to display a screen including the plurality of display devices (300) based on the priorities among the plurality of display devices (300). Here, the performance information can include at least one of whether each of the plurality of display devices (300) supports a hardware codec, CPU (central processing unit) performance, RAM capacity, or GPU (graphics processing unit) support.

[0126] The processor (140) can identify priorities among multiple display devices (300) based on priorities among each element included in the performance information. For example, as illustrated in FIG. 8, the processor (140) can identify whether hardware codec is supported (S810). If the display device supports a hardware codec, the processor (140) can identify whether the CPU performance is 1.6 GHz or higher and the RAM capacity is 2 GB or higher (S820). If the CPU performance is 1.6 GHz or higher and the RAM capacity is 2 GB or higher, the processor (140) can identify whether a GPU is supported (S830). If the display device supports a GPU, the processor (140) can identify it as a first-priority terminal (S840), and if the display device does not support a GPU, the processor (140) can identify it as a second-priority terminal (S850). If the CPU performance is less than 1.6 GHz or the RAM capacity is less than 2 GB, the processor (140) can identify it as a third-priority terminal (S860).

[0127] The processor (140) can identify whether the CPU performance is 1.6 GHz or higher and the RAM capacity is 2 GB or higher if the display device does not support a hardware codec (S870). If the CPU performance is 1.6 GHz or higher and the RAM capacity is 2 GB or higher, the processor (140) can identify the terminal as a third-priority terminal (S870), and if the CPU performance is lower than 1.6 GHz or the RAM capacity is lower than 2 GB, the processor can identify the terminal as a low-performance terminal (S890).

[0128] However, the present invention is not limited thereto, and the processor (140) may identify priorities among the plurality of display devices (300) in any number of different ways. For example, the processor (140) may identify priorities among the plurality of display devices (300) by weighting factors such as whether each of the plurality of display devices (300) supports a hardware codec, CPU performance, RAM capacity, and GPU support.

[0129] Additionally, the processor (140) may identify priorities among multiple display devices (300) by taking into account any number of other factors. For example, the processor (140) may identify priorities among multiple display devices (300) by taking into account the resolution of the display included in the display device, the battery performance of the display device, etc.

[0130] FIG. 9 is a flowchart for explaining the operation of an electronic device (100) for screen expansion in one embodiment of the present disclosure.

[0131] The processor (140) can obtain information on multiple candidate display devices. For example, as illustrated in FIG. 9, when a first user command for changing the display device of one of multiple contents is received, the processor (140) can collect candidate display device (secondary terminal) discovery and candidate display device performance (S910).

[0132] The processor (140) can identify whether there is a candidate display device capable of distributing video (S920). For example, the processor (140) can identify at least one of whether the candidate display device is turned on, whether a preset application is installed, or whether the preset application is running.

[0133] If the processor (140) identifies that there are candidate display devices capable of distributing images, it recommends a list of candidate display devices in order of best performance (S930). If one or more candidate display devices are selected (S940), it distributes images to the candidate display devices and outputs the remaining images (S950). Alternatively, if it identifies that there are no candidate display devices capable of distributing images, the current output status may be maintained.

[0134] Although the above description describes collecting candidate display device performance after receiving a first user command, the present invention is not limited thereto. For example, the processor (140) may collect candidate display device performance before receiving the first user command, and upon receiving the first user command, identify whether a candidate display device capable of video distribution exists.

[0135] FIG. 10 is a flowchart illustrating a method for controlling an electronic device according to an embodiment of the present disclosure.

[0136] First, a plurality of contents streamed from a plurality of other electronic devices are displayed (S1010). Then, when a first user command for changing the display device of one of the plurality of contents is received, a screen including information about the plurality of display devices is displayed (S1020). Then, when a second user command for selecting one of the plurality of display devices is received, information about the other electronic device providing one content is transmitted to one of the display devices (S1030). Then, the remaining contents except one of the plurality of contents are displayed (S1040).

[0137] In addition, the step of displaying the screen (S1020) may, when a first user command is received, identify whether a plurality of candidate display devices registered to the same account as the electronic device are turned on, request performance information from a plurality of display devices that are turned on among the plurality of candidate display devices, and, when performance information is received from each of the plurality of display devices, display a screen including information about the plurality of display devices based on the performance information.

[0138] And, the second user command includes a user command for selecting a first display device and a second display device among a plurality of display devices, and the step (S1030) of transmitting to one display device may transmit information about another electronic device that provides one content based on performance information to a high-performance display device among the first display device and the second display device when the second user command is received.

[0139] Additionally, the step of displaying the screen (S1020) may display a screen listing multiple display devices in order of performance based on performance information.

[0140] Additionally, the performance information may include at least one of hardware codec support, CPU (central processing unit) performance, RAM capacity, or GPU (graphics processing unit) support for each of the plurality of display devices.

[0141] In addition, the step of displaying the remaining content (S1040) is such that when another electronic device providing one content streams one content to one display device, and another electronic device providing one content does not stream one content to the electronic device, the remaining content can be displayed except for one content among the plurality of contents.

[0142] And, the step of displaying the remaining contents (S1040) can change the display area of ​​the remaining contents except for one of the plurality of contents.

[0143] In addition, the step of displaying multiple contents (S1010) may downscale at least one content among the multiple contents, display the multiple contents with at least one content downscaled, and the step of displaying the remaining contents (S1040) may display the remaining contents excluding one content among the multiple contents without downscaling.

[0144] And, when the electronic device is moved from the first position to the second position, the step of updating the plurality of display devices by adding a fourth display device to the plurality of display devices, excluding a third display device among the plurality of display devices based on the second position, and the step of displaying a screen including information about the updated plurality of display devices may be further included.

[0145] Additionally, the step of displaying the screen (S1020) may display the screen based on at least one of the hardware specifications or resources of the electronic device.

[0146] According to various embodiments of the present disclosure as described above, the electronic device can secure resources of the electronic device by reducing the number of contents to be displayed by displaying some of the plurality of contents on the display device through screen expansion.

[0147] Additionally, the electronic device may be capable of optimal screen expansion by distributing content taking into account the performance of the display device.

[0148] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call instructions stored from the storage medium and operate according to the called instructions, and may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0149] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0150] Furthermore, according to one embodiment of the present disclosure, the various embodiments described above may be implemented in a computer-readable recording medium or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. In a software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0151] Meanwhile, computer instructions for performing processing operations of a device according to the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations in the device according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0152] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0153] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, Memory that stores one or more instructions; communication interface; display; and one or more processors connected to the memory, the communication interface and the display to control the electronic device; The one or more processors, by executing the one or more instructions, Controlling the display to display a plurality of contents each streamed from a plurality of other electronic devices through the communication interface; When a first user command for changing the display device of one of the plurality of contents is received, the display is controlled to display a screen including information about the plurality of display devices, When a second user command for selecting one of the plurality of display devices is received, the communication interface is controlled to transmit information about another electronic device providing the one content to the one display device, An electronic device that controls the display to display the remaining contents except for the one content among the plurality of contents.

2. In paragraph 1, The above processor, When the above first user command is received, it identifies whether a plurality of candidate display devices registered to the same account as the electronic device are turned on, Control the communication interface to request performance information from the plurality of display devices that are turned on among the plurality of candidate display devices; An electronic device that controls the display to display a screen including information about the plurality of display devices based on the performance information when performance information is received from each of the plurality of display devices through the communication interface.

3. In paragraph 2, The second user command above is, Includes a user command for selecting a first display device and a second display device among the plurality of display devices, The above processor, An electronic device that controls the communication interface to transmit information about another electronic device that provides the one content based on the performance information to a high-performance display device among the first display device and the second display device when the second user command is received.

4. In paragraph 2, The above processor, An electronic device that controls the display to display a screen listing the plurality of display devices in order of performance based on the performance information.

5. In paragraph 2, The above performance information is, An electronic device comprising at least one of hardware codec support, CPU (central processing unit) performance, RAM capacity, or GPU (graphics processing unit) support of each of the plurality of display devices.

6. In paragraph 1, The above processor, Controlling the communication interface to transmit information about another electronic device providing the above content to the above one display device; An electronic device that controls the display to display the remaining content excluding the one content among the plurality of contents when the other electronic device providing the one content streams the one content to the one display device and the other electronic device providing the one content does not stream the one content to the electronic device.

7. In paragraph 1, The above processor, An electronic device that controls the display to change the display area of ​​the remaining contents excluding the one content among the plurality of contents.

8. In paragraph 1, The above processor, Downscaling at least one of the plurality of contents, and controlling the display to display the plurality of contents in which the at least one content is downscaled; When the first user command is received, the display is controlled to display the screen, When the second user command is received, the communication interface is controlled to transmit information about another electronic device providing the one content to the one display device, An electronic device that controls the display to display the remaining contents, excluding the one content among the plurality of contents, without the downscaling.

9. In paragraph 1, The above processor, When the electronic device is moved from the first position to the second position, the plurality of display devices are updated by adding a fourth display device to the plurality of display devices, excluding a third display device from the plurality of display devices based on the second position, and An electronic device that controls the display to display a screen containing information about the plurality of updated display devices.

10. In paragraph 1, The above processor, An electronic device that controls the display to display the screen based on at least one of the hardware specifications or resources of the electronic device.

11. In a method for controlling an electronic device, A step of displaying a plurality of contents each streamed from a plurality of other electronic devices; A step of displaying a screen including information on a plurality of display devices when a first user command for changing a display device of one of the plurality of contents is received; When a second user command for selecting one of the plurality of display devices is received, a step of transmitting information about another electronic device providing the one content to the one display device; and A control method comprising: a step of displaying the remaining contents excluding the one content among the plurality of contents; 12. In paragraph 11, The steps to display the above screen are: When the above first user command is received, it identifies whether a plurality of candidate display devices registered to the same account as the electronic device are turned on, Requesting performance information from the plurality of display devices that are turned on among the plurality of candidate display devices, A control method for displaying a screen including information about the plurality of display devices based on performance information received from each of the plurality of display devices.

13. In paragraph 12, The second user command above is, Includes a user command for selecting a first display device and a second display device among the plurality of display devices, The step of transmitting to the above one display device is: A control method for transmitting information about another electronic device providing the one content based on the performance information to a high-performance display device among the first display device and the second display device when the second user command is received.

14. In paragraph 12, The steps to display the above screen are: A control method for displaying a screen listing the plurality of display devices in order of performance based on the above performance information.

15. In paragraph 12, The above performance information is, A control method comprising at least one of whether each of the plurality of display devices supports hardware codec, CPU (central processing unit) performance, RAM capacity, or GPU (graphics processing unit) support.

Images