Electronic device and control method thereof

Abstract

An electronic device includes a processor, which is configured to: display content as a first image including a playback area, send data of the first image to a first external device, and based on a first user input generated in the first external device, identify whether the first user input requests to change the playback area of ​​a second image, wherein in the first external device, based on the received data, the content is displayed as a second image including the same playback area as the playback area of ​​the first image, and based on identifying that the first user input requests to change the playback area of ​​the second image, a request to display the content as a third image including a playback area different from the playback area of ​​the first image is sent to the first external device, and the playback area of ​​the first image is maintained on the display.

Description

Technical Field

[0001] The present disclosure relates to providing image content to a display device, and more particularly, to performing an operation based on receiving a touch input while providing image content to the display device based on mirroring from another device. Background Art

[0002] In order to calculate and process information according to certain processes, the electronic device includes electronic components for calculation, such as a central processing unit (CPU), a chipset, and a memory. For example, the electronic device may be an information processing device such as a personal computer (PC), a server, etc. for processing information, an image processing device for processing image data, an audio device for audio processing, a home appliance for various household chores, etc. The image processing device may be implemented by a display device that displays an image on its display panel based on the processed image data.

[0003] The display device can be a fixed device installed in one place, such as a television (TV), and a portable device carried by a user, such as a mobile device. Specifically, with the development of wireless communication environments and mobile devices such as smart phones and tablet computers that can utilize wireless communication environments, many users use personalized mobile devices. In this regard, there is a growing demand for linking operations between multiple display devices, such as between mobile devices and TVs. This linking operation is based on wireless communication between devices, and mirroring is performed as an example of a linking operation.

[0004] Mirroring refers to a function in which an image being displayed on a display device is identically displayed on another display device. For example, when a content image is displayed on the screen of a mobile device, the mobile device sends content data to the display device in real time to display the content image. The display device displays the same image as the content image being displayed on the mobile device based on the content data received from the mobile device.

[0005] When the image displayed on the mobile device includes an on-screen display (OSD) menu for user input, the user can touch the OSD menu on the screen of the mobile device to perform an operation corresponding to the touched OSD menu. However, in the related art, the display device only displays the image received from the mobile device during mirroring. Therefore, although the screen of the display device is provided as a touch screen and the user touches the OSD menu on the screen of the display device, the user is not allowed to perform the corresponding operation. In other words, during mirroring, the playback state of the content image can be controlled by touching the OSD menu on the screen of the mobile device, but cannot be controlled through the screen of the display device.

[0006] In addition, in terms of mirroring the decoded image in the mobile device to the TV, the network environment between the mobile device and the TV may affect the mirrored image displayed on the TV. For example, when the user wants to display the mirrored image at a high resolution on the TV, the network bandwidth available for mirroring between the mobile device and the TV may not be sufficient to send such high-resolution image data.

[0007] Therefore, when a user input for viewing an image in a full screen mode is made through an OSD menu displayed on a screen of a TV during mirroring, there may be a problem that the operation is not properly performed. Summary of the invention

[0008] [Technical solution]

[0009] According to one aspect of the present disclosure, an electronic device is provided, comprising: a display; a communication circuit configured to communicate with a first external device; and a processor configured to: display content as a first image including a playback area on the display, send data of the first image to the first external device through the communication circuit, and in response to a first user input generated in the first external device, identify whether the first user input indicates a change in the playback area of ​​a second image, wherein in the first external device, based on the received data, the content is displayed as a second image including a playback area that is the same as the playback area of ​​the first image, and based on the first user input identified as indicating a change in the playback area of ​​the second image, the playback area of ​​the first image is maintained on the display, and a request is sent through the communication circuit to request the first external device to display the content as a third image including a playback area that is different from the playback area of ​​the first image.

[0010] The playback area of ​​the third image may be larger than the playback area of ​​the second image.

[0011] The request for displaying the third image may include at least one of a file of the content or address information from which the content can be obtained.

[0012] The second image may include a menu item for selecting a change in the playback area of ​​the second image, and the processor may be configured to identify that the first user input indicates a change in the playback area of ​​the second image based on whether coordinate information on the screen of the first external device obtained from the first user input corresponds to the position of the menu item.

[0013] The processor may be configured to send captured data of the first image to the first external device through the communication circuit based on a first transmission protocol, and to send a request for displaying a third image to the first external device through the communication circuit based on a second transmission protocol different from the first transmission protocol.

[0014] The electronic device may further include a user input receiver, wherein the processor may be configured to: identify whether a second user input generated by the user input receiver requests a change in a playback area of ​​the first image; based on the second user input identified as a request to change the playback area of ​​the first image, display content as a fourth image including a playback area different from the playback area of ​​the first image on the display; and send captured data of the fourth image to the first external device via the communication circuit.

[0015] The processor may be configured to: identify whether a third user input generated in a first external device displaying a third image requests restoration of a playback area of ​​a second image; and based on the third user input identified as a request to restore a playback area of ​​the second image, send the captured data of the first image to the first external device again through the communication circuit.

[0016] The electronic device may further include a user input receiver, wherein the processor may be configured to: based on a fourth user input generated through the user input receiver and recognized as a request for external broadcasting of content, display a list of connectable second external devices on the display and perform an external broadcasting operation for the content on the second external device, and based on a first user input generated in the external device and recognized as a request for external broadcasting of content, send a request for displaying a third image to the first external device through the communication circuit without displaying the list.

[0017] According to one aspect of the present disclosure, a method for controlling an electronic device is provided, including: displaying content as a first image including a playback area; sending data of the first image to a first external device; in response to a first user input generated in the first external device, identifying whether the first user input indicates a change in the playback area of ​​a second image, wherein in the first external device, based on the received data, the content is displayed as a second image including a playback area that is the same as the playback area of ​​the first image; and based on the first user input identified as indicating a change in the playback area of ​​the second image, the playback area of ​​the first image is maintained, and a request is sent to the first external device requesting the first external device to display the content as a third image including a playback area that is different from the playback area of ​​the first image.

[0018] According to one aspect of the present disclosure, an electronic device is provided, comprising: a display; a communication circuit configured to communicate with an external device; and a processor configured to: based on a user input for requesting motion image content to be displayed, display the motion image content as an image in a first form in a partial area of ​​a screen, while displaying the image in the first form on the display, send image data corresponding to the image in the first form to the external device through a communication unit, based on a user input generated in the external device while the image data is being sent to the external device, identify whether the user input indicates a change in the display form of the motion image content, and based on the user input identified as indicating a change in the display form of the motion image content, maintain the image in the first form to be displayed on the display of the electronic device, and send information for allowing the external device to display the motion image content being displayed on the display of the electronic device in a second form to the external device through the communication unit.

[0019] The change in the display form may include a change in an image size of the moving image content displayed on the screen, and the image size of the moving image content displayed in the second form may be larger than the image size of the moving image content displayed in the first form.

[0020] The information about the display in the second form may include at least one of a file corresponding to the moving image content stored in a storage unit of the electronic device or address information from which the moving image content can be obtained.

[0021] The image of the first form may include a user interface (UI) item that is provided to selectively change the display form of the moving image content, and the processor may be configured to perform an operation for changing the display form of the moving image content through user input based on whether coordinate information of the user input obtained from an external device corresponds to the position of the UI item.

[0022] The processor may be configured to transmit the image data to the external device through the communication circuit based on a first transmission protocol, and transmit information for displaying the moving image content in a second form to the external device through the communication circuit based on a second transmission protocol different from the first transmission protocol.

[0023] The electronic device may further include a user input receiver, wherein the processor is configured to identify whether a second user input received through the user input receiver requests a change in a display form of the motion image content; display the motion image content by changing the display form to a second form different from the first form based on the second user input identified as requesting a change in the display form of the motion image content; and send image data corresponding to the motion image content displayed in the second form to an external device through the communication circuit.

[0024] The processor may be configured to identify whether a third user input generated in an external device that displays motion image content in a second form requests restoration of the first form of the motion image content; and based on the third user input identified as requesting restoration of the first form of the motion image content, send image data corresponding to an image currently displayed on the electronic device to the external device through the communication circuit. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A mobile device and a display device according to an embodiment are shown;

[0026] Figure 2 is a block diagram of a display device and a mobile device according to an embodiment;

[0027] Figure 3 is a flow chart showing a control method of a mobile device according to an embodiment;

[0028] Figure 4 is a block diagram showing a signal flow based on a transmission protocol between a mobile device and a display device according to an embodiment;

[0029] Figure 5 shows images displayed on a mobile device and a display device, respectively, during mirroring according to an embodiment;

[0030] Figure 6 A method according to an embodiment is shown by which a mobile device identifies an event corresponding to a touch input occurring in a display device during mirroring;

[0031] Figure 7 According to an embodiment, when a user Figure 5 The image displayed when the full screen option of the first screen is touched in the state;

[0032] Figure 8 According to an embodiment, when a user Figure 5 The image displayed when you touch the full screen option of the second screen in the state;

[0033] Fig. 9 is a flow chart illustrating a process in which a mobile device receives a user input for expanding a play area of ​​an image during mirroring according to an embodiment;

[0034] Fig.10 It is shown that according to the embodiment Figure 5 When the outbound broadcast option on the first screen of the mobile device is touched in the state of being ...

[0035] Fig.11 It is shown that according to the embodiment Figure 5An image displayed when a broadcast option is touched on a second screen of a mobile device in a state of FIG.

[0036] Fig.12 is a flow chart illustrating a process by which a mobile device receives user input for requesting outbound broadcasting during mirroring according to an embodiment; and

[0037] Fig.13 is a flowchart illustrating a control method of a mobile device according to an embodiment. DETAILED DESCRIPTION

[0038] Below, certain embodiments will be described in detail with reference to the accompanying drawings. In addition, unless otherwise specified, the embodiments described with reference to the accompanying drawings are not mutually exclusive, and multiple embodiments can be selectively combined in one device. The combination of these multiple embodiments can be arbitrarily selected and applied by a person of ordinary skill in the art to implement the concept of the present invention.

[0039] As used herein, terms such as “first” or “first”, “second” or “second”, etc. may modify corresponding elements regardless of importance or order and are used to distinguish one element from another without limiting the elements.

[0040] Furthermore, the term “at least one” of a plurality of elements refers not only to all the elements but also to each of the elements, excluding other elements or all combinations of elements.

[0041] Figure 1 A mobile device and a display device according to an embodiment are shown.

[0042] like Figure 1 As shown, an electronic device and an external device are provided to communicate with each other. In one embodiment, the electronic device is used as a source device for providing image content, and the external device is used as a sink device for receiving image content from the electronic device. For example, the electronic device is a mobile device 110, and the external device is a display device 120. However, these terms are only for the convenience of distinguishing the two devices, and these terms do not limit the functions of the devices.

[0043] The mobile device 110 may include various devices having a screen for displaying an image, such as a smart phone, a tablet computer, a multimedia player, a wearable device, etc. The electronic device may be implemented by various devices, such as a TV installed in a fixed location, a digital sign, an electronic frame, a computer with a monitor, etc., as well as the portable mobile device 110.

[0044] The display device 120 includes a bracket 130 mounted on a mounting surface and rotatably supported on one side of the bracket 130. The bracket 130 includes a motor 131 connected to the back of the display device 120 and is driven by the motor 131 to rotate the display device 120. The operation of the motor 131 can be controlled by a received control signal. The control signal transmitted to the motor 131 can be generated by a controller set in the bracket 130, or can be received from the display device 120. In addition, an instruction for generating such a control signal can be received from a remote controller provided in the bracket 130 or the display device 120.

[0045] In addition, it can be designed to rotate only the display panel forming the screen of the display device. In other words, the configuration such as the processor other than the display panel can form the main body, and the display panel can be rotated relative to the main body.

[0046] The display device 120 is provided to pivot relative to the central area of ​​the screen for displaying images. The display device 120 can pivot between at least a P position and an L position. When the display device 120 is in the P position, the display device 120 enters a vertical or portrait mode, in which the vertical length is longer than the horizontal length, and when the display device 120 is in the L position, the display device 120 enters a horizontal or landscape mode, in which the horizontal length is longer than the vertical length.

[0047] The image or UI displayed on the display device 120 may correspond to one of the portrait mode and the landscape mode. Of course, the display device 120 may display a landscape image in the portrait mode, and may also display a portrait image in the landscape mode. However, in this case, the screen and the image do not match according to the ratio between the horizontal length and the vertical length. In other words, in this case, the area where the image is not displayed on the screen of the display device 120, i.e., the edge, is relatively large. In addition, the UI or image may be designed to be adapted to the landscape mode or the portrait mode.

[0048] Therefore, the display device 120 displays a portrait image in the portrait mode and displays a landscape image in the landscape mode. For example, the display device 120 operates in the landscape mode to display a broadcast image. In addition, the display device 120 operates in the portrait mode to display a mirror image of the mobile device 110.

[0049] The mobile device 110 may wirelessly communicate with the display device 120. The mobile device 110 may access the display device 120 via a wide area network (WAN) or a local area network (LAN). For example, the mobile device 110 may perform communication with the display device 120 based on Wi-Fi communication through an access point (AP), or may perform direct communication with the display device 120 based on Bluetooth Low Energy (BLE).

[0050] Through this communication route, the mobile device 110 can provide image content to the display device 120. The display device 120 can process the image content received from the mobile device 110 to display the image. There are many transmission protocols for allowing the mobile device 110 to provide image content to the display device 120. For example, there are mirroring transmission protocols and broadcasting transmission protocols, which will be described later.

[0051] Figure 2 is a block diagram of a display device and a mobile device according to an embodiment.

[0052] like Figure 2 As shown, the mobile device 110 includes a communicator 211, a display 213, a user input receiver 214, a storage unit 215, and a processor 216. The display device 120 includes a second communicator 221, a signal input / output unit 222, a second display 223, a second user input receiver 224, a second storage unit 225, and a second processor 226.

[0053] The communicator 211 refers to an interactive communication circuit, which includes at least one element corresponding to various wired and wireless communication protocols, such as a communication module, a communication chip, etc. The communicator 211 supports one or more wireless communication standards, and includes a communication circuit, a communication chip, or a communication module to perform communication based on a wireless communication standard. For example, the communicator 211 includes a communication circuit for one-to-one local area communication, such as Bluetooth, BLE, etc. that conform to the Institute of Electrical and Electronics Engineers (IEEE) 802.15 standard. The communicator 211 includes a communication circuit for WAN-based communication such as Wi-Fi or a wireless LAN communication conforming to IEEE 802.11. Therefore, the communicator 211 can communicate with the display device 120 and the server 240 through the AP 230.

[0054] The display 213 includes a display panel capable of displaying an image on a screen. The display panel is provided to have a light receiving structure such as a liquid crystal, or a self-emitting structure such as an organic light emitting diode (OLED). The display 213 may include additional elements according to the structure of the display panel. For example, when the display panel is a liquid crystal, the display 213 includes a liquid crystal display panel, a backlight unit configured to emit light, and a panel driving substrate configured to drive the liquid crystal of the liquid crystal display panel.

[0055] The user input receiver 214 includes circuits related to various input interfaces, which are provided to be controlled by the user to receive the user's input. The user input receiver 214 can be configured in various ways according to the type of the mobile device 110, and can include, for example, a mechanical or electronic button of the mobile device 110, a touch pad installed in the display 213, a touch screen, etc.

[0056] The storage unit 215 is configured to store various data segments. Depending on whether the stored data is retained, the storage unit 215 includes a non-volatile storage device and a volatile memory. The storage device refers to a medium that retains recorded data even when there is no power supply, and includes a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a read-only memory (ROM), etc. The memory refers to a medium in which recorded data is not retained when there is no power supply, and includes a buffer, a random access memory (RAM), etc.

[0057] The processor 216 includes one or more hardware processors implemented by a CPU, a chipset, a buffer, a circuit, etc. mounted on a printed circuit board (PCB). As an example, the processor 216 can be designed as a system on a chip (SoC). The processor 216 can execute an operating system of the mobile device 110 and applications corresponding to various functions. The processor 216 can play the image content stored in the storage unit 215 so as to be displayed on the display 213 as an image, or provide the image content to the display device 120.

[0058] The second communicator 221 refers to an interactive communication circuit, which includes at least one element corresponding to various wired and wireless communication protocols, such as a communication module, a communication chip, etc. The second communicator 221 supports wireless communication based on Wi-Fi, for example, so as to communicate with the mobile device 110 or the server 240 through the AP 230.

[0059] The signal input / output unit 222 is connected to an external device such as a set-top box or an optical media player through a one-to-one or one-to-many wired connection, thereby receiving data from the external device or outputting data to the external device. The signal input / output unit 222 may, for example, include a connector, a port, etc. based on a preset transmission protocol, such as a high-definition multimedia interface (HDMI) port, a display port, a universal serial bus (USB) port, etc.

[0060] The second display 223 displays an image based on the image signal processed by the second processor 226 .

[0061] The second user input receiver 224 includes circuits related to various input interfaces, which are provided to be controlled by the user to receive user input. For example, the second user input receiver 224 includes a physical or electronic button set in the display device 120, a remote controller separated from the display device 120, a touch screen structure set in the second display 223, etc.

[0062] The second storage unit 225 is configured to store data. The second storage unit 225 includes various storages and memories, such as flash memory, HDD, SSD, buffer, RAM, etc.

[0063] The second processor 226 includes one or more hardware processors implemented by a CPU, a chipset, a buffer, a circuit, etc. mounted on a PCB. As an example, the second processor 226 can be designed as a SoC. The second processor 226 includes modules corresponding to various processes, such as a demultiplexer, a decoder, a scaler, an audio digital signal processor (DSP), an amplifier, etc., to process images. In such modules, some or all of the modules can be implemented by SoC. For example, modules such as a demultiplexer, a decoder, a scaler, etc. related to image processing can be implemented as an image processing SoC, and an audio DSP can be implemented as a chipset separated from the SoC. In addition, as an example of a design method, when the display device 120 is provided to rotate by a motor of a bracket, the second processor 226 can be electronically connected to the motor so that a control signal for driving the motor can be sent to the motor.

[0064] Figure 3 is a flowchart illustrating a control method of a mobile device performed by a processor of the mobile device according to an embodiment.

[0065] The mobile device displays the content as a first image having a play area in operation 310. The play area of ​​the content refers to an area where the corresponding content is displayed within the entire area of ​​a screen provided to display an image.

[0066] In operation 320, the mobile device transmits capture data obtained by capturing the first image to the display device while displaying the first image. The capture data refers to data obtained by decoding and buffering image content in order to display the first image in the mobile device.

[0067] The display device receives the captured data and displays the content as a second image having a playback area that is the same as the playback area of ​​the first image displayed in the mobile device. The second image corresponds to a mirror image of the first image because the second image is displayed based on the data decoded in the mobile device for the first image.

[0068] In operation 330, the mobile device detects whether a user input for changing a play area of ​​the second image is received in the display device while the second image is displayed in the display device.

[0069] In operation 340, in response to the user input, the mobile device maintains the play area of ​​the first image being displayed in the mobile device. In other words, the mobile device maintains the current display state of the first image.

[0070] Furthermore, in operation 350, the mobile device transmits a request to the display device to request the display device to display the content as a third image having a play area different from that of the first image in response to the user input.

[0071] The display device displays a third image changed from the second image based on a request received from the mobile device. The third image is based on image content that is the same as the image content of the second image, but is displayed based on data of the image content transmitted through a transmission protocol that is different from the transmission protocol of the second image. The second image is displayed based on a mirroring of the image content, but the third image is displayed based on a broadcast of the image content.

[0072] In this way, when a user input for changing the playback area of ​​an image is received in the display device while the mobile device provides image content, so that the display device can display a mirror-based image, the mobile device controls the display device to be switched to display a broadcast-based image. Therefore, the mobile device can allow the display device to adaptively display a high-resolution and high-quality image in response to a user's request.

[0073] The processor of the mobile device can perform at least a portion of data analysis, data processing, and result information generation based on at least one of machine learning, neural networks, rule-based deep learning algorithms, or artificial intelligence (AI) algorithms, so as to perform operations for maintaining the playback area of ​​the first image and sending a request for the display device to display a third image in response to user input for changing the playback area of ​​the second image as described above.

[0074] For example, a processor of a mobile device may be used as a learner and a recognizer. The learner may perform a function of generating a learned neural network, and the recognizer may perform a function of recognizing (or inferring, predicting, estimating, and recognizing) data based on the learned neural network. The learner may generate or update a neural network. The learner may obtain learning data to generate a neural network. For example, the learner may obtain learning data from a storage unit of the mobile device or from the outside. The learning data may be data for learning a neural network, and the data subjected to the above-mentioned operation may be used as learning data to teach the neural network.

[0075] Before teaching the neural network based on the learning data, the learner may perform a preprocessing operation on the obtained learning data, or select data to be used in the learning from a plurality of learning data. For example, the learner may process the learning data to have a preset format, apply filtering to the learning data, or process the learning data to be suitable for learning by adding noise to the learning data and / or removing noise from the learning data. The learner may generate a neural network set using the preprocessed learning data to perform the operation.

[0076] The learned neural network may include multiple neural networks (or layers). The nodes of multiple neural networks have weights, and multiple neural networks may be connected to each other so that the output value of a particular neural network may be used as the input value of another neural network. As examples of neural networks, there are convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recursive DNNs (BRDNNs), and deep Q networks.

[0077] The recognizer may obtain target data to perform the above-mentioned operations. The target data may be obtained from a storage unit of a mobile device or from the outside. The target data may be data that is targeted for recognition by a neural network. Before applying the target data to the learned neural network, the recognizer may perform a preprocessing operation on the obtained target data, or select data to be used in recognition from a plurality of target data. For example, the recognizer may process the target data to have a preset format, apply filtering to the target data, or process the target data into data suitable for recognition by adding / removing noise. The recognizer may obtain an output value output from the neural network by applying the preprocessed target data to the neural network. In addition, the recognizer may obtain a random value or a reliability value as well as the output value.

[0078] Figure 4 is a block diagram illustrating a signal flow based on a transmission protocol between a mobile device and a display device according to an embodiment.

[0079] like Figure 4 As shown, the mobile device 110 obtains the predetermined image content data. The mobile device 110 may obtain the image content data from the outside or the inside. For example, the mobile device 110 may load the image content data stored in the storage unit or receive the image content data from the server 430.

[0080] The mobile device 110 includes a communicator 411, a display 412, a decoder 413, and a buffer 414. The decoder 413 may be implemented as a single chip together with the processor of the mobile device 110 or as a chip separate from the processor. The decoder 413 decodes the obtained image content data and buffers the decoded data to the buffer 414. The data buffered to the buffer 414 is output from the buffer 414 to the display 412 at the display timing of the image, so that the image can be displayed on the display 412. In other words, the data buffered to the buffer 414 is decoded by the decoder 413. For ease of description, the image displayed on the display 412 of the mobile device 110 is referred to as a first image.

[0081] The display device 120 includes a second communicator 421, a second display 422, and a second decoder 423. The display device 120 may display an image based on the same image content as that of the first image being displayed on the mobile device 110 by a mirroring method or a casting method.

[0082] According to the mirroring method, the mobile device 110 transmits the image content data buffered to the buffer 414 to the display device 120 through the communicator 411. In other words, the data buffered to the buffer 414 is transmitted to the display 412 and the communicator 411. When the data from the mobile device 110 is received in the second communicator 421, the display device 120 transmits the received data to the second display 422 without decoding the received data through the second decoder 423. For the convenience of description, the image displayed on the second display 422 through this process is referred to as a second image.

[0083] In the mirroring method, the first image and the second image are both decoded outputs of the decoder 413 of the mobile device 110. In other words, the second image is not a product of the second decoder 423 of the display device 120, but a product of the decoder 413 of the mobile device 110.

[0084] According to the broadcast method, the mobile device 110 does not transmit the image content data buffered to the buffer 414 to the display device 120 through the communicator 411, but transmits the original image content data that has not been decoded by the decoder 413 to the display device 120. As another example, the mobile device 110 does not directly transmit the content of the content to the display device 120, but may provide information about an address where the display device 120 can obtain the data of the content. When the data from the mobile device 110 is received in the second communicator 421 of the display device 120, the display device 120 decodes the received data through the second decoder 423 and transmits the decoded data to the second display 422. For ease of description, the image displayed on the second display 422 through this process is referred to as a third image.

[0085] In the casting method, the first image is a decoded output of the decoder 413 of the mobile device 110, and the third image is a decoded output of the second decoder 423 of the display device 120. In other words, the third image is an output caused by a different decoding than the second image, even if it is based on the same image content as the second image. For example, the casting method can be performed via the Digital Living Network Alliance (DLNA).

[0086] The broadcast method may include the first broadcast method of the aforementioned example and the second broadcast method as follows. In the case of the second broadcast method, the mobile device 110 does not send the undecoded original image content data to the display device 120, but sends the address information of the server 430 from which the data can be obtained to the display device 120 through the communicator 411. The display device 120 accesses the server 430 based on the received address information and receives the image content data stored in the corresponding address information. The display device 120 decodes the data received in this way through the second decoder 423 and displays the decoded data on the second display 422.

[0087] The image content data for displaying the third image may be directly transmitted by the mobile device 110 to the display device 120, or may be obtained by the display device 120 based on the address information provided by the mobile device 110. In both cases, the data obtained by the display device 120 is decoded by the second decoder 423.

[0088] As described in detail below, the mobile device 110 and the display device 120 display an image according to a transmission protocol, and change the image according to the switching of the transmission protocol.

[0089] Figure 5 Shown are images displayed on a mobile device and a display device, respectively, during mirroring according to an embodiment.

[0090] like Figure 5 As shown, the mobile device has a first screen for displaying an image, and the display device has a second screen 520 for displaying an image. When mirroring of an image is performed while the mobile device displays an image on the first screen 510, the display device displays a mirrored image of the image on the second screen 520. When the display device is pivotally provided to switch between a portrait mode and a landscape mode, the display device can control the display having the second screen 520 to enter the portrait mode in response to the mirroring.

[0091] The mobile device can display multiple images together in the entire playback area of ​​the first screen 510 capable of displaying images. The mobile device displays an A image 511 based on predetermined content in a partial playback area in the entire playback area of ​​the first screen 510 capable of displaying images. The mobile device can display an image based on different content, i.e., a B image 512 in another playback area in the entire playback area of ​​the first screen 510.

[0092] For example, when the mobile device is accessing a motion picture content service provided through the Internet, A image 511 may be a playback image based on the motion picture content, and B image 512 may be a content list provided by the service. In addition, A image 511 may display various options related to the content being played, or options for adjusting the playback status of the content. For example, A image 511 may display an outbound broadcast option 513 (i.e., Figure 5 UI TV in the display device) for displaying A image 511 on an external device such as a display device, a full screen option 514 (ie, Figure 5 UI Full) in the image 511 is used to display the A image 511 as a full screen, etc.

[0093] The external broadcast option 513 refers to a method for indicating the content of the image 511 to be broadcast to a device selected from a number of external devices such as a display device, so that the selected device can display the broadcast-based image.

[0094] The full screen option 514 refers to an option for indicating that the play area of ​​the A image 511 is changed to the full screen. One embodiment describes the full screen option 514, but other options for expanding the play area of ​​the A image 511 to a non-full screen but relatively large area or reducing the play area of ​​the A image 511 are also possible.

[0095] The mobile device may perform mirroring by executing an application supporting a mirroring function. The mobile device transmits capture data obtained by capturing the entire image of the first screen 510 including the A image 511 and the B image 512 to the display device. The capture data may be data decoded by a decoder and buffered to display the entire image including the A image 511 and the B image 512 on the first screen 510. The display device displays a mirror image on the second screen 520 based on the capture data received from the mobile device.

[0096] The image displayed on the second screen 520 is based on mirroring and thus has the same form as the image displayed on the first screen 510. In this state, when the user touches the outward broadcast option 513 or the full screen option 514 displayed on the first screen 510, the operations of the mobile device and the display device may be different from the operations when the user touches the outward broadcast option 523 or the full screen option 524 (i.e., UI TV or UI Full, respectively) displayed on the second screen 520, such as Figure 5 shown.

[0097] In one embodiment, the following four touch input scenarios are possible: the user may touch the full screen option 514 of the first screen 510; the outward broadcast option 513 of the first screen 510; the full screen option 524 of the second screen 520; or the outward broadcast option 523 of the second screen 520. In particular, the operation performed in the mobile device when the user touches the full screen option 514 of the first screen 510 is different from the operation performed in the mobile device when the user touches the full screen option 524 of the second screen 520.

[0098] The outward broadcast option 513 and the full screen option 514 displayed on the first screen 510 are menus generated by the mobile device. Therefore, when the user touches the outward broadcast option 513 or the full screen option 514, the mobile device can easily recognize that the outward broadcast option 513 or the full screen option 514 is touched. However, the mirror image displayed on the second screen 520 is not decoded by the display device, but decoded by the mobile device, and the display device only displays the mirror image. In other words, the outward broadcast option 523 and the full screen option 524 displayed on the second screen 520 are not menus generated by the display device.

[0099] Therefore, when the user touches the outward broadcast option 523 or the full screen option 524 displayed on the second screen 520, the mobile device or the display device recognizes such a touch input using the method described below.

[0100] Figure 6 A method according to an embodiment is shown by which a mobile device recognizes an event corresponding to a touch input occurring in a display device during mirroring.

[0101] like Figure 6 As shown, the mobile device 110 and the display device 120 perform communication to exchange information with each other.

[0102] First, the display device 120 detects a touch input on the second screen. Various techniques may be applied to the structure for detecting the touch input. For example, the display device 120 may have a structure in which a pressure-sensitive or capacitive touch sensing film is stacked on a display having a second screen. As another example, the display device 120 may have a structure including a plurality of infrared sensors arranged around the second screen, and detect infrared interference occurring when a user touches.

[0103] The mobile device 110 displays a play image of content and an image including an OSD menu in operation 610. The mobile device 110 displays an option with an OSD menu on the play image of content while playing the content.

[0104] In operation 620 , the mobile device 110 provides the display device 120 with the captured data of the displayed image.

[0105] In operation 630 , the display device 120 displays a mirror image of the image displayed on the mobile device 110 based on the provided capture data.

[0106] At operation 640 , the display device 120 detects a touch input on the second screen.

[0107] At operation 650, the display device 120 transmits coordinate information about the position on the second screen where the touch input is detected to the mobile device 110. The coordinate information may include, for example, X and Y coordinates of the touch position on the second screen, or coordinates of a pixel area corresponding to the touch position among all pixels of the second screen.

[0108] At operation 660, the mobile device 110 identifies a position on the first screen corresponding to the coordinate information received from the display device 120. When the second screen and the first screen are different in size from each other, the mobile device 110 may identify the touch position by matching the coordinate information on the second screen with the first screen based on a ratio between the size of the second screen and the size of the first screen.

[0109] The mobile device 110 identifies the OSD menu displayed at the identified position on the first screen at operation 670. Therefore, the mobile device 110 identifies that the user touches the OSD menu displayed on the second screen of the display device 120 during mirroring.

[0110] Figure 7 According to an embodiment, when a user Figure 5 An image displayed when the full screen option of the first screen is touched during a mirroring operation is shown.

[0111] Reference Figure 5 and Figure 7 As shown, when the user touches the full screen option 514 of the first screen 510, the mobile device displays a C image 711. The C image 711 is an image that plays the same image content as the A image 511, and has a playback area enlarged from the playback area of ​​the A image. In other words, the C image 711 is a full screen image obtained by enlarging the A image. When the display device is pivotally provided to switch between the portrait mode and the landscape mode, the display device can control the display having the second screen 520 to enter the landscape mode in response to the touch input.

[0112] The C image 711 is an image whose play area is expanded to the full screen, and may additionally display a size restoration option 714, i.e., a UI Small, for receiving an input requesting restoration to the original size of the play area (e.g., the size during the mirroring operation). As another example, the C image 711 may display a UI for receiving an input requesting a reduction in the size of the play area compared to the original size. When the size restoration option 714 is touched, the play area of ​​the C image 711 is reduced to the play area of ​​the A image.

[0113] In this way, when a touch input is made to the full screen option on the first screen 510, the mobile device continues to maintain the transmission protocol in the mirroring state, and changes the state of the image on the first screen 510 based on the full screen option. While maintaining the mirroring state, the display device continues to receive and display the capture data of the C image 711 from the mobile device. Therefore, the state of the image displayed on the first screen 510 is cleverly reflected in the image displayed on the second screen 520, and thus the mirror image of the second screen 520 is in the form of the C image 711 of the first screen 510 as shown previously.

[0114] As described in detail below, an operation when the full screen option 524 of the second screen 520 is touched during a mirroring operation is different from an operation when the user touches the full screen option 514 of the first screen.

[0115] Figure 8 According to an embodiment, when a user Figure 5 Image displayed when the full screen option of the second screen is touched during mirroring operation shown.

[0116] Reference Figure 5 and Figure 8 As shown, when the user touches the full screen option 524 of the second screen 520 , the mobile device controls the image displayed on the first screen 510 to be different from the image displayed on the second screen 520 .

[0117] When the display device detects a touch input, coordinate information about the position on the second screen where the touch input is detected is sent to the mobile device. The mobile device recognizes that the full screen option for the second screen is touched based on the received coordinate information. This recognition method is equivalent to the above reference Figure 6 Describe the method.

[0118] The mobile device continues to display the A image 511 and the B image 512, which are displayed on the first screen 510, even after the touch input is detected. In other words, the image state of the first screen 510 is not changed by the touch input. Therefore, the full screen option 514 for the first screen 510 remains displayed as before.

[0119] However, the mobile device changes the image state of the second screen 520 in response to the touch input. The mobile device terminates the mirroring function of providing the captured data of the image on the first screen 510 to the display device, and changes the transmission protocol to the casting method. When the transmission protocol is changed from mirroring to casting, the mobile device sends a request to the display device, through which the display device is requested to display the D image 821 based on the casting method.

[0120] As described above, in the aforementioned embodiment, the broadcast method includes a method in which the mobile device provides the display device with undecoded image content data, and a method in which the display device provides the address information for obtaining the image content data. In the former method, the mobile device sends the image content data to the display device in the background while displaying the image on the first screen 510.

[0121] When the display device receives the request from the mobile device and obtains the image content data through the broadcast method, the display device decodes the obtained data and displays the D image 821 in full screen mode on the second screen 520. In response to the touch input of the full screen option, the D image 821 has a full screen playback area, which is larger than the D image displayed on the second screen 520. Figure 5 The playback area of ​​the A image is further expanded.

[0122] The data of the image content of the D image 821 is the same as the data of the image content of the A image 511. However, the data of the D image 821 is decoded by the display device, but the data of the A image 511 is decoded by the mobile device.

[0123] The D image 821 may further display a size restoration option 822, i.e., a UI Small, for receiving an input requesting restoration to the original size of the play area (e.g., the size during the mirroring operation). As another example, the D image 821 may display a UI for receiving an input requesting a reduction in the size of the play area compared to the original size. When the size restoration option 822 is touched in the D image 821, the mobile device restores the second screen 520 to Figure 5 In other words, the mobile device changes the transmission protocol from broadcasting to mirroring so that the state of the image displayed on the first screen 510 can be displayed on the second screen 520 as before.

[0124] For example, when the mobile device is a smart phone and the display device is a TV, the smart phone has a smaller screen size than the TV and often performs many processes simultaneously. Although a mobile device suitable for one hand has a screen size of about 10 inches, a TV has been developed with a large screen size of more than 50 inches, exceeding 40 inches. Therefore, in some cases, the decoding performance of the mobile device may not be sufficient to cover the screen of the TV.

[0125] In mirroring, the image displayed on the second screen of the display device is the output of the decoder of the mobile device. This means that enlarging the image displayed on the second screen may result in a significant reduction in the quality of the image. When the playback area of ​​the image displayed on the second screen is expanded, the problem of image quality may become considerable.

[0126] In this regard, when a user input for enlarging the play area of ​​the mirrored image is detected from the display device during mirroring, the mobile device cleverly maintains the image state of the first screen, but switches the mirroring to the broadcast, thereby expanding the play area of ​​the image on the second screen. Therefore, the quality of the image with the expanded play area can be ensured on the second screen.

[0127] In addition, according to one embodiment, an operation intuitively desired by a user is performed for each of a touch input to a first screen of a mobile device and a touch input to a second screen of a display device, thereby providing a mobile device and a display device improved in convenience of user control.

[0128] In an embodiment, user input is performed by touch input performed on a first screen of a mobile device or a second screen of a display device. However, in addition to touch input for an OSD menu on the screen, user input can be performed by various user input interfaces provided in the mobile device or display device. For example, user input for indicating a full screen mode of an image can be implemented by a button, a remote controller, a touch pad, an on-screen keyboard, a stylus, etc. provided in the mobile device or display device.

[0129] Fig. 9 is a flow chart illustrating a method performed by a processor of a mobile device when the mobile device receives a user input for expanding a play area of ​​an image during mirroring according to an embodiment.

[0130] In operation 910, a mobile device displays a first image based on image content data.

[0131] In operation 920, the mobile device provides the capture data of the first image to the display device so that the display device can display the second image as a mirror image of the first image based on the capture data.

[0132] In operation 930, the mobile device detects a user input for indicating that the play area of ​​the first or second image is to be expanded. The expansion of the play area of ​​the image may, for example, mean that the play area of ​​the image currently being displayed is increased in size and displayed as a full screen mode.

[0133] In operation 940, the mobile device identifies whether the detected user input is performed in the mobile device.

[0134] When it is recognized that a user input is made in the mobile device, in operation 950 , the mobile device expands and displays a play area of ​​the first image.

[0135] In operation 960, the mobile device continuously provides the display device with the captured data of the first image displayed together with the expanded play area, so that the second image of the display device can also be displayed together with the expanded play area.

[0136] When it is recognized that the user input is not made in the mobile device but is made in the display device, the mobile device maintains the current play state of the first image without expanding the play area of ​​the first image (operation 970).

[0137] In operation 980, the mobile device performs a broadcast-based operation to expand the playback area of ​​the second image in the display device. The broadcast-based operation includes a method of sending image content data to the display device in the background when the mobile device is displaying the first image, or a method of sending address information storing image content data to the display device and requesting the display device to obtain the image content data based on the address information.

[0138] Below, in Figure 5 In the state of being touched, an operation when the outward broadcast option 513 is touched on the first screen 510 and an operation when the outward broadcast option 523 is touched on the second screen 520 will be described.

[0139] Fig.10 According to one embodiment, when Figure 5 A list of devices displayed on a first screen when the Outcast option is touched on a first screen of a mobile device during a mirroring operation is shown.

[0140] Reference Figure 5 and Fig.10 As shown, when the outward casting option 513 is touched on the first screen 510 during mirroring, the mobile device additionally displays a list 1012 of many external devices with which the mobile device can communicate on the first screen 510. The list 1012 provides options of external devices that are capable of displaying images and support a transmission protocol of a casting method between external devices with which the mobile device can communicate.

[0141] When the user selects a display device that is currently in mirroring or another external device in the list 1012, the mobile device stops mirroring and starts an outward playback operation on the selected display device. In other words, the mobile device can send undecoded image content data to the display device, or provide the display device with address information about a server in which the image content data is stored, so that the display device can obtain the image content data based on the address information.

[0142] For example, when the mobile device displays the A image 511 received from the motion image content service of the Internet, the mobile device performs the following outcasting operation. In response to the selection of the display device on the list 1012, the mobile device stops sending the captured data of the A image 511 to the display device. The mobile device sends a request to the display device for an instruction for executing a service for providing the image content of the A image 511, address information for obtaining the image content, play time information about the current play time point within the total running time of the A image 511, and the like.

[0143] The display device executes an application prepared for accessing the motion picture content service in response to a request from the mobile device, thereby accessing the corresponding service. The display device receives the image content based on the address information of the accessed service, and plays the received image content from the corresponding time point, thereby displaying the D image 821 on the second screen 520.

[0144] As a result, the A image 511 is displayed on the first screen 510, but the D image 821, which is different from the A image 511 in the decoded source, is displayed on the second screen 520. In addition, a control menu for controlling the play state of the D image 821 displayed on the second screen 520 may be displayed on the first screen 510 of the mobile device. In other words, the mobile device may be used as a remote controller for controlling the D image 821 displayed on the display device.

[0145] Fig.11 It is shown that according to an embodiment, when Figure 5 An image displayed when the outcast option is touched on a second screen of a display device during mirroring is shown.

[0146] Reference Figure 5 and 11 As shown, when the outward casting option 523 of the second screen 520 is touched during mirroring, the mobile device performs the outward casting operation as follows. The mobile device does not display a list 1012 (see FIG. 1 ) for selecting an external device to which the casting operation is applied. Fig.10 ). Instead, the mobile device identifies that the display device is selected as a target to undergo a casting operation based on user input being made in the display device.

[0147] The mobile device stops sending the captured data of the A image 511 to the display device, thereby terminating the already executed mirroring. The mobile device sends a request to the display device for an instruction for executing a service for providing the image content of the A image 511, address information for obtaining the image content, play time information about the current play time point within the total running time of the A image 511, and the like.

[0148] The display device executes an application prepared for accessing the motion picture content service in response to a request from the mobile device, thereby accessing the corresponding service. The display device receives the image content based on the address information of the accessed service, and plays the received image content from the corresponding time point, thereby displaying the D image 821 on the second screen 520.

[0149] A control menu 1112 for controlling the play status of the D image 821 displayed on the second screen 520 may be displayed on the first screen 510 of the mobile device. The control menu 1112 may be displayed even on the Fig.10 The control menu 1112 may include various options, such as a play button for indicating the playback of the D image 821, a stop button for stopping the playback of the D image 821, other content play buttons for playing the previous content or the next content in the content block provided by the service, a volume controller for controlling the volume of the D image 821, etc. Therefore, the mobile device may send a control signal for controlling the playback state of the D image 821 to the display device in response to the user's input to the control menu 1112.

[0150] When it is recognized that a user input for an outbound broadcast request is made in the mobile device, the mobile device displays the list on the first screen 510. On the other hand, when it is recognized that a user input for an outbound broadcast request is not made in the mobile device but is made in the display device, the mobile device does not display the list on the first screen 510.

[0151] Fig.12 is a flow chart illustrating a method performed by a processor of a mobile device when the mobile device receives user input requesting outbound broadcasting during mirroring according to an embodiment.

[0152] In operation 1210, the mobile device displays a first image based on image content data.

[0153] In operation 1220, the mobile device provides the capture data of the first image to the display device so that the display device can display the second image, which is a mirror image of the first image based on the capture data.

[0154] At operation 1230, the mobile device detects a user input for requesting an outbound broadcast.

[0155] In operation 1240, the mobile device identifies whether the detected user input is made in the mobile device.

[0156] When it is recognized that a user input is made in the mobile device, the mobile device displays a list of external devices with which communication is possible (operation 1250).

[0157] In operation 1260, the mobile device receives a selection of a display device from among the external devices in the list.

[0158] In operation 1270, the mobile device stops mirroring with respect to the corresponding display device and starts an outward broadcast operation with respect to the corresponding display device. The outward broadcast operation of the mobile device is equivalent to the aforementioned operation.

[0159] When it is identified that no user input is made in the mobile device but a user input is made in the display device, the mobile device proceeds to operation 1270 .

[0160] Below, the operation of the mobile device will be described under the condition that the mobile device and the display device are connected for communication. The mobile device is one of the types of electronic devices, and for ease of description, the display device may be referred to as an external device to be distinguished from the electronic device (eg, mobile device).

[0161] Fig.13 is a flowchart illustrating a control method of a mobile device according to an embodiment.

[0162] like Fig.13 As shown, the following operations are performed by the processor of the mobile device.

[0163] In operation 1310 , the mobile device receives a first user input for requesting display of predetermined moving image content.

[0164] At operation 1320, the mobile device displays an image in a first form in which moving image content is displayed in a partial area of ​​the screen in response to the received first user input. The first form corresponds to a state in which the mobile device can display an image of moving image content in a partial area within the entire screen on which the image is displayed.

[0165] In operation 1330, the mobile device sends image data corresponding to the image in the first form to the display device while displaying the image in the first form. Therefore, the display device can display the same image in the first form as the image displayed on the mobile device, that is, display a mirror image. The mobile device sends the image data corresponding to the image in the first form to the display device based on the first transmission protocol. The image in the first form displayed by the mobile device may include a UI item that is provided to be selectable by the user to change the display form of the motion image content.

[0166] In operation 1340 , the mobile device receives a second user input made in the display device while transmitting image data to the display device and / or while displaying an image on the display device.

[0167] At operation 1350, the mobile device identifies whether the second user input indicates a change in the display form of the moving image content, for example, when the user touches a UI item provided to change the display form of the moving image content on the mirror image displayed on the display device. The display device sends the second user input related to the touched coordinate information to the mobile device. When it is identified that the coordinate information associated with the second user input corresponds to the display position of the UI item, the mobile device can identify that the corresponding UI item is touched.

[0168] When recognizing that the second user input does not indicate a change in the display form of the moving image content, the mobile device maintains the current mirroring operation.

[0169] On the other hand, when it is recognized that the second user input indicates a change in the display form of the motion image content, the mobile device cleverly maintains the display state of the image in the first form (operation 1360). In addition, in operation 1370, the mobile device sends information for displaying the motion image content in a second form different from the first form to the display device. The mobile device sends the information based on a second transmission protocol different from the first transmission protocol. In this case, the display device displays an image of the motion image content based on the received information, wherein the image is not a mirror image of the image being displayed in the mobile device.

[0170] Here, the information for displaying the moving image content in the second form may be, for example, a file of the moving image content stored in a storage unit of the mobile device, or address information of the moving image content may be obtained therefrom. When the information is given as a file of the moving image content, the second transmission protocol may include, for example, a broadcast transmission method.

[0171] When the display device displays an image of the moving image content based on the received information, the mobile device may receive the image displayed in the display device, for example, Figure 8 When the third user input indicates that the moving image content in the first form is to be restored, the mobile device sends image data corresponding to the image currently being displayed in the mobile device to the display device. Therefore, the display device displays a mirror image of the image displayed in the mobile device.

[0172] When the mobile device is displaying an image in a first form and sending image data corresponding to the image in the first form to the display device, a second user input may be made in the mobile device. When it is recognized that the second user input made in the mobile device indicates a change in the display form of the motion image content, for example, UI Full is activated, the mobile device displays the image by changing the display form of the motion image content to a second form, and sends image data corresponding to the motion image content displayed in the second form to the display device. In other words, in this case, the display device cleverly displays a mirror image of the image displayed in the mobile device.

[0173] The operation of the above-mentioned devices can be performed by artificial intelligence provided in the corresponding devices. Artificial intelligence can be applied to various related technical systems by using machine learning algorithms. An artificial intelligence system refers to a computer system with human intelligence or intelligence second only to that of humans. In such a system, a machine, device or system performs learning and recognition autonomously, and improves the accuracy of discrimination and recognition based on accumulated experience. Artificial intelligence is based on basic technology by utilizing machine learning (deep running) technology and algorithms based on autonomous classification and learning features of input data to imitate the functions of human brain perception, recognition, etc.

[0174] For example, the basic technologies may include at least one of language understanding technology for recognizing human language and text, visual understanding technology for recognizing things like human vision, reasoning and prediction technology for recognizing information and logically making inferences and predictions, knowledge representation technology for processing human experience information into knowledge data, and motion control technology for controlling autonomous driving of a vehicle or the movement of a robot.

[0175] Language understanding refers to the recognition, application and processing technology of human language or text, including natural language processing, machine translation, dialogue systems, question and answer, speech recognition and synthesis, etc.

[0176] Reasoning and prediction refer to techniques for identifying information and making logical predictions, including knowledge- and probability-based reasoning, optimization forecasting, preference-based planning, recommendations, etc.

[0177] Knowledge representation refers to the technology of automatically converting human experience information into knowledge data, including knowledge construction such as data creation and classification, knowledge management such as data utilization, etc.

[0178] Embodiments may be implemented as software containing one or more instructions stored in a machine-readable (e.g., computer-readable) storage medium (e.g., internal memory or external memory). The processor may call instructions from the storage medium and may operate according to the called instructions, and the processor includes an electronic device. When the instructions are executed by the processor, the processor may use other components to directly or under the control of the processor to perform functions corresponding to the instructions. The instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. The computer-readable medium may include program instructions, data files, data structures, etc. or a combination thereof. For example, a computer-readable medium may be stored in a non-volatile storage unit such as a universal serial bus (USB) memory, regardless of whether it is removable or rewritable, such as a RAM, ROM, flash memory, memory chip, integrated circuit (IC) or other memory, or an optically recordable or magnetically recordable or machine (e.g., computer) readable storage unit medium. For example, a compact disc (CD), a digital versatile disc (DVD), a magnetic disk, a magnetic tape, etc. It should be understood that the memory that may be included in the mobile terminal is an example of a machine-readable storage unit medium suitable for storing a program having instructions for implementing the embodiments. The program instructions recorded in the storage unit medium may be specifically designed and configured according to the embodiment, or may be used by those skilled in the art. The computer program instructions may be implemented by a computer program product.

[0179] Although certain embodiments have been specifically shown and described with reference to the accompanying drawings, the embodiments are provided for illustrative purposes, and those skilled in the art will appreciate that various modifications and other equivalent embodiments may be made according to the present disclosure. Therefore, the true technical scope of the present disclosure is defined by the technical spirit of the appended claims.

Claims

1. An electronic device, include: A display having a first screen; a communication circuit configured to communicate with an external device; as well as The processor is configured as: displaying a first content image corresponding to the content on a portion of the first screen; transmitting first data corresponding to the first screen to an external device through the communication circuit so that a second content image corresponding to the content is displayed on a part of a second screen of the external device based on the first data, the part of the first screen corresponding to a part of the second screen, and In response to receiving a signal corresponding to a user input for requesting to change the display size of the second content image from the external device, second data is sent to the external device through the communication circuit, so that a third content image corresponding to the content and having a display size different from the display size of the second content image is displayed on the second screen of the external device based on the second data; and the first content image is maintained on a portion of the first screen. 2 . The electronic device according to claim 1 , wherein a display size of the third content image is larger than a display size of the second content image. 3 . The electronic device according to claim 1 , wherein the request to display the content as the third content image includes at least one of a file of the content or address information at which the content can be obtained.

4. The electronic device according to claim 1, wherein the second content image includes a menu item for selecting a change in a display size of the second content image, and The processor is further configured to recognize the user input to request a change in a display size of the second content image based on whether coordinate information on the screen of the external device obtained from the user input corresponds to a position of the menu item.

5. The electronic device of claim 1, wherein the processor is further configured to: sending the first data to the external device through the communication circuit based on a first transmission protocol, and A request to display the content as the third content image is sent based on a second transmission protocol different from the first transmission protocol.

6. The electronic device of claim 1, further comprising a user input receiver, The processor is further configured to: identifying whether a second user input generated by the user input receiver requests a change in a display size of the first content image, displaying the content to include a fourth content image having a display size different from the display size of the first content image on the display based on the second user input being identified as requesting a change in the display size of the first content image, The data of the fourth content image is transmitted to the external device through the communication circuit.

7. The electronic device of claim 1, wherein the processor is further configured to: identifying whether a second user input generated in the external device displaying the third content image requests restoration of a display size of the second content image, and Based on the second user input recognized as requesting restoration of the display size of the second content image, the first data is transmitted again to the external device through the communication circuit.

8. The electronic device of claim 1, further comprising a user input receiver, in, The external device is one of a plurality of external devices connectable to the electronic device, and The processor is further configured to: Based on a second user input generated by the user input receiver and recognized as a request for outbound broadcasting of the content, a list of the plurality of external devices is displayed on the display, and based on a third user input of selecting one of the plurality of external devices in the list, an outbound broadcasting operation of the content is performed to the one of the plurality of external devices.

9. A method for controlling an electronic device, the method include: displaying a first content image corresponding to the content on a portion of the first screen; transmitting first data corresponding to the first screen to an external device so that a second content image corresponding to the content is displayed on a portion of a second screen of the external device based on the first data, the portion of the first screen corresponding to a portion of the second screen, and In response to receiving a signal corresponding to a user input for requesting to change the display size of the second content image from the external device, second data is sent to the external device, so that a third content image corresponding to the content and having a display size different from the display size of the second content image is displayed on the second screen of the external device based on the second data; and the first content image is maintained on a portion of the first screen. 10 . The method of claim 9 , wherein a display size of the third content image is larger than a display size of the second content image. 11 . The method of claim 9 , wherein the request to display the content as the third content image includes at least one of a file of the content or address information at which the content can be obtained.

12. The method according to claim 9, wherein the second content image includes a menu item for selecting a change in a display size of the second content image, and The user input is recognized to request a change in a display size of the second content image based on whether coordinate information on the screen of the external device obtained from the user input corresponds to a position of the menu item.

13. The method according to claim 9, wherein sending the first data to the external device further comprises sending the first data to the external device based on a first transmission protocol, and Sending a request further includes sending a request to display the content as the third content image based on a second transfer protocol different from the first transfer protocol.

14. The method according to claim 9, further comprising: include: identifying whether a second user input generated in the electronic device requests a change in a display size of the first content image; Based on the second user input recognized as requesting a change in the display size of the first content image, displaying the content to include a fourth content image having a display size different from the display size of the first content image; and The data of the fourth content image is transmitted to the external device.

15. The method according to claim 9, further comprising: include: identifying whether a second user input generated in the external device displaying the third content image requests restoration of a display size of the second content image; as well as Based on the second user input being recognized as requesting restoration of the display size of the second content image, the first data is transmitted to the external device again.

Images