Electronic device sharing screen with external electronic device, and control method thereof

Abstract

An electronic device is disclosed. The electronic device may comprise: a communication module; a display; a processor; and a memory including instructions which, when executed by the processor, cause the electronic device to perform operations given below, wherein the instructions are configured to: generate a virtual screen for mirroring a screen displayed on at least a partial region of the display, on the basis of configuration information related to a screen of an external electronic device connected through the communication module; transmit the generated virtual screen to the external electronic device through the communication module; receive changed configuration information from the external electronic device through the communication module while the virtual screen is transmitted; change the virtual screen on the basis of the changed configuration information; and transmit the changed virtual screen to the external electronic device through the communication module.

Description

Electronic device sharing a screen with an external electronic device and a method for controlling the same

[0001] Embodiments of the present disclosure relate to an electronic device that shares a screen with an external electronic device and a method for controlling the same.

[0002] The variety of services and additional features offered through electronic devices, such as smartphones, is steadily increasing. To enhance the utility of these devices and satisfy the diverse needs of users, telecommunications service providers and electronic device manufacturers are competitively developing electronic devices to offer a variety of features and differentiate themselves from competitors. Consequently, the various functions offered through electronic devices are also becoming increasingly sophisticated.

[0003] The variety of services and additional features offered through electronic devices, such as smartphones, is steadily increasing. To enhance the utility of these devices and satisfy the diverse needs of users, telecommunications service providers and electronic device manufacturers are competitively developing electronic devices to offer a variety of features and differentiate themselves from competitors. Consequently, the various functions offered through electronic devices are also becoming increasingly sophisticated.

[0004] When surfing the web or using multimedia features, using an electronic device with a larger screen can be more convenient. While larger displays can be installed on electronic devices to provide a larger screen, the portability of electronic devices can limit the ability to expand the display size.

[0005] To this end, a mirroring function is used to share the screen of the first device by transmitting the display screen of the first device (source) to the second device (sink).

[0006] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0007] According to one embodiment, an electronic device may include a communication module, a display, a processor, and a memory (e.g., memory (130) of FIG. 1) that includes instructions that, when executed by the processor, cause the electronic device to perform the following operations.

[0008] According to one embodiment, the instructions may be configured to generate a virtual screen for mirroring a screen displayed on at least a portion of the display based on configuration information related to a screen of an external electronic device connected via the communication module.

[0009] According to one embodiment, the instructions may be configured to transmit the generated virtual screen to the external electronic device via the communication module.

[0010] According to one embodiment, the instructions may be configured to receive changed setting information from the external electronic device via the communication module while the virtual screen is being transmitted.

[0011] In one embodiment, the instructions may be configured to change the virtual screen based on the changed setting information.

[0012] According to one embodiment, the instructions may be configured to transmit the changed virtual screen to the external electronic device via the communication module.

[0013] According to one embodiment, an electronic device may be configured to include a communication module, a display, a processor, and a memory including instructions that, when executed by the processor, cause the electronic device to perform the following operations.

[0014] According to one embodiment, the instructions may be configured to receive first screen information related to a virtual screen mirroring a screen of an external electronic device connected via the communication module from the external electronic device.

[0015] According to one embodiment, the instructions may be configured to display a first screen generated based on the first screen information on at least a portion of the display.

[0016] According to one embodiment, the instructions may be configured to transmit changed setting information to the external electronic device based on a change in setting information of the display while the first screen information is being received.

[0017] According to one embodiment, the instructions may be configured to generate a second screen corresponding to the changed setting information based on second screen information received through the communication module after transmitting the changed setting information.

[0018] In one embodiment, the instructions may be configured to display the second screen on the display.

[0019] According to one embodiment, a method for controlling an electronic device may include generating a virtual screen for mirroring a screen displayed on at least a portion of a display of the electronic device based on configuration information related to a screen of an external electronic device connected via a communication module of the electronic device.

[0020] According to one embodiment, a method for controlling an electronic device may include an operation of transmitting the generated virtual screen to the external electronic device through the communication module.

[0021] According to one embodiment, a method for controlling an electronic device may include an operation of receiving changed setting information from the external electronic device through the communication module while the virtual screen is transmitted.

[0022] According to one embodiment, a method for controlling an electronic device may include an operation of changing the virtual screen based on the changed setting information.

[0023] According to one embodiment, a method for controlling an electronic device may include an operation of transmitting the changed virtual screen to the external electronic device through the communication module.

[0024] According to one embodiment, a method for controlling an electronic device may include receiving first screen information related to a virtual screen mirroring a screen of an external electronic device connected via a communication module of the electronic device from the external electronic device.

[0025] According to one embodiment, a method of controlling an electronic device may include an operation of displaying a first screen generated based on the first screen information on at least a portion of the display.

[0026] According to one embodiment, a method for controlling an electronic device may include an operation of transmitting changed setting information to an external electronic device based on a change in setting information of a display of the electronic device while the first screen information is being received.

[0027] According to one embodiment, a method for controlling an electronic device may include an operation of generating a second screen corresponding to the changed setting information based on second screen information received through the communication module after transmitting the changed setting information.

[0028] According to one embodiment, a method of controlling an electronic device may include an action of displaying the second screen on the display.

[0029] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs, wherein the one or more programs enable an electronic device to generate a virtual screen for mirroring a screen displayed on at least a portion of a display of the electronic device based on configuration information related to a screen of an external electronic device connected through a communication module of the electronic device.

[0030] According to one embodiment, the one or more programs may include commands for the electronic device to transmit the generated virtual screen to the external electronic device via the communication module.

[0031] According to one embodiment, the one or more programs may enable the electronic device to receive changed setting information from the external electronic device via the communication module while the virtual screen is being transmitted.

[0032] According to one embodiment, the one or more programs may include commands for the electronic device to change the virtual screen based on the changed setting information.

[0033] According to one embodiment, the one or more programs may include commands for the electronic device to transmit the changed virtual screen to the external electronic device via the communication module.

[0034] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs may include instructions for causing an electronic device to receive first screen information related to a virtual screen mirroring a screen of an external electronic device connected through a communication module of the electronic device from the external electronic device.

[0035] According to one embodiment, the one or more programs may include commands that cause the electronic device to display a first screen generated based on the first screen information on at least a portion of a display of the electronic device.

[0036] In one embodiment, the one or more programs may include commands for the electronic device to transmit changed setting information to the external electronic device based on a change in setting information of the display while the first screen information is being received.

[0037] According to one embodiment, the one or more programs may include commands for the electronic device to generate a second screen corresponding to the changed setting information based on the second screen information received through the communication module after transmitting the changed setting information.

[0038] According to one embodiment, the one or more programs may include commands that cause the electronic device to display the second screen on the display.

[0039] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0040] FIG. 2A is a diagram for explaining a mirroring operation when the screen settings of an external electronic device are changed during mirroring between an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0041] FIG. 2b is a diagram for explaining a mirroring operation when the screen settings of an external electronic device are changed during mirroring between an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0042] FIG. 3A is a drawing for briefly explaining the configuration of an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0043] FIG. 3b is a flowchart illustrating an operation for mirroring connection between an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0044] FIG. 3c is a flowchart illustrating a capability check process of an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0045] FIG. 4 is a flowchart illustrating an operation of an electronic device that changes a mirroring screen according to a change in the settings of an external electronic device, according to an embodiment of the present disclosure.

[0046] FIG. 5 is a drawing for explaining a plurality of mirrorable areas of a screen displayed on an electronic device according to an embodiment of the present disclosure.

[0047] FIG. 6A is a drawing for explaining a plurality of mirrorable areas of a screen displayed on an electronic device according to an embodiment of the present disclosure.

[0048] FIG. 6b is a drawing for explaining a plurality of mirrorable areas of a screen displayed on an electronic device according to an embodiment of the present disclosure.

[0049] FIG. 6c is a drawing for explaining a plurality of mirrorable areas of a screen displayed on an electronic device according to an embodiment of the present disclosure.

[0050] FIG. 7 is a diagram illustrating an operation of mirroring additional information based on a screen setting of an external electronic device in an electronic device according to an embodiment of the present disclosure.

[0051] FIG. 8 is a flowchart illustrating an operation of displaying a changed mirroring screen according to a setting change of an external electronic device according to an embodiment of the present disclosure.

[0052] FIG. 9A is a flowchart illustrating an operation of applying a transition effect when displaying a mirroring screen changed according to a setting change of an external electronic device according to an embodiment of the present disclosure.

[0053] FIG. 9b is a flowchart illustrating an operation of applying a transition effect based on a time stamp of an external electronic device according to an embodiment of the present disclosure.

[0054] FIG. 10 is a drawing for explaining an operation of applying a transition effect when displaying a mirroring screen changed according to a setting change of an external electronic device according to an embodiment of the present disclosure.

[0055] FIG. 11A is a drawing for explaining an area displaying a mirroring screen of an external electronic device according to an embodiment of the present disclosure.

[0056] FIG. 11b is a drawing for explaining an area displaying a mirroring screen of an external electronic device according to an embodiment of the present disclosure.

[0057] FIG. 12 is a drawing for explaining an area displaying a mirroring screen of an external electronic device according to an embodiment of the present disclosure.

[0058] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one embodiment. Referring to FIG. 1 , in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0059] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0060] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0061] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0062] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0063] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0064] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0065] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0066] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0067] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0068] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0069] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0070] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0071] The camera module (180) can capture still images and moving images. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0072] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0073] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0074] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0075] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0076] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0077] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0078] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0079] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0080] FIG. 2A is a diagram illustrating a mirroring operation when the external electronic device changes its screen settings during mirroring between an electronic device and an external electronic device according to one embodiment of the present disclosure. For example, FIG. 2 is a diagram illustrating a mirroring operation when the screen settings change as the external electronic device rotates.

[0081] Referring to FIG. 2A, an electronic device (101) (e.g., the electronic device (101) of FIG. 1) (e.g., a source device) may perform screen sharing with an external electronic device (104) (e.g., the electronic device (104) of FIG. 1) (e.g., a sink device). For example, the electronic device (101) may transmit screen information of a first screen (210) displayed on a display (e.g., a display module (160) of FIG. 1) (e.g., a source screen) of the electronic device (101) to the external electronic device (104). For example, the screen information may be information on a virtual screen that mirrors at least a portion of the first screen (210). For example, the information on the virtual screen may include a stream of video data to be displayed on the external electronic device (104).

[0082] According to one embodiment, the external electronic device (104) may display a second screen (220) on the display (e.g., a sync screen) of the external electronic device (104) based on screen information received from the electronic device (101). According to one embodiment, the second screen (220) may be a screen that mirrors the first screen (210).

[0083] According to one embodiment, configuration information of the external electronic device (104) may change as the external electronic device (104) rotates. For example, the configuration information may include the aspect ratio of the screen displayed on the external electronic device (104), the width / height of the screen, and / or the resolution of the screen.

[0084] According to one embodiment, the electronic device (101) may receive changed configuration information of the external electronic device (104) from the external electronic device (104). The electronic device (101) may change the screen information of the virtual screen mirroring the first screen (210) based on the changed configuration information of the external electronic device (104). For example, the electronic device (101) may change the ratio of the virtual screen mirroring the first screen (210), the horizontal / vertical length of the virtual screen, and / or the resolution of the virtual screen based on the changed configuration information of the external electronic device (104). According to one embodiment, as illustrated in FIG. 2A, the electronic device (101) may change the horizontal length of the virtual screen to be shortened and the vertical length to be lengthened based on the horizontal length becoming shorter and the vertical length becoming longer depending on the rotation of the external electronic device (104).

[0085] According to one embodiment, the electronic device (101) may transmit changed screen information to an external electronic device (104). The external electronic device (104) may display a third screen (230) based on the changed screen information. According to one embodiment, the third screen (230) may be a screen that mirrors at least a portion of the first screen (210).

[0086] FIG. 2B is a diagram illustrating a mirroring operation when the screen settings of an external electronic device change during mirroring between an electronic device and an external electronic device according to an embodiment of the present disclosure. For example, FIG. 2B is a diagram illustrating a mirroring operation when the screen settings change according to folding or unfolding of the external electronic device.

[0087] Referring to FIG. 2B, an electronic device (101) (e.g., the electronic device (101) of FIG. 1) can perform screen sharing with an external electronic device (104) (e.g., the electronic device (104) of FIG. 1). For example, the electronic device (101) can transmit screen information of a first screen (210) displayed on a display (e.g., the display module (160) of FIG. 1) (e.g., a source screen) of the electronic device (101) to the external electronic device (104). For example, the screen information may be information on a virtual screen that mirrors at least a portion of the first screen (210). According to one embodiment, the information on the virtual screen may include, for example, a stream of video data that the electronic device (101) is to display on the external electronic device (104).

[0088] According to one embodiment, the external electronic device (104) may display a second screen (240) on the display (e.g., a sync screen) of the external electronic device (104) based on screen information received from the electronic device (101). According to one embodiment, the second screen (320) may be a screen that mirrors the first screen (210).

[0089] According to one embodiment, configuration information of the external electronic device (104) may change as the external electronic device (104) is folded (or unfolded). For example, the configuration information may include the aspect ratio of the screen displayed on the external electronic device (104), the width / height of the screen, and / or the resolution of the screen.

[0090] According to one embodiment, the electronic device (101) may receive changed configuration information of the external electronic device (104) from the external electronic device (104). The electronic device (101) may change screen information of a virtual screen mirroring the first screen (210) based on the changed configuration information of the external electronic device (104). For example, the electronic device (101) may change the ratio of the virtual screen mirroring the first screen (210), the horizontal / vertical length of the virtual screen, and / or the resolution of the virtual screen based on the changed configuration information of the external electronic device (104). According to one embodiment, as illustrated in FIG. 2B, the electronic device (101) may change the horizontal length of the virtual screen to be maintained and the vertical length to be shortened based on the horizontal length being maintained and the vertical length being shortened according to the folding of the external electronic device (104).

[0091] According to one embodiment, the electronic device (101) may transmit changed screen information to an external electronic device (104). The external electronic device (104) may display a third screen (250) based on the changed screen information. According to one embodiment, the third screen (250) may be a screen that mirrors at least a portion of the first screen (210).

[0092] In FIGS. 2A and 2B, the external electronic device (104) is depicted as being a bar-shaped or foldable device, but the type of external electronic device (104) may be a VR (or AR) device or a rollable device.

[0093] According to one embodiment, although the entire screen of the electronic device (101) is illustrated as being shared with the external electronic device (104) in FIGS. 2A and 2B, one window or pop-up screen among the multi-window screens of the electronic device (101) may be mirrored, and the mirrored screen may be displayed on a portion of the display of the external electronic device (104).

[0094] In this way, the electronic device can provide screen information for mirroring based on the setting information of the external electronic device that is sharing the screen, and can change and provide the screen information for mirroring according to changes in the setting information of the external electronic device.

[0095] FIG. 3A is a drawing for briefly explaining the configuration of an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0096] Referring to FIG. 3A, an electronic device (101) (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) (e.g., a source device) may include a socket communication module (310) (e.g., the communication module (190) of FIG. 1), a configuration module (311), a virtual display manager (312), and a Wi-Fi Display (313).

[0097] According to one embodiment, the socket communication module (310) may include a TCP network socket communication module for exchanging configuration information in a message format. According to one embodiment, the socket communication module (310) may be separate from a communication module (e.g., a Wi-Fi display (313)) that transmits and receives display data.

[0098] According to one embodiment, the socket communication module (310) may receive configuration information of an external electronic device (104) (e.g., the electronic device (104) of FIG. 1) (e.g., a sink device). According to one embodiment, the socket communication module (310) may transmit the configuration information of the external electronic device (104) received from the external electronic device (104) to the configuration module (311).

[0099] According to one embodiment, the configuration information may be configuration information related to a display area of ​​an external electronic device (104) on which a virtual screen is to be displayed. According to one embodiment, the configuration information may include at least one of the number, position, size, and resolution of the display area on which the virtual screen is to be displayed.

[0100] According to one embodiment, the configuration module (311) can perform normalization for different configuration ranges and values ​​between heterogeneous devices, and settings for configuration values ​​for which correction work has been completed.

[0101] According to one embodiment, the setting module (311) may include a judgment unit and a setting unit. According to one embodiment, the judgment unit of the setting module (311) may correct the received setting information to be standardized based on the difference between the setting range and the setting value for each setting information.

[0102] According to one embodiment, the setting unit of the setting module (311) can set the setting value for each setting information based on the corrected setting range and setting value.

[0103] According to one embodiment, the virtual screen management unit (312) manages multiple virtual screens that are created or deleted according to the number of displays and / or the number of windows of the external electronic device (104), and when two or more virtual screens are created, the content to be displayed in the ROI area to be transmitted to each screen can be set.

[0104] According to one embodiment, when the electronic device (101) includes multiple displays, multiple windows, or mirrors only a portion of a screen displayed on the electronic device (101), the region to be created as a virtual screen can be set by the ROI setting unit of the virtual screen management unit (312). For example, the ROI setting unit of the virtual screen management unit (312) can set a display, window, or region that the user is viewing as the region to be created as a virtual screen by tracking the user's gaze.

[0105] According to one embodiment, the virtual screen management unit (312) may determine a display and / or window of the external electronic device (104) on which at least one generated virtual screen will be displayed, if the setting information received from the external electronic device (104) includes two or more displays and / or windows and information about the location of each display and / or window.

[0106] For example, the virtual screen management unit (312) may perform a 1:N, N:1, or N:N mirroring operation based on the number of screens (e.g., displays and / or windows) of the electronic device (101) to be created as virtual screens and the number of screens of the external electronic device (104) on which the virtual screens will be displayed. According to an embodiment, when there are multiple external electronic devices (104) that perform the mirroring operation with the electronic device (101), or when there are multiple displays and / or windows that perform mirroring on one external electronic device (104), applicable setting information (capability check) may be checked on each external electronic device (104). According to an embodiment, each external electronic device (104) may manage changes in settings and perform mirroring with the electronic device (101) according to set judgment conditions. According to an embodiment, the 1:N, N:1, or N:N mirroring operation will be described in more detail below with reference to FIG. 4.

[0107] According to one embodiment, the virtual screen management unit (312) may transmit screen information for at least one generated virtual screen to an external electronic device (103) via a Wi-Fi display (313). For example, the Wi-Fi display (313) may encode at least one virtual screen generated by the virtual screen management unit (312) and transmit it to an external electronic device (104) via a network session.

[0108] According to one embodiment, an external electronic device (104) (e.g., the electronic device (104) of FIG. 1) (e.g., a sink device) may include a socket communication module (320), a Wi-Fi Display (321), a configuration module (322), and a display / window module (323).

[0109] According to one embodiment, the socket communication module (320) can transmit setting information to the electronic device (101). According to one embodiment, the socket communication module (320) can transmit setting information received from the setting module (322) to the electronic device (101).

[0110] According to one embodiment, the configuration information may be configuration information related to an area where a virtual screen of an external electronic device (104) is to be displayed. According to one embodiment, the configuration information may include at least one of the number, position, size, and resolution of the area where the virtual screen is to be displayed.

[0111] According to one embodiment, the Wi-Fi display (321) can decode virtual screen information received from the electronic device (101) and perform rendering. According to one embodiment, the Wi-Fi display (321) can check the rendering time of a video frame to which changed settings have been applied among the screen information received from the electronic device (101) through a frame rendering listener. According to one embodiment, the frame rendering listener can transmit information about the rendering time of a video frame to which changed settings have been applied to the setting module (322).

[0112] According to one embodiment, when the setting module (322) receives information about a video frame to which a changed setting has been applied, it can determine whether to apply a screen transition effect and the type of transition effect.

[0113] In one embodiment, the configuration module (322) may recognize changes in settings for each display and / or window. For example, the configuration module (322) may recognize creation, deletion, or change of a display and / or window based on receiving change information from the display / window module (323) based on the creation or deletion of a display and / or window, the change of a display and / or window due to folding or rotation, or the change of a ROI where a virtual screen is to be displayed due to a user's touch input or gaze tracking.

[0114] According to one embodiment, the setting module (322) can transmit changed settings of the display / window to the socket communication module (320). The external electronic device (104) can transmit changed settings information of the display / window to the electronic device (101) through the socket communication module (320).

[0115] FIG. 3b is a flowchart illustrating an operation for mirroring connection between an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0116] Referring to FIG. 3b, an electronic device (101) (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) (e.g., a source device) may hold off on an RTSP Network connection process (340) for mirroring with an external electronic device (104) (e.g., the electronic device (104) of FIG. 1) (e.g., a sink device), and may perform a capability check process (330) before the RTSP Network connection process (340).

[0117] According to one embodiment, the capability check process (330) may be performed through a TCP connection separate from the communication for transmitting and receiving display data for the exchange of screen setting information. According to one embodiment, the capability check process (330) may be performed by sharing the screen setting information of the electronic device (101) and the screen setting information of the external electronic device (104) through the TCP connection. According to one embodiment, the capability check process (330) will be described in more detail below with reference to FIG. 3c.

[0118] FIG. 3c is a flowchart illustrating a capability check process of an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0119] Referring to FIG. 3c, in operation 350, the electronic device (101) (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may connect a separate TCP IP socket to exchange configuration information with an external electronic device (104) (e.g., the electronic device (104) of FIG. 1). For example, the electronic device (101) may connect a TCP IP socket separate from a communication module that transmits and receives display data with the external electronic device (104).

[0120] According to one embodiment, in operation 351, the electronic device (101) may share a capability with an external electronic device (104) for capability check. According to one embodiment, the capability may include changeable configuration type information related to the screen (e.g., configuration capability). For example, the changeable configuration type information related to the screen may include the size, ratio, resolution, rotation angle, mirroring target, and mirroring count of the mirroring screen.

[0121] According to one embodiment, the electronic device (101) may transmit the capability to an external electronic device (104) in the form of a message (e.g., a string).

[0122] [Method] [Version] [CRLF]

[0123] [Content-Length] [CRLF]

[0124] [Body] [CRLF]

[0125] According to one embodiment, the electronic device (101) may perform a message exchange for sharing the capability of the electronic device (101) and acquiring the capability of an external electronic device (104) through the GET method during the capability check process. According to one embodiment, a message transmitted by the electronic device (101) to the external electronic device (104) through the GET method may be as shown in the example below.

[0126] GET SECOND-SCREEN / 1.0

[0127] Content-Length:41

[0128] Configuration:Rotation|FontSize|Bluelight

[0129] According to one embodiment, the external electronic device (104) can store the setting type information of the received electronic device (101).

[0130] According to one embodiment, in operation 352, the external electronic device (104) may transmit an INFO method to the electronic device (101) in response to sharing the capability of the electronic device (101). According to one embodiment, the INFO method may include configuration information related to the screen of the external electronic device (104). For example, the external electronic device (104) may transmit, to the electronic device (101), configuration information applicable to the external electronic device (104) among the received capabilities of the electronic device (101). For example, the external electronic device (104) may transmit, to the electronic device (101), configuration information including the size, ratio, resolution information of the area where the mirroring screen is to be displayed, and / or folding or rolling sensing data. According to one embodiment, a message transmitted by the external electronic device (104) to the electronic device (101) via the INFO method may be as shown in the example below.

[0131] INFO SECOND-SCREEN / 1.0

[0132] Content-Length:31

[0133] Rotation:0

[0134] FontSize:3,0-4

[0135] According to one embodiment, the electronic device (101) can store the setting information of the received external electronic device (104).

[0136] According to one embodiment, the electronic device (101) and the external electronic device (104) may perform negotiation for mirroring.

[0137] According to one embodiment, even if a WiFi-Direct P2P connection with an external electronic device (104) is completed, the electronic device (101) may hold off on immediately proceeding with the WiFi-Display RTSP Negotiation process in order to check information related to the type of changeable screen settings (e.g., configuration capability).

[0138] According to one embodiment, in the RTSP Network connection process for WiFi-Display connection, the external electronic device (104) as a sink device becomes a Network Socket Client, and the electronic device (101) as a source device becomes a Network Socket Server. Instead of performing a direct connection from the external electronic device (104) to the electronic device (101), the connection can be put on hold for a display setting capability check process.

[0139] According to one embodiment, the electronic device (101) may transmit information related to the type of changeable screen settings (e.g., configuration capability) to the external electronic device (104), and then perform negotiation for mirroring after receiving the configuration information from the external electronic device (104).

[0140] According to one embodiment, in operation 353, the electronic device (101) may perform initial setup based on the INFO method. For example, the electronic device (101) may generate a virtual screen based on setup information received from an external electronic device (101).

[0141] In one embodiment, in operation 354, the electronic device (101) may connect to an RTSP server. In one embodiment, the external electronic device (104) may become a Network Socket Client, and the electronic device (101) may become a Network Socket Server.

[0142] In one embodiment, in operation 355, the electronic device (101) and the external electronic device (104) may be connected to a Wi-Fi display. In one embodiment, the electronic device (101) may transmit a virtual screen to the external electronic device (104).

[0143] According to one embodiment, the electronic device (101) may transmit and receive setting information of the external electronic device (104) through a channel (or a separate TCP socket) separate from the channel of the communication module that transmits and receives screen information for mirroring with the external electronic device (104). For example, the electronic device (101) may transmit and receive screen information for mirroring (e.g., mirroring stream) through a first port of Wi-Fi connected to the external electronic device (104), and may transmit and receive setting information through a second port of Wi-Fi.

[0144] According to one embodiment, the electronic device (101) may transmit and receive setting information with an external electronic device (104) through a communication module that communicates in a different manner from the communication module that transmits and receives screen information for mirroring. For example, the electronic device (101) may transmit and receive screen information for mirroring with the external electronic device (104) through Wi-Fi, and may also transmit and receive setting information through Bluetooth.

[0145] According to one embodiment, in operation 356, the external electronic device (104) may register a listener when the W-Fi Display connection is completed. According to one embodiment, the external electronic device (104) may only register a listener for the setting items received from the electronic device (101). The external electronic device (104) may monitor whether the corresponding setting has been changed through the registered listener.

[0146] According to one embodiment, the external electronic device (104) can display a virtual screen received from the electronic device (101). According to one embodiment, since the virtual screen is generated based on the setting information of the external electronic device (104), the virtual screen can be displayed to fill the area where the mirroring screen of the external electronic device (104) is to be displayed.

[0147] In one embodiment, in operation 357, the external electronic device (104) may change its configuration. For example, the external electronic device (104) may change configuration information based on rotation, folding / unfolding, rolling / unrolling, touch input, gaze tracking, adding an area to display the mirrored screen, changing the area to display the mirrored screen to another area, or changing the size and / or ratio of the area to display the mirrored screen. In one embodiment, the external electronic device (104) may recognize the configuration change through a listener.

[0148] In one embodiment, in operation 358, the external electronic device (104) may transmit a message regarding a configuration change to the electronic device (101). In one embodiment, the external electronic device (104) may transmit the name and value of the changed configuration to the electronic device (101) in the form of a 'key:value' message.

[0149] For example, an external electronic device (104) can transmit a message of the following form to an electronic device (101).

[0150] SET SECOND-SCREEN / 1.0

[0151] Content-Length:11

[0152] Rotation:90

[0153] According to one embodiment, the external electronic device (104) may transmit changed setting information to the electronic device (101) through a second port different from the first port through which the mirroring stream of the communication module is transmitted and received.

[0154] According to one embodiment, the operation after receiving a message related to changed setting information from an external electronic device (104) will be described in more detail with reference to FIG. 4 below.

[0155] According to one embodiment, the operation of the external electronic device (104) after transmitting the changed setting information to the electronic device (101) will be described in more detail with reference to FIG. 8 below.

[0156] FIG. 4 is a flowchart illustrating an operation of an electronic device that changes a mirroring screen according to a change in the settings of an external electronic device, according to an embodiment of the present disclosure.

[0157] Referring to FIG. 4, in operation 410, an electronic device (e.g., the electronic device (101) of FIG. 1 or the processor (120) of FIG. 1) may generate a virtual screen for mirroring a screen displayed on at least a portion of a display (e.g., the display module (160) of FIG. 1) based on setting information related to a screen of an external electronic device (e.g., the electronic device (104) of FIG. 1).

[0158] According to one embodiment, an electronic device may negotiate with an external electronic device to perform mirroring (e.g., Miracast). According to one embodiment, before negotiating with the external electronic device for mirroring, the electronic device may transmit information related to the types of changeable screen settings (e.g., configuration capability) to the external electronic device via a communication module (e.g., the communication module (190) of FIG. 1 ).

[0159] According to one embodiment, even if a WiFi-Direct P2P connection with an external electronic device is completed, the electronic device may hold off on performing the WiFi-Display RTSP Negotiation process directly in order to check information (e.g., configuration capability) related to the types of changeable screen settings. According to one embodiment, in the RTSP Network connection process for WiFi-Display connection, the external electronic device (104), which is a sink device, becomes a Network Socket Client, and the electronic device (101), which is a source device, becomes a Network Socket Server. Instead of performing a direct connection from the external electronic device (104) to the electronic device (101), the connection may be held off for the display setting capability check process.

[0160] According to one embodiment, the electronic device may, during a capability check process, transmit information related to the types of changeable screen settings (e.g., configuration capability) to the external electronic device and then receive configuration information from the external electronic device. According to one embodiment, the configuration information received from the external electronic device may include the number of displays / windows capable of displaying a mirrored screen on the external electronic device, their positions, the screen ratio, the horizontal / vertical length of the screen, and / or the screen resolution.

[0161] According to one embodiment, the electronic device may generate a virtual screen for mirroring based on configuration information received from an external electronic device. According to one embodiment, the electronic device (e.g., the virtual screen management unit (312) of FIG. 3A) may generate a virtual screen corresponding to the horizontal length, vertical length, and / or resolution of the display and / or window of the external electronic device based on the received configuration information.

[0162] According to one embodiment, when there are multiple displays and / or windows in the received setting information, the electronic device (e.g., the ROI setting unit of the virtual screen management unit (312) of FIG. 3A) can set an area to be displayed on each display and / or window among the contents currently displayed on the electronic device (101).

[0163] According to one embodiment, based on the fact that the electronic device and the external electronic device are heterogeneous devices, the electronic device (e.g., the judgment unit of the configuration module (311) of FIG. 3A) can perform a correction operation (e.g., a standardization operation) on the configuration range and values.

[0164] According to one embodiment, the electronic device may transmit and receive configuration information of the external electronic device through a separate channel (or separate TCP socket) from the channel of the communication module that transmits and receives screen information for mirroring with the external electronic device. For example, the electronic device may transmit and receive screen information for mirroring (e.g., a mirroring stream) through a first Wi-Fi port connected to the external electronic device, and transmit and receive configuration information through a second Wi-Fi port.

[0165] According to one embodiment, when an electronic device connects to a separate channel (e.g., a TCP socket) for exchanging configuration information, the electronic device may perform a capability check process through the channel. According to one embodiment, during the capability check process, the electronic device may transmit information on screen configuration types supported by the electronic device to an external electronic device. According to one embodiment, after transmitting the screen configuration type information to the external electronic device, the electronic device may receive, in response, configuration information applicable to the external electronic device among the transmitted screen configuration types. According to one embodiment, the capability check process is the same as operations 350, 351, and 352 of FIG. 3C, and thus, redundant descriptions are omitted.

[0166] According to one embodiment, the electronic device may transmit and receive configuration information with an external electronic device via a communication module that communicates in a different manner than the communication module that transmits and receives screen information for mirroring. For example, the electronic device may transmit and receive screen information for mirroring with the external electronic device via Wi-Fi, and may also transmit and receive configuration information via Bluetooth.

[0167] According to one embodiment, the electronic device may perform mirroring with a plurality of external electronic devices, or may perform an operation of setting an area to be mirrored among the screens displayed on the electronic device based on the number of displays or the number of windows of one external electronic device.

[0168] In one embodiment, in operation 420, the electronic device may transmit the generated virtual screen to an external electronic device.

[0169] In one embodiment, in operation 430, the electronic device may receive changed setting information from an external electronic device while the virtual screen is being transmitted.

[0170] According to one embodiment, the changed setting information may include a first ratio of the horizontal length and vertical length of the screen of the external electronic device.

[0171] In one embodiment, the changed configuration information may include rotation information related to a rotation angle of the external electronic device.

[0172] In one embodiment, the changed configuration information may further include second configuration information that is different from the configuration information. For example, the second configuration information may be configuration information for an additional or created display or window other than the existing display or window of the external electronic device. For example, if a new window is created on the screen of the external electronic device, 1:N or N:N mirroring may be performed.

[0173] In one embodiment, the configuration information received from the external electronic device may be a message, and the message may be in a format of Key:Value. In one embodiment, the format of the configuration information corresponding to Value may include a defined value (e.g., a specific number such as the degree of rotation, defined as on or off such as blue light), a defined form (e.g., a form composed of (number) x (number) such as resolution), and a value / value range (e.g., a minimum-maximum range when values ​​are standardized for compatibility between heterogeneous devices such as font size and screen zoom).

[0174] According to one embodiment, the form of the setting information corresponding to the Key may include rotation information (e.g., 0, 90, 180, 270), blue light filter (e.g., on or off), dark theme (e.g., on or off), resolution (e.g., width x height), font size (e.g., 1 and 0 (minimum) - 5 (maximum) range), screen magnification (e.g., 100, 100 (minimum) - 500 (maximum) range).

[0175] According to one embodiment, the external electronic device may obtain rotation information through a sensor, and identify the display base direction as one of values ​​0, 90, 180, and 270 based on the rotation information, with '0' as the reference.

[0176] According to one embodiment, when an electronic device receives a message regarding configuration information from an external electronic device, the electronic device may obtain (e.g., parse) and store the configuration information contained in the message.

[0177] In one embodiment, if the electronic device and the external electronic device are of the same type, the electronic device can directly apply the acquired setting information without a separate standardization operation. In another embodiment, if the electronic device and the external electronic device are of different types, the electronic device can perform a standardization operation depending on the difference in the value range. For example, the electronic device can obtain a standardized value by calculating the ratio of the setting value of the external electronic device and the (maximum value - minimum value), and can obtain the setting value of the corresponding setting by multiplying the (maximum value - minimum value) of the corresponding setting of the electronic device by the standardized value.

[0178] For example, if the value related to the font size included in the setting information received from the external electronic device is 1 and the range is 0-5, the electronic device can obtain 0.2, which is the ratio of the setting value 1 and (5-0), as a standardized value.

[0179] In one embodiment, if the range of the font of the electronic device is 0-6, the standardized value of (6-0) can be multiplied by 0.2 to obtain the standardized value of 1.2.

[0180] According to one embodiment, the electronic device can transmit the acquired values ​​and ranges to an external electronic device by including them in the screen information of a virtual screen for mirroring.

[0181] In one embodiment, in operation 440, the electronic device may change the virtual screen based on the changed setting information.

[0182] In one embodiment, the electronic device can change the horizontal and vertical ratios of the virtual screen based on the horizontal and vertical ratios of the screen of the external electronic device.

[0183] According to one embodiment, the electronic device can change the horizontal and vertical lengths of the virtual screen based on rotation information. According to one embodiment, the electronic device can set the orientation of the display before rotation as '0' and, based on the rotation information acquired by the sensor, identify the rotation information as one of 0, 90, 180, and 270. According to one embodiment, the electronic device can maintain the horizontal and vertical lengths when the rotation information is 0 or 180. According to one embodiment, the electronic device can change the horizontal length before rotation to the vertical length and the vertical length before rotation to the horizontal length when the rotation information is 90 or 270.

[0184] According to one embodiment, the electronic device may generate a second virtual screen based on second configuration information for a new display or window of the external electronic device. According to one embodiment, the second virtual screen may be the same content as the existing virtual screen with a different ratio, or may be separate content. According to one embodiment, in the case of separate content, the second virtual screen may be a virtual screen with a different ROI from the existing virtual screen within the entire area of ​​the same content, or may be separate content displayed in a different area from the content of the existing virtual screen.

[0185] According to one embodiment, the changed virtual screen may further include information indicating that the changed virtual screen reflects the changed configuration information. For example, the information indicating that the changed configuration information has been reflected may be a timestamp. For example, the electronic device may include a timestamp in a frame of a virtual screen generated by reflecting the changed configuration information. According to one embodiment, the timestamp may be a PTS included in a PES Header. The PTS (presentation time stamp) is a 33-bit field and is output time information. The value of the PTS is a value whose unit is a cycle obtained by dividing the system clock frequency (usually 90 kHz) by 300, and only the upper 33 bits are used. The external electronic device can obtain the output time from the external electronic device by adding the time difference between the PTS and the PCR (program clock reference) value indicating the reference time to the system time (RTC) at the time of PTS reception.

[0186] According to one embodiment, in operation 450, the electronic device may transmit the changed virtual screen to an external electronic device. For example, the electronic device may transmit a virtual screen with a changed screen ratio, size, or resolution to the external electronic device based on configuration information of the external electronic device.

[0187] In one embodiment, when a new virtual screen, a second virtual screen, is created, the electronic device can transmit the second virtual screen to an external electronic device.

[0188] In one embodiment, the electronic device may change the virtual screen to a virtual screen for mirroring a screen of the second area based on receiving changed setting information from an external electronic device that includes information related to a first area corresponding to the virtual screen and a second area different from the first area.

[0189] According to one embodiment, the electronic device may transmit information about multiple regions (e.g., multiple ROIs) of a display of the electronic device to an external electronic device.

[0190] According to one embodiment, the electronic device can transmit information about each of a plurality of areas of the display of the electronic device to different external electronic devices.

[0191] According to one embodiment, the information regarding the plurality of regions transmitted to the external electronic device may include thumbnails of the mirrored screens of each region. For example, the electronic device may transmit thumbnail screens of the plurality of mirrorable regions from the external electronic device.

[0192] For example, the multiple regions may be multiple ROIs within a single content as illustrated in FIG. 5, or may be multiple windows displayed on a display of an electronic device as illustrated in FIGS. 6A to 6C. For example, N:1 mirroring or N:N mirroring may be performed, mirroring the screens of two or more regions among the multiple regions onto the screens of one or more external electronic devices.

[0193] According to one embodiment, the types of multiple areas of the display of the electronic device will be described in more detail with reference to FIGS. 5, 6a, 6b and 6c below.

[0194] In one embodiment, the external electronic device may display thumbnail screens of the received plurality of areas and prompt the user to select a screen to mirror.

[0195] In one embodiment, if the selected screen is different from the screen being mirrored, the electronic device may change the existing screen for mirroring to a virtual screen for mirroring the screen in the area corresponding to the selected screen.

[0196] In one embodiment, when a window is added (e.g., in a PIP format or a multi-window format) from an external electronic device, the electronic device may receive the same setup information from the external electronic device as the display added.

[0197] For example, when a window is added, such as in the form of a PIP, on an external electronic device, the existing window is maintained, so the electronic device can only receive information about the new window (e.g., second screen).

[0198] For example, information about a new window might be:

[0199] SET SECOND-SCREEN / 1.0

[0200] Content-Length:75

[0201] Display:Add

[0202] DisplayNumber:1

[0203] Resolution: 1920x1080

[0204] Rotation:0

[0205] FontSize:3,0-4

[0206] With reference to this, information about the new window may include the size of the window (content-length), whether the window is added (display:add), display number (display number), resolution information (resolution), rotation information (rotation), and font size (fontsize).

[0207] In one embodiment, when a new window operates as a multi-window, a resize of an existing window occurs, so that the electronic device can receive information about the new window (e.g., second screen) and information about the changed existing window.

[0208] For example, information about a new window (e.g., a second screen) and information about an existing window that has been changed might be as follows:

[0209] SET SECOND-SCREEN / 1.0

[0210] Content-Length:110

[0211] Display:Add

[0212] DisplayNumber:1

[0213] Resolution: 640x480

[0214] Rotation:0

[0215] FontSize:3,0-4

[0216] DisplayNmbuer:0

[0217] Resolution: 640x480

[0218] In this context, information about a new window may include the window's size (content-length), whether the window has been added (display:add), the display number (display number) (e.g., 1), resolution information (resolution), rotation information (rotation), and font size (fontsize). Information about an existing window that has been changed may include the display number (e.g., 0) and resolution information.

[0219] In one embodiment, upon receiving information about a new window from an external electronic device or receiving user input (e.g., a long click) to create a new window through the display of the electronic device, the electronic device may additionally create a virtual screen.

[0220] In one embodiment, when receiving information about a new window from an external electronic device, the electronic device may additionally create a virtual screen that mirrors the home screen if the electronic device is a VR (or AR) device, the background screen if the electronic device is a smartphone or tablet PC, or the extended home screen including the taskbar if the electronic device is a PC.

[0221] According to one embodiment, a virtual screen transmitted to an external electronic device can be displayed and controlled by the external electronic device. According to one embodiment, when the configuration information of the external electronic device is changed (e.g., rotation, folding / unfolding, rolling / unrolling, or window resizing), the changed configuration information can be transmitted to the electronic device. The electronic device can change the virtual screen based on the changed configuration information.

[0222] In one embodiment, when the electronic device is a VR (or AR) device that is free to create windows, or a smartphone or tablet PC, the electronic device can additionally create a virtual screen to be mirrored based on user input (e.g., long-clicking a second screen function button) while connected to an external electronic device.

[0223] In one embodiment, when an electronic device receives a user input for mirroring multiple screens (e.g., a long click on a second screen function button), the electronic device may additionally create a virtual screen identical to the configuration information of an existing virtual screen. In one embodiment, the additionally created virtual screen may be a virtual screen that mirrors the home screen if the electronic device is a VR (or AR) device, the background screen if the electronic device is a smartphone or tablet PC, or the extended home screen including the taskbar if the electronic device is a PC.

[0224] According to one embodiment, the electronic device may transmit information about the added virtual screen to an external electronic device. For example, the information about the added virtual screen may be as follows.

[0225] INFO SECOND-SCREEN / 1.0

[0226] Content-Length:24

[0227] Display:Add

[0228] DisplayNumber:1

[0229] According to one embodiment, an external electronic device that receives information about an added virtual screen may provide a menu for selecting a screen to be mirrored. For example, the menu may include a number of virtual screens and / or thumbnail images of the electronic device. For example, the number of virtual screens and / or thumbnail images of the electronic device may each correspond to a plurality of areas of the electronic device's display.

[0230] In one embodiment, when a number and / or thumbnail screen of a virtual screen is selected through control from an external electronic device, the external electronic device may transmit the following information to the electronic device to perform mirroring to the selected virtual screen.

[0231] SET SECOND-SCREEN / 1.0

[0232] Content-Length:12

[0233] DisplayNumber:1

[0234] According to one embodiment, when the electronic device receives information from an external electronic device to change an added virtual screen to a selected screen, the electronic device can change the virtual screen generated based on area 0 of the display to be generated based on area 1 of the display.

[0235] This allows the electronic device to change areas and share its screen with external electronic devices.

[0236] In this way, the electronic device can perform 1:N, N:1 or N:N mirroring operations even in 1:1 mirroring operations without additional connection to external electronic devices.

[0237] In one embodiment, a 1:N mirroring operation may mirror a single source to multiple sinks. For example, the same source screen may be mirrored to multiple sink screens, and the sink screens may be mirrored differently depending on the ROI area (e.g., a face) of the source screen or the aspect ratio (e.g., the width-to-height ratio) of the sink screen.

[0238] In one embodiment, the electronic device may mirror different areas of the source screen on each of the sync screens based on preset conditions (e.g., mirroring the entire source screen on the sync screen being viewed by the user, mirroring each object of interest included in the source screen on other sync screens, or performing mirroring based on objects of interest of multiple users).

[0239] In one embodiment, the N:1 mirroring operation may be an operation that mirrors multiple source screens together on a single sync screen. For example, since multiple source screens must be merged and mirrored on a single sync screen, the electronic device may configure the mirrored source screens differently in various ways depending on the aspect ratio (e.g., horizontal / vertical ratio) of each source screen or the configuration of the sync screen.

[0240] In one embodiment, when it is not necessary to display all of the multiple source screens at once, the source screens with the highest priority may be mirrored sequentially on the sync screen, or only some of the multiple source screens may be selectively mirrored on the source screen based on preset conditions.

[0241] In one embodiment, the N:N mirroring operation may mirror multiple source screens onto multiple sink screens. In one embodiment, the electronic device may make a decision (e.g., a policy decision) on how to arrange the multiple source screens based on various logic and judgment criteria.

[0242] For example, the electronic device may display a source screen with a higher priority among a plurality of source screens (e.g., including an object of user interest or content with a higher priority) on a sync screen with the highest priority among a plurality of sync screens (e.g., a sync screen in the direction of the user's gaze through gaze tracking).

[0243] According to one embodiment, when a change in the settings of the sync screen of an external electronic device occurs during 1:N, N:1, and N:N mirroring operations, the changed settings information is transmitted to the electronic device, and the electronic device can change the virtual screen for mirroring based on the changed settings information and transmit the changed virtual screen to the external electronic device.

[0244] Due to this, the electronic device can provide a mirrored screen that matches the sync screen of the external electronic device even in 1:N, N:1 and N:N mirroring operations.

[0245] FIG. 5 is a drawing for explaining a plurality of mirrorable areas of a screen displayed on an electronic device according to an embodiment of the present disclosure.

[0246] Referring to FIG. 5, an electronic device (e.g., an electronic device (101) of FIG. 1 or a processor (120) of FIG. 1) can mirror an entire area (511) of content (510) displayed on a display (e.g., a display module (160) of FIG. 1) and / or a partial area (512) of content (510).

[0247] For example, the electronic device may transmit a first virtual screen (520) for mirroring the entire area (511) of the content (510) to the external electronic device, or transmit a second virtual screen (521) for mirroring a partial area (512) of the content (510) to the external electronic device, based on setting information of the external electronic device (e.g., the electronic device (104) of FIG. 1).

[0248] According to one embodiment, when there are multiple external electronic devices or the external electronic devices are multi-window, the electronic device may transmit both a first virtual screen (520) for mirroring the entire area (511) of the content (510) and a second virtual screen (521) for mirroring a partial area (512) of the content (510) to the external electronic device based on setting information of the multiple screens. According to one embodiment, the first virtual screen (520) and the second virtual screen (521) may be transmitted to the same external electronic device or may be transmitted to different external electronic devices, respectively.

[0249] FIG. 6A is a drawing for explaining a plurality of mirrorable areas of a screen displayed on an electronic device according to an embodiment of the present disclosure.

[0250] Referring to FIG. 6A, an electronic device (e.g., an electronic device (101) of FIG. 1 or a processor (120) of FIG. 1) can mirror the screen of a first window (610) and / or the screen of a second window (611) to at least one external electronic device (e.g., an electronic device (104) of FIG. 1) when a display (e.g., a display module (160) of FIG. 1) operates as a multi-window.

[0251] For example, the electronic device can transmit at least one of a virtual screen for mirroring the screen of the first window (610) or a virtual screen for mirroring the screen of the second window (611) to one external electronic device. The electronic device can transmit the virtual screen for mirroring the screen of the first window (610) to the first external electronic device and the virtual screen for mirroring the screen of the second window (611) to the second external electronic device. The electronic device can transmit the virtual screen for mirroring the screen of the first window (610) and the virtual screen for mirroring the screen of the second window (611) to the first external electronic device and the second external electronic device.

[0252] FIG. 6b is a drawing for explaining a plurality of mirrorable areas of a screen displayed on an electronic device according to an embodiment of the present disclosure.

[0253] Referring to FIG. 6b, the electronic device can mirror the screen of the entire area (620) of the content displayed on the display, the screen of the first ROI area (621), and / or the screen of the second ROI area (622) to at least one external electronic device.

[0254] For example, the electronic device can transmit at least one of a virtual screen for mirroring the screen of the entire area (620) of the content, a virtual screen for mirroring the screen of the first ROI area (621), or a virtual screen for mirroring the screen of the second ROI area (622) to one external electronic device. The electronic device can transmit the virtual screen for mirroring the screen of the entire area (620) of the content, the virtual screen for mirroring the screen of the first ROI area (621), or the virtual screen for mirroring the screen of the second ROI area (622) to different external electronic devices, respectively. The electronic device can transmit the virtual screen for mirroring the screen of the entire area (620) of the content, the virtual screen for mirroring the screen of the first ROI area (621), or the virtual screen for mirroring the screen of the second ROI area (622) to multiple external electronic devices.

[0255] FIG. 6c is a drawing for explaining a plurality of mirrorable areas of a screen displayed on an electronic device according to an embodiment of the present disclosure.

[0256] Referring to FIG. 6C, if the electronic device includes a pop-up screen on the display, the electronic device can mirror the entire screen (630) excluding the pop-up screen (631) and / or the pop-up screen (631) to at least one external electronic device. For example, the entire screen (630) excluding the pop-up screen (631) may include objects covered by the pop-up screen (631) when the pop-up screen (631) is not displayed.

[0257] For example, the electronic device can transmit at least one of a virtual screen for mirroring the entire screen (630) excluding the pop-up screen (631) or a virtual screen for mirroring the pop-up screen (631) to one external electronic device. The electronic device can transmit the virtual screen for mirroring the entire screen (630) excluding the pop-up screen (631) to the first external electronic device and the virtual screen for mirroring the pop-up screen (631) to the second external electronic device. The electronic device can transmit the virtual screen for mirroring the entire screen (630) excluding the pop-up screen (631) and the virtual screen for mirroring the pop-up screen (631) to the first external electronic device and the second external electronic device.

[0258] According to one embodiment, when an execution screen running on an electronic device is displayed on a pop-up screen (631) (e.g., Appcast), the electronic device may use the pop-up screen (631) as a sync screen without a separate external electronic device, and may mirror the execution screen and display it on the pop-up screen (631). According to one embodiment, the electronic device may omit communication connections and message exchanges for transmitting and receiving setting information, and directly control setting information of a virtual screen of the execution screen. For example, when the screen size and / or screen ratio of the pop-up screen (631) is changed, the electronic device may change the setting information of the virtual screen of the execution screen based on the changed screen size and / or screen ratio, and display the changed setting information on the changed pop-up screen.

[0259] FIG. 7 is a diagram illustrating an operation of mirroring additional information based on a screen setting of an external electronic device in an electronic device according to an embodiment of the present disclosure.

[0260] Referring to FIG. 7, an electronic device (e.g., electronic device (101) of FIG. 1 or processor (120) of FIG. 1) can generate a virtual screen for mirroring based on the size of a display of an external electronic device (e.g., electronic device (104) of FIG. 1).

[0261] According to one embodiment, when mirroring an execution screen (710) of an application displayed on a display of an electronic device (e.g., a display module (160) of FIG. 1) on a display of an external electronic device (104) that is larger or has a longer vertical length than the display of the electronic device, the electronic device may generate a virtual screen (720) that includes additional information, rather than a virtual screen that simply enlarges the execution screen (710) of the application to fit the size of the display of the external electronic device (104).

[0262] According to one embodiment, the virtual screen (720) may include an area (721) corresponding to the execution screen (710) of the application and a portion (722) of the screen that is not visible on the display of the electronic device but is visible when the execution screen (710) of the application is scrolled.

[0263] This can increase screen space utilization when mirroring to a large external electronic device.

[0264] For example, when the screen zoom is configured in 5 steps, the default resolution of 1080p size matches the default zoom value of 3 steps, and as the display resolution increases thereafter, the screen zoom steps can decrease by 1 step. For example, 1920 x 1080 can match with 3 steps, 2560 x 1440 can match with 2 steps, and 3840 x 2160 can match with 1 step.

[0265] In one embodiment, the electronic device may change the screen magnification value to match the corresponding resolution range when it reaches the corresponding resolution range.

[0266] In one embodiment, the electronic device may transmit to an external electronic device that the resolution has been changed. In another embodiment, when connected to an external electronic device, such as a VR (or AR) device, the electronic device may change the screen magnification level to match the resolution along with the resolution change and transmit the change information to the external electronic device. For example, the information regarding the resolution and screen magnification level change may be as follows.

[0267] INFO SECOND-SCREEN / 1.0

[0268] Content-Length:56

[0269] Resolution: 3840x2160

[0270] ScreenZoom:1

[0271] TimeStamp:12246772345212

[0272] FIG. 8 is a flowchart illustrating an operation of displaying a changed mirroring screen according to a setting change of an external electronic device according to an embodiment of the present disclosure.

[0273] Referring to FIG. 8, in operation 810, an external electronic device (e.g., electronic device (104) of FIG. 1) may receive first screen information related to a virtual screen that mirrors the screen of a connected electronic device (e.g., electronic device (101) of FIG. 1) from the electronic device.

[0274] In one embodiment, the external electronic device may negotiate with the electronic device to perform mirroring (e.g., Miracast). In one embodiment, before negotiating with the electronic device for mirroring, the external electronic device may receive information related to the types of changeable screen settings (e.g., configuration capability) from the electronic device via a communication module (e.g., the communication module (190) of FIG. 1 ).

[0275] According to one embodiment, even if a WiFi-Direct P2P connection with the electronic device is completed, the external electronic device may hold off on performing the WiFi-Display RTSP Negotiation process immediately in order to check information related to the types of changeable screen settings (e.g., configuration capability). According to one embodiment, in the RTSP Network connection process for WiFi-Display connection, the external electronic device as a sink device becomes a Network Socket Client, and the electronic device as a source device becomes a Network Socket Server. Instead of performing a direct connection from the external electronic device (104) to the electronic device, the connection may be held off for the display setting capability check process.

[0276] In one embodiment, the external electronic device may receive information related to the types of changeable screen settings (e.g., configuration capabilities) from the electronic device and then transmit configuration information to the electronic device. For example, the configuration information may include information on the size, aspect ratio, and / or resolution of an area (e.g., full screen, window, or pop-up screen) on which a mirrored screen is to be displayed on the display of the external electronic device. In one embodiment, if there are multiple areas on which a mirrored screen is to be displayed on the display of the external electronic device, the configuration information may further include identification information for the areas (e.g., display number).

[0277] In one embodiment, the external electronic device may transmit configuration information to the electronic device and then receive first screen information related to the mirrored virtual screen from the electronic device. For example, the first screen information related to the mirrored virtual screen may be generated based on configuration information of the external electronic device.

[0278] In one embodiment, in operation 820, the external electronic device may display a first screen generated based on the first screen information on at least a portion of the display.

[0279] According to one embodiment, the first screen information is generated based on setting information of the external electronic device, and the first screen can be displayed to fill the entire display of the external electronic device, a window, or a pop-up screen.

[0280] According to one embodiment, in operation 830, the external electronic device may transmit changed setting information to the electronic device based on a change in setting information of the display while the first screen information is received.

[0281] In one embodiment, the external electronic device may change the configuration information based on rotation, folding / unfolding, or rolling / unrolling of the electronic device.

[0282] In one embodiment, the external electronic device may change the configuration information based on a change in the area (e.g., a window) in which the mirroring screen (e.g., a screen generated based on screen information received from the electronic device) is to be displayed.

[0283] In one embodiment, the external electronic device may change the configuration information based on the rotation of the display and a change in the window in which the screen is displayed (e.g., by touch or eye tracking) based on the screen information.

[0284] For example, the setup information may include the size of the area in which the mirrored screen will be displayed, the aspect ratio, the resolution, and / or the rotation angle.

[0285] In one embodiment, if the external electronic device is a foldable electronic device or a rollable electronic device, the configuration information may change depending on the folding angle or degree of rolling. For example, the external electronic device, which is a foldable electronic device, may be completely folded so that the external display displays a mirrored screen, or may be folded halfway and operated in flex mode so that only half of the display displays a mirrored screen. In one embodiment, the external electronic device may determine that the resolution has changed, change the configuration information to a value that reduces the width of the area where the mirrored screen is to be displayed by half, and transmit the changed configuration information to the electronic device.

[0286] In one embodiment, when the area in which the mirroring screen is to be displayed is changed from the entire area of ​​the display to a PIP screen, the external electronic device may transmit information about the PIP screen to the electronic device.

[0287] In one embodiment, the external electronic device may change the configuration information based on a user input received while the first screen information is being received to change the area in which the first screen is displayed. For example, the user input for changing the area in which the first screen is displayed may be selecting a different area among multiple areas or changing the ROI.

[0288] According to one embodiment, the external electronic device may receive information about multiple regions of the electronic device's display from the electronic device. For example, the information about the multiple regions may include identification information (e.g., window identification information, ROI identification information) of mirrorable regions on the electronic device and / or thumbnail screens.

[0289] In one embodiment, the external electronic device may display information about multiple areas on a display.

[0290] In one embodiment, the external electronic device may change configuration information to include information related to the second region based on receiving a user input selecting a second region different from the first region corresponding to the first screen among multiple regions. For example, the second region may be different content from the first region corresponding to the first screen, which is the existing mirroring screen, or may be a different ROI within the same content.

[0291] In one embodiment, in operation 840, the external electronic device may generate a second screen corresponding to the changed setting information based on the second screen information received after transmitting the changed setting information.

[0292] In one embodiment, the second screen information may further include information indicating that the changed configuration information has been reflected. For example, the information indicating that the changed configuration information has been reflected may be a timestamp and may be included in the frame in which the changed configuration information has been reflected.

[0293] In one embodiment, in operation 850, the external electronic device may display a second screen on the display.

[0294] According to one embodiment, the external electronic device may provide a transition effect to the first screen after changing the display setting information. For example, the transition effect may include a blur transition effect that blurs the first screen. According to one embodiment, if the resolution changes and the size of the mirrored screen needs to be changed, the transition effect may further include a transition effect that adjusts the size of the first screen. According to one embodiment, the transition effect provided before the second screen information is received will be described in more detail with reference to FIGS. 9A, 9B, and 10 below.

[0295] In one embodiment, the external electronic device may terminate the transition effect and display the second screen based on receiving second screen information including a timestamp.

[0296] In this way, by providing a transition effect before receiving the mirrored screen reflecting the changed display setting information after changing the display setting information, the temporary screen ratio abnormality experience that occurs when changing the setting information can be reduced.

[0297] In one embodiment, if the external electronic device is a rollable electronic device, continuous changes in configuration information may occur depending on rolling or unrolling. In one embodiment, the external electronic device may delay the release of the transition effect by a set period of time (e.g., 300 ms) other than the time at which the second screen information including the timestamp is received.

[0298] According to one embodiment, the external electronic device may check whether there is a change in the setting information before releasing the transition effect, and if there is no change, the transition effect may be released. According to one embodiment, if there is a change in the setting information before releasing the transition effect, the external electronic device may not release the transition effect, but may delay the set time and then re-check whether there is a change in the setting information.

[0299] FIG. 9A is a flowchart illustrating an operation of applying a transition effect when displaying a mirroring screen changed according to a setting change of an external electronic device according to an embodiment of the present disclosure.

[0300] FIG. 9b is a flowchart illustrating an operation of applying a transition effect based on a time stamp of an external electronic device according to an embodiment of the present disclosure.

[0301] Referring to FIG. 9A, in operation 910, an external electronic device (e.g., an electronic device (104) of FIG. 1) may transmit changed setting information to an electronic device (e.g., an electronic device (101) of FIG. 1) during mirroring and then receive screen information. For example, the external electronic device may receive screen information in which the changed setting information is not reflected after transmitting the changed setting information.

[0302] According to one embodiment, in operation 920, the external electronic device may apply a screen transition effect (e.g., a blur transition effect, a quick slide transition effect, or an after transition effect). For example, the blur transition effect may be an effect that makes the screen appear blurry. The blur transition effect is only an example of a screen transition effect and is not limited thereto.

[0303] In one embodiment, in operation 930, the external electronic device can determine whether the resolution of the received screen information has increased or decreased.

[0304] According to one embodiment, when the resolution of the display increases or decreases (operation 930 - Yes), in operation 940, the external electronic device may apply a mirroring screen size increase or decrease transition effect. For example, if the change in setting information includes an increase or decrease in resolution, the external electronic device may apply a transition effect that increases or decreases the size of the received screen information. According to one embodiment, the external electronic device may first apply a blur transition effect to the received screen information and then apply a dynamic transition effect that gradually increases or decreases the size. For example, the external electronic device may first apply a blur transition effect to the received screen information and then apply an animation that increases the size after the rendering of the frame to which the changed setting is applied is completed.

[0305] According to one embodiment, the external electronic device may check the size and coordinates of the area where the mirroring screen is to be displayed according to a change in resolution, and gradually change the size and position of the generated screen based on the received screen information to the checked size and coordinates over a set period of time (e.g., 300 ms). For example, the external electronic device may check the size and coordinates of a layout that need to be changed according to the changed resolution, and gradually change the size of the screen from the x,y coordinates of each corner of the screen (surface view) generated based on the video stream received from the electronic device to the x1,y1 coordinates of each corner that need to be changed over a set period of time (e.g., 300 ms).

[0306] According to one embodiment, even if the external electronic device is a foldable device with different screen ratios for the internal display and the external display, the registered listener (e.g., the frame rendered listener of the Wi-Fi display (321) of FIG. 3A) recognizes the change in the resolution of the display that changes when folding or unfolding and transmits it to the electronic device, so that the electronic device can provide a virtual screen with the changed resolution setting of the virtual screen. According to one embodiment, the external electronic device can apply a screen transition effect until it checks the information on the virtual screen with the changed resolution setting after folding or unfolding.

[0307] In one embodiment, if there is no increase or decrease in the resolution of the received screen information (operation 930 - No), in operation 950, the external electronic device may not apply the mirroring screen size reduction transition effect.

[0308] In one embodiment, the external electronic device may only apply a blur transition effect to the mirrored screen and not apply a size increase transition effect if the resolution is not changed.

[0309] In one embodiment, in operation 960, the external electronic device may verify a timestamp included in the received screen information.

[0310] In one embodiment, the external electronic device can verify a timestamp included in the frame of the mirrored screen. For example, the timestamp may indicate that changed setting information has been reflected.

[0311] According to one embodiment, as illustrated in FIG. 9b, the electronic device (101) may transmit a video stream (e.g., frame) and a time stamp with changed setting information applied to the external electronic device (104) in operation 631.

[0312] In one embodiment, the timestamp may be a PTS included in the PES Header. In one embodiment, the external electronic device (104) may check whether the PTS is included in the PES Header of the video stream received from the electronic device (101).

[0313] In one embodiment, if the received screen information does not include a timestamp (action 960 - No), the external electronic device may repeat an operation to determine whether the received screen information includes a timestamp. For example, the electronic device may repeat the operation to determine whether the received screen information includes a timestamp while providing a blur transition effect and / or a size increase transition effect to the received screen information.

[0314] In one embodiment, if the received screen information includes a timestamp (operation 960 - yes), the external electronic device can, in operation 970, end the transition effect and display the second screen.

[0315] According to one embodiment, the external electronic device may terminate the transition effect based on receiving screen information reflecting the changed setting information and display a second screen generated based on the screen information reflecting the changed setting information.

[0316] In one embodiment, as illustrated in FIG. 9B, in operation 971, the external electronic device (104) may terminate the transition effect when a video frame having a presentation time stamp (PTS) (e.g., a timestamp) greater than or equal to the timestamp is rendered.

[0317] According to one embodiment, the external electronic device (104) can obtain the output time of the video frame from the external electronic device (104) by adding the time difference between the PTS and the PCR value indicating the reference time to the system time (RTC) at the time of PTS reception.

[0318] In this way, external electronic devices can provide effects that make abnormal screen output appear relatively natural before changed configuration information is reflected.

[0319] FIG. 10 is a drawing for explaining an operation of applying a transition effect when displaying a mirroring screen changed according to a setting change of an external electronic device according to an embodiment of the present disclosure.

[0320] Referring to FIG. 10, an external electronic device (104) (e.g., the electronic device (104) of FIG. 1) may display a first mirroring screen (1010). According to one embodiment, when setting information is changed based on rotation, the external electronic device (104) may transmit the changed setting information to an electronic device (e.g., the electronic device (101) of FIG. 1) and display a second mirroring screen (1020) that provides a blur transition effect based on screen information received before the changed setting information is reflected.

[0321] In one embodiment, the second mirroring screen (1020) may be reduced in size to display the entire mirroring screen based on screen information received by the rotated external electronic device. In one embodiment, the external electronic device may display a monochrome (e.g., black) screen in an area other than the area where the second mirroring screen (1020) is displayed.

[0322] In one embodiment, the external electronic device may display a third mirroring screen (1030) that gradually increases in size by reflecting a blur transition effect and a size increase transition effect. In one embodiment, as the second mirroring screen (1030) gradually increases in size, some portions of the mirroring screen based on the received screen information may not be displayed.

[0323] According to one embodiment, the external electronic device may, upon receiving screen information reflecting changed setting information while the third mirroring screen (1030) with the blur transition effect and the size increase transition effect is displayed, end the transition effect and display a fourth mirroring screen (1040) generated based on the screen information reflecting the changed setting information. According to one embodiment, since the fourth mirroring screen (1040) is generated by reflecting the changed setting information, it may be displayed to fill an area for displaying a mirroring screen (e.g., the entire display or a portion of the display).

[0324] FIG. 11A is a drawing for explaining an area displaying a mirroring screen of an external electronic device according to an embodiment of the present disclosure.

[0325] Referring to FIG. 11A, an external electronic device (e.g., electronic device (104) of FIG. 1) can display multiple mirroring screens received from one electronic device (e.g., electronic device (101) of FIG. 1) or multiple electronic devices.

[0326] For example, when the screen of an external electronic device is in a multi-window state including multiple windows, the external electronic device can display a first mirroring screen (1110) in the first window and a second mirroring screen (1111) in the second window.

[0327] For example, if the first mirroring screen (1110) and the second mirroring screen (1111) are received from one electronic device and are screens corresponding to multiple ROIs included in one source screen, the external electronic device can perform a 1:N mirroring operation. For example, if the first mirroring screen (1110) and the second mirroring screen (1111) are received from one electronic device and are screens corresponding to multiple source screens of one electronic device, the external electronic device can perform an N:N mirroring operation.

[0328] In one embodiment, when the first mirroring screen (1110) is received from the first electronic device and the second mirroring screen (1111) is received from the second electronic device, the external electronic device can perform an N:N mirroring operation.

[0329] FIG. 11b is a drawing for explaining an area displaying a mirroring screen of an external electronic device according to an embodiment of the present disclosure.

[0330] Referring to FIG. 11b, the external electronic device can display multiple mirroring screens received from one electronic device or multiple electronic devices.

[0331] For example, if the screen of the external electronic device includes a pop-up screen, the external electronic device can display the first mirroring screen (1120) on the entire display area and the second mirroring screen (1121) on the pop-up screen.

[0332] For example, if the first mirroring screen (1120) and the second mirroring screen (1121) are received from one electronic device and are screens corresponding to multiple ROIs included in one source screen, the external electronic device can perform a 1:N mirroring operation. For example, if the first mirroring screen (1120) and the second mirroring screen (1121) are received from one electronic device and are screens corresponding to multiple source screens of one electronic device, the external electronic device can perform an N:N mirroring operation.

[0333] In one embodiment, when the first mirroring screen (1120) is received from the first electronic device and the second mirroring screen (1121) is received from the second electronic device, the external electronic device can perform an N:N mirroring operation.

[0334] FIG. 12 is a drawing for explaining an area displaying a mirroring screen of an external electronic device according to an embodiment of the present disclosure.

[0335] Referring to FIG. 12, an external electronic device (e.g., electronic device (104) of FIG. 1) may be a VR (or AR) device that displays a virtual environment (1210).

[0336] In one embodiment, the external electronic device may display a mirroring screen (1220) in the form of a pop-up screen within the virtual environment (1210).

[0337] In one embodiment, the pop-up screen may include a rotation button and a zoom button (e.g., + / -) for changing the settings of the pop-up screen. For example, the external electronic device may rotate the pop-up screen by +90 degrees each time the rotation button is selected based on a gesture input. For example, the external electronic device may increase or decrease the size of the pop-up screen by a certain amount between a minimum size and a maximum size each time the zoom button is selected based on a gesture input.

[0338] In one embodiment, the external electronic device changes the pop-up screen by a certain angle or size depending on the selection of the rotation button or the zoom button, so that even if the external electronic device does not know the setting information of the changed pop-up screen, the setting information including the changed angle or size can be transmitted to the electronic device when the rotation button or the zoom button is selected.

[0339] In one embodiment, the external electronic device can change the screen size and aspect ratio through drag and drop of a pop-up screen via gesture input.

[0340] According to one embodiment, when the setting information of a pop-up screen is changed through drag and drop, the external electronic device can determine the size information, ratio information, and / or resolution information of the changed pop-up screen after the change of the pop-up screen is completed, and then transmit the changed setting information to the electronic device.

[0341] In this way, the electronic device of the present disclosure can recognize the sync screen settings of an external electronic device and apply them to the mirrored screen, thereby providing the user with an optimized mirroring screen experience without interruption. Furthermore, the electronic device can apply transition effects to abnormal screens that occur during the configuration information change process, thereby providing the user with an optimized mirroring screen experience without interruption.

[0342] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1) may include a communication module (e.g., communication module (190) of FIG. 1), a display (e.g., display module (160) of FIG. 1), a processor (e.g., processor (120) of FIG. 1), and a memory (e.g., memory (130) of FIG. 1) that includes instructions that, when executed by the processor, cause the electronic device to perform the following operations.

[0343] According to one embodiment, the instructions may be configured to generate a virtual screen for mirroring a screen displayed on at least a portion of the display based on configuration information related to a screen of an external electronic device connected via the communication module.

[0344] According to one embodiment, the instructions may be configured to transmit the generated virtual screen to the external electronic device via the communication module.

[0345] According to one embodiment, the instructions may be configured to receive changed setting information from the external electronic device via the communication module while the virtual screen is being transmitted.

[0346] In one embodiment, the instructions may be configured to change the virtual screen based on the changed setting information.

[0347] According to one embodiment, the instructions may be configured to transmit the changed virtual screen to the external electronic device via the communication module.

[0348] According to one embodiment, the changed setting information may include a ratio of the horizontal and vertical lengths of the screen of the external electronic device.

[0349] In one embodiment, the instructions may be configured to change the horizontal and vertical lengths of the virtual screen based on the ratio.

[0350] According to one embodiment, the changed setting information may include rotation information related to a rotation angle of the external electronic device.

[0351] According to one embodiment, the instructions may be configured to change the horizontal and vertical lengths of the virtual screen based on the rotation information.

[0352] According to one embodiment, the changed virtual screen may further include information indicating that the changed virtual screen reflects the changed setting information.

[0353] According to one embodiment, the changed setting information may further include second setting information different from the setting information.

[0354] In one embodiment, the instructions may be configured to generate a second virtual screen based on the second setting information.

[0355] According to one embodiment, the instructions may be configured to transmit the second virtual screen to the external electronic device via the communication module.

[0356] In one embodiment, the instructions may be configured to transmit information about a plurality of areas of the display to the external electronic device.

[0357] According to one embodiment, the instructions may be configured to change the virtual screen into a virtual screen for mirroring a screen of the second area based on receiving the changed setting information including information related to a second area different from a first area corresponding to the virtual screen from the external electronic device.

[0358] In one embodiment, the instructions may be configured to transmit information related to the types of changeable screen settings to the external electronic device before performing a mirroring negotiation with the external electronic device.

[0359] According to one embodiment, an electronic device may be configured to include a communication module, a display, a processor, and a memory including instructions that, when executed by the processor, cause the electronic device to perform the following operations.

[0360] According to one embodiment, the instructions may be configured to receive first screen information related to a virtual screen mirroring a screen of an external electronic device connected via the communication module from the external electronic device.

[0361] According to one embodiment, the instructions may be configured to display a first screen generated based on the first screen information on at least a portion of the display.

[0362] According to one embodiment, the instructions may be configured to transmit changed setting information to the external electronic device based on a change in setting information of the display while the first screen information is being received.

[0363] According to one embodiment, the instructions may be configured to generate a second screen corresponding to the changed setting information based on second screen information received through the communication module after transmitting the changed setting information.

[0364] In one embodiment, the instructions may be configured to display the second screen on the display.

[0365] In one embodiment, the instructions may be configured to change the setup information based on at least one of rotation of the display or changing a window in which to display a screen generated based on the screen information.

[0366] According to one embodiment, the second screen information may further include information indicating that the changed setting information has been reflected.

[0367] According to one embodiment, the instructions may be configured to provide a transition effect to the first screen after changing the setting information of the display.

[0368] In one embodiment, the instructions may be configured to terminate the transition effect and display the second screen based on the second screen information being received.

[0369] In one embodiment, the instructions may be configured to change the setting information based on receiving a user input for changing an area in which the first screen is displayed while the first screen information is being received.

[0370] In one embodiment, the instructions may be configured to receive information about a plurality of areas of a display of the external electronic device from the external electronic device.

[0371] In one embodiment, the instructions may be configured to display the information about the plurality of areas on the display.

[0372] According to one embodiment, the instructions may be configured to change the setting information to include information related to the second area based on receiving a user input selecting a second area different from the first area corresponding to the first screen among the plurality of areas.

[0373] In one embodiment, the instructions may be configured to receive information from the external electronic device regarding the type of changeable screen settings prior to performing a mirroring negotiation with the external electronic device.

[0374] According to one embodiment, a method for controlling an electronic device may include generating a virtual screen for mirroring a screen displayed on at least a portion of a display of the electronic device based on configuration information related to a screen of an external electronic device connected via a communication module of the electronic device.

[0375] According to one embodiment, a method for controlling an electronic device may include an operation of transmitting the generated virtual screen to the external electronic device through the communication module.

[0376] According to one embodiment, a method for controlling an electronic device may include an operation of receiving changed setting information from the external electronic device through the communication module while the virtual screen is transmitted.

[0377] According to one embodiment, a method for controlling an electronic device may include an operation of changing the virtual screen based on the changed setting information.

[0378] According to one embodiment, a method for controlling an electronic device may include an operation of transmitting the changed virtual screen to the external electronic device through the communication module.

[0379] According to one embodiment, the changed setting information may include a ratio of the horizontal and vertical lengths of the screen of the external electronic device.

[0380] According to one embodiment, the operation of changing the virtual screen may change the horizontal and vertical lengths of the virtual screen based on the ratio.

[0381] According to one embodiment, the changed setting information may include rotation information related to a rotation angle of the external electronic device.

[0382] According to one embodiment, the operation of changing the virtual screen may change the horizontal and vertical lengths of the virtual screen based on the rotation information.

[0383] According to one embodiment, the changed virtual screen may further include information indicating that the changed virtual screen reflects the changed setting information.

[0384] According to one embodiment, the changed setting information may further include second setting information different from the setting information.

[0385] According to one embodiment, the action of changing the virtual screen may include an action of generating a second virtual screen based on the second setting information.

[0386] According to one embodiment, the operation of transmitting the changed virtual screen to the external electronic device may include an operation of transmitting the second virtual screen to the external electronic device via the communication module.

[0387] According to one embodiment, the control method of the electronic device may further include an operation of transmitting information about a plurality of areas of the display to the external electronic device.

[0388] According to one embodiment, the operation of changing the virtual screen may include an operation of changing the virtual screen to a virtual screen for mirroring a screen of the second area, based on receiving the changed setting information including information related to a second area different from a first area corresponding to the virtual screen from the external electronic device.

[0389] According to one embodiment, the method for controlling the electronic device may further include an operation of transmitting information related to the types of changeable screen settings to the external electronic device before performing negotiation for mirroring with the external electronic device.

[0390] According to one embodiment, a method for controlling an electronic device may include receiving first screen information related to a virtual screen mirroring a screen of an external electronic device connected via a communication module of the electronic device from the external electronic device.

[0391] According to one embodiment, a method of controlling an electronic device may include an operation of displaying a first screen generated based on the first screen information on at least a portion of the display.

[0392] According to one embodiment, a method for controlling an electronic device may include an operation of transmitting changed setting information to an external electronic device based on a change in setting information of a display of the electronic device while the first screen information is being received.

[0393] According to one embodiment, a method for controlling an electronic device may include an operation of generating a second screen corresponding to the changed setting information based on second screen information received through the communication module after transmitting the changed setting information.

[0394] According to one embodiment, a method of controlling an electronic device may include an action of displaying the second screen on the display.

[0395] According to one embodiment, the control method of the electronic device may further include an operation of changing the setting information based on at least one of rotation of the display or changing a window in which a screen generated based on the screen information is to be displayed.

[0396] According to one embodiment, the second screen information may further include information indicating that the changed setting information has been reflected.

[0397] According to one embodiment, the control method of the electronic device may further include an operation of providing a transition effect to the first screen after changing the setting information of the display.

[0398] According to one embodiment, the operation of displaying the second screen on the display may end the transition effect and display the second screen based on the second screen information being received.

[0399] According to one embodiment, a control method of an electronic device may further include an operation of changing the setting information based on a user input for changing an area in which the first screen is displayed while the first screen information is being received.

[0400] According to one embodiment, a method of controlling an electronic device may further include receiving information about a plurality of areas of a display of the external electronic device from the external electronic device.

[0401] According to one embodiment, the control method of the electronic device may further include an operation of displaying the information about the plurality of areas on the display.

[0402] According to one embodiment, a control method of an electronic device may further include an operation of changing the setting information to include information related to the second area based on receiving a user input for selecting a second area different from the first area corresponding to the first screen among the plurality of areas.

[0403] According to one embodiment, a method for controlling an electronic device may further include, before performing a negotiation for mirroring with the external electronic device, receiving information related to a type of changeable screen settings from the external electronic device.

[0404] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs, wherein the one or more programs enable an electronic device to generate a virtual screen for mirroring a screen displayed on at least a portion of a display of the electronic device based on configuration information related to a screen of an external electronic device connected through a communication module of the electronic device.

[0405] According to one embodiment, the one or more programs may include commands for the electronic device to transmit the generated virtual screen to the external electronic device via the communication module.

[0406] According to one embodiment, the one or more programs may enable the electronic device to receive changed setting information from the external electronic device via the communication module while the virtual screen is being transmitted.

[0407] According to one embodiment, the one or more programs may include commands for the electronic device to change the virtual screen based on the changed setting information.

[0408] According to one embodiment, the one or more programs may include commands for the electronic device to transmit the changed virtual screen to the external electronic device via the communication module.

[0409] According to one embodiment, the changed setting information may include a ratio of the horizontal and vertical lengths of the screen of the external electronic device.

[0410] In one embodiment, the one or more programs may include commands for the electronic device to change the horizontal and vertical lengths of the virtual screen based on the ratio.

[0411] According to one embodiment, the changed setting information may include rotation information related to a rotation angle of the external electronic device.

[0412] In one embodiment, the one or more programs may include commands for the electronic device to change the horizontal and vertical lengths of the virtual screen based on the rotation information.

[0413] According to one embodiment, the changed virtual screen may further include information indicating that the changed virtual screen reflects the changed setting information.

[0414] According to one embodiment, the changed setting information may further include second setting information different from the setting information.

[0415] In one embodiment, the one or more programs may include commands that cause the electronic device to generate a second virtual screen based on the second setting information.

[0416] According to one embodiment, the one or more programs may include commands for the electronic device to transmit the second virtual screen to the external electronic device via the communication module.

[0417] In one embodiment, the one or more programs may include commands that cause the electronic device to transmit information about a plurality of areas of the display to the external electronic device.

[0418] In one embodiment, the one or more programs may include commands for changing the virtual screen into a virtual screen for mirroring a screen of the second area, based on the electronic device receiving the changed setting information including information related to a second area different from a first area corresponding to the virtual screen from the external electronic device.

[0419] In one embodiment, the one or more programs may include commands for transmitting information related to the types of changeable screen settings to the external electronic device before the electronic device performs negotiation for mirroring with the external electronic device.

[0420] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs may include instructions for causing an electronic device to receive first screen information related to a virtual screen mirroring a screen of an external electronic device connected through a communication module of the electronic device from the external electronic device.

[0421] According to one embodiment, the one or more programs may include commands that cause the electronic device to display a first screen generated based on the first screen information on at least a portion of a display of the electronic device.

[0422] In one embodiment, the one or more programs may include commands for the electronic device to transmit changed setting information to the external electronic device based on a change in setting information of the display while the first screen information is being received.

[0423] According to one embodiment, the one or more programs may include commands that cause the electronic device to generate a second screen corresponding to the changed setting information based on the second screen information received through the communication module after transmitting the changed setting information.

[0424] According to one embodiment, the one or more programs may include commands that cause the electronic device to display the second screen on the display.

[0425] In one embodiment, the one or more programs may include commands that cause the electronic device to change the setting information based on at least one of rotation of the display or changing a window in which to display a screen generated based on the screen information.

[0426] According to one embodiment, the second screen information may further include information indicating that the changed setting information has been reflected.

[0427] According to one embodiment, the one or more programs may enable the electronic device to provide a transition effect on the first screen after changing the setting information of the display.

[0428] According to one embodiment, the one or more programs may include commands that cause the electronic device to terminate the transition effect and display the second screen based on receiving the second screen information.

[0429] In one embodiment, the one or more programs may include commands for the electronic device to change the setting information based on receiving a user input for changing an area in which the first screen is displayed while the first screen information is being received.

[0430] In one embodiment, the one or more programs may include instructions for the electronic device to receive information about a plurality of areas of a display of the external electronic device from the external electronic device.

[0431] In one embodiment, the one or more programs may include instructions for the electronic device to display the information about the plurality of areas on the display.

[0432] In one embodiment, the one or more programs may include commands for changing the setting information to include information related to the second area based on the electronic device receiving a user input selecting a second area different from the first area corresponding to the first screen among the plurality of areas.

[0433] In one embodiment, the one or more programs may include instructions for the electronic device to receive information from the external electronic device regarding the types of changeable screen settings before negotiating with the external electronic device for mirroring.

[0434] Electronic devices according to the embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0435] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0436] The term "module" used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0437] One embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0438] According to one embodiment, the method according to one embodiment disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0439] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In electronic devices, Communication module; display; processor; and A memory comprising instructions that, when executed by the processor, cause the electronic device to perform the following operations; The above instructions are, Generating a virtual screen for mirroring a screen displayed on at least a portion of the display based on configuration information related to the screen of an external electronic device connected through the communication module; Through the above communication module, the generated virtual screen is transmitted to the external electronic device, While the above virtual screen is being transmitted, changed setting information is received from the external electronic device through the communication module, Change the virtual screen based on the changed setting information above, An electronic device configured to transmit the changed virtual screen to the external electronic device via the communication module.

2. In paragraph 1, The above changed settings information is: Including the ratio of the horizontal and vertical lengths of the screen of the external electronic device, The above instructions are, An electronic device set to change the horizontal and vertical lengths of the virtual screen based on the above ratio.

3. In paragraph 1, The above changed settings information is: Contains rotation information related to the rotation angle of the external electronic device, The above instructions are, An electronic device set to change the horizontal and vertical lengths of the virtual screen based on the rotation information.

4. In paragraph 1, The above changed virtual screen is, An electronic device further comprising information indicating that the changed virtual screen has reflected the changed setting information.

5. In paragraph 1, The above changed settings information is: Including second setting information other than the above setting information, The above instructions are, Generate a second virtual screen based on the second setting information above, An electronic device configured to transmit the second virtual screen to the external electronic device via the communication module.

6. In paragraph 1, The above instructions are, Transmitting information about multiple areas of the above display to the external electronic device, An electronic device configured to change the virtual screen into a virtual screen for mirroring a screen of the second area based on receiving the changed setting information including information related to a second area different from a first area corresponding to the virtual screen from the external electronic device.

7. In paragraph 1, The above instructions are, An electronic device configured to transmit information regarding the types of changeable screen settings to the external electronic device prior to negotiating with the external electronic device for mirroring.

8. In electronic devices, Communication module; Display; Processor; and A memory comprising instructions that, when executed by the processor, cause the electronic device to perform the following operations; The above instructions are, Receive first screen information related to a virtual screen mirroring the screen of an external electronic device connected through the above communication module from the external electronic device, Displaying a first screen generated based on the first screen information on at least a portion of the display, While the above first screen information is being received, based on the change in the setting information of the display, the changed setting information is transmitted to the external electronic device, After transmitting the changed setting information, a second screen corresponding to the changed setting information is generated based on the second screen information received through the communication module, An electronic device set to display the second screen on the display.

9. In paragraph 8, The above instructions are, An electronic device configured to change said setup information based on at least one of rotation of said display or changing a window in which a screen generated based on said screen information is displayed.

10. In paragraph 8, The above second screen information is, Including further information indicating that the above changed setting information has been reflected, The above instructions are, After changing the setting information of the above display, a transition effect is provided on the first screen, An electronic device set to terminate the transition effect and display the second screen based on receiving the second screen information.

11. In paragraph 8, The above instructions are, An electronic device configured to change the setting information based on a user input received for changing an area in which the first screen is displayed while the first screen information is being received.

12. In paragraph 8, The above instructions are, Receive information about a plurality of areas of a display of the external electronic device from the external electronic device, Displaying the above information about the above multiple areas on the above display, An electronic device configured to change the setting information to include information related to the second area based on receiving a user input selecting a second area different from the first area corresponding to the first screen among the plurality of areas.

13. In paragraph 8, The above instructions are, An electronic device configured to receive information from the external electronic device regarding the types of changeable screen settings prior to performing negotiation for mirroring with the external electronic device.

14. In a method for controlling an electronic device, An operation of generating a virtual screen for mirroring a screen displayed on at least a part of a display area of ​​the electronic device based on configuration information related to a screen of an external electronic device connected via a communication module of the electronic device; An operation of transmitting the generated virtual screen to the external electronic device through the communication module; An action of receiving changed setting information from the external electronic device through the communication module while the above virtual screen is transmitted; An action of changing the virtual screen based on the changed setting information; and A method for controlling an electronic device, comprising: an operation of transmitting the changed virtual screen to the external electronic device through the communication module.

15. In paragraph 14, The above changed settings information is: Including the ratio of the horizontal and vertical lengths of the screen of the external electronic device, The action of changing the above virtual screen is: A control method of an electronic device for changing the horizontal and vertical lengths of the virtual screen based on the above ratio.

Images