Electronic device and control method therefor

Abstract

This electronic device comprises: an output unit; a sensor; a memory for storing one or more instructions; and one or more processors, wherein the one or more processors execute the one or more instructions to: control the output unit to output a screen including a background area and a plurality of display areas when a multi-view mode is executed; identify whether at least one of the plurality of display areas is covered by a peripheral object on the basis of data obtained by the sensor; and when it is identified that at least one of the plurality of display areas is covered by the peripheral object, control the output unit to output the screen by replacing a first display area covered by the peripheral object with a second display area.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device for outputting an image and a method for controlling the same.

[0002] Advances in electronic technology have led to the development and proliferation of various types of electronic devices. In particular, electronic devices used in various settings, such as homes, offices, and public spaces, have been continuously evolving in recent years.

[0003] In particular, beam projectors are being used in a variety of settings, including offices, theaters, homes, and stores, and the market is continuously expanding. The increasing efficiency and power savings of LED light sources have led to a proliferation of diverse form factors for home / portable beam projectors, and the market is growing by 10-15% annually.

[0004] Additionally, with the recent increase in wired and wireless interface capabilities between beam projector devices and external devices, it has become possible to select and output various contents in real time from various locations via Wi-Fi and 5G networks.

[0005] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0006] According to one or more embodiments, an electronic device includes an output unit, a sensor, a memory storing one or more instructions, and one or more processors, wherein the one or more processors control the output unit to output a screen including a background area and a plurality of display areas when a multi-view mode is executed by executing the one or more instructions, identify whether at least one of the plurality of display areas is covered by a surrounding object based on data acquired by the sensor, and control the output unit to output a first display area covered by the surrounding object as a second display area when it is identified that at least one of the plurality of display areas is covered by the surrounding object.

[0007] According to one or more embodiments, the second display area may be provided with an image that is different from the first display area in at least one of a display size and a display position and is related to an image corresponding to the first display area.

[0008] According to one or more embodiments, the one or more processors may, by executing the one or more instructions, identify whether at least a portion of the second display area is covered by the surrounding object when replacing the first display area with the second display area based on a relationship between the surrounding object and a position at which the second display area is to be provided, and, if it is determined that at least a portion of the second display area is covered by the surrounding object even after replacing the first display area with the second display area, control the output unit to adjust and output at least one of a display size and a display position of the first display area without replacing the first display area with the second display area.

[0009] According to one or more embodiments, the one or more processors may control the output unit to adjust and output at least one of a display size and a display position of the second display area when, by executing the one or more commands, it is determined that at least a portion of the second display area is covered by the surrounding object after being replaced with the second display area.

[0010] According to one or more embodiments, the output unit includes an image projection unit, and the one or more processors can control the image projection unit to identify a vanishing point of a projection surface on which the screen is output based on sensing data acquired by the sensor by executing the one or more commands, generate a virtual surface area having the same vanishing point as the vanishing point of the projection surface, and output a screen including the virtual surface area in an area outside the background area.

[0011] According to one or more embodiments, the one or more processors may control the image projector to acquire an image outside the projection surface based on the sensing data, generate the virtual surface area based on the image outside the projection surface, reduce the background area, and output a screen including the virtual surface area in an area outside the reduced background area by executing the one or more commands.

[0012] According to one or more embodiments, the one or more processors can control the output unit to identify the second display area based on at least one of user preference or content usage history and output the first display area as the second display area by executing the one or more commands.

[0013] According to one or more embodiments, the one or more processors can control the output unit to identify characteristic information of a space where the screen is output based on the sensing data, identify the second display area based on the characteristic information of the space, and output the first display area by replacing it with the second display area by executing the one or more commands.

[0014] According to one or more embodiments, the background area is an area having a sense of space, and the one or more processors, by executing the one or more commands, identify a floor area included in the background area, and when it is determined that at least a portion of the second display area is covered by the surrounding object after replacing the first display area with the second display area, control the output unit to provide an effect in which the second display area is moved in a front direction or a rear direction in the floor area included in the background area.

[0015] According to one or more embodiments, the output unit may include at least one of a display or an image projection unit.

[0016] A method for controlling an electronic device according to one or more embodiments includes, when a multi-view mode is executed, a step of outputting a screen including a background area and a plurality of display areas, a step of identifying whether at least one of the plurality of display areas is covered by a surrounding object based on data acquired by the sensor, and a step of replacing a first display area covered by the surrounding object with a second display area and outputting the same.

[0017] A non-transitory computer-readable medium storing computer instructions that, when executed by a processor of an electronic device according to one or more embodiments, cause the electronic device to perform an operation, the operation comprising: when a multi-view mode is executed, outputting a screen including a background area and a plurality of display areas; identifying whether at least one of the plurality of display areas is occluded by a surrounding object based on data acquired by the sensor; and when it is identified that at least one of the plurality of display areas is occluded by the surrounding object, outputting a first display area occluded by the surrounding object as a second display area.

[0018] The above and other aspects and features of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0019] FIG. 1 is a drawing for explaining an implementation example of an electronic device according to one or more embodiments.

[0020] FIG. 2A is a block diagram illustrating a configuration of an electronic device according to one or more embodiments.

[0021] FIG. 2b is a drawing for explaining a detailed configuration of an electronic device according to one or more embodiments.

[0022] FIG. 3 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0023] FIGS. 4A and 4B are drawings illustrating associated images according to one or more embodiments.

[0024] FIG. 5 is a drawing for explaining a method for providing a multi-view mode according to one or more embodiments.

[0025] FIG. 6 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments.

[0026] FIG. 7 is a drawing for explaining a method for providing a multi-view mode according to one or more embodiments.

[0027] FIG. 8 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0028] FIG. 9 is a drawing for explaining a method for providing a multi-view mode according to one or more embodiments.

[0029] FIG. 10 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0030] FIG. 11 is a drawing for explaining a method for providing a multi-view mode according to one or more embodiments.

[0031] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0032] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions or cases of those skilled in the art, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0033] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0034] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to cases where (1) only A is included, (2) only B is included, or (3) both A and B are included.

[0035] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0036] When it is said that a component (e.g., a first component) is “operatively or communicatively coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0037] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0038] In some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0039] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0040] In the embodiments, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of "modules" or "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "module" or "part" that needs to be implemented as specific hardware.

[0041] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0042] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0043] FIG. 1 is a drawing for explaining an implementation example of an electronic device according to one or more embodiments.

[0044] An electronic device (100) according to one embodiment may have a function of projecting an image, i.e., a projector function. In particular, the electronic device (100) may be a projector device that projects an image onto a wall or a projection surface, and the projector device may be an LCD projector or a DLP (digital light processing) projector using a DMD (digital micromirror device).

[0045] In addition, the electronic device (100) may be implemented as a home or industrial display device, a lighting device used in daily life, an audio device including an audio module, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a wearable device, or a home appliance device. However, the electronic device (100) is not limited to the above-described devices, and the electronic device (100) may be implemented as an electronic device (100) having two or more functions of the above-described devices. For example, the electronic device (100) may be used as a display device, a lighting device, or an audio device by turning off the projector function and turning on the lighting function or the speaker function according to the operation of the processor, and may be used as an AI speaker including a microphone or a communication device.

[0046] According to FIG. 1, when a multi-view mode is executed while a general projection screen (10) is provided, a projection screen (20) including a plurality of display areas (21, 22) and a background area (23) may be provided. The multi-view mode may be a mode in which a plurality of content screens are provided simultaneously. According to an example, in the multi-view mode, not only a plurality of display areas (21, 22) for providing a plurality of contents but also a background area (230) may be provided. For example, the background area may be provided with a preset theme image. The preset themes may include various types of themes such as a nature theme, a city theme, a countryside theme, and a home space theme.

[0047] For example, when one of multiple display areas (21, 22) is selected in multi-view mode, various UX experiences can be provided to increase the user's concentration on the content provided in the selected display area.

[0048] Accordingly, below, various embodiments for increasing the user's concentration on selected content among multiple contents provided in multi-view mode will be described.

[0049] FIG. 2A is a block diagram illustrating a configuration of an electronic device according to one or more embodiments.

[0050] According to FIG. 2A, the electronic device (100) may include an output unit (110), a memory (120), and one or more processors (140). The electronic device (100) may be implemented as a projector that projects an image onto a wall, a projection surface, or a projection surface, or as various types of devices with an image projection function. However, the electronic device (100) is not limited thereto, and may also be implemented as a display device such as a TV.

[0051] According to FIG. 2a, the electronic device (100) may include an output unit (110), a memory (120), a sensor (130), and one or more processors (140).

[0052] The output unit (110) may include at least one of an image projector or a display, depending on the implementation example of the electronic device (100). In one example, if the electronic device (100) is implemented as a projector, the output unit (110) may be implemented as an image projector. In another example, if the electronic device (100) is implemented as a TV, the output unit (110) may be implemented as a display. In another example, if the electronic device (100) is implemented as a TV, the output unit (110) may be implemented as a display. In the case where the output unit (110) is implemented as a display, the display may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, it can be implemented as various types of displays such as LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diodes) display, LED (Light Emitting Diodes), micro LED, Mini LED, PDP (Plasma Display Panel), QD (Quantum dot) display, QLED (Quantum dot light-emitting diodes), etc. According to an example, the display can be implemented as a flat display, a curved display, a flexible display that can be folded or / and rolled, etc.

[0053] For convenience of explanation, the following description assumes that the output unit (110) is implemented as an image projection unit.

[0054] The output unit (110) may perform a function of projecting light to the outside to express an image and outputting the image on a projection surface. Here, the projection surface may be part of the physical space where the image is output or a separate projection surface. The output unit (110) may include various detailed configurations such as a light source of at least one of a lamp, an LED, and a laser, a projection lens, and a reflector.

[0055] The output unit (110) can project an image using one of various projection methods (e.g., CRT (cathode-ray tube) method, LCD (Liquid Crystal Display) method, DLP (Digital Light Processing) method, laser method, etc.). The output unit (110) can include at least one light source.

[0056] The output unit (110) can output images in a 4:3 screen ratio, a 5:4 screen ratio, or a 16:9 wide screen ratio depending on the purpose of the electronic device (100) or the user's settings, and can output images in various resolutions such as WVGA (854*480), SVGA (800*600), XGA (1024*768), WXGA (1280*720), WXGA (1280*800), SXGA (1280*1024), UXGA (1600*1200), and Full HD (1920*1080) depending on the screen ratio.

[0057] The output unit (110) can perform various functions for adjusting the projection image under the control of the processor (140). For example, the output unit (110) can perform a zoom in / out function, a lens shift function, etc. The zoom in / out function can include a hardware method for adjusting the screen size by moving the lens and a software method for adjusting the screen size by cropping the image, etc. Meanwhile, when the zoom in / out function is performed, it is necessary to adjust the focus of the image. For example, the method for adjusting the focus can include a manual focus method, an electric method, etc.

[0058] The output unit (110) can automatically analyze the surrounding environment and projection environment without user input to provide zoom / keystone / focus functions. Specifically, the output unit (110) can automatically provide zoom / keystone / focus functions based on the distance between the electronic device (100) and the projection surface, information about the space where the electronic device (100) is currently located, information about the amount of ambient light, etc.

[0059] The memory (120) can store data required for various embodiments. Depending on the purpose of data storage, the memory (120) may be implemented as a memory embedded in the electronic device (100) or as a memory detachable from the electronic device (100). For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding the functions of the electronic device (100) may be stored in a memory detachable from the electronic device (100). Meanwhile, in the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD). In addition, in the case of memory that can be attached or detached to the electronic device (100'), it may be implemented as at least one of memory cards (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc. It can be implemented in the form of.

[0060] For example, the memory (120) can store various theme images for providing a background area in multi-view mode.

[0061] For example, the memory (120) can store various display area images for providing display areas in multi-view mode. For example, the display area images can include images identical to or similar to actual display product images. For example, the display area images can include various display product images, such as a stand TV image, a wall-mounted TV image, and a kiosk image.

[0062] For example, the memory (120) can store various sensing data acquired through the driving O / S and sensor (130).

[0063] For example, the memory (120) may map and store images associated with each display area image. For example, a wall-mounted TV image may be mapped and stored as an associated image for a stand TV image. For example, a portrait mode TV image may be mapped and stored as an associated image for a landscape mode TV image.

[0064] The sensor (130) may include a camera. The camera may be turned on and take pictures based on a preset event. The camera may convert the captured image into an electrical signal and generate image data based on the converted signal. For example, a subject may be converted into an electrical image signal through a semiconductor optical element (CCD; Charge Coupled Device), and the converted image signal may be amplified and converted into a digital signal, and then subjected to signal processing.

[0065] In addition, the sensor (130) may include various types of sensors such as a distance sensor, a light sensor, an image sensor, a touch sensor, a proximity sensor, an acceleration sensor, a geomagnetic sensor, a gyro sensor, a pressure sensor, a position sensor, etc.

[0066] One or more processors (140) control the overall operation of the electronic device (100). Specifically, one or more processors (140) may be connected to each component of the electronic device (100) to control the overall operation of the electronic device (100). For example, one or more processors (140) may be operatively connected to the display (130) and the memory (120). The processor (140) may be composed of one or more processors.

[0067] One or more processors (140) may perform operations of the electronic device (100) according to various embodiments by executing at least one instruction stored in the memory (120).

[0068] The one or more processors (140) may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors (140) may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors (140) may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform a method according to one or more embodiments of the present disclosure by executing one or more instructions stored in a memory.

[0069] When a method according to one or more embodiments of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0070] One or more processors (140) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (140) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to one or more embodiments of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to one or more embodiments of the present disclosure.

[0071] When a method according to one or more embodiments of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0072] In the embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. Hereinafter, for the convenience of description, one or more processors (140) will be referred to as a processor (140).

[0073] According to one embodiment, when the multi-view mode is executed, the processor (140) can control the output unit (110) to output a screen including a background area and multiple display areas.

[0074] For example, multi-view mode may be a mode in which multiple content screens are presented simultaneously. For example, in multi-view mode, multiple display areas for presenting multiple contents may be provided, along with a background area. For example, the background area may be provided with a preset theme image. The preset themes may include various types of themes, such as nature, city, countryside, and home.

[0075] For example, the display area may include images identical to or similar to actual display product images. For example, the display area may include various display product images, such as images of a stand TV, images of a wall-mounted TV, and images of a kiosk.

[0076] According to one embodiment, the processor (140) may identify whether at least one of the plurality of display areas is obscured by a surrounding object based on data acquired by the sensor (130). For example, the processor (140) may identify whether at least a portion of at least one of the plurality of display areas is obscured by a surrounding object based on a captured image acquired through a camera.

[0077] According to one embodiment, when the processor (140) identifies that at least one of the plurality of display areas is covered by a surrounding object, the processor (140) may control the output unit (110) to output a second display area as a replacement for the first display area covered by the surrounding object. According to one example, the second display area may be different from the first display area in at least one of a display size and a display position, and may be provided as an image related to an image corresponding to the first display area.

[0078] According to one embodiment, the processor (140) may identify whether at least a portion of the second display area is covered by a surrounding object when replacing the first display area with the second display area based on the relationship between the surrounding objects and the location where the second display area is to be provided. In this case, if the processor (140) determines that at least a portion of the second display area is covered by the surrounding object even after replacing the first display area with the second display area, the processor (140) may control the output unit (110) to adjust at least one of the display size and the display position of the first display area and output the information without replacing the first display area with the second display area. Alternatively, the processor (140) may control the output unit (110) to adjust the size of the projection screen itself and output the information.

[0079] According to one embodiment, if the processor (140) determines that at least a portion of the second display area is covered by a surrounding object after being replaced with the second display area, the processor (140) may control the output unit (110) to adjust at least one of the display size and display position of the second display area and output the same.

[0080] According to one embodiment, the processor (140) can identify a vanishing point of a space where a screen is projected based on sensing data acquired by the sensor (130), generate a virtual surface area having the same vanishing point as the vanishing point of the space, and control an output unit (110) (e.g., an image projector) to output a screen including the virtual surface area in an area outside the background area.

[0081] According to one embodiment, the processor (140) may control the output unit (110) (e.g., an image projector) to acquire an image outside the projection surface based on sensing data, generate a virtual surface area based on the image outside the projection surface, reduce the background area, and output a screen including the virtual surface area in an area outside the reduced background area.

[0082] According to one embodiment, the processor (140) can identify a second display area based on at least one of user preference or content usage history, and control the output unit (110) (e.g., an image projector) to output the second display area by replacing the first display area.

[0083] According to one embodiment, the processor (140) may identify characteristic information of a space in which a screen is output based on sensing data. For example, the characteristic information of the space may include various information related to the space, such as information on the size of the space, information on the shape of the space, and information on the type, number, color, and arrangement of objects included in the space. In this case, the processor (140) may identify a second display area based on the characteristic information of the space, and control the output unit (110) (e.g., an image projector) to output the first display area by replacing it with the second display area.

[0084] According to one embodiment, the processor (140) identifies a floor area included in a background area having a sense of space, and if it is determined that at least a portion of the second display area is covered by a surrounding object after replacing the first display area with the second display area, the processor (140) may control the output unit (110) (e.g., the image projector) to provide an effect of the second display area moving in the front or rear direction in the floor area. For example, the output unit (110) (e.g., the image projector) may be controlled to move and output the second display area to a position at a preset ratio in the width of the floor area (or floor surface) of the space included in the background area. For example, the output unit (110) may be controlled to provide an animation effect of gradually moving in the front or rear direction in the floor area.

[0085] According to one embodiment, the processor (140) may identify a floor area included in a background area having a sense of space, and if it is determined that at least a portion of the second display area is covered by a surrounding object after replacing the first display area with the second display area, the processor may control the output unit (110) (e.g., an image projector) to adjust (e.g., reduce) the size of the second display area and output it. Alternatively, the processor (140) may control the output unit (110) to adjust the size of the projection screen itself including the second display area and output it.

[0086] FIG. 2b is a drawing for explaining a detailed configuration of an electronic device according to one or more embodiments.

[0087] According to FIG. 2b, the electronic device (100') includes an output unit (110), a memory (120), a sensor (130), one or more processors (140), a communication interface (150), a user interface (160), a speaker (170), and a power supply unit (180). Among the configurations illustrated in FIG. 2b, a detailed description of configurations that overlap with those illustrated in FIG. 2a will be omitted.

[0088] The communication interface (150) includes a circuit and can communicate with an external device (server or user terminal). For example, the processor (140) can receive various data or information from an external device connected via the communication interface (150) and can also transmit various data or information to the external device.

[0089] The communication interface (150) may include at least one of a WiFi module, a Bluetooth module, a wireless communication module, an NFC module, and a UWB module (Ultra Wide Band). At this time, the wireless communication module may perform communication according to various communication standards such as IEEE, Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), 5G (5th Generation), etc.

[0090] In addition, the communication interface (150) may perform communication according to a communication method such as Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc., depending on the implementation example of the electronic device (100').

[0091] The user interface (160) may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or may be implemented as a touch projection surface, a remote control transmitter / receiver, etc. that can also perform the above-described display function and operation input function. The remote control transmitter / receiver may receive a remote control signal from an external remote control device or transmit a remote control signal through at least one of infrared communication, Bluetooth communication, and Wi-Fi communication.

[0092] The speaker (170) can output audio signals. In one example, the speaker (170) may include multiple speaker units (or amplifiers) capable of outputting multi-channels. For example, the speaker (170) may include an L channel speaker and an R channel speaker that respectively reproduce the L channel and the R channel.

[0093] The power supply unit (180) can receive power from an external source and supply power to various components of the electronic device (100). As an example, the power supply unit (190) can receive power using a connector. For example, the power supply unit (180) can receive power using a 220V DC power cord. However, the present invention is not limited thereto, and the electronic device (100) can receive power using a USB power cord or a wireless charging method.

[0094] The power supply unit (180) can be powered by an internal battery or an external battery. For example, the power supply unit (180) can charge the power of the internal battery using at least one of a 220V DC power cord, a USB power cord, and a USB C-Type power cord, and can be powered by the charged internal battery. For example, the power supply unit (180) can be powered by the external battery when the electronic device (100) is connected to the external battery through various wired communication methods such as a USB power cord, a USB C-Type power cord, a socket home, etc. For example, the power supply unit (180) can be powered directly from the external battery, or can charge the internal battery through the external battery and be powered by the charged internal battery.

[0095] In addition, the electronic device (100') may include a microphone or the like, depending on the implementation example. The microphone is configured to receive user voice or other sounds and convert them into audio data. However, according to another embodiment, the electronic device (100') may receive user voice input via an external device through the communication interface (160).

[0096] FIG. 3 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0097] According to FIG. 3, in operation 310, when the multi-view mode is executed, the electronic device (100) may output a screen including a background area and multiple display areas. For example, the multiple display areas may be provided in the form of a PIP (Picture In Picture), but is not limited thereto.

[0098] For example, the multiview mode may be executed (or entered) based on a user command selecting the multiview mode or a preset event. For example, the user command may be received via at least one of a remote control signal, voice input, gesture input, or eye tracking. For example, the preset event may include various events for providing a multiview screen, such as an event in which multiple contents are selected or an event in which additional contents are selected during the playback of a specific content.

[0099] For example, the UI images, sizes, etc. of multiple display areas may be preset. However, they may be changed according to user commands, the context of the electronic device (100), etc. For example, the context of the electronic device (100) may include the current time, weather, selected content, number of users, user location, etc.

[0100] In one example, the background area may be manually selected based on a user command. In another example, the background area may be automatically selected based on user usage history, user profile, user preference information, the type of content selected in the display area, etc.

[0101] In operation 320, the electronic device (100) can identify whether at least one of the plurality of display areas is covered by a surrounding object based on data acquired by the sensor (130). For example, the surrounding object may be an actual object such as furniture, interior products, etc.

[0102] For example, the electronic device (100) can identify a plurality of display areas and surrounding objects included in a projection screen from a captured image acquired through a camera, and identify whether at least one of the plurality of display areas is covered by a surrounding object.

[0103] For example, the electronic device (100) may identify whether at least one of the plurality of display areas is covered by a surrounding object based on at least one of object recognition, object detection, object tracking, or image segmentation. For example, the electronic device (100) may identify whether at least one of the plurality of display areas is covered by a surrounding object using techniques such as semantic segmentation, which classifies and extracts objects included in a captured image by type as needed, instance segmentation, which recognizes objects by classifying them by type even if they are the same type, and a bounding box in the shape of a rectangle that includes the detected object when detecting an object included in an image.

[0104] In operation 320, if the electronic device (100) determines that a first display area among a plurality of display areas is covered by a surrounding object (or obstacle) based on data acquired by the sensor (130) (S320:Y), in operation 330, the electronic device (100) can output the first display area covered by the surrounding object by replacing it with a second display area.

[0105] In one example, the second display area may be an area related to the first display area.

[0106] For example, a second display area associated with a first display area may differ from the first display area in at least one of a display size and a display position, and may be provided as an associated image associated with an image corresponding to the first display area. For example, an associated image for a stand TV image may be an image of a wall-mounted TV. For example, an associated image for a landscape mode TV image may be an image of a portrait mode TV.

[0107] According to one embodiment, the electronic device (100) can identify a second image in a second display area according to preset criteria.

[0108] In one example, the electronic device (100) can identify a second image of a second display area that is mapped to a first image of a first display area.

[0109] In one example, the electronic device (100) may identify a second image of a second display area mapped to a first image of a first display area based on at least one of user preference and content usage history. For example, the electronic device (100) may select an associated image identified as preferred by the user based on multi-view mode usage history information of the electronic device (100).

[0110] According to an example, the electronic device (100) may identify a second image of a second display area that is pre-mapped to a first image of a first display area based on at least one of user preference and content usage history based on characteristic information of a space where a screen is output. For example, the electronic device (100) may obtain characteristic information of a space through a sensor (130) and select an associated image appropriate for the characteristic information of the space. For example, the characteristic information of a space may include information such as the size of a space, the arrangement of objects, and the size of objects.

[0111] In one example, the electronic device (100) can identify priorities for a plurality of associated images, and identify a second image of a second display area pre-mapped to a first image of a first display area based on the priorities for the plurality of associated images. For example, the electronic device (100) can identify priorities for a second image to replace the first image based on the priorities of the plurality of associated images, and when replacing the first image with the second image, can sequentially identify whether the second display area is covered by a surrounding object to identify a final associated image.

[0112] Meanwhile, in Fig. 3, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0113] FIGS. 4A and 4B are drawings illustrating associated images according to one or more embodiments.

[0114] For example, as illustrated in FIG. 4A, a landscape mode TV image (411) and a portrait mode TV image (412) may be mapped and pre-stored as associated images. For example, the landscape mode TV image (411) and the portrait mode TV image (412) may have their sizes and positions on the screen (410) pre-set.

[0115] For example, as illustrated in FIG. 4b, a stand TV image (413) and a wall-mounted TV image (414) may be mapped and stored as associated images. For example, the size and position of the stand TV image (413) and the wall-mounted TV image (414) on the screen (410) may be preset.

[0116] FIG. 5 is a drawing for explaining a method for providing a multi-view mode according to one or more embodiments.

[0117] According to an example, as illustrated on the left side of FIG. 5, the electronic device (100) may project a screen (510) including a plurality of display areas (511, 512) onto a projection surface (e.g., a wall surface) of a specific space. According to an example, if a portion of the display areas (511, 512) included in the projection screen (510) is covered by surrounding objects (521, 522), viewing of content provided through the display areas (511, 512) may be hindered. For example, the surrounding objects (521, 522) may be obstacles such as furniture located in an actual space.

[0118] According to one example, the electronic device (100) can identify whether a surrounding object (521, 522) covers the display area (511, 512) based on a captured image acquired through the sensor (130). For example, the electronic device (100) can identify whether not only the surrounding object (521, 522) but also the shadow of the surrounding object (521, 522) covers the display area (511, 512).

[0119] According to one example, when the electronic device (100) identifies that the wall-mounted TV-type display area (511) is covered by a surrounding object (521), it can project a screen (520) in which the wall-mounted TV-type display area (511) is replaced with a stand TV-type display area (511-1), as shown on the right side of FIG. 5.

[0120] According to one example, when the electronic device (100) identifies that the horizontal display area (512) is covered by a surrounding object (522), it can project a screen (520) in which the horizontal display area (512) is replaced with a vertical display area (512-1), as shown on the right side of FIG. 5.

[0121] FIG. 6 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments.

[0122] According to FIG. 6, in operation 610, when the multi-view mode is executed, the electronic device (100) can output a screen including a background area and multiple display areas.

[0123] In operation 620, the electronic device (100) can identify whether at least one of the plurality of display areas is covered by a surrounding object based on data acquired by the sensor (130). For example, the surrounding object may be an actual object such as furniture, interior products, etc.

[0124] In operation 620, if it is determined that at least one of the plurality of display areas, the first display area, is covered by a surrounding object (S620:Y), in operation 630, the electronic device (100) may identify a second display area to replace the first display area. In one example, the second display area may be an area associated with the first display area.

[0125] In operation 640, the electronic device (100) can identify whether at least a portion of the second display area is obscured by a surrounding object when replacing the first display area with the second display area based on a relationship between the surrounding object and the location where the second display area is to be provided. For example, the electronic device (100) can identify whether at least a portion of the second display area is obscured by a surrounding object when replacing the first display area with the second display area before replacing the first display area with the second display area.

[0126] In operation 650, if the electronic device (100) determines that at least a portion of the second display area is covered by a surrounding object even after replacing the first display area with the second display area, the electronic device may adjust and output at least one of the display size and display position of the first display area without replacing the first display area with the second display area.

[0127] Meanwhile, in Fig. 6, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0128] FIG. 7 is a drawing for explaining a method for providing a multi-view mode according to one or more embodiments.

[0129] According to an example, as illustrated on the left side of FIG. 7, the electronic device (100) may project a screen (710) including a plurality of display areas (711, 712) onto a projection surface (e.g., a wall surface) of a specific space. According to an example, if a portion of the display area (711) included in the projection screen (710) is covered by a surrounding object (721), viewing of content provided through the display area (711) may be hindered. For example, the surrounding object (721) may be an obstacle such as furniture located in an actual space.

[0130] For example, if the electronic device (100) determines that the wall-mounted TV-type display area (711) is covered by a surrounding object (721), the electronic device (100) may adjust the position of the wall-mounted TV-type display area (711) to a position that is not covered by the surrounding object (721), as illustrated on the right side of FIG. 7. However, although not illustrated in the drawing, it may also be possible to adjust the size of the wall-mounted TV-type display area (711). For example, the electronic device (100) may adjust the size to a size that is not covered by the surrounding object (721) while maintaining the ratio of the wall-mounted TV-type display area (711).

[0131] FIG. 8 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0132] According to FIG. 8, in operation 810, when the multi-view mode is executed, the electronic device (100) can output a screen including a background area and multiple display areas.

[0133] In operation 820, the electronic device (100) can identify whether at least one of the plurality of display areas is covered by a surrounding object based on data acquired by the sensor (130). For example, the surrounding object may be an actual object such as furniture, interior products, etc.

[0134] In operation 820, if it is determined that at least a portion of a first display area among a plurality of display areas is covered by a surrounding object (S820:Y), in operation 830, the electronic device (100) may output the first display area covered by the surrounding object by replacing it with a second display area. In one example, the second display area may be an area associated with the first display area.

[0135] In operation 840, if the electronic device (100) determines that at least a portion of the second display area is covered by a surrounding object after being replaced with the second display area (S840:Y), in operation 850, at least one of the display size and display position of the second display area can be adjusted and output.

[0136] Meanwhile, in Fig. 8, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0137] FIG. 9 is a drawing for explaining a method for providing a multi-view mode according to one or more embodiments.

[0138] According to an example, as illustrated in the upper left of FIG. 9, the electronic device (100) may project a screen (910) including a plurality of display areas (911, 912) onto a projection surface (e.g., a wall surface) of a specific space. According to an example, if a portion of the display area (911) included in the projection screen (910) is covered by a surrounding object (921), viewing of content provided through the display area (911) may be hindered. For example, the surrounding object (921) may be an obstacle such as furniture located in an actual space.

[0139] For example, if the electronic device (100) identifies that the wall-mounted TV-type display area (911) is covered by a surrounding object (721), it may output the wall-mounted TV-type display area (911) by replacing it with a stand-type display area (911-1) as shown in the upper right of FIG. 9.

[0140] According to one example, if the electronic device (100) determines that at least a portion of the stand-type display area (911-1) is covered by a surrounding object (921) after replacing the wall-mounted TV-type display area (911) with the stand-type display area (911-1), the electronic device (100) may adjust the position of the stand-type display area (911-1) to a position that is not covered by the surrounding object (921), as illustrated in the lower center of FIG. 9. However, although not illustrated in the drawing, it may also be possible to adjust the size of the stand-type display area (911-1). For example, the electronic device (100) may adjust the size to a size that is not covered by the surrounding object (921) while maintaining the ratio of the stand-type display area (911-1).

[0141] For example, the background area may be an area having a sense of space. For example, the background area may be a 2D image but may include an image having a sense of space. For example, the background area may be an image including a floor area, a wall area, and a ceiling area, but is not limited thereto. In this case, when the electronic device (100) replaces the first display area with a second display area and determines that at least a portion of the second display area is obscured by a surrounding object, the electronic device (100) may provide an effect in which the second display area moves forward or backward from the floor area.

[0142] However, in some cases, the background area may include a 3D image. Even when the background area is a 3D image, the same / similar embodiments as when it is a 2D image may be applied.

[0143] FIG. 10 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0144] According to FIG. 10, in operation 1010, when the multi-view mode is executed, the electronic device (100) can output a screen including a background area and multiple display areas.

[0145] In operation 1020, the electronic device (100) can identify a vanishing point of a space where a screen is output based on data acquired by the sensor (130). As an example, the electronic device (100) can identify a vanishing point of an actual space (e.g., a wall surface) based on the location of the electronic device (100) through a camera and a ToF sensor. For example, the vanishing point may be a point where an extension line of an actual wall surface is focused and converges at a single point according to a perspective method.

[0146] In operation 1030, the electronic device (100) can generate a virtual surface area having a vanishing point identical to the vanishing point of the projection surface of the space where the screen is output.

[0147] In operation 1030, the electronic device (100) can output a screen including a virtual surface area in an area outside the background area.

[0148] According to one embodiment, the electronic device (100) may acquire an image outside the projection surface based on sensing data, and generate a virtual surface area based on the image outside the projection surface. According to one example, the electronic device (100) may reduce the background area and output a screen including the virtual surface area in an area outside the reduced background area.

[0149] For example, the electronic device (100) may generate a virtual surface area image based on data acquired by the sensor (130). For example, the electronic device (100) may acquire an image corresponding to an actual space surface (e.g., a ceiling surface, a left wall surface, a right wall surface, a floor surface, etc.) based on data acquired by the sensor (130), and generate a virtual surface area image based on the acquired image.

[0150] In one example, the image of the virtual surface area may be a preset image. For example, the image of the virtual surface area may be an image similar to the background area. In this case, the virtual surface area may have the effect of expanding the background area.

[0151] FIG. 11 is a drawing for explaining a method for providing a multi-view mode according to one or more embodiments.

[0152] According to an example, as shown on the right side of FIG. 11, when the multi-view mode is executed, the electronic device (100) can output a screen (1110) including multiple display areas (1111, 1112) and a background area (1113).

[0153] According to an example, as illustrated on the left side of FIG. 11, the electronic device (100) can identify a vanishing point of a space where a screen is output based on data acquired through a sensor (130). According to an example, the electronic device (100) can identify a vanishing point of an actual wall based on the position of the electronic device (100) based on a captured image acquired through a camera and distance information acquired through a ToF sensor. For example, the distance information acquired through the ToF sensor can include distance information for a space constituting an actual space (e.g., a distance to an actual wall, a distance to an actual ceiling, a distance to an actual floor, etc.), a distance to a projection surface, a distance to a surrounding object, etc. For example, the electronic device (110) can identify a vanishing point corresponding to four corners of an actual wall.

[0154] For example, the electronic device (100) may generate a virtual wall area having the same vanishing point as the actual wall area, and output a screen (1120) including the virtual wall area (1114) in an area outside the background area (1113'). For example, the electronic device (100) may reconfirm whether the connecting line between the actual wall area and the virtual wall area is a straight line based on an image captured by a camera. Meanwhile, since the size of the projection screen is maintained, the size of the background area (1113') may be reduced, and the virtual wall area (1114) may be included in the remaining area.

[0155] Meanwhile, although the above-described embodiment mainly describes an embodiment in which the electronic device (100) is implemented as a projector, embodiments identical to / similar to the above-described embodiment may also be applied when the electronic device (100) is implemented as a display device such as a TV. For example, sensing data acquired through an external device (e.g., a user's mobile phone)

[0156] According to the various embodiments described above, when providing a multi-view mode, a natural background area and display area suitable for the space are provided, and the visibility of the content screen can be enhanced while improving the sense of space.

[0157] Meanwhile, the methods according to the various embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device.

[0158] Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in an electronic device, or an external server of the electronic device.

[0159] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call instructions stored from the storage medium and operate according to the called instructions, and may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0160] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0161] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0162] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, Output section; sensor; A memory that stores one or more instructions; and comprising one or more processors; The one or more processors, by executing the one or more instructions, When the multi-view mode is executed, the output unit is controlled to output a screen including a background area and multiple display areas, Identifying whether at least one of the plurality of display areas is obscured by a surrounding object based on data acquired by the sensor; An electronic device that controls the output unit to output a first display area covered by the surrounding object as a second display area when it is identified that at least one of the plurality of display areas is covered by the surrounding object.

2. In paragraph 1, The above second display area is, An electronic device, wherein at least one of the display size and display position is different from the first display area and an image is provided related to an image corresponding to the first display area.

3. In paragraph 1, The one or more processors, by executing the one or more instructions, Identifying whether at least a portion of the second display area is obscured by the surrounding object by replacing the first display area with the second display area based on the relationship between the surrounding object and the location where the second display area is to be provided; An electronic device that controls the output unit to adjust and output at least one of the display size and display position of the first display area without replacing the first display area with the second display area, if it is determined that at least a portion of the second display area is covered by the surrounding object even after being replaced with the second display area.

4. In paragraph 1, The one or more processors, by executing the one or more instructions, An electronic device that controls the output unit to adjust and output at least one of the display size and display position of the second display area when it is identified that at least a part of the second display area is covered by the surrounding object after being replaced with the second display area.

5. In paragraph 1, The above output section includes an image projection section, The one or more processors, by executing the one or more instructions, Identifying the vanishing point of the projection surface on which the screen is output based on the sensing data acquired by the above sensor, Create a virtual surface area having the same vanishing point as the vanishing point of the above projection surface, An electronic device that controls the image projector to output a screen including the virtual surface area in an area outside the background area.

6. In paragraph 5, The one or more processors, by executing the one or more instructions, Based on the above sensing data, an image outside the projection surface is acquired, Generate the virtual surface area based on an image outside the projection surface, An electronic device that controls the image projector to reduce the background area and output a screen including the virtual surface area in an area outside the reduced background area.

7. In paragraph 1, The one or more processors, by executing the one or more instructions, An electronic device that identifies the second display area based on at least one of user preference and content usage history, and controls the output unit to output the second display area as a replacement for the first display area.

8. In paragraph 1, The one or more processors, by executing the one or more instructions, Identifying the characteristic information of the space where the screen is output based on the above sensing data, An electronic device that identifies the second display area based on characteristic information of the space and controls the output unit to output by replacing the first display area with the second display area.

9. In paragraph 1, The above background area is an area that has a sense of space. The one or more processors, by executing the one or more instructions, Identify the floor area included in the above background area, An electronic device that controls the output unit to provide an effect in which the second display area is moved in a front direction or a rear direction in a floor area included in the background area when it is determined that at least a portion of the second display area is covered by the surrounding object after replacing the first display area with the second display area.

10. In paragraph 1, The above output section, An electronic device comprising at least one display or image projection unit.

11. In a method for controlling an electronic device, When the multi-view mode is executed, a step of outputting a screen including a background area and multiple display areas; A step of identifying whether at least one of the plurality of display areas is covered by a surrounding object based on data acquired by the sensor; and A control method, comprising: a step of outputting a first display area covered by the surrounding object by replacing the first display area covered by the surrounding object with a second display area, if at least one of the plurality of display areas is identified as being covered by the surrounding object.

12. In paragraph 11, The above second display area is, A control method, wherein at least one of the display size and display position is different from the first display area and an image is provided related to an image corresponding to the first display area.

13. In paragraph 11, The step of identifying whether at least one of the plurality of display areas is covered comprises: A step of identifying whether at least a portion of the second display area is covered by the surrounding object by replacing the first display area with the second display area based on the relationship between the surrounding object and the location where the second display area is to be provided; The above control method is, A control method further comprising: a step of adjusting and outputting at least one of a display size and a display position of the first display area without replacing the first display area with the second display area, if it is determined that at least a portion of the second display area is obscured by the surrounding object even after being replaced with the second display area; 14. In paragraph 11, A control method further comprising: a step of adjusting and outputting at least one of a display size and a display position of the second display area if it is determined that at least a part of the second display area is covered by the surrounding object after being replaced with the second display area; 15. A non-transitory computer-readable medium storing computer instructions that, when executed by a processor of an electronic device, cause the electronic device to perform an operation, The above actions are, When the multi-view mode is executed, a step of outputting a screen including a background area and multiple display areas; A step of identifying whether at least one of the plurality of display areas is covered by a surrounding object based on data acquired by the sensor; and A non-transitory computer-readable medium, comprising: a step of outputting, when it is identified that at least one of the plurality of display areas is covered by the surrounding object, a first display area covered by the surrounding object as a second display area;

Images