Electronic device and method for controlling same

Abstract

An electronic device is disclosed. The electronic device comprises: a communication interface; a projection unit; a memory storing information about a projection range ratio for each aspect ratio of content provided to an external display device; and one or more processors that, when image information and aspect ratio information corresponding to the content displayed on the external display device are received via the communication interface, identify a projection area on the basis of the received aspect ratio information corresponding to the content and information stored in the memory, identify a projection image to be output to the projection area on the basis of the aspect ratio information corresponding to the content, the image information, and the information stored in the memory, and control the projection unit so that the identified projection image is output to the identified projection area.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device and a control method thereof that provide expanded content in conjunction with an external device by taking into account the aspect ratio of the content and an object within a projection surface.

[0002] Advances in electronic technology have led to the development and widespread adoption of various types of electronic devices. However, when content with an aspect ratio different from that of the display device being produced is displayed, letterboxing can occur alongside the content. This leads to wasted space within the display. To address this issue, a method is needed to display content using a projector linked to the display device, thereby providing users with content without letterboxing.

[0003] An electronic device according to one embodiment of the present disclosure may include a communication interface, a projection unit, a memory storing information on a projection range ratio for each aspect ratio of content provided to an external display device, and one or more processors that, when aspect ratio information and image information corresponding to content displayed on the external display device are received through the communication interface, identify a projection area based on the aspect ratio information corresponding to the received content and the information stored in the memory.

[0004] The one or more processors can identify a projection image to be output to the projection area based on aspect ratio information corresponding to the content, the image information, and information stored in the memory.

[0005] The one or more processors can control the projection unit so that the identified projection image is output to the identified projection area.

[0006] A method for controlling an electronic device according to an embodiment of the present disclosure may include, when aspect ratio information and image information corresponding to content displayed on an external display device are received, a step of identifying a projection area based on aspect ratio information corresponding to the received content and information on a projection range ratio for each aspect ratio of the content provided to the external display device stored in a memory.

[0007] The above control method may include a step of identifying a projection image to be output to the projection area based on aspect ratio information corresponding to the content, the image information, and information stored in the memory.

[0008] The above control method may include a step of outputting the identified projection image to the identified projection area.

[0009] In a non-transitory computer-readable recording medium storing a computer instruction that causes an electronic device to perform an operation when executed by a processor of an electronic device according to an embodiment of the present disclosure, the operation may include a step of, when aspect ratio information and image information corresponding to content displayed on an external display device are received, identifying a projection area based on aspect ratio information corresponding to the received content and information on a projection range ratio for each aspect ratio of content provided to the external display device stored in a memory.

[0010] The above operation may include a step of identifying a projection image to be output to the projection area based on aspect ratio information corresponding to the content, the image information, and information stored in the memory.

[0011] The above operation may include a step of outputting the identified projection image to the identified projection area.

[0012] FIG. 1 is a drawing schematically illustrating a control method of an electronic device according to one embodiment.

[0013] Figure 2 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0014] Figure 3 is a flowchart for explaining a control method of an electronic device according to one embodiment.

[0015] Figures 4a to 4c are drawings for explaining a method of displaying a projection image.

[0016] FIG. 5 is a drawing for explaining a method for identifying a first area and a second area according to one embodiment.

[0017] FIG. 6 is a diagram illustrating a method for identifying a viewing obstruction area according to one embodiment.

[0018] FIG. 7 is a drawing for explaining a method of outputting a guide UI (User Interface) according to one embodiment.

[0019] FIGS. 8A and 8B are drawings for explaining a method of outputting an image according to one embodiment.

[0020] FIG. 9 is a flowchart illustrating a method for controlling an electronic device according to an embodiment.

[0021] FIG. 10 is a diagram illustrating a method for re-identifying a projection image and a display image according to one embodiment.

[0022] FIG. 11 is a drawing for explaining a method for providing a UI menu according to one embodiment.

[0023] Fig. 12 is a block diagram showing a detailed configuration of an electronic device according to one embodiment.

[0024] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0025] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0026] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0027] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0028] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0029] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0030] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0031] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0032] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0033] FIG. 1 is a drawing schematically illustrating a control method of an electronic device according to one embodiment.

[0034] According to FIG. 1, according to one embodiment, when the electronic device (10) receives aspect ratio information of content (30) displayed on the external display device (1), the electronic device (10) can identify a projection area corresponding to an image to be projected through the electronic device (10) within a projection surface based on the aspect ratio information. The projection surface may be a part of a physical space where an image is output or a preset area within the physical space. According to one example, the electronic device (10) can identify projection range ratio information corresponding to the aspect ratio of the content, and can identify a projection area corresponding to an image to be projected through the electronic device (10) based on the identified projection range ratio. The projection range ratio may be an aspect ratio value corresponding to a projection image output through the electronic device (100).

[0035] According to one embodiment, the electronic device (10) can identify a projection image (20) to be projected through the electronic device (10) based on image information of the content and image information corresponding to the content. The electronic device (10) can output the identified projection image (20) to an identified projection area within a projection surface.

[0036] Hereinafter, various embodiments that can provide extended content to a user in conjunction with an external display device (1) by displaying a portion of the content (30) (or, a projection image (20)) displayed on an external display device (1) in a preset area within a projection surface using the aspect ratio and projection range ratio of the content (30) will be described.

[0037] Figure 2 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0038] According to FIG. 2, the electronic device (100) may include a communication interface (110), a projection unit (120), a memory (130), and one or more processors (140).

[0039] The electronic device (100) may be implemented as a projector according to one embodiment, but is not limited thereto, and may be implemented as a different type of device having a function of projecting an image, i.e., a projection function.

[0040] Meanwhile, the electronic device (100) can communicate with an external display device in various ways. According to one embodiment, a communication module for communication with the external display device may be implemented in the same manner. For example, the electronic device (100) may communicate with the external display device using a Bluetooth module. The external display device may be implemented as various devices equipped with a display, such as a smart TV, a tablet, a monitor, a smart phone, a desktop computer, a laptop computer, etc., according to one embodiment. The external display device according to one embodiment of the present disclosure is not limited to the above-described devices, and the external display device may be implemented as an external display device having the functions of two or more of the above-described devices.

[0041] According to one embodiment, the communication interface (110) can input and output various types of data. For example, the communication interface (110) can transmit and receive various types of data to and from an external device (e.g., a source device), an external storage medium (e.g., a USB memory), an external server (e.g., a web hard drive) through a communication method such as AP-based Wi-Fi (Wireless Local Area Network), Bluetooth, Zigbee, wired / wireless Local Area Network (LAN), Wide Area Network (WAN), Ethernet, IEEE 1394, High-Definition Multimedia Interface (HDMI), Universal Serial Bus (USB), Mobile High-Definition Link (MHL), Audio Engineering Society / European Broadcasting Union (AES / EBU), Optical, Coaxial, etc.

[0042] For example, the communication interface (110) may include a BLE (Bluetooth Low Energy) module. BLE refers to Bluetooth technology that enables transmission and reception of low-power, low-capacity data in the 2.4 GHz frequency band with a range of about 10 m. However, the present invention is not limited thereto, and the communication interface (110) may also include a Wi-Fi communication module. That is, the communication interface (110) may include at least one of a BLE (Bluetooth Low Energy) module or a Wi-Fi communication module.

[0043] According to one embodiment, the projection unit (120) performs a function of projecting light to the outside to express an image and outputting the image on a projection surface. The projection surface may be a part of a physical space where an image is output or a preset area within the physical space. The projection unit (120) may include various detailed configurations such as at least one light source of a lamp, an LED, a laser, a projection lens, a reflector, etc. The projection unit (120) may project an image using one of various projection methods (for example, a CRT (cathode-ray tube) method, an LCD (Liquid Crystal Display) method, a DLP (Digital Light Processing) method, a laser method, etc.). The projection unit (120) may include at least one light source. The projection unit (120) may output an image with a 4:3 screen ratio, a 5:4 screen ratio, a 16:9 wide screen ratio depending on the purpose of the electronic device (100) or a user's settings, but is not limited thereto. The projection unit (120) can output images in various resolutions such as WVGA (854*480), SVGA (800*600), XGA (1024*768), WXGA (1280*720), WXGA (1280*800), SXGA (1280*1024), UXGA (1600*1200), Full HD (1920*1080), etc. depending on the screen ratio. In addition, the projection unit (120) can perform various functions for adjusting the projection image by the control of the processor (140). For example, the projection unit (120) can perform a zoom function, a lens shift function, etc.

[0044] According to one embodiment, the memory (130) can store data required for various embodiments. The memory (130) may be implemented in the form of a memory embedded in the electronic device (100) or may be implemented in the form of a memory that can be detachably attached to the electronic device (100) depending on the purpose of data storage. For example, data for driving the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding the functions of the electronic device (100) may be stored in a memory that can be detachably attached to the electronic device (100).

[0045] Meanwhile, in the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)). In addition, in the case of memory that can be detachably attached to the electronic device (100), it may be implemented as at least one of memory cards (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc. It can be implemented.

[0046] According to one embodiment, the memory (130) may store information on the projection range ratio for each aspect ratio of content provided to an external display device. The projection range ratio may be an aspect ratio value corresponding to a projection image output through the electronic device (100). For example, the projection range ratio corresponding to content having an aspect ratio of 18:9 may be 9:1. Alternatively, the projection range ratio corresponding to content having an aspect ratio of 21:9 may be 9:2.5. This is not limited thereto, and it goes without saying that information on projection range ratios for different aspect ratios of content may be stored in the memory (130). This will be described in detail with reference to FIG. 4C.

[0047] According to one embodiment, one or more processors (140) (hereinafter, “processors”) are electrically connected to a communication interface (110), a projection unit (120), and a memory (130) to control the overall operation of the electronic device (100). The processor (140) may be composed of one or more processors. Specifically, the processor (140) may perform operations of the electronic device (100) according to various embodiments of the present disclosure by executing at least one instruction stored in the memory.

[0048] According to one embodiment, the processor (140) may be implemented as a digital signal processor (DSP), a microprocessor, a graphics processing unit (GPU), an artificial intelligence (AI) processor, a neural processing unit (NPU), or a time controller (TCON) for processing a digital image signal. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), or an ARM processor, or may be defined by the relevant terminology. In addition, the processor (140) may be implemented as a system on chip (SoC) or large scale integration (LSI) having a built-in processing algorithm, or may be implemented in the form of an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA).

[0049] According to one embodiment, the processor (140) may identify a projection area based on aspect ratio information corresponding to content displayed on an external display device. According to one example, when aspect ratio information and image information corresponding to content displayed on an external display device are received through the communication interface (110), the processor (140) may identify a projection area based on aspect ratio information corresponding to the received content and information stored in the memory (130). Here, the image information refers to image information of the content displayed on the external display device, and the image information may be pixel information corresponding to the content.

[0050] For example, the electronic device (100) and the external display device may be in a communication connection, and the processor (140) may receive aspect ratio information and image information corresponding to content displayed on the external display device from the external display device through the communication interface (110). Based on the aspect ratio information corresponding to the received content, the processor (140) may identify a projection range ratio corresponding to the aspect ratio of the displayed content among information on projection range ratios for each aspect ratio of the content provided to the external display device. For example, when the aspect ratio of the displayed content is 18:9, the processor (140) may identify information on a projection range ratio (9:1) corresponding to the aspect ratio (18:9) based on information stored in the memory (130).

[0051] For example, the processor (140) may identify a projection area based on information about the identified projection range ratio. For example, if an area corresponding to an external display device within the projection surface is identified, the processor (140) may identify a projection area within the projection surface where a projection image will be output based on information about the identified projection range ratio. A specific method for identifying the projection area will be described in detail with reference to FIGS. 4A to 4C, FIGS. 5 and 6.

[0052] According to one embodiment, the processor (140) may identify a projection image to be output to the projection area. According to one example, the projection image refers to an image output through an electronic device, and the display image refers to an image displayed through an external display device. According to one example, the processor (140) may identify a projection image to be output to the projection area based on aspect ratio information corresponding to the content, image information, and information stored in the memory (130). For example, when information on a projection range ratio for the content is identified based on aspect ratio information corresponding to the content and information on a projection range ratio for each aspect ratio of the content provided to an external display device stored in the memory (130), the processor (140) may identify a pixel area to be output through the electronic device among images corresponding to the content based on the identified projection range ratio information and image information of the content, and may identify an image corresponding to the identified pixel area as a projection image.

[0053] According to one embodiment, the processor (140) can control the projection unit (120) to output an identified projection image to an identified projection area.

[0054] Figure 3 is a flowchart for explaining a control method of an electronic device according to one embodiment.

[0055] According to FIG. 3, first, according to one embodiment, the control method may determine whether aspect ratio information and image information corresponding to content displayed on an external display device are received (S310). According to one example, the electronic device (100) and the external display device may be communicatively connected, and the processor (140) may receive aspect ratio information and image information corresponding to content displayed on the external display device from the external display device via the communication interface (110).

[0056] Next, according to one embodiment, when aspect ratio information and image information corresponding to content displayed on an external display device are received (S310: Y), the control method can identify a projection area based on aspect ratio information corresponding to the received content and information on projection range ratios for each aspect ratio of content provided to the external display device stored in the memory (130) (S320).

[0057] For example, the memory (130) may store information on the projection range ratio for each aspect ratio of the content, and the processor (140) may identify the projection range ratio corresponding to the received aspect ratio information. The processor (140) may identify a projection area in which a projection image is to be output within the projection surface based on the identified projection range ratio. For example, when an area corresponding to an external display device within the projection surface is identified, the processor (140) may identify a projection area in which a projection image is to be output within the projection surface based on the size of the area corresponding to the identified external display device and the identified projection range ratio.

[0058] Next, according to one embodiment, the control method can identify a projection image to be output to the projection area based on aspect ratio information corresponding to the content, image information, and information stored in the memory (130) (S330). According to one example, the processor (140) can identify a pixel area corresponding to the projection image within the image corresponding to the content based on image information corresponding to the displayed content and information on the projection range ratio.

[0059] Next, according to one embodiment, the control method can output the identified projection image to the identified projection area (S340). According to one example, when the projection area and the projection image are identified, the processor (140) can output the identified projection image to the identified projection area within the projection surface through the projection unit (120).

[0060] According to the above-described example, the electronic device (100) can display a portion of the content displayed on the external display device in a preset area within the projection surface. The electronic device (100) can provide expanded content to the user by linking with the external display device. Accordingly, the electronic device (100) can provide an immersive viewing experience.

[0061] Figures 4a to 4c are drawings for explaining a method of displaying a projection image.

[0062] According to one embodiment, the processor (140) can identify a projection image to be output through the electronic device (400) and a display image to be displayed on an external display device (4).

[0063] According to one embodiment, it is assumed that content (40) is provided through an external display device (4) as illustrated in FIG. 4A. According to one example, the processor (140) may receive aspect ratio information and image information corresponding to the content displayed on the external display device (4) through the communication interface (110). The processor (140) may identify a projection range ratio corresponding to the content (40) based on the aspect ratio information corresponding to the received content and information stored in the memory (130).

[0064] For example, as illustrated in FIG. 4c, information on projection range ratios corresponding to different content aspect ratios (or aspect ratios of content) may be stored. For example, let's assume that the aspect ratio of the external display device (4) is 16:9. The memory (130) may store a projection range ratio (9:1) value corresponding to the content aspect ratio of 18:9. However, this is merely an example, and the memory (130) may also store information on projection range ratios corresponding to aspect ratios of the external display device (4) having different values.

[0065] For example, when a projection range ratio corresponding to content (40) is identified, the processor (140) may identify a projection area based on the identified projection range ratio. For example, when the projection range ratio is identified as 9:8, the processor (140) may identify a preset area (e.g., an area adjacent to the first area) within the projection surface as a projection area based on information about the identified projection range ratio (9:8) and an area (or, the first area) corresponding to the external display device (4) within the projection surface. In this case, for example, the area of ​​the projection area may be an area proportional to the size of the area (the first area) corresponding to the external display device (4). For example, as illustrated in FIG. 4C, the vertical length of the external display device (4) and the vertical length of the projection area may have the same value, but are not limited thereto.

[0066] According to one embodiment, the processor (140) may identify a projection image to be output to the projection area based on aspect ratio information corresponding to the content, image information, and information stored in the memory (130). When the projection image to be output to the projection area is identified, the processor (140) may identify a display image to be displayed on the external display device (4) based on the identified projection image and image information of the content. According to one embodiment, the processor (140) may identify a pixel area corresponding to the projection image within an image corresponding to the content, and identify an area excluding the identified pixel area within the image corresponding to the content as a display image. According to one embodiment, the processor (140) may transmit information on the identified display image to the external display device (4) via the communication interface (110).

[0067] According to one embodiment, as illustrated in FIG. 4b, the electronic device (400) can control the projection unit (120) to output the projection images (41 and 42) to the identified projection areas within the projection surface. The external display device (4) can display the identified display images (43).

[0068] According to the example described above, the electronic device (400) can provide expanded content in conjunction with an external display device (4). In this case, by providing expanded content based on the aspect ratio of the content, the user can view the video without letterboxing. This can enhance the user's sense of immersion.

[0069] Returning to FIG. 2, in one embodiment, an external display device, rather than the electronic device (100), may be the main device for performing the operation. In one example, the external display device may identify a projection area based on aspect ratio information and image information corresponding to the displayed content, and may identify a projection image to be output through the electronic device (100) based on aspect ratio information, image information, and projection range ratios for each aspect ratio of the content corresponding to the content. In this case, the memory of the external display device may store information on projection range ratios for each aspect ratio of the content provided to the external display device. The external display device may transmit information on the identified projection area and information on the identified projection image to the electronic device (100) through a communication interface, and the electronic device (100) that receives the information on the identified projection area and information on the identified projection image may control the projection unit (120) so that the identified projection image is output to the identified projection area based thereon.

[0070] According to one embodiment, the external display device can identify a display image to be displayed on the external display device based on the identification of a projection image to be output to the projection area. The external display device can display the identified display image through the display.

[0071] FIG. 5 is a drawing for explaining a method for identifying a first area and a second area according to one embodiment.

[0072] According to FIG. 5, according to one embodiment, the processor (140) can identify information about a first area (520) within a projection surface and information about a second area (510). The first area (520) may be a first area (520) corresponding to a position of an external display device (5) within the projection surface, and the information about the first area (520) may be coordinate information within the projection surface corresponding to the first area (520). The second area (510) may be an area excluding the first area (520) among the maximum projection areas of the electronic device (500), and the information about the second area (510) may be coordinate information within the projection surface corresponding to the second area (510).

[0073] According to an example, the electronic device (500) may include at least one sensor, and the processor (140) may identify information about a first area (520) corresponding to a location of an external display device (5) within a projection surface based on sensing data acquired based on the at least one sensor. For example, if the projection surface is a space within a home where an external display device (5) is installed, the processor (140) may acquire sensing data corresponding to the external display device (5) through a vision sensor included in the electronic device (500). The processor (140) may identify coordinate information of a vertex corresponding to the external display device (5) within the projection surface based on the acquired sensing data corresponding to the external display device (5), and identify information about the first area (520) based on the coordinate information of the identified vertex.

[0074] According to an example, the processor (140) may identify information about a second area (510) corresponding to a maximum projection area within a projection surface based on information about the first area (520). According to an example, the processor (140) may identify projection angle information of the projection unit (120). The projection angle of the projection unit (120) refers to an angle formed between the ground and the projection unit (120). According to an example, when information about the first area (520) is identified, the processor (140) may identify information about the second area (510) based on the projection angle information of the identified projection unit (120) and the information about the identified first area (520). For example, the processor (140) may identify information about the second area (510) through coordinate information of a vertex corresponding to the maximum projection area and coordinate information of a vertex corresponding to the first area (520). For example, the processor (140) can control the projection unit (120) so that the projection image is output within the second area (510).

[0075] According to one embodiment, the external display device (5) can receive sensing data from the electronic device (500), and identify information about a first area (520) corresponding to a position of the external display device (5) within a projection surface and information about a second area (510) corresponding to a maximum projection area within the projection surface based on the received sensing data. The external display device (5) can transmit information about the identified first area (520) and information about the second area (510) to the electronic device (500), and the electronic device (500) can control the projection unit (120) based on the information so that a projection image is output within the second area (510).

[0076] FIG. 6 is a diagram illustrating a method for identifying a viewing obstruction area according to one embodiment.

[0077] According to FIG. 6, according to one embodiment, the processor (140) can identify a viewing obstruction area (630). The viewing obstruction area (630) refers to an area where a user's viewing is obstructed when an image is output within the projection surface due to an object (60) existing within the second area (620). According to one example, when an object (60) within the second area (620) is identified based on information about the second area (620) and acquired sensing data, the processor (140) can identify the viewing obstruction area (630) within the projection surface based on an area corresponding to the identified object (60). For example, the electronic device may include a vision sensor, and the processor (140) may identify a location of an object (60) existing within the second area (620) based on sensing data acquired through the vision sensor. When the location of the object (60) is identified, the processor (140) can identify an area corresponding to the identified object (60) within the second area (620) as a viewing obstruction area (630). The processor (140) can identify coordinate information of a vertex corresponding to the viewing obstruction area (630) as information about the viewing obstruction area (630).

[0078] According to one embodiment, the processor (140) can update the projection image based on information about the viewing obstruction area (630). According to one example, the processor (140) can identify a third area (610) within the projection surface based on information about the viewing obstruction area (630) and update the projection image to correspond to the identified third area (610). The third area (610) refers to a rectangular area within the second area (620) from which the viewing obstruction area (630) is excluded. The processor (140) can identify, within the identified third area (610), a horizontal length (610-1) of an area reduced by the viewing obstruction area (630) among the second area (620). The processor (140) can update the projection image and the display image to correspond to the identified horizontal length (610-1). For example, the processor (140) can identify a reduced-size projection image and a displayed image without changing the aspect ratio of the output projection image and the displayed image, as illustrated in FIG. 8a.

[0079] According to one embodiment, the processor (140) may control the projection unit (120) to output the updated projection image within an area excluding the viewing obstruction area (630) in the second area (620). According to one example, the processor (140) may control the projection unit (120) to output the updated projection image in the third area (610), and may transmit information about the updated display image to an external display device (6) via the communication interface (110).

[0080] According to one embodiment, the external display device (6) can obtain sensing data from the electronic device (600). The external display device (6) can identify whether an object (60) is identified within the second area (620) based on the obtained sensing data. The external display device (6) can identify a viewing obstruction area (630) within the projection surface based on this, and can update the projection image based on the identified viewing obstruction area (630). The external display device (6) can transmit the updated projection image to the electronic device (600), and the electronic device (600) can control the projection unit (120) to output the received projection image.

[0081] FIG. 7 is a drawing for explaining a method of outputting a guide UI (User Interface) according to one embodiment.

[0082] According to FIG. 7, according to one embodiment, the processor (140) may provide a guide UI (User Interface) 70 for selecting either the projection image or the updated projection image. According to one example, when a viewing obstruction area within the projection surface is identified, the processor (140) may control the projection unit (120) to output a guide UI (70) for selecting either the projection image or the updated projection image. For example, the processor (140) may control the projection unit (120) to output a guide UI (70) including text information corresponding to 'Viewing may be obstructed due to surrounding structures. Would you like to enlarge in the fourth output method?' Here, the fourth output method refers to a method of projecting content such that a letter box is removed, as illustrated in FIG. 4b. Alternatively, the processor (140) may transmit information about the identified guide UI (70) to an external display device (7) via a communication interface (110).

[0083] In one embodiment, the external display device (7) may provide a guide UI (User Interface) 70 for selecting either the projected image or the updated projected image. In one example, when a viewing obstruction area within the projection surface is identified, the external display device (7) may output a guide UI (70) for selecting either the projected image or the updated projected image.

[0084] FIGS. 8A and 8B are drawings for explaining a method of outputting an image according to one embodiment.

[0085] In one embodiment, the processor (140) can output either a projected image or an updated projected image based on user input.

[0086] According to FIG. 8A, according to an example, when a user input for projecting content while avoiding a viewing obstruction area is received, the external display device (8) can transmit the received user input to the electronic device (800). When a user input for projecting content while avoiding a viewing obstruction area is received through a communication interface, the processor (140) can control the projection unit (120) to output updated projection images (81-1 and 82-1). Since a method for obtaining the updated projection images (81-1 and 82-1) has been described with reference to FIG. 6, a detailed description thereof will be omitted.

[0087] For example, when a user input for projecting content such that a letter box is removed within the external display device (8) is received, the external display device (8) may transmit the received user input to the electronic device (800). When a user input for projecting content such that a letter box is removed is received via the communication interface, the processor (140) may control the projection unit (120) to output updated projection images (81-2 and 82-2) as illustrated in FIG. 8B.

[0088] In one embodiment, the processor (140) can identify either a display image or an updated display image based on received user input, and can transmit information about the identified either image to an external display device (8) via a communication interface. The external display device (8) can display either an output image or an updated output image.

[0089] For example, as illustrated in FIG. 8A, when a user input is received to project content while avoiding a viewing obstruction area, the processor (140) may transmit an updated display image (83-1) to the external display device (8) via the communication interface (110). Alternatively, as illustrated in FIG. 8B, when a user input is received to project content while removing a letter box within the external display device (8), the processor (140) may transmit an updated display image (83-2) to the external display device (8) via the communication interface (110).

[0090] FIG. 9 is a flowchart illustrating a method for controlling an electronic device according to an embodiment.

[0091] According to FIG. 9, first, according to an embodiment, a control method may identify a center region of an external display device based on information about a first region (S910). According to an example, the processor (140) may identify information about a first region corresponding to an external display device within a projection surface based on sensing data acquired through at least one sensor. Once the information about the first region is identified, the processor (140) may identify a location corresponding to the center of the first region as the center region of the external display device. For example, the processor (140) may identify a location corresponding to the center of the first region based on the location of a vertex corresponding to the first region.

[0092] Next, according to one embodiment, the control method may control the driving unit of the electronic device (100) so that the electronic device (100) is positioned within a preset distance from the center region based on the identified center region (S920). According to one example, the electronic device (100) may further include a driving unit for moving the electronic device (100). When the center region of the external display device is identified, the processor (140) may control the driving unit so that the center (e.g., center of gravity) of the electronic device (100) is positioned within a preset distance from the center region of the external display device, as illustrated in FIGS. 8A and 8B .

[0093] FIG. 10 is a diagram illustrating a method for re-identifying a projection image and a display image according to one embodiment.

[0094] First, according to FIG. 10, according to one embodiment, the control method can determine whether the PPI (Pixels Per Inch) value of the projection image is less than the PPI value for the external display device (S1010).

[0095] In one example, the processor (140) may identify a PPI value for an external display device. For example, the processor (140) may identify a PPI value for the external display device from the external display device via the communication interface (110). Alternatively, the memory of the electronic device (100) may store PPI values ​​for an external display device that is communicatively connected to the electronic device (100), and the processor (140) may identify a PPI value for the external display device based on the information stored in the memory.

[0096] For example, when a projection image is identified based on image information corresponding to content, the processor (140) can identify a PPI value of the identified projection image, and compare the PPI value of the identified projection image with a PPI value for an external display device to determine whether the PPI (Pixels Per Inch) value of the projection image is less than the PPI value for the external display device.

[0097] Next, according to one embodiment, if the PPI (Pixels Per Inch) value of the projection image is less than the PPI value for the external display device (S1010: Y), the control method can re-identify the projection image to be output from the electronic device (100) and the display image to be displayed on the external display device based on the PPI value for the external display device (S1020). According to one example, the processor (140) can identify a first image, of which the PPI value of the image corresponding to the content is the same as the PPI value for the external display device, based on image information corresponding to the content. The processor (140) can identify the projection image and the display image corresponding to the first image based on information about the identified first image, aspect ratio information corresponding to the content, and information stored in the memory.

[0098] Next, according to one embodiment, the control method can output the re-identified projection image (S1030). According to one example, the processor (140) can control the projection unit (120) so that the projection image corresponding to the identified first image is output to the projection area.

[0099] Next, according to one embodiment, the control method may transmit the re-identified display image to an external display device (S1040). According to one example, the processor (140) may transmit information about the display image corresponding to the first image to the external display device via the communication interface (110).

[0100] FIG. 11 is a drawing for explaining a method for providing a UI menu according to one embodiment.

[0101] According to FIG. 11, according to one embodiment, the processor (140) can identify at least one of a projection image to be output through the projection unit (120) and a display image to be displayed on the external display device (11) based on a user command received through the UI menu (1100) for selecting an output method of content. According to one example, the external display device (11) can display the UI menu (1100) for selecting an output method of content, and based on this, the external display device (11) can receive a user command for selecting an output method of content from the user. When the user command is received, the processor (140) can receive the user command from the external display device (8) through the communication interface (110). The processor (140) can identify the projection image to be output through the projection unit (120) and the display image to be displayed on the external display device (11) based on the received user command.

[0102] According to one embodiment, the UI menu (1100) may include at least one of a first output method (e.g., “TV mode”) that outputs content only on an external display device (11), a second output method (e.g., “maximum size mode”) that projects content in a maximum projection area, a third output method (e.g., “appropriate size mode”) that projects content while avoiding a viewing obstruction area, and a fourth output method (e.g., “letter box removal mode”) that projects content such that letter boxes are removed.

[0103] Fig. 12 is a block diagram showing a detailed configuration of an electronic device according to one embodiment.

[0104] According to FIG. 12, the electronic device (100') may include a communication interface (110), a projection unit (120), a memory (130), one or more processors (140), a microphone (150), a speaker (160), a display (170), a user interface (180), and at least one sensor (190). Among the configurations illustrated in FIG. 12, a detailed description of configurations that overlap with those illustrated in FIG. 2 will be omitted.

[0105] A microphone (150) may refer to a module that acquires sound and converts it into an electrical signal, and may be a condenser microphone, a ribbon microphone, a moving coil microphone, a piezoelectric element microphone, a carbon microphone, or a MEMS (Micro Electro Mechanical System) microphone. In addition, it may be implemented in an omnidirectional, bidirectional, unidirectional, subcardioid, supercardioid, or hypercardioid manner.

[0106] The speaker (160) may be composed of a tweeter for reproducing high-frequency sounds, a midrange for reproducing mid-frequency sounds, a woofer for reproducing low-frequency sounds, a subwoofer for reproducing ultra-low-frequency sounds, an enclosure for controlling resonance, and a crossover network for dividing the frequency of an electric signal input to the speaker into bands.

[0107] The speaker (160) can output an audio signal to the outside of the electronic device (100'). The speaker (160) can output multimedia playback, recording playback, various notification sounds, voice messages, etc. The electronic device (100') may include an audio output device such as the speaker (160), but may also include an output device such as an audio output terminal. In particular, the speaker (160) can provide acquired information, information processed and produced based on acquired information, response results to user voice, operation results, etc. in voice form.

[0108] The display (170) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, a QLED (Quantum dot light-emitting diodes), etc. The display (170) may also include a driving circuit, a backlight unit, etc., which may be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (170) may be implemented as a touch screen combined with a touch sensor, a flexible display, a rollable display, a 3D display, a display in which a plurality of display modules are physically connected, etc. The processor (140) can control the display (170) to output an output image obtained according to the various embodiments described above. Here, the output image may be a high-resolution image of 4K or 8K or higher. The output image may also be a game image according to one embodiment.

[0109] The user interface (180) is a configuration for the electronic device (100') to interact with the user. For example, the user interface (180) may include, but is not limited to, at least one of a touch sensor, a motion sensor, a button, a jog dial, a switch, a microphone, or a speaker.

[0110] At least one sensor (190, hereinafter referred to as a sensor) may include a plurality of sensors of various types. The sensor (190) may measure a physical quantity or detect an operating state of an electronic device (100') and convert the measured or detected information into an electrical signal. The sensor (190) may include a camera, and the camera may include a lens that focuses visible light or other optical signals reflected by an object and received onto an image sensor, and an image sensor that may detect visible light or other optical signals. Here, the image sensor may include a 2D pixel array divided into a plurality of pixels. Alternatively, at least one sensor (190) may include a temperature sensor or an infrared sensor.

[0111] According to the above-described example, the electronic device (100') can display a portion of the content displayed on the external display device in a preset area within the projection surface. The electronic device (100') can then link with the external display device to provide the user with expanded content. Accordingly, the electronic device (100') can provide an immersive viewing experience.

[0112] Meanwhile, the methods according to the various embodiments of the present disclosure described above may be implemented in the form of applications that can be installed on existing electronic devices. Alternatively, the methods according to the various embodiments of the present disclosure described above may be performed using a deep learning-based trained neural network (or a deep learned neural network), i.e., a learning network model. Furthermore, the methods according to the various embodiments of the present disclosure described above may be implemented only through a software upgrade or a hardware upgrade of an existing electronic device. Furthermore, the various embodiments of the present disclosure described above may also be performed through an embedded server provided in an electronic device or an external server of the electronic device.

[0113] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including commands stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The machine is a device that can call commands stored from the storage medium and operate according to the called commands, and may include a display device (e.g., display device (A)) according to the disclosed embodiments. When a command is executed by a processor, the processor can perform a function corresponding to the command directly or by using other components under the control of the processor. The command may include code provided or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0114] Furthermore, according to one embodiment, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily provided in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0115] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0116] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, communication interface; Projection section; A memory storing information about the projection range ratio for each aspect ratio of content provided to an external display device; and When aspect ratio information and image information corresponding to content displayed on the external display device are received through the communication interface, a projection area is identified based on the aspect ratio information corresponding to the received content and the information stored in the memory, Identifying a projection image to be output to the projection area based on aspect ratio information corresponding to the content, the image information, and information stored in the memory, An electronic device comprising: one or more processors for controlling the projection unit so that the identified projection image is output to the identified projection area.

2. In paragraph 1, One or more of the above processors, When a projection image to be output to the projection area is identified, a display image to be displayed on the external display device is identified based on the identified projection image and the image information of the content, An electronic device that transmits information about the identified display image to the external display device through the communication interface.

3. In paragraph 1, further comprising at least one sensor; One or more of the above processors, Identifying information about a first area corresponding to a position of the external display device within the projection surface based on sensing data acquired through at least one sensor; Identify information about a second area corresponding to the maximum projection area within the projection surface based on the size information of the external display device and the information about the first area, An electronic device that controls the projection unit so that the projection image is output within the second area.

4. In paragraph 3, One or more of the above processors, When an object within the second area is identified based on the information about the second area and the acquired sensing data, an area of ​​obstructed viewing within the projection surface is identified based on an area corresponding to the identified object, Update the projection image based on information about the viewing obstruction area and aspect ratio information of the content, An electronic device that controls the projection unit so that the updated projection image is output within an area excluding the viewing obstruction area in the second area.

5. In paragraph 4, One or more of the above processors, An electronic device that controls the projection unit to output a guide UI (User Interface) for selecting one of the projection image and the updated projection image when an obstructed viewing area within the projection surface is identified, or transmits information about the identified guide UI to the external display device through the communication interface.

6. In paragraph 5, One or more of the above processors, An electronic device that controls the projection unit to output one of the projection image and the updated projection image based on the received user input when a user input for selecting one of the projection image and the updated projection image is received through the communication interface.

7. In paragraph 3, Further comprising a driving unit for moving the above electronic device; One or more of the above processors, Identifying a central area of ​​the external display device based on information about the first area; An electronic device that controls the driving unit so that the electronic device is positioned within a preset distance from the central region based on the identified central region.

8. In paragraph 1, One or more of the above processors, If the PPI (Pixels Per Inch) value of the above projection image is identified as being less than the PPI value for the external display device, the projection image to be output from the electronic device and the display image to be displayed on the external display device are re-identified based on the PPI value for the external display device, Controlling the projection unit to output the re-identified projection image; An electronic device that transmits the re-identified display image to the external display device through the communication interface.

9. In paragraph 1, One or more of the above processors, Identifying at least one of a projection image to be output through the projection unit and a display image to be displayed on the external display device based on a user command received through a UI menu for selecting an output method of the content, The above UI menu is, An electronic device comprising at least one of a first output method for outputting content only on the external display device, a second output method for projecting content on a maximum projection area, a third output method for projecting content while avoiding a viewing obstruction area, and a fourth output method for projecting content so as to remove letter boxes.

10. In a method for controlling an electronic device, When aspect ratio information and image information corresponding to content displayed on an external display device are received, a step of identifying a projection area based on aspect ratio information corresponding to the received content and information on a projection range ratio for each aspect ratio of the content provided to the external display device stored in a memory; A step of identifying a projection image to be output to the projection area based on aspect ratio information corresponding to the content, the image information, and information stored in the memory; and A control method, comprising: a step of outputting the identified projection image to the identified projection area.

11. In paragraph 10, When a projection image to be output to the projection area is identified, a step of identifying a display image to be displayed on the external display device based on the identified projection image and image information of the content; and A control method further comprising: a step of transmitting information about the identified display image to the external display device.

12. In paragraph 10, A step of identifying information about a first area corresponding to a position of the external display device within a projection surface based on sensing data acquired through at least one sensor; and Further comprising a step of identifying information on a second area corresponding to a maximum projection area within the projection surface based on size information of the external display device and information on the first area; The above output step is, A control method for outputting the above projection image within the second area.

13. In paragraph 12, A step of identifying a viewing obstruction area within the projection surface based on an area corresponding to the identified object when an object within the second area is identified based on information about the second area and the acquired sensing data; and Further comprising a step of updating the projection image based on information about the viewing obstruction area and aspect ratio information of the content; The above output step is, A control method for outputting the updated projection image within an area excluding the viewing obstruction area in the second area.

14. In paragraph 13, A control method further comprising: a step of outputting a guide UI (User Interface) for selecting one of the projection image and the updated projection image, or transmitting information about the identified guide UI to the external display device when a viewing obstruction area within the projection surface is identified.

15. A non-transitory computer-readable recording medium storing computer instructions that, when executed by a processor of an electronic device, cause the electronic device to perform an operation, wherein the operation comprises: When aspect ratio information and image information corresponding to content displayed on an external display device are received, a step of identifying a projection area based on aspect ratio information corresponding to the received content and information on a projection range ratio for each aspect ratio of the content provided to the external display device stored in a memory; A step of identifying a projection image to be output to the projection area based on aspect ratio information corresponding to the content, the image information, and information stored in the memory; and A computer-readable recording medium comprising: a step of outputting the identified projection image to the identified projection area;

Images