Electronic device for projecting image by chaneging position of partial region of image and control method therefor

Abstract

Disclosed is an electronic device. The electronic device comprises a memory for storing instructions, a projection unit, a camera, and one or more processors including processing circuitry, wherein the instructions, when executed individually or collectively by the one or more processors, may cause the electronic device to: control the projection unit to project an image onto a projection area; acquire a captured image obtained by photographing the projection area by means of the camera; and if an object that is covering the image projected onto the projection area is identified from the captured image, control the projection unit to project the image with the position of a partial area of the image chaneged on the basis of the area in which the object is identified.

Description

Electronic device for projecting by changing the position of a portion of an image and a control method thereof

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more specifically, to an electronic device and a method for controlling the same that change the position of a portion of an image and project it.

[0002] With the advancement of electronic technology, electronic devices offering various functions are being developed. In particular, various types of projectors have recently become widespread.

[0003] A projector is a device that projects images. By projecting light onto a projection area to create an image, it has the advantage of easily enabling large screens compared to other types of display devices.

[0004] Recently, projectors or accessory devices attached to projectors that detect user touch interactions on a specific area of ​​the projection area are also being developed.

[0005] However, there is a problem where part of the image being projected onto the projection area is obscured depending on the user's touch interaction.

[0006] According to one embodiment of the present disclosure for achieving the above objectives, an electronic device comprises one or more processors including a memory for storing instructions, a projection unit, a camera, and processing circuitry, and when the instructions are executed individually or collectively by the one or more processors, the projection unit is controlled to project an image onto a projection area, and a captured image of the projection area is obtained through the camera, and when an object obscuring the image projected onto the projection area is identified from the captured image, the projection unit is controlled to change the position of a part of the image based on the area where the object is identified and project it.

[0007] In addition, when the above instructions are executed individually or collectively by the one or more processors, the projection unit can be controlled to change the position of at least one of the plurality of icons included in the image and project it based on the area where the object is identified.

[0008] And, when the above instructions are executed individually or collectively by the one or more processors, the area to be modified and projected can be identified based on the shape of the object.

[0009] Additionally, the infrared emitting unit is further included, and the instructions, when executed individually or collectively by the one or more processors, control the infrared emitting unit to emit a plurality of infrared rays into the projection area, identify an infrared distribution in the captured image, and identify the object based on the infrared distribution.

[0010] And, when the above instructions are executed individually or collectively by the one or more processors, prior to projecting the image, a third image is obtained by capturing the area on which the image is to be projected through the camera, a third object is identified in the third image, and if the third object is larger than a preset size, a message is provided to guide the movement of the third object, and if the third object is smaller than the preset size, at least one of the size or position of the projection area can be changed.

[0011] Additionally, the system further includes a sensor, and when the instructions are executed individually or collectively by the one or more processors, the projection unit can acquire attitude information of the electronic device through the sensor and control the projection unit to invert the image based on the attitude information and project it onto the projection area.

[0012] And, when the above instructions are executed individually or collectively by the one or more processors, the sensor identifies whether the posture of the electronic device is a first posture or a second posture, and if the posture of the electronic device is the first posture, the projection unit controls the projection unit to project the image onto the projection area, and if the posture of the electronic device is the second posture, the projection unit controls the projection unit to invert the image and project it onto the projection area.

[0013] Additionally, when the above instructions are executed individually or collectively by the one or more processors, the projection unit can be controlled to change the position of a part of the image and project it based on the area where the object is identified and the pose information.

[0014] And, when the above instructions are executed individually or collectively by one or more processors, if the electronic device is connected to a touch sensing device including an infrared emitting unit, an infrared distribution in the captured image can be identified and the object can be identified based on the infrared distribution.

[0015] Additionally, when the above instructions are executed individually or collectively by the one or more processors, they may provide a message guiding the reconnection of the electronic device and the touch sensing device based on the position of the projection area in the captured image.

[0016] Meanwhile, according to one embodiment of the present disclosure, a control method for an electronic device may include the steps of projecting an image onto a projection area, acquiring a captured image of the projection area, and, if an object obscuring the image projected onto the projection area is identified from the captured image, changing the position of a part of the image based on the area where the object is identified and projecting.

[0017] In addition, the step of changing and projecting above may change the position of at least one icon among a plurality of icons included in the image based on the area where the object is identified and project it.

[0018] And, the step of changing and projecting above can identify the area to be changed and projected based on the shape of the object.

[0019] Additionally, the method further includes a step of emitting a plurality of infrared rays into the projection area, and the step of changing and projecting can identify an infrared distribution in the captured image and identify the object based on the infrared distribution.

[0020] Additionally, before projecting the image, the method may further include the steps of acquiring a captured image of the area on which the image is to be projected, identifying a third object in the captured image, and providing a message guiding the movement of the third object if the third object is larger than a preset size, and changing at least one of the size or position of the projection area if the third object is smaller than the preset size.

[0021] Additionally, the method further includes the step of acquiring attitude information of the electronic device, and the step of projecting to the projection area may project the image to the projection area by inverting it based on the attitude information.

[0022] And, the step of acquiring the posture information identifies whether the posture of the electronic device is a first posture or a second posture, and the step of projecting to the projection area projects the image to the projection area if the posture of the electronic device is the first posture, and projects the image inverted to the projection area if the posture of the electronic device is the second posture.

[0023] In addition, the step of changing and projecting above may change the position of a part of the image and project based on the area where the object is identified and the pose information.

[0024] And, in the step of changing and projecting above, when the electronic device is connected to a touch sensing device including an infrared emitting part, the infrared distribution in the captured image can be identified, and the object can be identified based on the infrared distribution.

[0025] Additionally, the method may further include a step of providing a message guiding the reconnection of the electronic device and the touch sensing device based on the position of the projection area in the captured image.

[0026] FIG. 1 is a block diagram showing the configuration of an electronic device according to one embodiment of the present disclosure.

[0027] FIG. 2 is a block diagram showing the detailed configuration of an electronic device according to one embodiment of the present disclosure.

[0028] FIGS. 3 and FIGS. 4 are drawings for explaining an operation according to posture information according to one embodiment of the present disclosure.

[0029] FIGS. 5 and 6 are drawings for explaining a method of projecting by changing the position of a portion of an image based on posture information according to one embodiment of the present disclosure.

[0030] FIGS. 7 and 8 are drawings for explaining an operation to identify an area to be projected by changing based on the shape of an object according to one embodiment of the present disclosure.

[0031] FIG. 9 is a drawing for explaining the object of position change according to one embodiment of the present disclosure.

[0032] FIGS. 10 and FIGS. 11 are drawings for explaining an infrared emission operation according to one embodiment of the present disclosure.

[0033] FIGS. 12 and FIGS. 13 are drawings for explaining the operation according to the identification of other objects according to one embodiment of the present disclosure.

[0034] FIG. 14 is a drawing for explaining the operation according to the combination of an electronic device and a touch sensing device according to one embodiment of the present disclosure.

[0035] FIG. 15 is a flowchart illustrating the overall operation of an electronic device according to one embodiment of the present disclosure.

[0036] FIG. 16 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0037] The purpose of the present disclosure is to provide an electronic device that projects by changing the position of a portion of an image and a method for controlling the same.

[0038] The present disclosure will be described in detail below with reference to the attached drawings.

[0039] The terms used in the embodiments of this disclosure have been selected to be as widely used as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section of this disclosure. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0040] In this specification, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, operations, or components such as parts) and do not exclude the presence of additional features.

[0041] The expression "at least one of A or / and B" should be understood as representing either "A" or "B" or "A and B".

[0042] Expressions such as "first," "second," "first," or "second" used in this specification may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0043] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "consisting of" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0044] In this specification, the term "user" may refer to a person using the display device or a device using the display device (e.g., an artificial intelligence electronic device).

[0045] Various embodiments of the present disclosure will be described in more detail below with reference to the attached drawings.

[0046] FIG. 1 is a block diagram showing the configuration of an electronic device (100) according to one embodiment of the present disclosure.

[0047] The electronic device (100) may be a device that projects an image onto a projection area. For example, the electronic device (100) may be an ultra-short throw projector that projects an image. An ultra-short throw projector can project a large screen even when the distance from the projection area is 0.5 m or less. However, it is not limited to this, and the electronic device (100) may be a general projector rather than an ultra-short throw projector.

[0048] The electronic device (100) may identify an object that obstructs an image projected onto a projection area and project by changing the position of a part of the image based on the area where the object is identified. Alternatively, the electronic device (100) may receive information about an object that obstructs an image projected onto a projection area from an external device and project by changing the position of a part of the image based on the received information.

[0049] However, it is not limited thereto, and the electronic device (100) may be a device that provides an image to a projector outside the electronic device (100), and when an object obstructing the image projected onto the projection area is identified, provides an image to the projector in which the position of a part of the image is changed based on the area where the object is identified. Alternatively, the electronic device (100) may be a device that provides an image to a projector outside the electronic device (100), and when an object obstructing the image projected onto the projection area is identified, provides information about the area where the object is identified to the projector.

[0050] According to FIG. 1, the electronic device (100) includes a memory (110), a projection unit (120), a camera (130), and a processor (140).

[0051] Memory (110) may refer to hardware that stores information, such as data, in an electrical or magnetic form so that a processor (140), etc. can access it. To this end, memory (110) may be implemented as at least one piece of hardware among non-volatile memory, volatile memory, flash memory, hard disk drive (HDD) or solid-state drive (SSD), RAM, ROM, etc.

[0052] At least one instruction required for the operation of an electronic device (100) or a processor (140) may be stored in the memory (110). Here, the instruction is a unit of code that directs the operation of the electronic device (100) or the processor (140), and may be written in machine language, which is a language that a computer can understand. Alternatively, a plurality of instructions that perform a specific task of the electronic device (100) or the processor (140) may be stored in the memory (110) as an instruction set.

[0053] Data in bit or byte units, which can represent characters, numbers, images, etc., can be stored in the memory (110). For example, images, object identification modules, neural network models, etc., can be stored in the memory (110).

[0054] The memory (110) is accessed by the processor (140), and the processor (140) may perform read / write / modify / delete / update, etc. on instructions, instruction sets, or data.

[0055] The projection unit (120) can project an image onto a projection area. Specifically, the projection unit (120) can project an image or video onto a projection area using a light source such as a lamp or an LED, the image or video including at least one of an image received from a source device and an image stored in advance.

[0056] The camera (130) is configured to capture still images or video. The camera (130) can capture a still image at a specific point in time, but can also capture a series of still images.

[0057] For example, the camera (130) can photograph the front of the electronic device (100) to photograph the area where the content is projected. The processor (140) can identify an object that obscures the image based on the captured image taken through the camera (130).

[0058] The camera (130) includes a lens, a shutter, an aperture, a solid-state image sensor, an AFE (Analog Front End), and a TG (Timing Generator). The shutter controls the time when light reflected from a subject enters the camera (130), and the aperture controls the amount of light incident on the lens by mechanically increasing or decreasing the size of the opening through which light enters. When light reflected from a subject accumulates as photocharge, the solid-state image sensor outputs an image based on the photocharge as an electrical signal. The TG outputs a timing signal for reading out pixel data from the solid-state image sensor, and the AFE samples and digitizes the electrical signal output from the solid-state image sensor.

[0059] The processor (140) controls the overall operation of the electronic device (100). Specifically, the processor (140) can control the overall operation of the electronic device (100) by being connected to each component of the electronic device (100). For example, the processor (140) can control the operation of the electronic device (100) by being connected to components such as memory (110), projection unit (120), camera (130), etc.

[0060] One or more processors (140) may include one or more of a CPU, a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), a NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. One or more processors (140) may control one or any combination of other components of the electronic device (100) and may perform operations or data processing related to communication. One or more processors (140) may execute one or more programs or instructions stored in memory (110). For example, one or more processors (140) may perform a method according to one embodiment of the present disclosure by executing one or more instructions stored in memory (110).

[0061] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an artificial intelligence dedicated processor).

[0062] One or more processors (140) may be implemented as a single-core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When one or more processors (140) are implemented as multicore processors, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0063] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one of the plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in a multi-core processor, or the first operation and the second operation may be performed by a first core included in a multi-core processor and the third operation may be performed by a second core included in a multi-core processor.

[0064] In the embodiments of the present disclosure, one or more processors (140) may refer to a system-on-chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, GPU, APU, MIC, NPU, hardware accelerator, or machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. However, for convenience of explanation, the operation of the electronic device (100) is described below using the expression "processor (140)."

[0065] The processor (140) can control the projection unit (120) to project an image onto a projection area. Here, the projection area may be an area where light emitted through the projection unit (120) is projected.

[0066] The processor (140) can obtain a captured image of the projection area through the camera (130). For example, when the processor (140) controls the projection unit (120) to project an image, it can turn on the camera (130) and obtain a captured image of the projection area through the camera (130). Here, the captured image may be an image of the projection area and the area surrounding the projection area.

[0067] When the processor (140) identifies an object that obscures the image projected onto the projection area from the captured image, it can control the projection unit (120) to change the position of a part of the image based on the area where the object is identified and project it.

[0068] For example, the electronic device (100) may be a device capable of identifying touch interactions on a projection area, or a device combined with an accessory capable of identifying touch interactions on a projection area. In this case, the user may touch a part of the image projected onto the projection area. The processor (140) may identify an object that obscures the image projected onto the projection area from a captured image, and control the projection unit (120) to change the position of a part of the image based on the area where the object is identified and project it.

[0069] The processor (140) can control the projection unit (120) to change the position of at least one of the multiple icons included in the image and project it based on the area where the object is identified.

[0070] For example, the processor (140) may control the projection unit (120) to change the position of at least one icon included in the area where the object is identified and project it. Alternatively, the processor (140) may control the projection unit (120) to change the position of at least one icon included within a preset distance from the area where the object is identified and project it.

[0071] The processor (140) can identify the area to be projected by changing it based on the shape of the object. For example, if the object is the user's right arm, the processor (140) can identify the area to be projected by changing the left side of the touch point.

[0072] However, it is not limited thereto, and the processor (140) may identify the area to be projected by changing based on the type of object included in the image. For example, the processor (140) may identify the area to be projected by changing the area among the objects included in the image that does not have an icon. Alternatively, the processor (140) may identify the area to be projected by changing based on the shape of the object and the type of object included in the image. For example, if the object is the user's right arm, the processor (140) may identify the left area of ​​the touch point as a candidate area, and identify the area among the objects included in the candidate area that does not have an icon as the area to be projected by changing. Here, the type of object may include an icon, an image, text, etc., and the processor (140) may identify the area to be projected by changing based on information regarding the priority of each type of object.

[0073] The electronic device (100) further includes an infrared emitting unit, and the processor (140) controls the infrared emitting unit to emit a plurality of infrared rays into a projection area, identifies an infrared distribution in a captured image, and can identify an object based on the infrared distribution.

[0074] However, it is not limited to this, and the processor (140) may also identify objects by inputting a captured image into a neural network model.

[0075] Alternatively, the electronic device (100) may be connected to a touch sensing device including an infrared emitting unit. In this case, the processor (140) may identify an infrared distribution in a captured image and identify an object based on the infrared distribution.

[0076] The processor (140) can identify whether the electronic device (100) is connected to a touch sensing device including an infrared emitter. If the processor (140) identifies that the electronic device (100) is connected to a touch sensing device including an infrared emitter, it identifies an infrared distribution in a captured image and identifies an object based on the infrared distribution, and if the electronic device (100) is identified that it is not connected to a touch sensing device including an infrared emitter, it may identify an object from the captured image itself.

[0077] The processor (140) may provide a message guiding the reconnection of the electronic device (100) and the touch detection device when the connection between the electronic device (100) and the touch detection device is incomplete. For example, the processor (140) may provide a message guiding the reconnection of the electronic device (100) and the touch detection device based on the position of the projection area in the captured image.

[0078] Before projecting the image, the processor (140) obtains a captured image of the area to be projected through the camera (130), identifies other objects in the captured image, and if the other objects are larger than a preset size, provides a message guiding the movement of the other objects, and if the other objects are smaller than a preset size, changes at least one of the size or position of the projection area.

[0079] However, it is not limited thereto, and the processor (140) may provide a message by further considering the location of other objects, or change at least one of the size or location of the projection area. For example, if another object is identified within a preset distance from the center of the projection area, the processor (140) may provide a message guiding the movement of the other object, and if another object is identified outside the preset distance from the center of the projection area, it may change at least one of the size or location of the projection area.

[0080] The electronic device (100) further includes a sensor, and the processor (140) can obtain posture information of the electronic device (100) through the sensor and control the projection unit (120) to invert the image based on the posture information and project it onto a projection area.

[0081] For example, the processor (140) can identify whether the posture of the electronic device (100) is a first posture or a second posture through a sensor, and if the posture of the electronic device (100) is a first posture, control the projection unit (120) to project an image onto a projection area, and if the posture of the electronic device (100) is a second posture, control the projection unit (120) to project an inverted image onto a projection area. Here, the first posture and the second posture may be determined based on the angle formed between the electronic device (100) and the horizontal plane of the floor.

[0082] The processor (140) can control the projection unit (120) to change the position of a portion of the image and project it based on the area and posture information where the object is identified. For example, if the posture of the electronic device (100) is one of the first posture and the second posture, the processor (140) can control the projection unit (120) to change the position of an icon included in the area where the object is identified and project it, and if the posture of the electronic device (100) is the other of the first posture and the second posture, the processor (140) can control the projection unit (120) to change the position of an icon included in the area where the object is identified and the area where the image is not projected onto the projection area by the object and project it.

[0083] The processor (140) can control the projection unit (120) to change the position of a part of the image and project it when an object is identified and the position of a part of the image is changed and projected, and when the object is not identified again, restore the position of a part of the image and project it.

[0084] Meanwhile, the artificial intelligence-related functions according to the present disclosure can be operated through the processor (140) and memory (110).

[0085]

[0086] The processor (140) may be composed of one or more processors. In this case, the one or more processors may be a general-purpose processor such as a CPU, AP, DSP, etc., a graphics-dedicated processor such as a GPU, VPU (Vision Processing Unit), or an artificial intelligence-dedicated processor such as an NPU.

[0087] One or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory (110). Alternatively, if one or more processors are dedicated artificial intelligence processors, the dedicated artificial intelligence processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model. The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0088] Here, "created through learning" means that a basic artificial intelligence model is trained using multiple learning data by a learning algorithm, thereby creating a predefined rule of operation or an artificial intelligence model configured to perform a desired characteristic (or objective). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0089] An artificial intelligence model can be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated during the learning process so that the loss or cost values ​​obtained by the artificial intelligence model are reduced or minimized.

[0090] Artificial neural networks may include deep neural networks (DNNs), such as, but are not limited to, Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), Generative Adversarial Networks (GANs), or Deep Q-Networks.

[0091] FIG. 2 is a block diagram showing the detailed configuration of an electronic device (100) according to one embodiment of the present disclosure. The electronic device (100) may include a memory (110), a projection unit (120), a camera (130), and a processor (140). Additionally, according to FIG. 2, the electronic device (100) may further include an infrared emitting unit (150), a sensor (160), a communication interface (170), a user interface (175), a display (180), a microphone (185), and a speaker (190). Detailed descriptions of parts of the components shown in FIG. 2 that overlap with the components shown in FIG. 1 are omitted.

[0092] The infrared emitting unit (150) is configured to emit infrared rays and can emit multiple infrared rays into a projection area. For example, when an electronic device (100) is placed on the floor and the electronic device (100) projects an image onto a projection area on the floor, the processor (140) can control the infrared emitting unit (150) to emit multiple infrared rays into the projection area.

[0093] However, it is not limited thereto, and the electronic device (100) may be connected to a touch sensing device including an infrared emitting unit. In this case, when the processor (140) controls the projection unit (120) to project an image, it may also control the communication interface (170) to transmit an infrared emission command or information that the image is being projected to the touch sensing device. Additionally, when the image is not being projected, the processor (140) may control the communication interface (170) to transmit an infrared emission stop command or information that the image is not being projected to the touch sensing device. In this case, the touch sensing device may emit infrared or stop emitting based on a signal received from the electronic device (100).

[0094] The sensor (160) may include a configuration for acquiring attitude information of the electronic device (100). For example, the sensor (160) may include at least one of a gyroscope sensor, an accelerometer sensor, or a magnetometer sensor.

[0095] A gyro sensor is a sensor for detecting the rotation angle of an electronic device (100), and can measure changes in the orientation of an object by utilizing the property of always maintaining a constant direction initially set with high accuracy regardless of the rotation of the Earth. A gyro sensor is also called a gyroscope and can be implemented in a mechanical manner or an optical manner using light.

[0096] A gyroscope sensor can measure angular velocity. Angular velocity refers to the angle of rotation per unit of time, and the measurement principle of a gyroscope sensor is as follows. For example, in a horizontal state (stationary state), the angular velocity is 0 degrees / sec. If an object tilts by 50 degrees while moving for 10 seconds, the average angular velocity over those 10 seconds is 5 degrees / sec. If the tilt angle of 50 degrees is maintained while stationary, the angular velocity becomes 0 degrees / sec. Through this process, the angular velocity changes from 0 to 5 to 0, and the angle increases from 0 degrees to 50 degrees. To calculate the angle from the angular velocity, integration must be performed over the entire time. Since the gyroscope sensor measures angular velocity in this manner, the tilt angle can be calculated by integrating this angular velocity over the entire time. However, errors occur in the gyroscope sensor due to the influence of temperature, and as these errors accumulate during the integration process, the final value may drift. Accordingly, the electronic device (100) may further be equipped with a temperature sensor and can compensate for the error of the gyro sensor using the temperature sensor.

[0097] An acceleration sensor is a sensor that measures the acceleration or the intensity of an impact of an electronic device (100), and is also called an accelerometer. An acceleration sensor detects dynamic forces such as acceleration, vibration, and impact, and can be implemented as an inertial type, a gyro type, a silicon semiconductor type, etc., depending on the detection method. That is, an acceleration sensor is a sensor that senses the degree of tilt of an electronic device (100) using gravitational acceleration, and can typically be composed of a 2-axis or 3-axis flux gate.

[0098] A magnetometer sensor generally refers to a sensor that measures the strength and direction of the Earth's magnetic field; however, in a broader sense, it also includes sensors that measure the magnetization strength of an object and is also called a magnetometer. Magnetometer sensors can be implemented by suspending a magnet horizontally within a magnetic field and measuring the direction of its movement, or by rotating a coil within the field and measuring the induced electromotive force generated in the coil to measure the strength of the magnetic field.

[0099] In particular, a geomagnetic sensor, which measures the strength of the Earth's magnetic field as a type of magnetometer, can generally be implemented as a fluxgate-type geomagnetic sensor that detects geomagnetism using a fluxgate. A fluxgate-type geomagnetic sensor refers to a device that uses a high-permeability material such as permalloy as a magnetic core and applies an excitation field through a driving coil wound around the core; by measuring the second harmonic component proportional to the external magnetic field generated according to the magnetic saturation and nonlinear magnetic characteristics of the core, it measures the magnitude and direction of the external magnetic field. By measuring the magnitude and direction of the external magnetic field, the current azimuth angle is detected, and accordingly, the degree of rotation can be measured. The geomagnetic sensor can be composed of a 2-axis or 3-axis fluxgate. A 2-axis fluxgate sensor, that is, a 2-axis sensor, means a sensor composed of mutually orthogonal X-axis fluxgates and Y-axis fluxgates, and a 3-axis fluxgate, that is, a 3-axis sensor, means a sensor in which a Z-axis fluxgate is added to the X-axis and Y-axis fluxgates.

[0100] By using the geomagnetic sensor and acceleration sensor as described above, attitude information of the electronic device (100) can be obtained. For example, the attitude information of the electronic device (100) can be expressed as pitch angle, roll angle, and azimuth angle.

[0101] Azimuth (yaw) refers to an angle that changes in the left-right direction on a horizontal plane, and by calculating the azimuth, it is possible to determine which direction the electronic device (100) is facing. For example, if a geomagnetic sensor is used, the azimuth can be measured using the following formula.

[0102] ψ=arctan(sinψ / cosψ)

[0103] Here, ψ represents the azimuth angle, and cosψ and sinψ represent the X-axis and Y-axis fluxgate output values.

[0104] The roll angle refers to the angle at which a horizontal plane tilts to the left or right, and by calculating the roll angle, the left or right tilt of the electronic device (100) can be determined. The pitch angle refers to the angle at which a horizontal plane tilts up or down, and by calculating the pitch angle, the angle of inclination of the electronic device (100) to the upper or lower side can be determined. For example, using an accelerometer, the roll angle and pitch angle can be measured through the following formula.

[0105] φ=arcsin(ay / g)

[0106]

[0107] *θ=arcsin(ax / g)

[0108] Here, g represents the acceleration due to gravity, φ represents the roll angle, θ represents the pitch angle, ax represents the X-axis acceleration sensor output value, and ay represents the Y-axis acceleration sensor output value.

[0109] For convenience of explanation, the sensor (160) has been described above as including at least one of a gyroscope sensor, an accelerometer sensor, or a magnetometer sensor. However, it is not limited thereto, and the sensor (160) may be any sensor capable of acquiring attitude information of the electronic device (100).

[0110] The sensor (160) may include a sensor for identifying the distance from the electronic device (100) to the projection area. For example, the sensor (160) may include a ToF sensor. However, it is not limited thereto, and the sensor (160) may be any sensor that can identify the distance from the electronic device (100) to the projection area. Additionally, the processor (140) may identify the distance from the electronic device (100) to the projection area through the camera (130).

[0111] The communication interface (170) is a configuration that performs communication with various types of external devices according to various types of communication methods. For example, an electronic device (100) can perform communication with a server or a user terminal device through the communication interface (170).

[0112] The communication interface (170) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, and a wireless communication module. Here, each communication module may be implemented in the form of at least one hardware chip.

[0113] The Wi-Fi module and Bluetooth module perform communication using the Wi-Fi and Bluetooth methods, respectively. When using the Wi-Fi or Bluetooth module, various connection information, such as the SSID and session key, is transmitted and received first; after establishing a communication connection using this information, various data can be transmitted and received. The infrared communication module performs communication according to infrared communication (IrDA, Infrared Data Association) technology, which wirelessly transmits data over short distances using infrared rays that lie between visible light and millimeter waves.

[0114] In addition to the communication method described above, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

[0115] Alternatively, the communication interface (170) may include a wired communication interface such as HDMI, DP, Thunderbolt, USB, RGB, D-SUB, DVI, etc.

[0116] In addition, the communication interface (170) may include at least one of a LAN (Local Area Network) module, an Ethernet module, or a wired communication module that performs communication using a pair cable, a coaxial cable, or a fiber optic cable.

[0117] The user interface (175) may be implemented as a button, touchpad, mouse, and keyboard, or as a touch screen capable of performing display functions and operation input functions. Here, the button may be a various type of button, such as a mechanical button, touchpad, or wheel, formed in any area of ​​the exterior of the main body of the electronic device (100), such as the front, side, or back.

[0118] The display (180) is configured to display content and can be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, and a PDP (Plasma Display Panel). The display (180) may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, and an OTFT (organic TFT). Meanwhile, the display (180) can be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, etc.

[0119] The microphone (185) is configured to receive sound input and convert it into an audio signal. The microphone (185) is electrically connected to the processor (140) and can receive sound under the control of the processor (140).

[0120] For example, the microphone (185) may be formed as an integrated unit on the upper side, front side, or side side of the electronic device (100). Alternatively, the microphone (185) may be provided in a remote control or the like, separate from the electronic device (100). In this case, the remote control may receive sound through the microphone (185) and provide the received sound to the electronic device (100).

[0121] The microphone (185) may include various configurations such as a microphone that collects analog sound, an amplifier circuit that amplifies the collected sound, an A / D conversion circuit that samples the amplified sound and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0122] Meanwhile, the microphone (185) may be implemented in the form of a sound sensor, and any configuration capable of collecting sound is acceptable.

[0123] The speaker (190) is a component that outputs various audio data processed by the processor (140), as well as various notification sounds or voice messages.

[0124] As described above, the electronic device (100) can improve user convenience by changing the position of some areas of the image and projecting them even if some areas are obscured due to touch interaction or the like during the projection process.

[0125] By compensating for the external areas resulting from keystone correction, the sense of unfamiliarity to the user is minimized, thereby enhancing user immersion.

[0126] The operation of the electronic device (100) will be described in more detail below through FIGS. 3 to 15. FIGS. 3 to 15 describe individual embodiments for convenience of explanation. However, the individual embodiments of FIGS. 3 to 15 may be implemented in any combination.

[0127] FIGS. 3 and FIGS. 4 are drawings for explaining an operation according to posture information according to one embodiment of the present disclosure.

[0128] The processor (140) can obtain posture information of the electronic device (100) through the sensor (160). For example, when the processor (140) receives a user command to project an image, it can obtain posture information of the electronic device (100) through the sensor (160). Alternatively, the processor (140) can obtain posture information of the electronic device (100) when the posture of the electronic device (100) changes while projecting the image.

[0129] The processor (140) can control the projection unit (120) to project an image onto a projection area when the electronic device (100) is in a first position, and can control the projection unit (120) to project an inverted image onto a projection area when the electronic device (100) is in a second position.

[0130] For example, the processor (140) can control the projection unit (120) to project an image inverted and projected into a projection area when the electronic device (100) is positioned parallel to the floor or tilted as in FIG. 3, and can control the projection unit (120) to project an image into a projection area without inversion when the electronic device (100) is positioned tilted as in FIG. 4.

[0131] When the processor (140) controls the projection unit (120) to project an image, it can turn on the infrared emission unit (150) and the camera (130). For example, when the processor (140) controls the projection unit (120) to project an image, it can control the infrared emission unit (150) to emit a plurality of infrared rays to the projection area and obtain a captured image of the projection area through the camera (130). When the projection of the image is finished, the processor (140) can turn off the infrared emission unit (150) and the camera (130).

[0132] The processor (140) can identify an infrared distribution in a captured image and identify an object based on the infrared distribution. For example, a user may touch a portion of the projection area with a finger, in which case some of the infrared emitted from the infrared emitter (150) may be reflected by the finger. The processor (140) can identify an infrared distribution in a captured image and identify a point where the infrared is identified differently from the surroundings by the reflected infrared as the location of the object. That is, the processor (140) can identify the location of the object as a touch point and perform an action corresponding to the touch interaction. For example, if there is an icon at the touch point, the processor (140) can control the projection unit (120) to project a web page corresponding to the icon.

[0133] In FIGS. 3 and 4, it is described that the projection method of an image is determined based on the posture information of the electronic device (100), but it is not limited thereto. For example, the processor (140) may identify the position of a user through the camera (130) and determine the projection method of an image based on the position of the user. For instance, as shown in FIG. 3, the processor (140) may acquire a captured image through the camera (130), and if a user is identified in the captured image, the projection unit (120) may control the projection unit (120) to invert the image and project it onto the projection area. Alternatively, as shown in FIG. 4, the processor (140) may acquire a captured image through the camera (130), and if a user is not identified in the captured image, the projection unit (120) may control the projection unit (120) to project the image onto the projection area without inversion.

[0134] FIGS. 5 and 6 are drawings for explaining a method of projecting by changing the position of a portion of an image based on posture information according to one embodiment of the present disclosure.

[0135] First, the processor (140) can identify an object from a captured image. For example, the processor (140) can identify an object by inputting the captured image into a neural network model. Alternatively, the processor (140) can identify an infrared distribution in the captured image and identify an object based on the infrared distribution.

[0136] The processor (140) can control the projection unit (120) to change the position of a part of the image and project it by taking into account not only the area where the object is identified but also the pose information.

[0137] For example, FIG. 5 assumes a case where the electronic device (100) projects an image from the upper side of the projection area and the user performs a touch interaction on the 510 area. The processor (140) identifies an infrared distribution in the captured image and can identify the 510 area corresponding to the user's touch interaction as an object based on the infrared distribution. In this case, since the electronic device (100) projects an image from the upper side of the projection area, an image may not be projected onto the 520 area depending on the user's touch interaction. However, the 520 area may be an area that the user is not very conscious of. For instance, when the user touches the 510 area, the 520 area may be an area that is covered by the user's arm, and since the user's eyes are above the user's arm, it may be natural for the 520 area to be covered by the user's arm. Accordingly, the processor (140) can control the projection unit (120) to change the position of icons, etc., included only in the 510 area and project them.

[0138] On the other hand, FIG. 6 assumes a case where the electronic device (100) projects an image from the lower side of the projection area and the user performs a touch interaction on the 610 area. The processor (140) identifies an infrared distribution in the captured image and can identify the 610 area corresponding to the user's touch interaction as an object based on the infrared distribution. In this case, since the electronic device (100) projects an image from the lower side of the projection area, an image may not be projected onto the 620 area depending on the user's touch interaction. However, the 620 area may be an area where the user feels a sense of unfamiliarity. For example, when the user touches the 610 area, the area below the 610 area is an area covered by the user's arm, but the 620 area is not an area covered by the user's arm, yet no information is provided, which may cause the user to feel a sense of unfamiliarity. Accordingly, the processor (140) can control the projection unit (120) to change the position of icons, etc. included in the 620 area as well as the 610 area and project them. Alternatively, the processor (140) can control the projection unit (120) to project the 630-1 area, which includes icons or menus, into the 630-2 area based on the 610 area and the 620 area.

[0139] The above description describes a case where the electronic device (100) includes an infrared emitting unit (150), but it is not limited thereto. For example, the electronic device (100) may not include an infrared emitting unit (150). In this case, the processor (140) may identify a projection area based on a captured image before the image and object are identified, and then identify an object by comparing the projection area included in the image and the captured image. For instance, the processor (140) may control the projection unit (120) to project frame A in the image and acquire a captured image A that includes a projection area where frame A is projected. The processor (140) may identify a projection area corresponding to frame A in the captured image A by comparing frame A and the captured image A. That is, the processor (140) may identify the relative position of the projection area in the captured image. Subsequently, the processor (140) may identify an object by comparing a frame after frame A with a captured image after the captured image A. For example, the processor (140) can identify a pixel group as an object if, by comparing a frame after frame A and a captured image after captured image A in pixel units, an adjacent pixel group having a pixel value greater than or equal to a preset difference is identified. However, the pixel group may not exactly contain only objects. That is, the processor (140) may identify it as one object in the 510 and 520 regions of FIG. 5. In this case, the processor (140) may identify that the electronic device (100) is in a state of projecting an image from the upper side of the projection area based on the posture information of the electronic device (100), and may control the projection unit (120) to project by changing the position of an icon, etc., with a certain weight or more by assigning a weight to the upper side among the areas identified as objects. Alternatively, the processor (140) may identify it as one object in the 610 and 620 regions of FIG. 6.In this case, the processor (140) identifies that the electronic device (100) is projecting an image from the lower side of the projection area based on the posture information of the electronic device (100), and can control the projection unit (120) to change the position of an icon, etc. included in the area identified as an object and project it.

[0140] FIGS. 7 and 8 are drawings for explaining an operation to identify an area to be projected by changing based on the shape of an object according to one embodiment of the present disclosure.

[0141] The processor (140) can identify the area to be projected by changing based on the shape of the object. For example, the processor (140) can identify the shape of the object based on the captured image. For instance, the processor (140) can identify the infrared distribution in the captured image and identify the object based on the infrared distribution. However, the object identified here is merely the area where the object touches in the touch area and does not represent the shape of the object in three-dimensional space. The processor (140) can identify the entire shape of the object by analyzing the captured image based on the identified object, that is, the area where the object touches in the touch area. For instance, as illustrated at the top of FIG. 7, the processor (140) can identify the infrared distribution in the captured image and identify the object based on the infrared distribution, and the object identified here may be the area where the end of the stick touches the projection area. The processor (140) can identify the area where the end of the stick touches the projection area in the captured image and identify the entire stick in the captured image based on the pixel value of the area. The processor (140) can identify that the bar is positioned in a direction from the bottom left to the top right.

[0142] The processor (140) can determine whether to change the position of a part of the image based on the degree to which an object covers an icon, etc. For example, the processor (140) does not change the position of a part of the image in the case of the top of FIG. 7 because the object is a thin bar, but in the case of the bottom of FIG. 7 because the object is a thick bar, the projection unit (120) can control the projection unit to change the 710-1 area of ​​the image to the 710-2 area and project it.

[0143] Additionally, the processor (140) can identify that the bar is positioned in a direction from the bottom right to the top left, as shown at the top of FIG. 8. In the case of the top of FIG. 8, the processor (140) does not change the position of a part of the image because the object is a thin bar, but in the case of the bottom of FIG. 8, the object is a thick bar, so the projection unit (120) can control the projection unit (120) to change the 810-1 area of ​​the image to the 810-2 area and project it.

[0144] If the electronic device (100) does not include an infrared emitting unit (150), the processor (140) can identify the shape of an object in the captured image by comparing a frame of the image with the captured image. In this case, the processor (140) may also identify an area to be projected by changing it based on the shape of the object identified in the captured image.

[0145] FIG. 9 is a drawing for explaining the object of position change according to one embodiment of the present disclosure.

[0146] The processor (140) can identify an area to be projected by changing based on at least one of the area where the object is identified, the shape of the object, or the posture information of the electronic device (100).

[0147] For example, the processor (140) can change and project the area of ​​920-1 containing the first icon into the area of ​​920-2 based on the area where the object is identified (indicated in the form of a cylinder), as shown in FIG. 9. Additionally, the processor (140) can change and project the area of ​​930-1 containing the second icon adjacent to the first icon into the area of ​​930-2. Additionally, the processor (140) can change and project the area of ​​910-1, which is the area where information is provided according to the touch of the first icon, into the area of ​​910-2.

[0148] FIGS. 10 and FIGS. 11 are drawings for explaining an infrared emission operation according to one embodiment of the present disclosure.

[0149] The processor (140) may turn on the infrared emitter (150) and the camera (130) before projecting an image to obtain information about the projection area. For example, when a user command to project an image is received, as shown in FIG. 10, the processor (140) controls the infrared emitter (150) to emit a plurality of infrared rays into the projection area, obtains a captured image of the projection area through the camera (130), identifies an infrared distribution in the captured image, and identifies other objects based on the infrared distribution. The processor (140) may provide a message to the user or project an image based on the other objects.

[0150] The processor (140) can control the infrared emitter (150) to emit a plurality of infrared rays into a projection area, as illustrated in FIG. 11. Here, the propagation direction (1110) of the plurality of infrared rays can be maintained at an angle less than a preset angle with respect to the projection area. For example, the plurality of infrared rays can be emitted while maintaining an angle of less than 10 with respect to the projection area. Through this operation, infrared rays can be emitted throughout the entire projection area even if the projection area is uneven.

[0151]

[0152] The processor (140) can acquire a captured image including a projection area through the camera (130) and identify an infrared distribution in the captured image. For example, the processor (140) can obtain an infrared distribution for the captured area by image processing the captured image. When a touch interaction such as 1120 exists, the processor (140) can acquire an infrared distribution such as the right side of FIG. 11 and identify that an object exists in an area where the infrared distribution is not uniform.

[0153] FIGS. 12 and FIGS. 13 are drawings for explaining the operation according to the identification of other objects according to one embodiment of the present disclosure.

[0154] When a user command to project an image is received, as shown on the left side of FIG. 12, the processor (140) controls the infrared emitter (150) to emit a plurality of infrared rays into the projection area, obtains a captured image of the projection area through the camera (130), identifies the infrared distribution in the captured image, and can identify other objects (1210) based on the infrared distribution.

[0155] The processor (140) provides a message guiding the movement of the other object (1210) if the other object (1210) is larger than a preset size, and can change at least one of the size or location of the projection area if the other object (1210) is smaller than a preset size. For example, if the other object (1210) is larger than a preset size, the processor (140) provides a message such as "Please remove the other object from the projection area," and if the other object (1210) is smaller than a preset size, the processor can reduce the size of the projection area from 1220 to 1230, as shown on the right side of FIG. 12, and change the projection area to an area where the other object (1210) is not present. Alternatively, if the other object (1210) is smaller than a preset size, the processor (140) may maintain the size of the projection area but change the location of the projection area to an area where the other object (1210) is not present.

[0156] Alternatively, when another object is identified, the processor (140) may update the area of ​​the other object based on the reflection of infrared light, provide a message based on the updated area of ​​the other object, or change at least one of the size or location of the projection area.

[0157] For example, as illustrated in FIG. 13, other objects may correspond to the 1310 area, but the 1320 area may be identified due to the reflection of infrared light. That is, the processor (140) may identify the 1320 area as an object due to the reflection of infrared light. However, the processor (140) may update the 1320 area to the 1310 area by removing the effect of the reflection of infrared light, provide a message based on the 1310 area, or change at least one of the size or location of the projection area.

[0158] FIG. 14 is a drawing for explaining the operation of a combination of an electronic device (100) and a touch detection device according to one embodiment of the present disclosure. For convenience of explanation, FIG. 14 assumes that the touch detection device is placed on the floor and the electronic device (100) is connected above it. FIG. 14 omits the illustration of the touch detection device to highlight that the electronic device (100) is tilted due to a problem with the combination with the touch detection device.

[0159] The processor (140) can provide a message guiding the reconnection of the electronic device (100) and the touch sensing device based on the position of the projection area in the captured image.

[0160] For example, the electronic device (100) may be connected to a touch detection device as shown in the upper drawing of FIG. 14, but if connected as shown in the middle or lower drawing of FIG. 14, the electronic device (100) may be tilted.

[0161] The projection area of ​​the projection unit (120) is the 1410 area, and the shooting range according to the field of view of the camera (130) may be the 1420, 1430, and 1440 areas. When the electronic device (100) is tilted, the change in the shooting range may be greater than the change in the projection area because the field of view of the camera (130) is larger. In FIG. 14, for convenience of explanation, the projection area is depicted as not changing. That is, when the electronic device (100) is tilted due to combination with the touch detection device, the position of the projection area in the captured image may change. When the position of the projection area in the captured image deviates from a preset position by more than a preset length, the processor (140) may provide a message guiding the reconnection of the electronic device (100) and the touch detection device.

[0162] FIG. 15 is a flowchart for explaining the overall operation of an electronic device (100) according to one embodiment of the present disclosure.

[0163] The processor (140) can identify whether it is connected to a touch detection device (touch acc) (S1510). For example, when the electronic device (100) is connected to a touch detection device, a closed circuit may be formed, and the processor (140) can identify whether it is connected to a touch detection device based on the current flow. However, it is not limited thereto, and the processor (140) may identify whether it is connected to a touch detection device (touch acc) in any number of different ways.

[0164] When a touch sensing device is combined, the processor (140) can acquire posture information of the electronic device (100) (S1520). The processor (140) can identify whether the projection area is flat when the posture of the electronic device (100) is horizontal or the projection direction is downward (S1530). If the projection area is flat, the processor (140) can start a projection operation, and if the projection area is partially non-planar, it can provide a message guiding reinstallation (S1540).

[0165] When the projection direction is upward, the processor (140) projects an inverted image (S1550) and can identify an object (S1560).

[0166] The processor (140) identifies the moving speed / direction / size of the object (S1570), and may not perform additional actions if the size is large but the speed is slow, or if the size is small. Alternatively, if the object enters the projection area, the size is greater than a preset size, and the speed is greater than a preset speed, the processor (140) may change the position of some areas and project (S1580). If the processor (140) identifies that the object has deviated (S1590), it may restore the parts where the position was changed.

[0167] FIG. 16 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0168] First, the image is projected onto the projection area (S1610). Then, a captured image of the projection area is obtained (S1620). Then, if an object obscuring the image projected onto the projection area is identified from the captured image, the position of a part of the image is changed based on the area where the object is identified and projected (S1630).

[0169] Additionally, the step of changing and projecting (S1630) may change the position of at least one of the multiple icons included in the image based on the area where the object is identified and project it.

[0170] And, the modification and projection step (S1630) can identify the area to be modified and projected based on the shape of the object.

[0171] Additionally, the step of emitting a plurality of infrared rays into a projection area and changing and projecting (S1630) can identify an infrared distribution in a captured image and identify an object based on the infrared distribution.

[0172] Additionally, before projecting the image, the method may further include the steps of acquiring a captured image of the area to be projected, identifying an object in the captured image, and providing a message guiding the movement of the object if the object is larger than a preset size, and changing at least one of the size or position of the projection area if the object is smaller than a preset size.

[0173] Additionally, the method further includes the step of acquiring attitude information of the electronic device, and the step of projecting to a projection area (S1610) may project an image to a projection area by inverting it based on the attitude information.

[0174] And, the step of acquiring posture information identifies whether the posture of the electronic device is a first posture or a second posture, and the step of projecting to a projection area (S1610) projects an image to a projection area if the posture of the electronic device is a first posture, and projects an inverted image to a projection area if the posture of the electronic device is a second posture.

[0175] Additionally, the step of changing and projecting (S1630) can change the position of a part of the image and project it based on the area and pose information where the object is identified.

[0176] And, in the step of changing and projecting (S1630), when the electronic device is connected to a touch sensing device including an infrared emitting part, the infrared distribution in the captured image is identified, and an object is identified based on the infrared distribution.

[0177] Additionally, it may further include a step of providing a message guiding the reconnection of the electronic device and the touch sensing device based on the position of the projection area in the captured image.

[0178] According to various embodiments of the present disclosure as described above, user convenience can be improved by changing the position of a portion of an image and projecting it even if a portion of the image is obscured due to touch interaction or the like during the projection process.

[0179] Meanwhile, according to the exemplary embodiments of the present disclosure, the various embodiments described above may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means only that the storage medium does not contain a signal and is tangible, and does not distinguish whether data is stored semi-permanently or temporarily on the storage medium.

[0180] Additionally, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0181] Additionally, according to one embodiment of the present disclosure, the various embodiments described above may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented as the processor itself. According to a software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software. Each of the software may perform one or more functions and operations described herein.

[0182] Meanwhile, computer instructions for performing processing operations of the device according to the various embodiments described above may be stored in a non-transitory computer-readable medium. When computer instructions stored in such a non-transitory computer-readable medium are executed by the processor of a specific device, they cause the specific device to perform processing operations in the device according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of a non-transitory computer-readable medium may include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, ROMs, etc.

[0183] Additionally, each component (e.g., module or program) according to the various embodiments described above may be composed of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in the various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the same or similar functions as those performed by each of the respective components prior to integration. The operations performed by the module, program, or other components according to the various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations added.

[0184] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, Memory for storing instructions; Projection section; Camera; and One or more processors including processing circuitry; and When the above instructions are executed individually or collectively by the one or more processors, Control the projection unit to project the image onto the projection area, and A captured image of the projection area captured through the camera is obtained, and An electronic device that controls the projection unit to change the position of a part of the image and project it based on the area where the object is identified, when an object that obscures the image projected onto the projection area is identified from the above-described image.

2. In Paragraph 1, When the above instructions are executed individually or collectively by the one or more processors, An electronic device that controls the projection unit to change the position of at least one of a plurality of icons included in the image and project it based on the area where the object is identified.

3. In Paragraph 1, When the above instructions are executed individually or collectively by the one or more processors, An electronic device that identifies the area to be changed and projected based on the shape of the object.

4. In Paragraph 1, It further includes an infrared emitting part; and When the above instructions are executed individually or collectively by the one or more processors, Control the infrared emitting unit to emit a plurality of infrared rays to the projection area, and Identify the infrared distribution in the above-mentioned captured image, and An electronic device that identifies the object based on the infrared distribution.

5. In Paragraph 1, When the above instructions are executed individually or collectively by the one or more processors, Before projecting the above image, another captured image of the area to be projected is obtained through the camera, and Identifying other objects in the above-mentioned other captured images, and If the above other object is larger than a preset size, a message guiding the movement of the above other object is provided, and An electronic device that changes at least one of the size or position of the projection area if the above other object is smaller than the above preset size.

6. In Paragraph 1, It further includes a sensor; When the above instructions are executed individually or collectively by the one or more processors, Attitude information of the electronic device is obtained through the sensor above, and An electronic device that controls the projection unit to invert the image based on the above posture information and project it onto the projection area.

7. In Paragraph 6, When the above instructions are executed individually or collectively by the one or more processors, Identify whether the posture of the electronic device is a first posture or a second posture through the sensor above, and An electronic device that controls the projection unit to project the image onto the projection area when the position of the electronic device is the first position, and controls the projection unit to project the image onto the projection area by inverting it when the position of the electronic device is the second position.

8. In Paragraph 6, When the above instructions are executed individually or collectively by the one or more processors, An electronic device that controls the projection unit to change the position of a portion of the image and project it based on the identified area of ​​the object and the pose information.

9. In Paragraph 1, When the above instructions are executed individually or collectively by the one or more processors, When the above electronic device is connected to a touch sensing device including an infrared emitting unit, it identifies an infrared distribution in the captured image, and An electronic device that identifies the object based on the infrared distribution.

10. In Paragraph 9, When the above instructions are executed individually or collectively by the one or more processors, An electronic device that provides a message guiding the reconnection of the electronic device and the touch sensing device based on the position of the projection area in the above-described captured image.

11. In a method for controlling an electronic device, A step of projecting an image onto a projection area; A step of acquiring a captured image of the above-mentioned projection area; and A control method comprising the step of, when an object obscuring the image projected onto the projection area is identified from the above-described image, changing the position of a part of the image based on the area where the object is identified and projecting it.

12. In Paragraph 11, The step of changing and projecting as described above is, A control method for changing the position of at least one icon among a plurality of icons included in the image and projecting it based on an area where the object is identified.

13. In Paragraph 11, The step of changing and projecting as described above is, A control method for identifying the area to be changed and projected based on the shape of the object.

14. In Paragraph 11, The method further includes the step of emitting a plurality of infrared rays to the projection area; The step of changing and projecting as described above is, Identify the infrared distribution in the above-mentioned captured image, and A control method for identifying the object based on the infrared distribution.

15. In Paragraph 11, A step of acquiring a different captured image of the area on which the image is to be projected, before projecting the above image; A step of identifying other objects in the above-mentioned other captured images; and A control method further comprising the step of providing a message guiding the movement of the other object if the other object is larger than or equal to a preset size, and changing at least one of the size or position of the projection area if the other object is smaller than the preset size.

Images