Electronic device and control method thereof

Abstract

An electronic device is disclosed. The present electronic device comprises: an output device that emits infrared rays from the front of the electronic device; a distance sensor that measures the distance to an object; a camera; a memory storing at least one instruction; and at least one processor that executes the at least one instruction, wherein the at least one processor controls the output device to emit infrared rays in the direction in which an image is projected, acquires information of an object approaching the projection space between a source and the projection area in which the image is projected, on the basis of distance information acquired by the distance sensor and the infrared distribution in a captured image acquired by the camera, and controls an image projection operation on the basis of the acquired object information.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device for controlling image projection operation and a method for controlling the same.

[0002] Thanks to advancements in electronic technology, various types of electronic devices are being used in daily life. Among these devices may be electronic devices that project images, including projectors.

[0003] For example, there may be a projector that projects an image onto a projection surface such as a wall.

[0004] A projector is a device that displays an image by projecting light containing an image onto an external screen or wall.

[0005] According to at least one embodiment of the present disclosure, an electronic device comprises an output device that emits infrared rays toward the front of the electronic device, a distance sensor for measuring the distance to an object, a camera, a memory for storing at least one instruction, and at least one processor for executing said at least one instruction. The processor controls the output device to emit infrared rays in a direction for projecting an image, acquires object information that approaches a projection space between a source source and a projection area where the image is projected based on distance information acquired from the distance sensor and infrared distribution within a captured image acquired through the camera, and controls an image projection operation based on said acquired object information.

[0006] Additionally, according to at least one embodiment of the present disclosure, a control method for an electronic device may include the steps of: emitting infrared rays in a direction for projecting an image; obtaining object information that approaches a projection space between a source source and a projection area where the image is projected, based on distance information of an object obtained through the electronic device and infrared distribution within a captured image obtained through the electronic device; and controlling an image projection operation based on the obtained object information.

[0007] Additionally, a non-transient readable recording medium according to at least one embodiment of the present disclosure stores a program for performing a method of controlling an electronic device, the program comprising the steps of: emitting infrared rays in a direction for projecting an image; obtaining object information that approaches a projection space between a source source and a projection area where the image is projected, based on distance information of an object obtained through the electronic device and infrared distribution within a captured image obtained through the electronic device; and controlling an image projection operation based on the obtained object information.

[0008] FIG. 1 is a diagram for schematically illustrating the operation of an electronic device according to at least one embodiment of the present disclosure.

[0009] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to at least one embodiment of the present disclosure.

[0010] FIG. 3 is a detailed block diagram for illustrating an electronic device according to at least one embodiment of the present disclosure.

[0011] FIG. 4 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure controlling an image projection operation.

[0012] FIG. 5 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure controlling an image projection operation.

[0013] FIG. 6 is a diagram illustrating an operation in which an electronic device according to at least one embodiment of the present disclosure controls an image projection operation based on posture information.

[0014] FIG. 7 is a diagram illustrating an operation in which an electronic device according to at least one embodiment of the present disclosure controls an image projection operation based on position information of an object.

[0015] FIG. 8 is a diagram illustrating an operation in which an electronic device according to at least one embodiment of the present disclosure controls an image projection operation based on position information of an object.

[0016] FIG. 9 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure controlling an image projection operation.

[0017] FIG. 10 is a flowchart illustrating the overall operation of an electronic device according to at least one embodiment of the present disclosure.

[0018] FIG. 11 is a flowchart illustrating the operation of an electronic device controlling an image projection operation according to at least one embodiment of the present disclosure.

[0019] The terms used in the various embodiments of this Disclosure have been selected to be as widely used and general as possible, taking into account their functions within this Disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant description section of this Disclosure. Therefore, terms used in this Disclosure should be defined not merely by their names, but based on their meanings and the overall content of this Disclosure.

[0020] The various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments.

[0021] In relation to the description of the drawings, similar reference numerals may be used for similar or related components.

[0022] The singular form of the noun corresponding to the item may include one or multiple items, unless the relevant context clearly indicates otherwise.

[0023] In the present disclosure, each of the phrases such as “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “at least one of A, B, or C” may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0024] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order).

[0025] Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that the component may be connected to the other component directly (e.g., via a wire), wirelessly, or through a third component.

[0026] Terms such as "include" or "have" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this disclosure, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0027] When it is said that a component is "connected," "combined," "supported," or "in contact" with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0028] When it is said that a component is located "on" another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0029] The term "and / or" includes a combination of multiple related described components or any of the multiple related described components.

[0030] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor, except for a "module" or "part" that needs to be implemented in specific hardware.

[0031] Meanwhile, various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present disclosure is not limited by the relative sizes or spacing depicted in the attached drawings.

[0032] In the present disclosure, the term "user" may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0033] An embodiment of the present disclosure will be described in more detail below with reference to the attached drawings.

[0034] FIG. 1 is a diagram for schematically illustrating the operation of an electronic device according to at least one embodiment of the present disclosure.

[0035] Referring to FIG. 1, the electronic device (100) may include a camera (130) and an output device (140).

[0036] The electronic device (100) can acquire a captured image of an object approaching the projection space through a camera (130). Here, "projection space" may refer to the space between the source source and the projection area where the image is projected.

[0037] According to one example of the present disclosure, the “image” may include various forms of visual information representing the movement of an object (or object) using a plurality of consecutive still images.

[0038] An image according to one example may include a captured image acquired using a sensor (e.g., a camera) equipped in an electronic device, an input image received from an external device through a communication unit, a graphic image generated by the electronic device, etc.

[0039] Images according to one example may include, depending on the aspect ratio, horizontal images where the width is longer than the height (e.g., landscape images, horizontal images), vertical images where the height is longer than the width (e.g., portrait images, vertical images), etc. For example, horizontal images may include images with a 16:9 aspect ratio, and vertical images may include images with a 9:16 aspect ratio. Specific numbers are examples for convenience of explanation and are not limited thereto.

[0040] An image according to one example may include various resolutions depending on the number of pixels constituting the image (the product of the number of pixels in the horizontal direction and the number of pixels in the vertical direction). For example, depending on the resolution, the image may include high-resolution images (FHD (1920X1080), 8K (7680X4320), etc.) and low-resolution images (640X480, etc.).

[0041] According to one example of the present disclosure, each of the plurality of still images included in the image may mean a frame (or, image frame).

[0042] According to one example, the video consists of multiple frames, and each of the multiple frames may correspond to one screen output by an electronic device. According to one example, the number of frames (or screens) output by an electronic device per second is referred to as the frame rate, and the frame rate can be expressed in frames per second (fps) or Hertz (Hz). According to one example, a video of 60 fps may mean a video containing 60 frames in 1000ms.

[0043] The electronic device (100) can emit infrared rays in a direction that projects an image through an output device (140). The electronic device (100) can acquire object information that approaches the projection space between the source and the projection area where the image is projected, based on the infrared distribution within the captured image.

[0044] Here, "object information" means information about an object accessing the projection space, and may include at least one of the object's shape, number of objects, location of objects, distance of objects, and area of ​​objects obscuring the projection space.

[0045] The electronic device (100) can control the image projection operation based on acquired object information. For example, if a person (10-1) is located at a safe distance from the projection space through the camera (130) of the electronic device (100) and an object approaching the projection space is identified as not being the person's face, the electronic device (100) can maintain the brightness of the image through the projection device.

[0046] On the other hand, if a person (10-2) is located at a dangerous distance from the projection space or an object approaching the projection space is identified as a person's face through the camera (130) of the electronic device (100), the electronic device (100) can reduce the brightness of the image through the projection device.

[0047] Here, "safe distance" may refer to a safe distance where there is no risk of damaging objects approaching the projection space. Here, "dangerous distance" may refer to a dangerous distance where the energy from a source (e.g., a light source) is concentrated and poses a risk of damaging objects approaching the projection space.

[0048] Here, brightness refers to what a person perceives when light strikes the rods and cones of the retina, and the brightness of electronic devices can be expressed as luminance. Luminance is the luminous intensity projected onto a given area and direction, and can be represented as the intensity of light emitted from a light source. Luminance can encompass the degree of brightness of an electronic device. The units used are cd / m² (candela) or nit, and 1 candela can represent the brightness emitted by one candle.

[0049] In FIG. 1, the electronic device (100) is shown in the form of a small projector, but is not limited thereto and may be implemented as a fixed, mounted, ceiling-mounted, or mobile projector. Here, a projector refers to a device that projects light containing an image onto an external screen or wall to display an image, and a mobile projector refers to a projector that can be moved using wheels and motors, etc.

[0050] In addition, the projector included in the electronic device (100) illustrated in FIG. 1 can be implemented as various types of projectors, such as a CRT (Cathode-Ray Tube) projector, an LCD (Liquid Crystal Display) projector, a DLP (Digital Light Processing) projector, an LED (Light Emitting Diode) projector, or an LCOS (Liquid Crystal on Silicon) projector, depending on the method of projecting images. However, it is not limited to these and can be implemented as various devices equipped with image projection functions.

[0051] Additionally, although the electronic device (100) illustrated in FIG. 1 is shown in the form of a projector, the electronic device (100) can be implemented as a device configured to be detachably attached to an external projection device that performs the function of projecting an image.

[0052] For example, the electronic device (100) may perform an image projection function in the form of being attached to an external projection device that performs the function of projecting an image.

[0053] The object depicted in FIG. 1 is shown in the form of a human, but is not limited thereto, and the object may include animals such as dogs and cats.

[0054] Hereinafter, the operation of an electronic device (100) according to various embodiments of the present disclosure will be described.

[0055] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to at least one embodiment of the present disclosure.

[0056] Referring to FIG. 2, the electronic device (100) includes a memory (110), a distance sensor (120), a camera (130), an output device (140), and at least one processor (150).

[0057] According to an embodiment, the memory (110) may store various programs, data, instructions, etc. used in the electronic device (100). In addition, the memory (110) may store various information according to various embodiments of the present disclosure.

[0058] Depending on the purpose of data storage, the memory (110) may be implemented in the form of a memory embedded in the electronic device (100) or in the form of a memory that can be attached to and detached from the electronic device (100).

[0059] For example, data for driving the electronic device (100) may be stored in memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in memory that is detachable from the electronic device (100).

[0060] In the case of memory embedded in an electronic device (100), it may be implemented in the form of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD).

[0061] In the case of a memory that can be attached to and detached from an electronic device (100), it can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.) or an external memory that can be connected to a USB port (e.g., USB memory).

[0062] The memory (110) may include various instructions required for the operation of the processor (150). Here, the instructions may include an instruction to emit infrared light in the direction of projecting an image, an instruction to acquire object information that accesses the projection space between the source source and the projection area where the image is projected, an instruction to control the image projection operation based on the acquired object information, an instruction to stop the operation of the distance sensor, an instruction to resume the operation of the distance sensor, an instruction to change the brightness of the image and project it, etc.

[0063] The memory (110) can store distance information obtained from the distance sensor (120) or captured images obtained through the camera (130).

[0064] Here, "distance information" may refer to distance information between an object accessing the projection space between a source and a projection area where an image is projected, and an electronic device (100).

[0065] Here, "captured image" may include an image of an object approaching the projection space and an image of the infrared distribution emitted from the output device (140).

[0066] The memory (110) can store object information, posture information or setting information of the electronic device (100).

[0067] Here, "object information" refers to various information about an object accessing the projection space, and may include at least one of the object's shape, number of objects, location of objects, distance of objects, and area of ​​objects obscuring the projection space.

[0068] Here, "attitude information" refers to information indicating the direction in which an electronic device is positioned in space, and may include at least one of roll information, pitch information, and yaw information. According to one example, roll information and pitch information may be obtained through an acceleration sensor (or gravity sensor) provided in the electronic device (100). Additionally, yaw information may be obtained based on the field of view information of a camera (130) used to photograph the projection surface in the electronic device (100).

[0069] Here, "setting information" refers to information regarding a setting that controls each configuration of an electronic device. For example, the setting information may include setting information for configuration control so that when the electronic device (100) identifies an object approaching the projection space, the distance sensor (120) and the camera (130) can be controlled simultaneously to obtain object information.

[0070] The setting information can be generated directly by the electronic device (100) based on user input, or it can receive and use setting information generated by another electronic device (e.g., user terminal device).

[0071] For example, at least one processor (150) can acquire setting information and, if it identifies an object approaching the projection space through the distance sensor (120), it can acquire object information by simultaneously controlling the distance sensor (120) and the camera (130) based on the setting information.

[0072] The distance sensor (120) is configured to measure the distance to an object. Based on the sensing value of the distance sensor (120), the distance between the object approaching the projection space and the electronic device (100) can be measured. For example, if the sensing value by the distance sensor (120) changes beyond a preset range (e.g., the distance change is 50 cm or more) or if it exceeds the set range of accessible distances through evaluation by the device itself, at least one processor (150) can perform an operation to darken the brightness of the image or temporarily turn off the light source.

[0073] The distance sensor (120) may include a 3D-ToF sensor, a ToF camera sensor, and a PIR sensor, but is not limited thereto, and may include various sensors capable of measuring the distance to an object.

[0074] When the distance sensor (120) acquires a sensing value for an object approaching the projection space, at least one processor (150) can identify the object approaching the projection space through the distance sensor (120).

[0075] The distance sensor (120) can stop or resume the operation of the distance sensor (120) based on the infrared emission operation or the emitted infrared distribution.

[0076] For example, when the electronic device (100) emits infrared rays in a direction that projects an image through the output device (140), the operation of the distance sensor (120) is stopped, and the operation of the stopped distance sensor (120) can be resumed based on the infrared distribution in the captured image obtained through the camera (130).

[0077] When the electronic device (100) emits infrared rays, the operation of the distance sensor (120) can be stopped to perform a touch function based on the infrared distribution. However, if the electronic device (100) cannot perform a touch function based on the infrared distribution, the operation of the distance sensor (120) can be resumed to simultaneously operate the camera (130), thereby more accurately identifying an object approaching the projection space and controlling the image projection operation.

[0078] Here, "infrared distribution" may refer to an infrared distribution in which a touch cannot be detected through a touch detection device including an output device (140) based on emitted infrared radiation. For example, if the distance between the projection surface on which an image is projected and the electronic device (100) is far, and the infrared radiation reaches the projection surface in a spread state, it may be difficult to accurately recognize the location of the touch, and thus a case may occur in which the touch cannot be detected.

[0079] The camera (130) is configured to photograph an object approaching the projection space or an infrared distribution. The camera (130) can photograph an infrared distribution emitted in the direction of projecting the image or an object approaching the projection space.

[0080] According to one example of the present disclosure, "shooting" may include the operation of an electronic device that controls a camera (e.g., a camera including an image sensor and a lens) equipped in an electronic device to convert an optical image formed through a lens into an electrical signal and acquire an image.

[0081] The camera (130) includes a lens, a shutter, an aperture, a solid-state image sensor, an AFE (Analog Front End), and a TG (Timing Generator). The shutter controls the time when light reflected from a subject enters the camera (130), and the aperture controls the amount of light incident on the lens by mechanically increasing or decreasing the size of the opening through which light enters. When light reflected from a subject accumulates as photocharge, the solid-state image sensor outputs an image based on the photocharge as an electrical signal. The TG outputs a timing signal for reading out pixel data from the solid-state image sensor, and the AFE samples and digitizes the electrical signal output from the solid-state image sensor.

[0082] The output device (140) is configured to emit infrared rays toward the front of the electronic device (100). The output device (140) may emit infrared rays toward a projection area of ​​the electronic device (100) to perform a touch function. The electronic device (100) may further include a touch sensing device comprising the output device (140) to perform a touch function.

[0083] For example, when an output device (140) emits infrared rays onto a projection surface on which an image is projected, and a user touches the projection surface on which the infrared rays are emitted, the electronic device (100) can perform a touch function by detecting the distribution of changing infrared rays through a camera (130).

[0084] In this case, if a captured image of the infrared distribution is obtained through the camera (130) and it is identified that a touch cannot be detected through the touch detection device, at least one processor (150) can resume the operation of the interrupted distance sensor (120) to obtain more accurate information about an object approaching the projection space.

[0085] According to an embodiment, at least one processor (150) controls the overall operation of the electronic device (100). Specifically, at least one processor (150) is connected to each component of the electronic device (100) and can control the overall operation of the electronic device (100).

[0086] At least one processor (150) can perform the operation of an electronic device (100) according to various embodiments by executing at least one instruction stored in memory.

[0087] According to an embodiment, at least one processor (150) may be implemented as a digital signal processor (DSP) that processes digital signals, a microprocessor, or a TCON (Timing controller). However, it is not limited thereto and may include or be defined by one or more of a central processing unit (CPU), a Micro Controller Unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, or an AI (Artificial Intelligence) processor. Additionally, at least one processor (150) may be implemented as a System on Chip (SoC) or Large Scale Integration (LSI) with a built-in processing algorithm, or may be implemented in the form of a Field Programmable Gate Array (FPGA). At least one processor (150) can perform various functions by executing computer executable instructions stored in memory.

[0088] At least one processor (150) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator.

[0089] At least one processor (150) can acquire setting information and, when it identifies an object approaching the projection space through the distance sensor (120), it can acquire object information by simultaneously controlling the distance sensor (120) and the camera (130) based on the setting information.

[0090] At least one processor (150) can control a camera (130) provided in the electronic device (100) to photograph the front of the electronic device (100) and obtain an image (e.g., a captured image) including one or more frames. Here, the image may include a live-view image.

[0091] At least one processor (150) can identify an object approaching the projection space based on a captured image taken through the camera (130).

[0092] For example, if at least one processor (150) identifies an object approaching the projection space through the distance sensor (120), it can obtain object information by simultaneously controlling the distance sensor (120) and the camera (130) based on setting information. Here, "setting information" refers to information about the settings that control each configuration of the electronic device.

[0093] Here, "object information" may include at least one of the shape of an object, the number of objects, the location of an object, the distance of an object, and the area of ​​an object that obscures the projection space. However, it is not limited thereto, and may also include the time during which an object accesses the projection space and remains in that accessed state.

[0094] For example, at least one processor (150) can control the projection operation differently by taking into account the shape of the object accessing the projection space and the time the object remains in the state of accessing the projection space.

[0095] For example, at least one processor (150) may change the brightness of the screen by setting a reference time to a longer time for the object to remain in the projection space when the shape of the object approaching the projection space is identified as a human arm than when the object is identified as a human face.

[0096] However, this is not limited thereto, and at least one processor (150) may change the brightness of the screen immediately without considering the time the object remains in the position of approaching the projection space in order to prevent accidents when the shape of the object approaching the projection space is identified as a human face.

[0097] At least one processor (150) can control one or any combination of other components of the electronic device and can perform operations or data processing related to communication. At least one processor (150) can execute one or more programs or instructions stored in memory. For example, at least one processor (150) can perform the method according to an embodiment of the present disclosure by executing one or more instructions stored in memory.

[0098] If the method according to the embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors.

[0099] For example, when the first operation, the second operation, and the third operation are performed by the method according to the embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence dedicated processor).

[0100] One or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory (110). Alternatively, if one or more processors are dedicated artificial intelligence processors, the dedicated artificial intelligence processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model. The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0101] Here, "created through learning" means that a basic artificial intelligence model is trained using multiple learning data by a learning algorithm, thereby creating a predefined rule of operation or an artificial intelligence model configured to perform a desired characteristic (or objective). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0102] An artificial intelligence model can be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated during the learning process so that the loss or cost values ​​obtained by the artificial intelligence model are reduced or minimized.

[0103] Artificial neural networks may include deep neural networks (DNNs), such as, but are not limited to, Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), Generative Adversarial Networks (GANs), or Deep Q-Networks.

[0104] At least one processor (150) may be implemented as a single core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore).

[0105] When at least one processor (150) is implemented as a multi-core processor, each of the multiple cores included in the multi-core processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by multiple cores may be included in the multi-core processor.

[0106] Each of the multiple cores (or some of the multiple cores) included in the multi-core processor may independently read and execute program instructions for implementing the method according to the embodiment of the present disclosure, or all (or some of) of the multiple cores may be linked together to read and execute program instructions for implementing the method according to the embodiment of the present disclosure.

[0107] FIG. 3 is a detailed block diagram for illustrating an electronic device according to at least one embodiment of the present disclosure.

[0108] Referring to FIG. 3, an electronic device (100) according to one embodiment of the present disclosure may include a memory (110), a distance sensor (120), a camera (130), an output device (140), at least one processor (150), a projection device (160), a second sensor (170), an input interface (180), and a communication unit (190). Parts that overlap with the description above will be omitted or abbreviated below.

[0109] The projection device (160) is configured to project light containing an image onto an external screen or wall to display an image. The projection device (160) can change the brightness of the image by adjusting the intensity of the light containing the image.

[0110] Specifically, the projection device (160) can project an image or video, including at least one of an image received from a source device and an image stored in advance, onto a projection area using a light source such as a lamp or an LED.

[0111] At least one processor (150) can control a projection device (160) to change the brightness of an image based on acquired object information and project it.

[0112] For example, if at least one processor (150) identifies the shape of an object approaching the projection space as at least one of a person and an animal, it can control the projection device (160) to change the brightness of the image and project it to prevent accidents.

[0113] The second sensor (170) is configured to acquire attitude information of the electronic device (100). Here, "attitude information" refers to information indicating the direction in which the electronic device is positioned in space, and may include at least one of roll information, pitch information, and yaw information.

[0114] The second sensor (170) may include at least one of an acceleration sensor, a gravity sensor, a gyroscope sensor, and a camera sensor. For example, the electronic device (100) may acquire roll information and pitch information through the acceleration sensor and acquire yaw information based on the field of view information of the camera (130).

[0115] An acceleration sensor is a sensor that measures the acceleration or the intensity of an impact of an electronic device (100), and is also called an accelerometer. An acceleration sensor detects dynamic forces such as acceleration, vibration, and impact, and can be implemented as an inertial type, a gyro type, a silicon semiconductor type, etc., depending on the detection method. That is, an acceleration sensor is a sensor that senses the degree of tilt of an electronic device (100) using gravitational acceleration, and can typically be composed of a 2-axis or 3-axis flux gate.

[0116] A gyro sensor is a sensor for detecting the rotation angle of an electronic device (100), and can measure changes in the orientation of an object by utilizing the property of always maintaining a constant direction initially set with high accuracy regardless of the rotation of the Earth. A gyro sensor is also called a gyroscope and can be implemented in a mechanical manner or an optical manner using light.

[0117] A gyroscope sensor can measure angular velocity. Angular velocity refers to the angle of rotation per unit of time, and the measurement principle of a gyroscope sensor is as follows. For example, in a horizontal state (stationary state), the angular velocity is 0 degrees / sec. If an object tilts by 50 degrees while moving for 10 seconds, the average angular velocity over those 10 seconds is 5 degrees / sec. If the tilt angle of 50 degrees is maintained while stationary, the angular velocity becomes 0 degrees / sec. Through this process, the angular velocity changes from 0 to 5 to 0, and the angle increases from 0 degrees to 50 degrees. To calculate the angle from angular velocity, integration must be performed over the entire time. Since a gyroscope sensor measures angular velocity in this manner, the tilt angle can be calculated by integrating this angular velocity over the entire time.

[0118] However, the gyro sensor may have an error due to the influence of temperature, and the error may accumulate during the integration process, causing the final value to drift. Accordingly, the electronic device (100) may further be equipped with a temperature sensor and can compensate for the error of the gyro sensor using the temperature sensor.

[0119] For convenience of explanation, the second sensor (170) has been described above as including at least one of an accelerometer, a gravity sensor, a gyroscope, and a camera sensor. However, it is not limited thereto, and the second sensor (170) may be any sensor capable of acquiring attitude information of the electronic device (100).

[0120] At least one processor (150) can control an image projection operation based on the attitude information and the acquired object information when it acquires attitude information of an electronic device through the second sensor (170).

[0121] At least one processor (150) can control the image projection operation by calculating the degree of damage to an object approaching the projection space based on the posture information of the electronic device (100).

[0122] For example, the projection surface of the electronic device (100) can be mounted so that it faces the floor or the wall. When the projection surface of the electronic device (100) is mounted so that it faces the wall rather than the floor, the action of changing the brightness of the screen can be performed more quickly as it identifies an object approaching the projection space.

[0123] However, the electronic device (100) may include not only the distance sensor (120) or the second sensor (170), but also various types of sensors such as a 3D-ToF sensor, a ToF camera sensor, a PIR sensor, a camera sensor, an IR camera sensor, a depth camera sensor, a LiDAR sensor, and an accelerometer.

[0124] The input interface (180) can receive various feedback from the user. For example, when the electronic device (100) receives user input regarding a setting that controls each configuration of the electronic device (100) through the input interface, it can operate each configuration in response to the user input.

[0125] For example, when an input interface (180) receives user input and obtains setting information, and at least one processor (150) identifies an object approaching the projection space through a distance sensor (120), the at least one processor (150) can obtain object information by simultaneously controlling the distance sensor (120) and the camera (130) based on the setting information.

[0126] The input interface (180) may include a microphone, a touchscreen, etc., but is not limited thereto, and may include various input interfaces capable of receiving user input.

[0127] Here, the microphone is a component for receiving sound input and converting it into an audio signal. The microphone is electrically connected to at least one processor (150) and can receive sound under the control of at least one processor (150).

[0128] For example, the microphone may be formed as an integrated unit on the upper side, front side, or other directions of the electronic device (100). Alternatively, the microphone may be provided in a remote control or the like, separate from the electronic device (100). In this case, the remote control may receive sound through the microphone and provide the received sound to the electronic device (100).

[0129] A microphone may include various components such as a microphone that collects analog sound, an amplifier circuit that amplifies the collected sound, an A / D converter circuit that samples the amplified sound and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0130] Meanwhile, the microphone may be implemented in the form of a sound sensor, and any configuration capable of collecting sound is acceptable.

[0131] The communication unit (190) can communicate with an external server or an external electronic device. In particular, the communication unit (190) can receive setting information for controlling each configuration of the electronic device (100) from an external server or an external electronic device. However, it is not limited thereto, and the setting information may be information directly generated by the electronic device (100) based on user input.

[0132] The communication unit (190) may include wired or wireless input / output interfaces (or input / output terminals) according to various standards. For example, one or more connection interfaces may include various interfaces such as HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), DVI (Digital Visual Interface), AP-based Wi-Fi (Wi-Fi, Wireless LAN Network), Bluetooth, Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, AES / EBU (Audio Engineering Society / European Broadcasting Union), Optical, Coaxial, etc.

[0133] The operation of the electronic device (100) will be described in more detail below through FIGS. 4 to 11. FIGS. 4 to 11 describes individual embodiments for the convenience of explanation. However, the individual embodiments of FIGS. 4 to 11 may be implemented in any combination.

[0134] FIG. 4 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure controlling an image projection operation.

[0135] At least one processor (150) can acquire object information approaching the projection space through a distance sensor (120) or a camera (130). For example, when an object approaches the projection space while projecting an image, at least one processor (150) can acquire object information such as the shape of the approaching object, the number of objects, the location of the objects, the distance of the objects, or the area where the objects block the projection space.

[0136] Referring to FIG. 4, at least one processor (150) can perform control operations on screen brightness, such as turning off the screen or dimming it, when it identifies that an object (411) approaches the projection space through a distance sensor (120) or a camera (130) and the distance of the object (412) has reached a preset distance (432).

[0137] Here, "dimming" may include adjusting the brightness of a lighting device. For example, when using LEDs as a backlight in a display, it may include a method of blinking the LEDs according to image characteristics through LED dimming. Alternatively, it may include local dimming, which divides the backlight into multiple areas and individually adjusts the brightness of each divided area.

[0138] At least one processor (150) can identify whether an object approaching the projection space is a person or an animal based on the number and shape of the object.

[0139] For example, at least one processor (150) can identify an object approaching the projection space as a person or animal if it is identified in the form of one circle and two cylinders as in FIG. 4.

[0140] At least one processor (150) can control the projection device (160) to change the brightness of the image and project it so as not to damage the object when the shape of the object approaching the projection space is identified as at least one of a person and an animal.

[0141] FIG. 5 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure controlling an image projection operation.

[0142] At least one processor (150) may turn on the output device (140) and the camera (130) before projecting an image to obtain object information accessing the projection space. For example, when at least one processor (150) receives a user command to project an image, it may control the output device (140) to emit infrared light into the projection space, obtain a captured image of the projection space through the camera (130), identify an infrared distribution in the captured image, identify an object based on the infrared distribution, and obtain object information.

[0143] At least one processor (150) can output notification information about the distance of an object based on information about an object approaching the projection space when the distance to the object approaching the projection space is identified as being less than a preset distance.

[0144] At least one processor (150) can control an output device (140) to emit infrared rays into a projection space as illustrated in FIG. 5. At least one processor (150) can acquire a captured image including the projection space through a camera (130) and identify an infrared distribution in the captured image.

[0145] For example, at least one processor (150) can obtain an infrared distribution for a shooting area by processing a captured image. If an object (510) approaches and a touch interaction occurs, at least one processor (150) can obtain an infrared distribution (530) such as the lower side of FIG. 5 and identify that an object (520) exists in an area where the infrared distribution is not uniform.

[0146] FIG. 6 is a diagram illustrating an operation in which an electronic device according to at least one embodiment of the present disclosure controls an image projection operation based on posture information.

[0147] At least one processor (150) can control an image projection operation based on the acquired posture information and object information when it acquires posture information of the electronic device (100) through the second sensor (170).

[0148] Here, "attitude information" refers to information indicating the direction in which an electronic device is positioned in space, and may include at least one of roll information, pitch information, and yaw information.

[0149] The second sensor (170) may include at least one of an acceleration sensor, a gravity sensor, a gyroscope sensor, and a camera sensor. For example, the electronic device (100) may acquire roll information and pitch information through the acceleration sensor and acquire yaw information based on the field of view information of the camera (130), thereby acquiring attitude information based on at least one of roll information, pitch information, and yaw information.

[0150] At least one processor (150) can perform control operations on the brightness of an image, such as turning off or dimming the image, by calculating the degree of damage to an object approaching the projection space based on the posture information of the electronic device (100).

[0151] Referring to FIG. 6, when the projection surface of the electronic device (100) is mounted so that it faces a wall as shown on the right, the operation of changing the brightness of the screen as it identifies an object approaching the projection space may be performed faster than when the projection surface is mounted so that it faces the floor as shown on the left.

[0152] FIG. 7 is a diagram illustrating an operation in which an electronic device according to at least one embodiment of the present disclosure controls an image projection operation based on position information of an object.

[0153] Referring to FIG. 7, at least one processor (150) can control the projection operation based on the position of an object approaching the projection space.

[0154] At least one processor (150) can perform control operations for projection operations faster as the distance between the object accessing the projection space and the electronic device (100) becomes closer.

[0155] For example, control operations for screen brightness, such as turning off or dimming the screen, can be performed more quickly when the object (720) approaches the projection space while maintaining a closer distance to the electronic device (100) as shown on the right, compared to when the object (710) approaches the projection space while maintaining a far distance from the electronic device (100) as shown on the left in Fig. 7.

[0156] Meanwhile, in FIG. 7, the electronic device (100) is shown projecting an image in the direction of a wall, but it is not limited to this and can be applied in cases where the electronic device (100) projects an image in various directions, such as the direction of the floor.

[0157] The operation of obtaining information about the shape and distance of an object in the case where the object (710) approaches the projection space while maintaining a long distance from the electronic device (100) as shown on the left in FIG. 7, and the case where the object (720) approaches the projection space while maintaining a closer distance from the electronic device (100) as shown on the right, is explained in detail in FIG. 8.

[0158] FIG. 8 is a diagram illustrating an operation in which an electronic device according to at least one embodiment of the present disclosure controls an image projection operation based on position information of an object.

[0159] Referring to FIG. 8, at least one processor (150) can obtain information about the shape, size, and distance of an object approaching the projection space through a camera (130) based on the position of the object approaching the projection space.

[0160] At least one processor (150) can increase the size value of the object acquired as the distance between the object accessing the projection space and the electronic device (100) becomes closer.

[0161] For example, at least one processor (150) may have a larger object size value when the object approaches the projection space while maintaining a distance from the electronic device (100) (811, 812, 813) and when the object approaches the projection space while maintaining a closer distance from the electronic device (100) as shown on the right (821, 822, 823), even if it is the same object.

[0162] FIG. 9 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure controlling an image projection operation.

[0163] The electronic device (100) may be a device capable of identifying touch interactions in a projection space based on infrared distribution, or a device combined with an accessory capable of identifying touch interactions in a projection space.

[0164] In this case, the user can touch an area of ​​the image projected into the projection space. At least one processor (150) can identify an object approaching the projection space from the captured image and control the projection device (160) to change the brightness of the image and project it based on the space where the object is identified.

[0165] Here, “interaction” includes interaction between a user and an electronic device. Interaction may include the act of a user inputting user input, such as touch, voice, or gestures, into an electronic device through a user interface to use the electronic device, or the user input itself. Additionally, interaction may include the response of the electronic device to the user input, and the response may consist of tactile, visual, auditory, and a combination of at least two of these.

[0166] The electronic device (100) further includes an output device (140), and at least one processor (150) controls the output device (140) to emit infrared rays into a projection space, identifies an infrared distribution in a shooting area, and can identify an object based on the infrared distribution.

[0167] However, it is not limited to this, and at least one processor (150) may input a captured image into a neural network model to identify an object.

[0168] Alternatively, the electronic device (100) may be connected to a touch sensing device including an output device. In this case, at least one processor (150) may identify an infrared distribution in the shooting space and identify an object based on the infrared distribution.

[0169] At least one processor (150) can identify whether the electronic device (100) is connected to a touch sensing device including an output device (140). If the electronic device (100) is identified as being connected to a touch sensing device including an output device (140), the at least one processor (150) identifies an infrared distribution in the captured image and identifies an object based on the infrared distribution, and if the electronic device (100) is identified as not being connected to a touch sensing device including an output device (140), the processor (150) may identify an object from the captured space itself.

[0170] However, as shown in FIG. 9, if the distance between the electronic device (100) and the projection surface exceeds a certain distance (910), the touch function may not be able to be performed based on the infrared distribution. For example, if the distance between the projection surface where the image is projected and the electronic device (100) is far, and the infrared rays reach the projection surface in a spread state, it may be difficult to accurately recognize the touched location, and thus the touch may not be detected.

[0171] In this case, by resuming the operation of the distance sensor (120) and simultaneously operating the camera (130), the object approaching the projection space can be identified more accurately and the image projection operation can be controlled.

[0172] This is also the case when the output device (140) emits infrared rays toward the wall, as shown on the right side of Fig. 9.

[0173] FIG. 10 is a flowchart illustrating the overall operation of an electronic device according to at least one embodiment of the present disclosure.

[0174] Referring to FIG. 10, at least one processor (150) can identify whether the camera (130) is coupled (S1010).

[0175] If at least one processor (150) identifies that touch detection is possible when the camera (130) is combined (S1020), the camera (130) can be controlled (S1030) to obtain object information based on infrared distribution (S1050).

[0176] At least one processor (150) can control the distance sensor (120) (S1080) to obtain object information (S1050) when an object approaches (S1070) when the camera (130) is not connected.

[0177] If at least one processor (150) identifies that touch detection is impossible when the camera (130) is combined (S1020), it can control the distance sensor (120) and the camera (130) simultaneously (S1040) to obtain object information (S1050).

[0178] However, it is not limited to cases where the distance sensor (120) and the camera (130) are controlled simultaneously, and the distance sensor (120) may be controlled first or the camera (130) may be controlled first.

[0179] At least one processor (150) can control the image projection operation based on acquired object information (S1060). For example, if the shape of an object approaching the projection space is identified as at least one of a person and an animal, the at least one processor (150) can turn off the image or reduce the brightness of the image and project it.

[0180] FIG. 11 is a flowchart illustrating the operation of an electronic device controlling an image projection operation according to at least one embodiment of the present disclosure.

[0181] The electronic device emits infrared rays in the direction of image projection (S1110).

[0182] The electronic device acquires object information approaching the projection space based on the object's distance information and the infrared distribution within the captured image (S1120). For example, the electronic device can identify that an object exists in an area where the infrared distribution is non-uniform among the emitted infrared regions.

[0183] The electronic device controls the image projection operation based on acquired object information (S1130). For example, the electronic device can control the projection device to change the brightness of the image and project it based on the acquired object information. Additionally, based on the acquired object information, if the distance to an object approaching the projection space is identified as being less than a preset distance, the electronic device may output notification information regarding the distance of the object.

[0184] The image projection method described in Fig. 11 can be performed by a device having various configurations such as Fig. 2 and Fig. 3 described above, but is not necessarily limited thereto and can be performed by a device having various configurations.

[0185] The various embodiments described above may be implemented as individual embodiments, or at least one embodiment may be combined with one another, either wholly or partially, to be implemented together in a single device.

[0186] According to the various embodiments described above, the electronic device can prevent damage to objects approaching the projection space or protect the user's field of vision by controlling the image projection operation based on acquired object information. Ultimately, the user's experience can be enhanced.

[0187] Various embodiments of the present disclosure may be implemented as software stored on a machine-readable storage media that can be mounted on or connected to a smartphone, a user terminal device, and various other electronic devices (e.g., a computer).

[0188] Specifically, a non-transient readable storage medium may be provided that stores software for sequentially performing the steps of: emitting infrared rays in the direction of projecting an image; acquiring object information that approaches the projection space between a source and a projection area where the image is projected, based on object distance information acquired through an electronic device and infrared distribution within a captured image acquired through an electronic device; and controlling an image projection operation based on the acquired object information.

[0189] A device equipped with such a non-transient readable medium can perform various operations, such as image projection, image brightness change, and distance sensor operation, as described in the various embodiments described above.

[0190] In non-transitory readable storage media, 'non-transitory' simply means that the storage medium does not contain a signal and is tangible; it does not distinguish whether data is stored semi-permanently or temporarily on the storage medium.

[0191] Alternatively, a program for performing the methods according to the various embodiments described above may be distributed online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0192] Each component (e.g., module or program) according to various embodiments may consist of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the same or similar functions as those performed by each of the respective components prior to integration.

[0193] Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0194] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, An output device that emits infrared rays toward the front of the electronic device; Distance sensor for measuring distance to an object; camera; Memory for storing at least one instruction; and It includes at least one processor that executes the above at least one instruction; and The above-mentioned at least one processor is, Control the output device to emit infrared rays in the direction of projecting the image, and Based on distance information acquired from the distance sensor and infrared distribution within the captured image acquired through the camera, object information approaching the projection space between the source and the projection area where the image is projected is acquired, and An electronic device that controls image projection operation based on the object information obtained above.

2. In Paragraph 1, It further includes a projection device; and The above-mentioned at least one processor is, An electronic device that controls the projection device to change the brightness of the image and project it based on the object information obtained above.

3. In Paragraph 2, The above-mentioned at least one processor is, An electronic device that controls the projection device to change the brightness of the image and project it when the shape of the object approaching the projection space is identified as at least one of a person and an animal.

4. In Paragraph 1, The above-mentioned at least one processor is, An electronic device that outputs notification information regarding the distance of an object when, based on the object information obtained above, the distance to the object approaching the projection space is identified as being less than a preset distance.

5. In Paragraph 1, The above electronic device is an electronic device configured to be detachably attached to an external projection device that performs the function of projecting an image.

6. In Paragraph 1, The above object information is, An electronic device comprising at least one of the shape of the object, the number of the object, the location of the object, the distance of the object, and the area of ​​the object that obscures the projection space.

7. In Paragraph 1, It further includes a second sensor for acquiring attitude information of the above electronic device, and The above-mentioned at least one processor is, Attitude information of the electronic device is obtained through the second sensor, and An electronic device that controls image projection operation based on the above posture information and the above acquired object information.

8. In Paragraph 1, The above-mentioned at least one processor is, An electronic device that acquires setting information and, upon identifying an object approaching the projection space through the distance sensor, simultaneously controls the distance sensor and the camera based on the setting information to acquire the object information.

9. In Paragraph 1, The above-mentioned at least one processor is, When infrared rays are emitted in the direction of projecting the above image, the operation of the distance sensor is stopped, and An electronic device that resumes the operation of the interrupted distance sensor based on the infrared distribution in the captured image acquired through the camera.

10. In Paragraph 9, A touch sensing device including an output device; further comprising, The above-mentioned at least one processor is, An electronic device that resumes the operation of the interrupted distance sensor when it is identified, based on the infrared distribution, that a situation in which a touch cannot be detected through the touch detection device is identified.

11. In a method for controlling an electronic device, A step of emitting infrared rays in the direction of projecting an image; A step of obtaining object information approaching the projection space between a source and a projection area where the image is projected, based on distance information of an object obtained through the electronic device and infrared distribution within a captured image obtained through the electronic device; and A control method comprising the step of controlling an image projection operation based on the object information obtained above.

12. In Paragraph 11, The step of controlling the above-mentioned image projection operation is, A control method comprising the step of controlling a projection device of the electronic device to change the brightness of the image and project it based on the object information obtained above.

13. In Paragraph 12, The step of controlling the above-mentioned image projection operation is, A control method comprising the step of controlling a projection device of the electronic device to change the brightness of the image and project it when the shape of the object approaching the projection space is identified as at least one of a person and an animal.

14. In Paragraph 11, A control method further comprising the step of outputting notification information regarding the distance of an object if, based on the object information obtained above, the distance to the object approaching the projection space is identified as being less than a preset distance.

15. A non-transient readable recording medium comprising a program for executing a method of controlling an electronic device, The control method of the above electronic device is, A step of emitting infrared rays in the direction of projecting an image; A step of obtaining object information approaching the projection space between a source and a projection area where the image is projected, based on distance information of an object obtained through the electronic device and infrared distribution within a captured image obtained through the electronic device; and A non-transient readable recording medium comprising the step of controlling an image projection operation based on the object information obtained above.

Images