Electronic device and control method therefor

Abstract

This electronic device comprises a projection unit for projecting an image onto a projection surface, a sensor for acquiring data corresponding to the projection surface, and a processor, wherein the acquired data includes infrared (IR) data acquired from IR reflected from the projection surface and data not acquired from IR reflection, and the processor, by executing instructions stored in a memory: controls the projection unit to project a test image onto the projection surface; acquires a first area on the projection surface on the basis of the data that is not acquired by the IR reflection and is included in the data that is acquired via the sensor while the test image is projected onto the projection surface and that corresponds to the projection surface onto which the test image is projected; on the basis of the IR data included in the data that corresponds to the projection surface and is acquired via the sensor, acquires a second area corresponding to an area of the projection surface onto which the image is projected by the projection unit; and acquires a projection area on the projection surface on the basis of the first area and the second area.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more specifically, to an electronic device and a method for controlling the same that acquire a projection area for projecting an image based on sensing data acquired by sensing a projection surface.

[0002] With the recent advancements in electronic and optical technologies, various types of projectors are being utilized. A projector refers to an electronic device that projects light onto a projection surface to form an image.

[0003] In particular, projectors that provide touch input functions by detecting movements or touches, such as those of a user's fingers, on the projection surface where the image is projected are being developed recently. In this case, the projector can detect touches of fingers or objects through an infrared sensor.

[0004] An electronic device according to one embodiment of the present disclosure comprises a memory storing at least one instruction, a projection unit for projecting an image onto a projection surface, at least one sensor for acquiring data corresponding to the projection surface, and at least one processor, wherein the acquired data includes infrared data acquired from infrared (IR) reflected from the projection surface and data not acquired from infrared reflection, and the at least one processor controls the projection unit to project a test image onto the projection surface by executing at least one instruction stored in the memory, and while the test image is projected onto the projection surface, acquires a first area on the projection surface based on data not acquired by infrared reflection, which is included in data corresponding to the projection surface on which the test image is projected, acquired through the at least one sensor, and acquires a second area corresponding to the area of ​​the projection surface on which the image is projected by the projection unit based on infrared data included in data corresponding to the projection surface acquired through the at least one sensor, and acquires a projection area on the projection surface based on the first area and the second area.

[0005] The first area above may be smaller in size than the area on the projection surface, which is entirely covered by the projection of the test image based on data corresponding to the projection surface and on which the test image is projected.

[0006] The above at least one processor can, by executing the above at least one instruction stored in the memory, acquire a cross image based on a plurality of images acquired from data corresponding to the projection surface, reduce the test image to a reduced test image based on at least one cross pixel of the acquired cross image, control the projection unit to project the reduced test image onto an area on the projection surface based on at least one cross pixel of the acquired cross image, and acquire the area on the projection surface where the reduced test image is projected as the first area.

[0007] The above at least one processor, by executing the above at least one instruction stored in the memory, acquires a first image including pixel data from data corresponding to the projection surface acquired by the above at least one sensor, acquires the cross image based on the first image and the second image acquired based on the pixel data included in the first image, and if the number of the at least one cross pixel having a pixel value greater than or equal to a first threshold value in the cross image is greater than or equal to a first threshold number, the test image is reduced into the reduced test image based on a preset first unit, and the projection unit is controlled to project the reduced test image onto an area on the projection surface based on the at least one cross pixel, updates the number corresponding to the at least one cross pixel based on the reduced test image, and if the number of the updated at least one cross pixel is less than the first threshold number, the area on the projection surface onto which the reduced test image is projected can be acquired as the first area.

[0008] The above at least one processor can identify at least one valid pixel having a pixel value greater than or equal to a second threshold value in the acquired infrared image from the infrared data corresponding to the area where the test image is projected by executing the above at least one instruction stored in the memory, and acquire the above second area on the projection surface based on the identified at least one valid pixel.

[0009] The above at least one processor, by executing the above at least one instruction stored in the memory, controls the projection unit to reduce the test image into a reduced test image based on a preset second unit if the at least one valid pixel is greater than or equal to a second threshold number, controls the projection unit to project the reduced test image onto an area of ​​the projection surface based on the at least one valid pixel, updates the number corresponding to the at least one valid pixel based on the reduced test image, and if the updated number corresponding to the at least one valid pixel is less than the second threshold number, the area on the projection surface onto which the reduced test image is projected can be acquired as the second area.

[0010] The above image is a primary test image, and the at least one processor controls the projection unit to project an initial test image including an infrared pattern before projecting the primary test image by executing the at least one instruction stored in the memory, and controls the projection unit to project the primary test image including an RGB pattern if a function related to correction based on the infrared pattern is not performed, and can acquire the second region based on the infrared data included in the data corresponding to the projection surface obtained through the projection surface on which the initial test image is projected.

[0011] The above at least one processor can acquire the small-sized area as the projection area by executing the above at least one instruction stored in the memory, if the small-sized area included in the first area and included in the second area is greater than or equal to a predetermined threshold size.

[0012] The electronic device further includes a moving part, and the at least one processor can control the moving part so that the electronic device moves from a current position corresponding to the projection of the test image to a second position corresponding to the image projection in the projection area by executing the at least one instruction stored in the memory.

[0013] The above at least one processor can identify at least one obstacle based on data corresponding to the projection surface by using an artificial intelligence model by executing the above at least one instruction stored in the memory, obtain a third area on the projection surface that is included in the area excluding the area corresponding to the obstacle, and obtain the projection area based on the first area, the second area, and the third area.

[0014] A method for controlling an electronic device according to one embodiment of the present disclosure comprises: a projection unit for projecting an image onto a projection surface; and at least one sensor for acquiring data corresponding to the projection surface, the data including infrared data acquired from infrared reflected from the projection surface and data not acquired from infrared reflection. The method comprises the steps of: controlling the projection unit to project a test image onto the projection surface; acquiring a first area on the projection surface based on data not acquired by infrared reflection, which is included in data corresponding to the projection surface on which the test image is projected, which is acquired through the at least one sensor while the test image is projected onto the projection surface; acquiring a second area corresponding to the area of ​​the projection surface on which the image is projected by the projection unit based on infrared data included in data corresponding to the projection surface acquired through the at least one sensor; and acquiring a projection area on the projection surface based on the first area and the second area.

[0015] The step of acquiring the first region based on data not acquired by infrared reflection, which is included in data corresponding to the projection surface on which the test image is projected, acquired through the at least one sensor while the test image is projected onto the projection surface, may include the step of acquiring the first region, which is smaller in size than the area on the projection surface on which the test image is projected and which is entirely covered by the projection of the test image.

[0016] The step of obtaining a first area, which is smaller in size than the area on the projection surface and is entirely covered by the projection of the test image, may include: obtaining a cross image based on a plurality of images obtained from data corresponding to the projection surface; reducing the test image into a reduced test image based on at least one cross pixel of the obtained cross image; controlling the projection unit to project the reduced test image onto the area on the projection surface based on the at least one cross pixel of the obtained cross image; and obtaining the area on the projection surface where the reduced test image is projected as the first area.

[0017] The step of acquiring an intersection image based on a plurality of images acquired based on data corresponding to the projection surface comprises acquiring a first image including pixel data from the data corresponding to the projection surface acquired by the at least one sensor, acquiring the intersection image based on the first image and a second image acquired based on the pixel data included in the first image, and the step of reducing the test image into a reduced test image based on at least one intersection pixel of the acquired intersection image comprises, if the number corresponding to the at least one intersection pixel having a pixel value greater than or equal to a first threshold value in the intersection image is greater than or equal to a first threshold number, reducing the test image into the reduced test image based on a preset first unit, and the step of acquiring the area on the projection surface where the reduced test image is projected as the first area comprises, if the number corresponding to the at least one intersection pixel is updated based on the reduced test image, and if the number corresponding to the updated at least one intersection pixel is less than the first threshold number, the area on the projection surface where the reduced test image is projected can be acquired as the first area.

[0018] The step of acquiring the second region may include identifying at least one valid pixel whose pixel value is greater than or equal to a second threshold value in an infrared image acquired from infrared data corresponding to the area where the test image is projected, and acquiring the second region based on the identified at least one valid pixel among the projection surfaces.

[0019] The step of acquiring the second region based on at least one identified valid pixel among the projection surfaces comprises, if the number of at least one valid pixel is greater than or equal to a second threshold number, reducing and projecting the test image to a preset second unit, updating the number of at least one valid pixel based on the reduced test image, and if the updated number of at least one valid pixel is less than the second threshold number, acquiring the area where the reduced test image is projected as the second region.

[0020] The step of projecting the test image involves projecting an initial test image including an infrared pattern, and if a function related to correction based on the infrared pattern is not performed, projecting the test image including an RGB pattern, and the step of acquiring the second region may acquire the second region based on the infrared data among the data corresponding to the projection surface acquired through the projection surface on which the initial test image is projected.

[0021] The step of obtaining a final area based on the first area and the second area can be performed by obtaining the smaller area among the first area and the second area as the projection area if the smaller area is greater than or equal to a predetermined threshold size.

[0022] The above control method may further include the step of controlling a moving part of the electronic device to move from a current position corresponding to the projection of the test image to a second position corresponding to the image projection in the projection area.

[0023] According to one embodiment of the present disclosure, a non-transient computer-readable recording medium storing computer instructions that cause an electronic device to perform an operation when executed by a processor of an electronic device comprising: a projection unit for projecting an image onto a projection surface; and at least one sensor for acquiring data corresponding to the projection surface, wherein the data includes infrared data acquired from infrared reflected from the projection surface and data not acquired from infrared reflection. The operation comprises: a step of controlling the projection unit to project a test image onto the projection surface; a step of acquiring a first area on the projection surface based on data not acquired by infrared reflection, which is included in data corresponding to the projection surface onto which the test image is projected, which is acquired through the at least one sensor while the test image is projected onto the projection surface; a step of acquiring a second area corresponding to the image projection among the areas of the projection surface on which the image is projected by the projection unit, based on infrared data included in data corresponding to the projection surface acquired through the at least one sensor; and a step of acquiring a projection area on the projection surface based on the first area and the second area.

[0024]

[0025] FIG. 1 is a diagram for schematically illustrating the operation of an electronic device according to one or more embodiments of the present disclosure.

[0026] FIG. 2 is a block diagram for explaining the configuration of an electronic device according to one or more embodiments of the present disclosure.

[0027] FIG. 3 is a detailed block diagram for explaining the detailed configuration of an electronic device according to one or more embodiments of the present disclosure.

[0028] FIG. 4 is a drawing for explaining the operation of an electronic device according to one or more embodiments of the present disclosure.

[0029] FIG. 5 is a drawing for explaining the movement of an electronic device according to one or more embodiments of the present disclosure.

[0030] FIG. 6 is a drawing for illustrating a maximum reduction scale according to one or more embodiments of the present disclosure.

[0031] FIG. 7 is a drawing for illustrating an obstacle detector according to one or more embodiments of the present disclosure.

[0032] FIG. 8 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments of the present disclosure.

[0033] FIG. 9 is a flowchart illustrating the operation of an electronic device using a surface size detector and an infrared emission detector according to one or more embodiments of the present disclosure.

[0034] FIG. 10 is a drawing for explaining the operation of an electronic device using an obstacle detector according to one or more embodiments of the present disclosure.

[0035] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0036] In describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description is omitted.

[0037] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0038] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0039] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components such as parts) and do not exclude the presence of additional features.

[0040] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0041] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0042] When it is stated that a certain component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the said certain component may be directly connected to the said other component or connected through another component (e.g., a third component).

[0043] On the other hand, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between said certain component and said other component.

[0044] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0045] Instead, in some situations, the expression “device configured to do something” may mean that the device is “capable of doing something” in conjunction with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” may refer to a dedicated processor for performing those operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing those operations by executing one or more software programs stored in a memory device.

[0046] In the embodiments, a 'module' or 'part' performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'parts' may be integrated into at least one module and implemented by at least one processor, except for the 'module' or 'part' that needs to be implemented in specific hardware.

[0047] Meanwhile, various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present disclosure is not limited by the relative sizes or spacing depicted in the attached drawings.

[0048] Hereinafter, embodiments according to the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them.

[0049] FIG. 1 is a drawing for explaining the operation of an electronic device and an external electronic device according to one or more embodiments of the present disclosure.

[0050] According to FIG. 1, an electronic device (100) is disclosed.

[0051] The electronic device (100) is a smartphone, tablet PC (Personal computer), desktop PC, laptop PC, PC, set-top box, OTT service (Over-the-top media service) server, console (video game console), Blu-ray player, DVD (Digital Video Disc or Digital Versatile Disc) player, home automation control panel, security control panel, media box (e.g., Samsung HomeSync TM , Apple TV TM , or Google TV TM ), game console (e.g., Xbox) TM PlayStation TM It can be implemented as at least one of the following. However, it is not limited to this.

[0052] For example, the electronic device (100) may correspond to a projector capable of projecting an image. Here, the projector may refer to a device that projects an image onto a wall or screen. The electronic device (100) may move around the surrounding space and project an image at an appropriate location.

[0053] Here, the surrounding space may refer to a space that includes a passageway through which the electronic device (100) can move and a wall or other object for projecting an image. For example, the surrounding space may be implemented as an interior space of a house, an office, a warehouse, etc.

[0054] Meanwhile, the electronic device (100) can sense the surrounding space. For example, the electronic device (100) can sense the surrounding space to determine a suitable projection surface for projecting an image among the surrounding space. Here, the electronic device (100) can use the sensing data obtained by sensing the surrounding space. The sensing data will be described in detail in the following section.

[0055] Here, the fact that the electronic device (100) senses the surrounding space may mean acquiring an image of the surrounding space. For example, the electronic device (100) may acquire an image by capturing the surrounding space based on user operation input for photographing the surrounding space.

[0056] The surrounding space captured here may correspond to a projection surface (e.g., wall, floor, ceiling, etc.) on which an image is projected. The projected image here may correspond to an image projected by an electronic device (100). However, it is not limited to this and may correspond to an image projected by an external device.

[0057] Meanwhile, the projected image may serve as a test image for correcting the projection area. Here, correction may refer to keystone correction. Keystone correction refers to a function that forcibly shifts the corners of the projected screen. In other words, keystone correction refers to a function that adjusts the projected screen to be closer to its original rectangular shape.

[0058] For example, the electronic device (100) can perform keystone correction through a test image. Here, a projection area on which the test image is projected is captured using a camera provided in the electronic device (100), and keystone correction can be performed based on the captured image.

[0059] However, it is not limited to this, and the electronic device (100) may use a test image to perform other corrections.

[0060] For example, the electronic device (100) may use a test image to identify the user's touch location. The electronic device (100) may project the test image and sense distortions or boundaries of the projected pattern. The electronic device (100) may analyze the results obtained by sensing and adjust the size and position of the screen.

[0061] After the projection area is adjusted, the electronic device (100) can identify the touch location from the location where the touch event occurred based on the adjusted projection area. In this case, the electronic device (100) can identify the touch location using infrared light (13).

[0062] Here, infrared light (13) may correspond to light emitted by an electronic device (100). After being emitted by the electronic device (100), infrared light (13) may be reflected by the user's finger (20). The electronic device (100) may receive the reflected light through a camera.

[0063] Here, the camera can receive light through an image sensor (e.g., an infrared sensor). The camera may be equipped in an electronic device (100), but is not necessarily limited thereto and may be equipped in an external device. When the camera is equipped in an electronic device (100), the camera can sense space according to the detection range (11). That is, the camera can sense light reflected by space within the detection range (11).

[0064] Here, the detection range (11) can be determined based on the projection range (12). Here, the projection range (12) can be determined by the projection area (or adjusted projection area). For example, the detection range (11) can be determined as a range for sensing the projection area. Here, the projection area may refer to an area for the electronic device (100) to project an image and receive a touch input.

[0065] The electronic device (100) can identify a touch location by mapping the position of the detected reflected light and the adjustment data of the projection area. Here, the adjustment data may refer to data obtained by analyzing the boundaries, distortions, etc. of a projected test image. For example, the adjustment data may include screen size and position information generated after the projection area is adjusted.

[0066] Meanwhile, the electronic device (100) can detect infrared rays reflected by the obstacle (30) when an obstacle (30) is present on the projection surface. In this case, the electronic device (100) can recognize that a touch has been input by mapping the position of the reflected infrared rays with the adjustment data of the projection area.

[0067] In this case, the electronic device (100) may recognize that a touch has been input at the location of the obstacle (30). The electronic device (100) may incorrectly recognize that a touch has been input at the location of the obstacle (30) even if the user's finger (20) has actually input a touch at a different location, or if there is no touch by the user's finger (20).

[0068] Accordingly, the electronic device (100) can adjust (or readjust) the projection area to obtain a projection area excluding the area where the obstacle (30) is located. If the electronic device (100) identifies the space located to the left of the obstacle (30) as the projection area, the electronic device (100) may not be affected by the obstacle (30).

[0069] For example, when the electronic device (100) receives a touch from the user's finger (20), the electronic device (100) can recognize that there is a touch input only at the location of the finger (20).

[0070] The operation of the electronic device (100) adjusting the projection area based on the location of obstacles (30), etc., as described above will be explained in detail in the following section.

[0071] FIG. 2 is a block diagram for explaining the configuration of an electronic device according to one or more embodiments of the present disclosure.

[0072] According to FIG. 2, the electronic device (100) may include a memory (110), a sensor (120), a projection unit (130), and a processor (140).

[0073] As the electronic device (100) has been described previously, a redundant description will be omitted.

[0074] In the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD), and in the case of memory that is detachable from the electronic device (100), it may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (e.g., USB memory).

[0075] Memory (110) may include one or more storage media (or one or more storage devices). For example, memory (110) may include a memory assembly comprising one or more storage media. For example, one or more storage media may include a hard drive, flash memory, permanent memory (e.g., non-volatile memory) such as ROM (read-only memory), semi-permanent memory (e.g., volatile memory) such as RAM (random access memory), any other suitable type of storage (or storage assembly), or any combination thereof. Memory (110) may include cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (100). As an example, but not limited to, cache memory may be included within the processor (140). The memory (110) can be fixedly embedded in the electronic device (100) or incorporated into one or more suitable types of components (e.g., a SIM (subscriber identity module) card and / or an SD (secure digital) card) that can be repeatedly inserted into and removed from the electronic device (100).

[0076] For example, memory (110) may store one or more software applications, such as operating system (or system) software applications, firmware software applications, driver software applications, plugin (e.g., add-in, add-on, and / or applet) software applications, and / or any other suitable software applications. For example, one or more software applications may include instructions executable by the processor (140). For example, memory (110) may store instructions that can be called by an application programming interface (API). For example, memory (110) may store instructions within a library.

[0077] In the present disclosure, the term memory (110) may be used to include a storage unit, a ROM (not shown), a RAM (not shown) within at least one processor (140), or a memory card (not shown) (e.g., a micro SD card, a memory stick) mounted in an electronic device (100).

[0078] A memory (110) according to one or more embodiments may store at least one instruction. When the at least one instruction stored in the memory (110) is executed individually or collectively by the processor (140), the electronic device (100) may perform operations.

[0079] For example, at least one instruction may correspond to a command for the electronic device (100) to acquire a projection area. Here, the projection area may correspond to an area acquired by the electronic device (100) to project an image based on a plurality of areas. Here, the plurality of areas will be described in detail in the following section.

[0080] Meanwhile, the memory (110) according to one or more embodiments can store at least one artificial intelligence model.

[0081] Artificial intelligence models consist of machine learning (deep learning) technology that uses algorithms to self-classify and learn the characteristics of input data, and elemental technologies that utilize machine learning algorithms to mimic functions such as cognition and judgment of the human brain.

[0082] Here, artificial intelligence models may also be referred to as learning models, neural network models, AI models, deep learning models, etc.

[0083] The elemental technologies may include, for example, at least one of linguistic understanding technology that recognizes human language / characters, visual understanding technology that recognizes objects like human vision, reasoning / prediction technology that judges information to logically reason and predict, and knowledge representation technology that processes human experience information into knowledge data.

[0084] The neural network model is not limited to the example described above, and the neural network model can be implemented with various models trained to perform the necessary actions for the electronic device (100) to acquire content corresponding to the currently spoken voice.

[0085] According to one embodiment, at least one neural network model stored in memory (110) may include a model trained to identify at least one obstacle included in the projection surface based on sensing data. Here, the obstacle may correspond to an object that interferes with or blocks projection when the electronic device (100) projects an image.

[0086] That is, at least one neural network model stored in memory (110) can be trained to detect at least one obstacle present on the projection surface using the sensing data as training data.

[0087] In this case, when new sensing data obtained by sensing the projection surface is input to at least one neural network model, at least one neural network model can output information such as the location and size of the obstacle. The sensing data will be explained in detail in the following section.

[0088] The sensor (120) can sense the surrounding space of the electronic device (100). The sensor (120) may include at least one of an image sensor (RGB sensor, infrared sensor, etc.), a LiDAR sensor, an obstacle detection sensor, and a driving detection sensor. The sensor (120) can sense the surrounding space of the electronic device (100) and generate sensing data.

[0089] The electronic device (100) may include one sensor (120), but is not limited thereto and may be implemented with a plurality of sensors (120). For convenience, the following description assumes a case where the sensor (120) is implemented as at least one sensor (120).

[0090] For example, the electronic device (100) can sense the surrounding space of the electronic device (100) through at least one sensor (120).

[0091] Here, the surrounding space may refer to the projection surface onto which the test image is projected. Since the test image has been explained previously, a redundant explanation will be omitted.

[0092] The electronic device (100) can acquire sensing data by sensing the projection surface. Here, the sensing data may correspond to data acquired by the electronic device (100) sensing the surrounding space through at least one sensor (120).

[0093] Here, if at least one sensor (120) is implemented as an image sensor, the sensing data may correspond to data (digital data) from visual information of the surrounding environment captured through a camera. For example, the sensing data may correspond to data including pixel information of the captured workspace (brightness and color values ​​of each pixel).

[0094] According to one example, at least one sensor (120) may include an RGB sensor and an infrared sensor.

[0095] An RGB sensor can be an optical sensor that generates a color image by detecting light through three color channels: red (R), green (G), and blue (B). Here, the sensing data acquired through the RGB sensor can be referred to as RGB data. In this context, RGB data may include pixel-level values ​​composed of red (R), green (G), and blue (B) color channels.

[0096] An infrared sensor can be defined as a sensor that detects light in the infrared (IR) band, which extends beyond the visible light range, to collect information regarding the reflection, distance, and depth of an object. Here, the sensing data acquired through the infrared sensor can be referred to as infrared data. In this context, infrared data may include information such as the intensity, location, and distance of the reflected light detected by the infrared (IR) sensor.

[0097] According to one embodiment, data (sensing data) obtained from at least one sensor (120) may include infrared data and data not obtained from infrared reflection.

[0098] For example, infrared data may correspond to data obtained from infrared (IR) reflected from a projection surface. Data not obtained from infrared reflection may correspond to RGB data.

[0099] However, the present disclosure is not limited thereto, and the data obtained from at least one sensor (120) may include only one of infrared data and RGB data.

[0100] Meanwhile, if at least one sensor (120) is implemented as a LiDAR sensor, the sensing data may correspond to data representing the distance (or 3D position) to an object based on the reflection time of a laser pulse. In this case, the sensing data may correspond to point cloud data.

[0101] The sensing data is not limited to the examples described above, and can be implemented in various forms of data if at least one sensor (120) is implemented as a different type of sensor.

[0102] The projection unit (130) can project an image. The projection unit (130) can project an image onto a projection area using a light source such as a lamp or an LED. For example, the projection unit (130) can project light corresponding to the image through a lens and / or a mirror. Accordingly, the projected light can be formed on the projection area.

[0103] Such a projection unit (130) can project an image in a hybrid manner capable of switching between an ultra-short focus method, a projection method, an ultra-short focus method, and a projection method.

[0104] For example, in the case of a hybrid projector, a plurality of lenses (e.g., ultra-short focal length lenses, projection lenses) are provided within the projection unit (130), and an image can be projected using a corresponding lens in response to a change in the projection method.

[0105] According to one or more embodiments, the electronic device (100) can project a test image through the projection unit (130). For example, the electronic device (100) can project a test image to adjust the projection area after turning on. Here, since the test image has been described above, a redundant description will be omitted.

[0106] The processor (140) can control the overall operations of the electronic device (100). For example, the processor (140) can control the operation of the electronic device (100) by being operatively connected to the memory (110), the sensor (120), and the projection unit (130). Additionally, the processor (140) can control the operation of the electronic device (100) according to the present disclosure by executing one or more instructions stored in the memory (110). The processor (140) may be composed of one or more processors.

[0107] The processor (140) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing operations. The processor (140) may include at least one electrical circuit and may process instructions (or programs, data, etc.) stored in memory (110) individually or collectively in a distributed manner. The processor (140) may include a processor assembly comprising one or more processing circuits. The processor (140) may include any processing circuit that is operative to control the performance and operations of one or more components of the electronic device (100) (e.g., memory (110), sensor (120), projection unit (130)). For example, the processor (140) (e.g., application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). For example, the processor (140) may be implemented with a plurality of cores (or at least one core circuit), a plurality of chips, or a plurality of chipsets. For example, the processor (140) may include one or more processing circuits. For example, the processor (140) may include one or more processing circuits configured to perform the various functions of the present disclosure individually and / or collectively. As an example without limitation, at least a portion of the processor (140) may be included in a first chip of the electronic device (100), and at least another portion of the processor (140) may be included in a second chip of the electronic device (100) different from the first chip of the electronic device (100).

[0108] For example, the processor (140) may include a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a display controller, a memory controller, a storage controller, a communication processor (CP), and / or a sensor interface. These components of the processor (140) are merely exemplary. For example, the processor (140) may include other components. For example, some components of the processor (140) may be omitted from the processor (140). For example, some components of the processor (140) may be included as separate components of the electronic device (100) outside the processor (140). For example, some components of the processor (140) (e.g., a memory controller) may be included within other components (e.g., at least a portion of the memory (110), an interface (e.g., available for connection to at least one component of the electronic device (100)), a projection unit (130)).

[0109] The processor (140) may cause other components of the electronic device (100) to perform various operations by executing instructions stored in memory (110). The CPU (or central processing circuit) may be configured to control the components of the processor (140) based on the execution of instructions stored in memory (110) (e.g., volatile memory and / or non-volatile memory). The GPU (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (or neural processing circuit, or AI (artificial intelligence) chip) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). The ISP (or image signal processing circuit) may be configured to process a raw image acquired through an image sensor into a format suitable for components within the electronic device (100) or components of the processor (140). A display controller (or display control circuit, or DPU (display processing unit)) may be configured to process an image obtained from a CPU, GPU, ISP, or memory (110) (e.g., volatile memory) into a format suitable for display. A memory controller (or memory control circuit) may be configured to control reading data from volatile memory and writing data to volatile memory. A storage controller (or storage control circuit) may be configured to control reading data from non-volatile memory and writing data to non-volatile memory. A CP (communication processing circuit) may be configured to process data obtained from a component of the processor (140) into a format suitable for transmitting to another electronic device via a communication circuit, or to process data obtained from another electronic device via a communication circuit into a format suitable for processing by a component of the processor (140).For example, the communication circuit may include one or more communication circuits. The sensor interface (or sensing data processing circuit, sensor hub) may be configured to process data regarding the state of the electronic device (100) and / or the state around the electronic device (100), obtained through the sensor, into a format suitable for the components of the processor (140).

[0110] For example, the processor (140) can perform the method of the electronic device (100) according to an embodiment of the present disclosure by executing one or more instructions stored in memory (110).

[0111] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an artificial intelligence dedicated processor).

[0112] The processor (140) may be implemented as a single-core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When the processor (140) is implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0113] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one of the plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in a multi-core processor, or the first operation and the second operation may be performed by a first core included in a multi-core processor and the third operation may be performed by a second core included in a multi-core processor.

[0114] In the embodiments of the present disclosure, a processor may mean a system-on-chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator, or machine learning accelerator, but the embodiments of the present disclosure are not limited thereto.

[0115] Meanwhile, when the instructions stored in the memory (110) described above are executed individually or collectively by the processor (140), the electronic device (100) may be made to perform the following operations.

[0116] According to one or more embodiments, the electronic device (100) can control the projection unit (130) to project a test image. For example, the electronic device (100) can control the projection unit (130) to project a test image onto a projection surface. Here, since the test image has already been described, a redundant description will be omitted.

[0117] According to one or more embodiments, the electronic device (100) can acquire a first region based on data corresponding to the projection surface on which the test image is projected, which is acquired through at least one sensor (120) while the test image is projected onto the projection surface.

[0118] For example, the electronic device (100) can acquire a first area on the projection surface based on data not acquired by infrared reflection. Here, the data not acquired by infrared reflection may be included in the data acquired through at least one sensor (120).

[0119] Here, the data corresponding to the projection surface on which the test image is projected may correspond to data (sensing data) obtained by the electronic device (100) sensing the projection surface on which the test image is projected through at least one sensor (120).

[0120] Here, the sensing data may include RGB data and infrared data obtained from the projection surface. For example, the sensing data may correspond to an image of the projection surface captured by the electronic device (100) through an RGB sensor and an infrared sensor.

[0121] The electronic device (100) can acquire infrared data by capturing a projection surface on which a test image is projected. If the electronic device (100) fails to perform correction based on the test image, it can detect obstacles, etc., on the projection surface based on the infrared data. Here, correction may refer to the keystone correction described above. However, it is not limited thereto. This will be explained in detail in the following section.

[0122] Even if the projectable area of ​​the electronic device (100) is smaller than the area on which the test image is projected, the correction based on the above-described test image may fail. In this case, the electronic device (100) may project a special test image onto the projection surface.

[0123] Here, the special test image may correspond to an image different from the test image described above. For example, if the test image is an image projected in infrared, the special test image may correspond to an image projected in visible light.

[0124] The electronic device (100) can acquire an image (IR+RGB image) by capturing a projection surface on which a special test image is projected. Here, the image may include both infrared data and RGB data.

[0125] The first area may correspond to an area for projecting an image among the projection surfaces. Here, the area for projecting an image may mean an area obtained by adjusting the area where the electronic device (100) projects a test image.

[0126] Here, adjusting the area where the test image is projected may mean that the electronic device (100) adjusts the area where the test image is currently projected into a projectable area. Here, a projectable area may mean an area where there is no distortion of the test image due to obstacles or sagged surfaces.

[0127] For example, the electronic device (100) can identify the user's touch location when an image (content image, etc., in addition to a test image) is projected onto a projectable area. That is, the electronic device (100) can identify the touch location based on received infrared light when there is a user's finger touch, etc., on a part of the projectable area.

[0128] Here, the received infrared light may refer to infrared light reflected by the user's finger touch, etc. and received by the infrared sensor of the electronic device (100). Since this has been explained in detail above, a redundant explanation will be omitted.

[0129] According to one embodiment, the electronic device (100) can obtain a first area that is smaller in size than the area on which a test image is projected from the projection surface, from data corresponding to the projection surface.

[0130] Here, when a test image is projected onto an area larger than the projectable area, the electronic device (100) can acquire a first area as the projectable area or an area smaller than the projectable area.

[0131] According to one example, an electronic device (100) can obtain a cross image based on a plurality of images obtained from data corresponding to a projection surface. Here, the electronic device (100) can obtain a cross image by comparing a plurality of images. Here, the cross image may correspond to an image generated by extracting common features or overlapping regions between a plurality of images.

[0132] For example, an electronic device (100) may obtain a first image including pixel data from data corresponding to a projection surface obtained by at least one sensor (120). Here, the pixel data obtained from the projection surface may correspond to data obtained by the electronic device (100) sensing the projection surface.

[0133] Here, pixel data may correspond to data containing information about the brightness, color, or intensity values ​​represented by each pixel of the first image. For example, in an RGB image, pixel data may include R (red), G (green), and B (blue) values. In an infrared image, a pixel may include infrared reflection intensity (value).

[0134] The first image may correspond to an image containing the pixel data described above. For example, the first image may correspond to an image obtained by sensing a projected test image and a projection surface.

[0135] Here, the first image may include pattern distortion. Here, pattern distortion may refer to a deformation of the pattern that appears due to obstacles, curved surfaces, or lens distortion, etc., in the projected correction pattern.

[0136] The electronic device (100) can obtain a cross image based on the first image and the second image.

[0137] Specifically, the electronic device (100) can obtain a cross image by comparing a first image and a second image. Here, the second image may correspond to an image obtained through pixel data of the first image. Here, the image obtained through pixel data of the first image may correspond to an image obtained by the electronic device (100) converting pixel data of the first image.

[0138] For example, the second image may correspond to an image in which the electronic device (100) corrects distortion in the pixel data of the first image using a camera intrinsic matrix. Here, the second image may include a plurality of pixels distinguished according to the boundary of the work area.

[0139] Specifically, the electronic device (100) can match the size and boundaries of the area where the test image is projected in the first image with the working area. Here, the working area may mean an area where the image is predicted to be properly projected on the projection surface.

[0140] At this time, the electronic device (100) can remove obstacles and noise through morphological operations (e.g., erosion and bulging, etc.). Here, morphological operations may refer to a process of processing boundaries, shapes, or noise by adding or removing pixels based on the structural shape of an image.

[0141] For example, erosion can be a morphological operation that reduces bright pixels (e.g., white) in an image, trims edges, and removes small noise or unnecessary parts. Dilating can be a morphological operation that expands bright pixels in an image to thicken edges and fills small holes or empty areas.

[0142] The electronic device (100) can obtain the boundary of the area where the test image is projected through morphological operations and adjust the boundary according to the boundary of the predicted work area.

[0143] Meanwhile, the electronic device (100) can obtain a cross image by comparing the first image and the second image. Here, the cross image may correspond to an integrated image generated from common pixel data between the first image and the second image.

[0144] Specifically, the electronic device (100) can compare pixel data of the first image and the second image, and set each pixel value such that common pixels are below a first threshold and non-common pixels are above a first threshold. Here, the first threshold may correspond to a reference pixel value set to distinguish between common pixels and non-common pixels in the cross image.

[0145] Here, common pixels may refer to pixels that belong to the boundary in the first image and also belong to the boundary in the second image. Here, the boundary in the first image may correspond to the boundary identified according to the area where the test image is projected. The boundary in the second image may correspond to the boundary of the aforementioned work area.

[0146] Meanwhile, non-common pixels may refer to pixels within the boundaries of the first image that do not belong to the boundaries of the second image. The existence of non-common pixels may mean that the boundaries of the first image and the boundaries of the second image do not coincide.

[0147] That is, the process of the electronic device (100) identifying non-common pixels may correspond to the process of distinguishing between a projectable area and an excess area. Through this, if the area where the test image is projected is larger than the working area, the electronic device (100) can identify the excess pixel value (pixel value of the non-common pixel) to determine whether the projection screen size needs to be adjusted.

[0148] The electronic device (100) can acquire a cross-image composed of multiple pixels (including common pixels and non-common pixels). In the cross-image, a relatively bright area may represent a non-common area. On the other hand, a relatively dark area may represent a common area.

[0149] However, this is merely an example, and relatively bright areas may represent common areas, while dark areas may represent non-common areas.

[0150] According to one example, the electronic device (100) can control the projection unit (130) to project a reduced test image based on at least one cross pixel included in the acquired cross image.

[0151] For example, the electronic device (100) can reduce the test image into a reduced test image based on at least one cross pixel included in the acquired cross image.

[0152] And, the electronic device (100) can control the projection unit (130) to project a reduced test image onto an area on the projection surface based on at least one cross pixel of the acquired cross image.

[0153] Here, the intersecting pixel may refer to the aforementioned non-common pixel. That is, the intersecting pixel may refer to a pixel in the intersecting image whose pixel value is greater than or equal to a first threshold value.

[0154] For example, if the number of cross pixels corresponding to at least one cross pixel having a pixel value greater than or equal to a first threshold value in the cross image is greater than or equal to a first threshold number, the electronic device (100) can reduce the test image into a reduced test image based on a first unit that is pre-set.

[0155] For example, the electronic device (100) can identify whether at least one cross pixel, in which the pixel value in the cross image is greater than or equal to a first threshold value, is greater than or equal to a first threshold number.

[0156] Here, the first threshold number may refer to the minimum number required to identify whether the current test image is projected beyond the working area.

[0157] The electronic device (100) can control the projection unit (130) to project a test image by reducing it to a preset first unit if the number of cross pixels is greater than or equal to a first threshold number. For example, the number of cross pixels may mean the number of at least one cross pixel.

[0158] Here, the first unit may correspond to a fixed reference value of the reduction ratio, length, or width used when reducing the test image. The electronic device (100) can reduce the size of the test image step by step according to the preset first unit.

[0159] The electronic device (100) can control the projection unit (130) to project test images sequentially while gradually reducing the size of the test images.

[0160] Meanwhile, the electronic device (100) can acquire a projection surface area on which a reduced test image is projected as a first area.

[0161] Specifically, when a test image with reduced size is projected, the electronic device (100) senses the projection surface, identifies cross pixels from the first image and the second image, and if the number of cross pixels is still greater than or equal to a first threshold number, can project the test image again with reduced size. The electronic device (100) can repeatedly perform the above-described process, such as sensing the projection surface again.

[0162] For example, the electronic device (100) can update the number of at least one cross pixel based on a reduced test image. Specifically, the electronic device (100) can sense a projection surface on which the reduced test image is projected to obtain sensing data, and can obtain a first image and a second image based on the sensing data.

[0163] The electronic device (100) can identify cross pixels again from a cross image obtained based on a first image and a second image, and change the number of existing cross pixels to a new number.

[0164] The electronic device (100) can identify whether the number of at least one updated cross pixel is less than a first threshold number. If the number of at least one cross pixel is less than a first threshold number, the electronic device (100) can acquire the area where the reduced test image is projected as the first area.

[0165] That is, the electronic device (100) can reduce the test image step by step and repeatedly calculate the number of cross pixels.

[0166] If it is identified that the number of intersecting pixels has decreased to less than a certain number, the electronic device (100) can identify that the test image almost coincides with or is included in the projectable area. In this case, the electronic device (100) can acquire the area where the test image is currently projected as the first area.

[0167] According to one or more embodiments, the electronic device (100) may acquire a second region corresponding to an image projection among the projection surfaces based on infrared data. Here, the infrared data may correspond to the data acquired by the electronic device (100) through infrared light reflected from the projection surface among the data corresponding to the projection surface. Here, the data corresponding to the projection surface may correspond to the aforementioned sensing data.

[0168] For example, infrared data may correspond to data obtained by an electronic device (100) sensing infrared (IR) reflected by a projection surface.

[0169] Here, the second region may correspond to an area (projectable area) for the electronic device (100) to project an image. The second region is an area obtained using only infrared data among the sensing data, and may correspond to an area different from the first region.

[0170] According to one embodiment, the electronic device (100) can identify at least one valid pixel in an infrared image in which the pixel value is greater than or equal to a second threshold value. Here, the infrared image may correspond to an image obtained from infrared data corresponding to an area where a test image is projected.

[0171] Here, the infrared image may correspond to an image that visually represents the reflection intensity of a surface, etc., based on infrared (IR) data acquired by infrared rays reflected from a projection surface.

[0172] For example, if infrared light is reflected by an obstacle on the projection surface, the infrared image may include obstacle regions exhibiting high infrared intensity. Here, high infrared intensity may appear as relatively high pixel values ​​in the infrared image.

[0173] An obstacle area may refer to a set of pixels where infrared data relatively strongly reflected by an obstacle is concentrated. Here, the pixels included in the pixel set may refer to a set of pixels with a reflection intensity higher than that of a normal projection surface (e.g., a flat surface such as a wall, ceiling, or floor).

[0174] Meanwhile, if infrared rays are reflected by a sagged surface among the projection surfaces, the infrared image may include a sag area that exhibits high infrared intensity.

[0175] Meanwhile, the second threshold value may correspond to a reference value of infrared intensity set to determine whether a pixel value in an infrared image indicates an obstacle or a curved surface. For example, the second threshold value may be set to 200 out of 255 pixel values ​​in an infrared image, and if the pixel value is 200 or higher, it can be identified as infrared light being reflected by an obstacle, etc.

[0176] A valid pixel may correspond to a pixel in an infrared image in which the pixel value is identified as being greater than or equal to a second threshold. For example, the area occupied by a valid pixel may correspond to a pixel representing an area corresponding to an obstacle or a curved surface.

[0177] Meanwhile, the electronic device (100) can acquire a second region after identifying valid pixels present in the infrared image.

[0178] According to one example, the electronic device (100) can acquire a second region based on at least one effective pixel identified among the projection surfaces.

[0179] For example, an electronic device (100) can identify whether the number of at least one effective pixel is greater than or equal to a second threshold number.

[0180] Here, the second threshold number may refer to the minimum number of valid pixels (pixels greater than or equal to the second threshold) in an infrared image. That is, the second threshold number may serve as a criterion for determining whether there is an obstacle or a curved area on the projection surface.

[0181] If the number of valid pixels is greater than or equal to the second threshold number, the electronic device (100) can control the projection unit (130) to project the test image by reducing it to a preset second unit.

[0182] For example, if at least one effective pixel is greater than or equal to a second threshold number, the electronic device (100) can control the projection unit (130) to reduce the test image into a reduced test image based on a preset second unit.

[0183] And, the electronic device (100) can control the projection unit (130) to project a reduced test image onto an area of ​​the projection surface based on at least one effective pixel.

[0184] Here, the second unit may refer to a fixed unit set to reduce the test image. For example, the second unit may correspond to a ratio, area, or length unit. The electronic device (100) can reduce the size of the test image step by step according to the pre-set second unit.

[0185] The electronic device (100) can control the projection unit (130) to project test images sequentially while gradually reducing the size of the test images.

[0186] Meanwhile, the electronic device (100) can acquire a second region based on at least one effective pixel identified among the projection surfaces.

[0187] Specifically, when a test image with reduced size is projected, the electronic device (100) identifies valid pixels from infrared data obtained by sensing the projection surface, and if the number of valid pixels is still greater than or equal to a second threshold number, the test image can be reduced again and projected. The electronic device (100) can repeatedly perform the above-described process, such as sensing the projection surface again.

[0188] For example, the electronic device (100) can update the number of at least one valid pixel based on a reduced test image.

[0189] Specifically, the electronic device (100) can sense a projection surface on which a reduced test image is projected to acquire infrared data, identify the number of valid pixels based on the infrared data, and change the number of valid pixels from the existing number to a new number.

[0190] The electronic device (100) can acquire the area where the reduced test image is projected as the second area if the number of at least one updated valid pixel is less than the second threshold number. Here, the area where the reduced test image is projected may be included in the area excluding the area occupied by obstacles or curved surfaces among the projection surfaces.

[0191] That is, the electronic device (100) can reduce the test image step by step and repeatedly calculate the number of valid pixels.

[0192] If the number of cross pixels decreases to less than a certain number, the electronic device (100) can identify that the test image was projected while avoiding an obstacle (or a curved surface). In this case, the electronic device (100) can acquire the area where the current test image is projected as a second area.

[0193] According to one or more embodiments, the electronic device (100) can obtain a projection area by comparing a first area and a second area. For example, the electronic device (100) can obtain the smaller of the first area and the second area as the projection area.

[0194] According to one embodiment, the electronic device (100) can acquire the smaller area as a projection area if the smaller area among the first area and the second area is greater than or equal to a predetermined threshold size. Here, the predetermined threshold size may correspond to the minimum size of the projection area set to ensure minimum visibility of the projection area.

[0195] If the smaller of the first and second regions is less than or equal to a predetermined threshold size, the electronic device (100) can terminate the operation to acquire a projection region.

[0196] Meanwhile, the electronic device (100) can acquire a third area through a separate process for identifying obstacles and acquire a projection area based on the first area, second area, and third area described above.

[0197] For example, an electronic device (100) can identify at least one obstacle included in the projection surface based on data corresponding to the projection surface through an artificial intelligence model, wherein the data corresponding to the projection surface may correspond to data obtained by sensing the projection surface (sensing data).

[0198] Here, the artificial intelligence model may correspond to an artificial intelligence model corresponding to at least one obstacle identification. The artificial intelligence model corresponding to obstacle identification may correspond to an artificial intelligence model trained to identify obstacles.

[0199] Specifically, the artificial intelligence model may correspond to a model trained to identify at least one obstacle included in the previous projection surface based on previous sensing data.

[0200] Here, the previous sensing data may correspond to sensing data obtained by the electronic device (100) or an external electronic device sensing a previous projection surface prior to the current time. Here, the previous projection surface may refer to a target space sensed by the electronic device (100) or an external electronic device prior to the current time.

[0201] For example, an electronic device (100) or an external electronic device can sense the previous projection surface to obtain the previous sensing data as training data. An artificial intelligence model can be trained to identify obstacles present on the previous projection surface using the previous sensing data as training data.

[0202] The electronic device (100) can obtain information about the location and size of obstacles existing on the projection surface (current projection surface) by inputting data corresponding to the projection surface (currently acquired sensing data) into an artificial intelligence model.

[0203] Meanwhile, the electronic device (100) can acquire sensing data based on each of a plurality of test images and acquire a first region and a second region based on the acquired sensing data.

[0204] According to one embodiment, the electronic device (100) can control the projection unit (130) to project an initial test image including an infrared pattern. Here, the initial test image may correspond to a test image projected before the above-described test image is projected.

[0205] For example, the initial test image may include an infrared pattern. Here, the infrared pattern may correspond to a pattern projected via infrared to analyze the size, boundaries, obstacles, or curved areas of the projection surface. Here, the pattern may be detectable via an infrared sensor when it is projected onto the projection surface and reflected.

[0206] According to one embodiment, the electronic device (100) can control the projection unit (130) to project a test image including an RGB pattern when a function related to an infrared pattern is not performed. Here, the function related to the infrared pattern may correspond to a correction operation that the electronic device (100) can perform using the infrared pattern.

[0207] For example, it can be identified that a correction operation using an infrared pattern (e.g., keystone correction operation) has failed. In this case, the electronic device (100) can control the projection unit (130) to project a test image containing an RGB pattern.

[0208] For example, if there is an obstacle or a curved surface on the projection surface, the electronic device (100) may fail to perform correction based on the infrared pattern. Or, if the reflection of the projected infrared (infrared pattern) is weak or there is severe ambient light interference, the electronic device (100) may fail to perform correction because the IR sensor cannot accurately detect the boundary of the projection surface.

[0209] Additionally, the electronic device (100) may fail to correct if the area on which the test image is projected is larger than the projectable surface (working area).

[0210] In this case, the electronic device (100) can control the projection unit (130) to project a new test image containing an RGB pattern. Here, the RGB pattern may correspond to a pattern for analyzing the size, boundaries, and distortion of the projection surface. The RGB pattern may be implemented in the form of a color image composed of red (R), green (G), and blue (B) channels.

[0211] Here, the test image may include both RGB patterns and infrared patterns. In this case, the sensing data acquired while the test image is projected may include both RGB data and infrared data.

[0212] For example, if the electronic device (100) fails to correct based on an infrared pattern, the electronic device (100) can project a full-white test image onto a projection surface. The electronic device (100) can perform correction on the projection area based on the full-white test image instead of the infrared pattern.

[0213] The electronic device (100) can project a test image containing an RGB pattern onto a projection surface and acquire a first region based on sensing data that senses the projection surface. Since the operation of acquiring the first region has been described in detail above, a redundant description will be omitted.

[0214] Afterwards, the electronic device (100) can acquire a second region based on infrared data among the data corresponding to the projection surface obtained through the projection surface on which the initial test image is projected.

[0215] For example, the electronic device (100) can acquire a second region based on infrared data among the sensing data acquired by sensing the projection surface on which the initial test image is projected. That is, when the electronic device (100) acquires the second region, it can acquire the second region by sensing the projection surface on which the initial test image is projected, instead of the projection surface on which the (new) test image is projected.

[0216] Meanwhile, the electronic device (100) can acquire a third area included in the area excluding the area corresponding to the obstacle among the projection surfaces. For example, the electronic device (100) can acquire a third area that is equal in proportion to the area where the current test image is projected, among the areas excluding the area occupied by the obstacle among the projection surfaces.

[0217] Here, the electronic device (100) can acquire a third area that is equal to the ratio of the area where the test image is projected and has the maximum size. The electronic device (100) can acquire a projection area by comparing the first area, the second area, and the third area.

[0218] Through this, the electronic device (100) can obtain a second area excluding obstacles on the projection surface, but can obtain an appropriate projection position by using an artificial intelligence model for obstacle detection to more accurately identify obstacles.

[0219] As described above, the electronic device (100) can acquire a first region and a second region when a test image is projected onto a projection surface, and acquire a projection region by comparing the two regions.

[0220] Through this, the electronic device (100) can obtain a projection area that is more suitable for projecting an image compared to obtaining a projection area through only a single process (e.g., one of the process of obtaining a first area and the process of obtaining a second area). That is, the electronic device (100) can obtain a more accurate projection area even if there are undetected obstructions (obstacles or curved surfaces, etc.), so the user's satisfaction with the electronic device (100) can be further increased.

[0221] Although the electronic device (100) in FIG. 2 is illustrated as having only a basic configuration, the electronic device (100) may include various additional configurations in addition to the configuration described above.

[0222] FIG. 3 is a detailed block diagram for explaining the detailed configuration of an electronic device according to one or more embodiments of the present disclosure.

[0223] According to FIG. 3, the electronic device (100) may include a memory (110), a sensor (120), a projection unit (130), a processor (140), a movement unit (150), and a communication unit (160). Here, the memory (110), the sensor (120), the projection unit (130), and the processor (140) have already been described, so they will be described without redundant descriptions.

[0224] The moving part (150) is configured to move the electronic device (100). To this end, the moving part (150) includes a motor, wheels, etc., and can move the electronic device (100) through the movement of the wheels. Meanwhile, when implementing, a caterpillar, etc., may be used instead of wheels, and if the electronic device (100) is implemented as a drone, propellers may be used instead of wheels.

[0225] According to one embodiment, the electronic device (100) can control the moving part (150) to move from the current position to a second projection position.

[0226] Here, the current position may refer to a position corresponding to the projection of the test image. Here, the position corresponding to the projection of the test image may correspond to a position for the electronic device (100) to project the test image.

[0227] The second position may correspond to a position corresponding to image projection in the projection area. Here, the position corresponding to image projection in the projection area may correspond to a position for the electronic device (100) to project an image (an image other than a test image) onto the projection area.

[0228] Specifically, the second position may correspond to a position where the electronic device (100) can face the center of the projection area. Here, the second position may exist within a focal range from the projection surface (e.g., a wall). Here, the focal range may refer to a range between a minimum distance and a maximum distance at which the electronic device (100) can project a clear and distortion-free image onto the projection surface.

[0229] For example, the second position may correspond to a position for projecting an image at the center of the projection area. Alternatively, the second position may correspond to a position for projecting an image onto the projection area while avoiding obstacles in the surroundings (e.g., the floor surface on which the electronic device (100) travels).

[0230] Specifically, the electronic device (100) can obtain a second location and a path toward the second location based on projection area and sensing data.

[0231] For example, the electronic device (100) can analyze sensing data acquired from the projection surface to generate map information including the location of the projection area and surrounding obstacle information. The electronic device (100) can obtain an optimal path from the current location to a second location based on the map information. Here, the map information may include the size of the projection surface, boundaries, obstacle locations, and coordinates of the projection area.

[0232] Meanwhile, the communication unit (160) is configured to communicate with various types of external devices according to various types of communication methods. The communication unit (160) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, and a wireless communication module. Here, each communication module may be implemented in the form of at least one hardware chip.

[0233] Wi-Fi modules and Bluetooth modules can perform communication using Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information, such as SSID and session key, is transmitted and received first; after establishing a communication connection using this information, various information can be transmitted and received.

[0234] The infrared communication module performs communication according to infrared communication (IrDA, Infrared Data Association) technology, which uses infrared rays located between visible light and millimeter waves to wirelessly transmit data over short distances.

[0235] In addition to the communication method described above, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

[0236] In addition, the communication unit (160) may include at least one wired communication module that performs communication using a LAN (Local Area Network) module, an Ethernet module, a pair cable, a coaxial cable, a fiber optic cable, or an UWB (Ultra Wide-Band) module. This communication unit (160) may also be referred to as a transceiver.

[0237] For example, the electronic device (100) can receive sensing data acquired by the external electronic device sensing the projection surface through the communication unit (160) from the external electronic device. Alternatively, the electronic device (100) can receive data regarding the first area and the second area acquired by the external electronic device (100) through the communication unit (160).

[0238] In this case, the electronic device (100) can sense the projection surface around the electronic device (100) and transmit the acquired sensing data to an external electronic device, and receive data regarding the first area and the second area from the external electronic device. However, it is not limited thereto.

[0239] Meanwhile, although an electronic device (100) including various additional configurations is illustrated in FIG. 3, some of the illustrated configurations may be implemented in an omitted form. Additionally, other configurations that are not illustrated may also be included.

[0240] For example, the electronic device (100) may further include a communication unit.

[0241] For example, the communication unit can perform data communication between an external device and an electronic device using at least one of the data communication methods including wired LAN, wireless LAN, Wi-Fi, Wi-Fi Direct, Bluetooth, ZigBee, WFD (Wi-Fi Direct), infrared communication (IrDA, infrared Data Association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), Wibro (Wireless Broadband Internet), WiMAX (World Interoperability for Microwave Access), SWAP (Shared Wireless Access Protocol), WiGig (Wireless Gigabit Alliances, WiGig), and RF communication.

[0242] In addition, the communication unit can perform communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation), in addition to the communication method described above.

[0243] Meanwhile, the processor (140) can receive a signal from a server, etc., through a communication unit to obtain a projection area based on a first area and a second area.

[0244] Alternatively, the processor (140) may transmit a signal requesting data for the first area and the second area to a server, etc., through a communication unit. For example, the processor (140) may obtain a projection area based on the received data for the first area and the second area, etc.

[0245] Meanwhile, the processor (140) may receive a control signal from a server device, etc., that causes the electronic device (100) to identify a projection area based on a first area and a second area. However, it is not limited thereto.

[0246] Meanwhile, the electronic device (100) may further include interfaces such as an HDMI port, DP, RGB, DVI, USB, and Thunderbolt for receiving video / audio signals connected to external devices or servers. HDMI, DP, and Thunderbolt are ports capable of simultaneously transmitting video and audio signals. The electronic device (100) can output distance information and second map data by performing various processing such as demuxing, decoding, and scaling on various signals received from external devices, servers communicating with external devices, etc., through the communication unit and these various interfaces.

[0247] Meanwhile, the electronic device (100) may include a display.

[0248] The display is a component for displaying the operating status, notification messages, UI screens, etc. of an electronic device (100). The display can be implemented in various forms such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, or a PDP (Plasma Display Panel). The display may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as an a-si TFT (amorphous silicon thin film transistor), an LTPS (low temperature poly silicon) TFT, or an OTFT (organic TFT). Meanwhile, the display can be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, or a three-dimensional display. Alternatively, the display may be implemented with only one or more light-emitting elements.

[0249] The electronic device (100) can change the display state of the display according to various states, such as when the electronic device (100) is turned on, when it is operating normally, when there is insufficient power, or when there is an error, so that the user can intuitively understand the state of the electronic device (100).

[0250] For example, the processor (140) can control the display to display a UI corresponding to the first area obtained. Here, the UI corresponding to the first area may correspond to a graphic element that visually represents the first area in a form overlaid on the projected surface captured.

[0251] However, it is not limited to this, and the processor (140) can control the display to display an image of the projection surface before acquiring the first area or the second area.

[0252] However, the display configuration is merely one of the various embodiments, and the display configuration may be omitted. That is, the electronic device (100) may be a device that directly has a display or a device connected to an external device.

[0253] For example, if the electronic device (100) is implemented as a set-top box, one-connect box, projector, etc., the operations of the electronic device (100) described above may be performed in an electronic device that does not include a display.

[0254] In this case, the processor (140) may acquire a UI, etc. based on a first projection area and provide the acquired UI to an external display device (a device equipped with a display) through a communication unit. The external display device may display the received UI through a display equipped on the external display device.

[0255] Meanwhile, the electronic device (100) may include a microphone.

[0256] The microphone can receive the user's voice when activated. For example, the microphone may be formed integrally on the upper side, front side, or side side of the electronic device (100).

[0257] The microphone may include various components such as a microphone that collects user voice in an analog form, an amplifier circuit that amplifies the collected user voice, an A / D conversion circuit that samples the amplified user voice and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0258] The microphone can transmit the received user voice to the electronic device (100). Subsequently, the electronic device (100) can input the received user voice into a speech recognition model to perform speech recognition. For example, the electronic device (100) can perform speech recognition on the user voice by performing STT (Speech to Text) on the user voice. The microphone receives the user voice and can transmit the received user voice to the electronic device (100). Subsequently, the electronic device (100) can input the received user voice into a speech recognition model to perform speech recognition. For example, the electronic device (100) can perform speech recognition on the user voice by performing STT (Speech to Text) on the user voice.

[0259] For example, the processor (140) can receive user input through a microphone. Here, the user input may correspond to user input for acquiring a projection area. However, it is not limited thereto.

[0260] FIG. 4 is a drawing for explaining the operation of an electronic device according to one or more embodiments of the present disclosure.

[0261] According to FIG. 4, an initial area (411) and a final area (421) are shown.

[0262] Here, the initial area (411) may correspond to the area where the electronic device (100) projects a test image. Since the test image has been described in detail previously, a redundant description will be omitted.

[0263] The electronic device (100) can adjust the initial area (411) to obtain a final area (421) that avoids obstacles (412). The electronic device (100) can project a test video or content video onto the final area (421).

[0264] Specifically, the electronic device (100) can project a test image onto a projection surface and sense an initial area (411) onto which the test image is projected to obtain sensing data. Here, the sensing data may include at least one of infrared data and RGB data.

[0265] The electronic device (100) can identify whether an obstacle (412) exists from the sensing data.

[0266] Specifically, the electronic device (100) can identify an obstacle (412) based on a plurality of images (e.g., a first image and a second image) obtained from sensing data. Alternatively, the electronic device (100) can identify an obstacle (412) based on an infrared image obtained from infrared data.

[0267] The electronic device (100) can gradually reduce the size of an initial area (411) based on the location of an obstacle (412) and obtain a final area (421) that satisfies a specific condition. Here, the final area (421) may correspond to a projection area obtained by the electronic device (100) based on a first area and a second area. Here, the specific condition may mean a condition in which the number of intersecting pixels or the number of valid pixels is less than a specific number.

[0268] As the operation of the electronic device (100) acquiring a final area (421) based on various types of pixel values ​​has been explained in detail above, a redundant explanation will be omitted.

[0269] The electronic device (100) can project an image onto the acquired final area (421), and subsequently, if a new obstacle occurs in the final area or a curved surface occurs, the final area (421) can be newly adjusted.

[0270] According to the example described above, the electronic device (100) can obtain a final area (421) by adjusting the initial area (411) at a fixed position without moving, and can project an image onto the obtained final area (421). However, it is not limited to this, and as described below, the electronic device (100) can move to a different position and project an image onto the final area (421).

[0271] FIG. 5 is a drawing for explaining the movement of an electronic device according to one or more embodiments of the present disclosure.

[0272] According to FIG. 5, an initial area (511) and a final area (521) are shown.

[0273] The electronic device (100) can obtain a final area (521) by adjusting the initial area (511). The electronic device (100) can project a test image or content image onto the final area (521).

[0274] Here, the electronic device (100) may move to a new location and project an image onto the final area (521). Here, the new location may refer to the aforementioned second location.

[0275] After moving, the electronic device (100') can project an image onto the final area (521) at a new location. Here, the new location may correspond to a location obtained based on the location of the final area (521).

[0276] The electronic device (100) can project an image onto the final area (521) from a position facing the front of the center of the final area (521). Here, the position facing the front can be located within the focal range from the center point of the final area (521). Since the focal range and the like have been explained previously, a redundant explanation will be omitted.

[0277] Meanwhile, the electronic device (100) can identify that an obstacle exists at a new location (e.g., a frontal location) based on sensing data. For example, the electronic device (100) can identify the obstacle based on map information obtained from the sensing data. In this case, the electronic device (100) can move to a location close to the obstacle.

[0278] Here, the destination and path for the electronic device (100) to move to have been explained previously, so a redundant explanation will be omitted.

[0279] After moving, the electronic device (100') can project an image onto the final area (521). At this time, if distortion (e.g., keystone) occurs in the final area (521) onto which the image is projected, additional correction (e.g., keystone correction) can be performed.

[0280] Accordingly, the electronic device (100) can project an image at a position facing the center of the final area (521) to minimize distortion caused by the projection position, and if there is an obstacle at that position, the electronic device (100) can move to a position that avoids the obstacle and project an image onto the final area.

[0281] FIG. 6 is a drawing for illustrating a maximum reduction scale according to one or more embodiments of the present disclosure.

[0282] According to FIG. 6, a surface size detector (610), an infrared emission detector (620), and an automatic screen size adjuster (630) are shown.

[0283] The electronic device (100) may include a surface size detector (610), an infrared emission detector (620), and an automatic screen size adjuster (630). In this case, the electronic device (100) can acquire the aforementioned final area by using a plurality of types of images as input data.

[0284] The surface size detector (610) can acquire the size and boundaries of the projectible area and generate data for adjusting the screen size. The surface size detector (610) can receive an IR+RGB image as input.

[0285] Here, the IR+RGB image may correspond to an image obtained by sensing a projection surface on which an arbitrary pattern is projected. Here, the arbitrary pattern may refer to an RGB pattern. For example, the RGB pattern may refer to the aforementioned full-white test image. For example, the electronic device (100) may project a full-white test image if it fails to correct based on an infrared pattern.

[0286] The electronic device (100) can output data including information about the size and boundaries of a projectable area based on an IR+RGB image. For example, the electronic device (100) can output data including information about the maximum reduction scale.

[0287] Here, the maximum reduction scale may refer to the maximum scale among the scales reduced to fit the projectible area when the pattern is projected beyond the projectible area. Here, the maximum reduction scale may correspond to the aforementioned first area.

[0288] For example, the maximum reduction scale may correspond to the maximum area that the reduced pattern (test image) can occupy within the projectible area while maintaining its original aspect ratio. However, it is not limited to this.

[0289] The infrared emission detector (620) can acquire data to adjust the projection area according to the area occupied by obstacles, etc. The surface size detector (610) can receive an IR image (infrared image).

[0290] Here, the IR image may correspond to an image obtained by sensing a projection surface onto which a calibration pattern is projected. Here, the calibration pattern may correspond to a pattern for analyzing the size, boundaries, obstacles, or curved areas of the projection surface.

[0291] For example, the electronic device (100) can project a calibration pattern onto a projection surface via infrared light to identify obstacles, etc. The infrared sensor of the electronic device (100) can receive infrared light reflected by the calibration pattern. As this has been previously described, a redundant explanation will be omitted.

[0292] The electronic device (100) can output data including information about the size and boundaries of a projectable area based on an IR image. Here, the projectable area may correspond to an area included in the area excluding obstacles, etc. identified by infrared data.

[0293] For example, the electronic device (100) may output data containing information about the maximum reduction scale. Here, the maximum reduction scale may refer to the maximum scale among the scales reduced to fit the projectable area when the area where the pattern is projected includes obstacles or curved surfaces. Here, the maximum reduction scale may correspond to the second area mentioned above.

[0294] For example, the maximum reduction scale may correspond to the maximum area that the reduced pattern (test image) can occupy within the projectible area while maintaining its original aspect ratio. However, it is not limited to this.

[0295] Meanwhile, the automatic screen size adjuster (630) can automatically adjust the screen size based on surface size and obstacle information to obtain an optimal area. Here, the surface size may refer to the size of the projectable area among the data output by the surface size detector (610).

[0296] Here, obstacle information may correspond to information included in the data output by the infrared emission detector (620). For example, obstacle information may refer to information regarding the location, size, and shape of an obstacle detected based on infrared data from the projection surface.

[0297] Meanwhile, the automatic screen size adjuster (630) can receive output data from the surface size detector (610) and the infrared emission detector (620), respectively.

[0298] For example, the automatic screen size adjuster (630) can receive both information about the maximum reduction scale output by the surface size detector (610) and information about the maximum reduction scale output by the infrared emission detector (620).

[0299] The automatic screen size adjuster (630) can obtain a minimum reduction scale based on the above input data. Here, the minimum reduction scale may refer to a minimum scale greater than or equal to the minimum scale (threshold size) among a plurality of maximum reduction scales output by each of the surface size detector (610) and the infrared emission detector (620).

[0300] The automatic screen size adjuster (630) can identify a minimum value by comparing two maximum reduction scales received. The automatic screen size adjuster (630) can compare the minimum value with a threshold size, and if the minimum value is greater than or equal to the threshold size, the minimum value can be output as the maximum reduction scale. Here, since the threshold size has been explained previously, a redundant explanation will be omitted.

[0301] As described above, the electronic device (100) can obtain two maximum reduction scales in different ways through the surface size detector (610) and the infrared emission detector (620), respectively. The electronic device (100) can obtain a minimum reduction scale corresponding to the final projection area by comparing the two maximum reduction scales.

[0302] Through this, the electronic device (100) can obtain a final area that can more accurately avoid elements that interfere with projection (obstacles or curved surfaces, etc.).

[0303] Meanwhile, the electronic device (100) can acquire a final area by additionally using a separate module in addition to the surface size detector (610) and the infrared emission detector (620).

[0304] FIG. 7 is a drawing for illustrating an obstacle detector according to one or more embodiments of the present disclosure.

[0305] According to FIG. 7, a surface size detector (710), an infrared emission detector (720), an obstacle detector (730), and an automatic screen size adjuster (740) are shown.

[0306] Here, the surface size detector (710) and the infrared emission detector (720) can perform the same functions as the surface size detector (610) and the infrared emission detector (620) described in FIG. 6, respectively. In FIG. 7, redundant descriptions of the above configurations are omitted, and the operation of the obstacle detector (730) and the automatic screen size adjuster (740) is described in detail.

[0307] The obstacle detector (730) can detect the location and size of the obstacle and acquire data to adjust the projection area according to the area occupied by the obstacle. The obstacle detector (730) can receive an IR+RGB image.

[0308] Here, the IR+RGB image may correspond to an image obtained by sensing a projection surface onto which a calibration pattern is projected. Here, the calibration pattern may refer to a pattern for identifying obstacles, etc. The electronic device (100) may project the calibration pattern onto a projection surface through at least one of infrared and visible light to identify obstacles, etc.

[0309] The electronic device (100) can output data containing information about the size and boundaries of a projectable area based on an IR+RGB image. Here, the projectable area may correspond to an area included in the area excluding obstacles, etc. identified through an artificial intelligence model.

[0310] Here, the neural network model may correspond to a model trained to identify obstacles present on the projection surface based on sensing data. Since details regarding such artificial intelligence models have been explained previously, a redundant explanation will be omitted.

[0311] For example, the electronic device (100) may output data containing information about the maximum reduction scale as output data. Here, the maximum reduction scale may refer to the maximum scale among the scales reduced to fit the projectable area when an obstacle is included in the area where the pattern is projected. Here, the maximum reduction scale may correspond to the aforementioned third area.

[0312] For example, the maximum reduction scale may correspond to the maximum area that the reduced pattern (test image) can occupy within the projectible area while maintaining its original aspect ratio. However, it is not limited to this.

[0313] Meanwhile, the automatic screen size adjuster (740) can receive output data from each of the surface size detector (710), infrared emission detector (720), and obstacle detector (730).

[0314] For example, the electronic device (100) can receive all of the information regarding the maximum reduction scale output by the surface size detector (710), the information regarding the maximum reduction scale output by the infrared emission detector (720), and the information regarding the maximum reduction scale output by the obstacle detector (730).

[0315] The automatic screen size adjuster (740) can obtain a minimum reduction scale based on the above input data. Here, the minimum reduction scale may refer to a minimum scale greater than or equal to the minimum scale (threshold size) among the multiple maximum reduction scales output by each of the surface size detector (710), the infrared emission detector (720), and the obstacle detector (730).

[0316] The automatic screen size adjuster (740) can identify a minimum value by comparing three maximum reduction scales received. The automatic screen size adjuster (740) can compare the minimum value with a threshold size, and if the minimum value is greater than or equal to the threshold size, it can output the minimum value as the maximum reduction scale.

[0317] As described above, the electronic device (100) can obtain a maximum reduction scale through the surface size detector (710) and the infrared emission detector (720), as well as through the obstacle detector (730). The electronic device (100) can obtain a minimum reduction scale corresponding to the final projection area by comparing the three maximum reduction scales.

[0318] Through this, the electronic device (100) can more accurately identify obstacles through an artificial intelligence model for obstacle detection and obtain a more optimized final area.

[0319] FIG. 8 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments of the present disclosure.

[0320] The electronic device (100) can project a test image (S810).

[0321] Next, the electronic device (100) can acquire a first region based on data corresponding to the projection surface on which the test image is projected (S820).

[0322] According to one or more embodiments, the electronic device (100) can acquire a first region based on data corresponding to a projection surface on which a test image is projected.

[0323] According to one embodiment, the electronic device (100) can acquire a first area that is smaller in size than the area on which a test image is projected among the projection surfaces, based on data corresponding to the projection surface.

[0324] Next, a second region can be obtained based on infrared data among the sensing data (S830).

[0325] According to one or more embodiments, the electronic device (100) can acquire a second region corresponding to an image projection among the projection surfaces based on infrared data acquired through infrared reflected from the projection surface among the data corresponding to the projection surface.

[0326] According to one embodiment, the electronic device (100) can identify at least one valid pixel in an infrared image obtained from infrared data corresponding to an area where a test image is projected, wherein the pixel value is greater than or equal to a second threshold value.

[0327] And the electronic device (100) can acquire a second region based on at least one effective pixel identified among the projection surfaces.

[0328] Next, the electronic device (100) can obtain a projection area based on the first area and the second area (S840).

[0329] According to one embodiment, the electronic device (100) can acquire the smaller area as a projection area if the smaller area among the first area and the second area is greater than or equal to a predetermined threshold size.

[0330] As described above, the electronic device (100) can acquire both the first region and the second region in different ways, and acquire the final region by comparing the first region and the second region. Below, regarding the operation of the electronic device (100) acquiring the final region, the operation of the surface size detector, the infrared emission detector, and the obstacle detector will be described.

[0331] FIG. 9 is a flowchart illustrating the operation of an electronic device using a surface size detector and an infrared emission detector according to one or more embodiments of the present disclosure.

[0332] The electronic device (100) may fail to calibrate (S910). For example, the electronic device (100) may fail to calibrate if distortion occurs on the projection surface. Here, distortion on the projection surface may mean that there is an obstacle or a curved surface on the projection surface, or that there is an object obstructing the projection between the electronic device (100) and the projection surface.

[0333] Next, the electronic device (100) can identify whether the available surface is smaller than the projected area (S920). For example, the electronic device (100) can identify whether the available surface is smaller than the projected area using a surface size detector.

[0334] Here, the projected area may refer to the area where the test image is projected. Here, the test image may refer to a calibration pattern. The available projection surface may refer to a projectable area. For example, a surface size detector may identify an available projection surface from multiple images (e.g., a first image and a second image) acquired from sensing data.

[0335] If the projected area matches the available projection surface or is larger than the available projection surface, the electronic device (100) can set the current scale as the output of the surface size detector (S921). Here, the current scale may mean the currently projected area, that is, the area where the current test image is projected onto the projection surface.

[0336] If the projected area is smaller than the available projection surface, the electronic device (100) can detect the maximum reduction scale for the available projection surface and set the maximum reduction scale as the output of the surface size detector (S922).

[0337] For example, the electronic device (100) can reduce the size of the test image stepwise and identify the maximum size of the test image that fits into the available projection surface. When the electronic device (100) identifies the maximum size of the test image that fits into the available projection surface, it can acquire the area on which the test image (reduced test image) is projected at the maximum reduced scale.

[0338] Next, the electronic device (100) can identify whether IR (infrared) is emitted due to an obstacle (S930). For example, the electronic device (100) can identify whether projected IR is emitted using an infrared emission detector.

[0339] Here, the case where IR emission is present may correspond to the case where infrared light is reflected by a projection surface and received by an electronic device (100). Alternatively, the electronic device (100) may also detect whether IR emission is present due to a curved surface.

[0340] If there is no IR emission due to an obstacle, the electronic device (100) can set the current scale to the output of the IR emission detector (S931).

[0341] On the other hand, if there is IR emission due to an obstacle, the electronic device (100) can detect a maximum reduction scale at which no IR emission occurs and set the maximum reduction scale as the output of the IR emission detector (S932).

[0342] For example, the electronic device (100) can gradually reduce the size of the test image and identify the maximum size test image where no IR emission occurs. Here, the absence of IR emission may mean that the number of effective pixels is less than a threshold number (e.g., a second threshold number). Here, the effective pixels and the second threshold number, etc., have been explained in detail above, so a redundant explanation will be omitted.

[0343] When the electronic device (100) identifies a test image of the maximum size where no IR emission occurs, it can acquire the area on which the test image (reduced test image) is projected at the maximum reduced scale.

[0344] Next, the electronic device (100) can identify the minimum value among the output of the surface size detector and the output of the IR emission detector (S940). For example, the electronic device (100) can identify the minimum value among the maximum reduction scale output by the surface size detector and the maximum reduction scale output by the IR emission detector.

[0345] Next, it is determined whether the minimum value is greater than or equal to a threshold value (S950), and the electronic device (100) can identify that the correction operation has failed if the minimum value is less than the threshold value (S951). Here, the correction operation may refer to the operation of adjusting the current projection area to obtain the final area.

[0346] If the minimum value is greater than or equal to the threshold value, the electronic device (100) can set the minimum value to the current screen scale (S952). The electronic device (100) can adjust the projection area based on the current screen scale and project the test image or content image onto the adjusted projection area (final area).

[0347] As described above, the electronic device (100) is described as an example of obtaining the output of a surface size detector and then obtaining the output of an IR emission detector, but is not necessarily limited thereto.

[0348] For example, the electronic device (100) may perform the process of acquiring the output of the surface size detector and the process of acquiring the output of the IR emission detector in parallel. Alternatively, the electronic device (100) may acquire the output of the IR emission detector first, and then acquire the output of the surface size detector sequentially.

[0349] Meanwhile, according to FIG. 9, the electronic device (100) can obtain a final area using a surface size detector and an IR emission detector, but can also obtain a final area using a separate module.

[0350] FIG. 10 is a flowchart illustrating the operation of an electronic device using an obstacle detector according to one or more embodiments of the present disclosure.

[0351] The electronic device (100) may fail to calibrate (S1010). Since the case of a calibration failure has been described in FIG. 9, a redundant description will be omitted.

[0352] Next, the electronic device (100) can identify whether the available projection surface is smaller than the projected area (S1020). If the projected area matches the available projection surface or is larger than the available projection surface, the electronic device (100) can set the current scale as the output of the surface size detector (S1021).

[0353] If the projected area is smaller than the available projection surface, the electronic device (100) can detect the maximum reduction scale for the available projection surface and set the maximum reduction scale as the output of the surface size detector (S1022). Since the above-described operation regarding the surface size detector has been described in FIG. 9, a redundant description will be omitted.

[0354] Next, the electronic device (100) can identify whether there is an obstacle in the projected area (S1030). For example, the electronic device (100) can identify whether there is an obstacle using an obstacle detector.

[0355] Here, the electronic device (100) can identify whether an obstacle exists and the location and size of the obstacle using a neural network model (or artificial intelligence model). Here, the neural network model may correspond to a model trained to identify obstacles on a projection surface based on sensing data obtained by sensing the projection surface.

[0356] If there are no obstacles in the projected area, the electronic device (100) can set the current scale to the output of the obstacle detector (S1031).

[0357] On the other hand, if there is an obstacle in the projected area, the electronic device (100) can detect a maximum reduction scale that avoids the obstacle and set the maximum reduction scale as the output of the obstacle detector (S1032).

[0358] For example, when an obstacle is identified by the neural network model described above, the electronic device (100) can obtain the largest area among the areas excluding the area occupied by the obstacle on the projection surface as the maximum reduction scale. Here, the maximum reduction scale may correspond to the third area described above.

[0359] When the electronic device (100) identifies a test image of the maximum size where no IR emission occurs, it can acquire the area on which the test image (reduced test image) is projected at the maximum reduced scale.

[0360] Meanwhile, the electronic device (100) can identify whether IR (infrared) is emitted due to an obstacle (S1040). If there is no IR emission due to an obstacle, the electronic device (100) can set the current scale to the output of the IR emission detector (S1041).

[0361] On the other hand, if IR emission due to an obstacle is present, the electronic device (100) can detect a maximum reduction scale at which no IR emission occurs and set the maximum reduction scale as the output of the IR emission detector (S1042). Since the above-described operation regarding the IR emission detector has been described in FIG. 9, a redundant description will be omitted.

[0362] Next, the electronic device (100) can identify the minimum value among the output of the surface size detector, the output of the obstacle detector, and the output of the IR emission detector (S1050). For example, the electronic device (100) can identify the minimum value among the maximum reduction scale output by the surface size detector, the maximum reduction scale output by the obstacle detector, and the maximum reduction scale output by the IR emission detector.

[0363] Next, it is determined whether the minimum value is greater than or equal to a threshold value (S1060), and the electronic device (100) can determine that the correction operation has failed if the minimum value is less than the threshold value (S1061). If the minimum value is greater than or equal to the threshold value, the electronic device (100) can set the minimum value to the current screen scale (S1062).

[0364] Next, the electronic device (100) can adjust the projection area based on the current screen scale and project a test image or content image onto the adjusted projection area (final area). Since the operation of the electronic device (100) identifying the final area based on a minimum value and a threshold value has been described in detail in FIG. 9, a redundant description will be omitted.

[0365] As described above, the electronic device (100) can obtain a maximum reduction scale through the surface size detector (710) and the infrared emission detector (720), as well as through the obstacle detector (730). The electronic device (100) can obtain a minimum reduction scale corresponding to the final projection area by comparing the three maximum reduction scales.

[0366] Through this, the electronic device (100) can more accurately identify obstacles through a neural network model for obstacle detection and obtain a more optimized final area. That is, by using multiple modules that output a maximum projectable area (maximum reduction scale) through different methods, the electronic device (100) can satisfy the user's need to automatically adjust the projection area to an optimal area to prevent touch errors.

[0367] Meanwhile, in FIGS. 8 to 10, the order of all steps has been mapped for convenience of explanation, but it goes without saying that the order of steps that are not related to the order or can be performed in parallel is not necessarily limited to that order.

[0368] Meanwhile, methods according to at least some of the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing electronic device.

[0369] In addition, methods according to at least some of the various embodiments of the present disclosure described above may be implemented by software upgrades or hardware upgrades alone for existing electronic devices.

[0370] In addition, methods according to at least some of the various embodiments of the present disclosure described above may also be performed through an embedded server equipped in an electronic device, or through at least one external server among the electronic devices.

[0371] Meanwhile, according to one embodiment of the present disclosure, the various embodiments described above may be implemented as software containing instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., an electronic device (100)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory storage medium" simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium. For example, a 'non-transient storage medium' may include a buffer in which data is temporarily stored. According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TMIt can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0372] Various embodiments of the present disclosure may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., an electronic device (100)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions.

[0373] When the above-described instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or an interpreter.

[0374] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, Memory that stores at least one instruction; A projection unit that projects an image onto a projection surface; At least one sensor for acquiring data corresponding to the projection surface; and It includes at least one processor; and The above-mentioned acquired data includes infrared data acquired from infrared (IR) reflected from the projection surface and data not acquired from infrared reflection, and The above at least one processor executes at least one instruction stored in the memory, Control the projection unit to project a test image onto the projection surface, and A first region on the projection surface is obtained based on data not obtained by infrared reflection, which is included in data corresponding to the projection surface on which the test image is projected, obtained through the at least one sensor while the test image is projected onto the projection surface, and Based on infrared data included in the data corresponding to the projection surface obtained through the above at least one sensor, a second area corresponding to the area of ​​the projection surface where the image is projected by the projection unit is obtained, and An electronic device that obtains a projection area on the projection plane based on the first area and the second area.

2. In Paragraph 1, The first area is an electronic device having a size smaller than the area on the projection surface, which is entirely covered by the projection of the test image, and is based on data corresponding to the projection surface and the test image is projected thereon.

3. In Paragraph 2, The above at least one processor executes the above at least one instruction stored in the memory, A cross image is obtained based on a plurality of images obtained from data corresponding to the projection surface, and Based on at least one intersection pixel of the above-mentioned intersection image, the test image is reduced into a reduced test image, and Control the projection unit to project the reduced test image onto an area on the projection surface based on at least one intersection pixel of the above-mentioned intersection image, and An electronic device that acquires the area on the projection surface on which the above-mentioned reduced test image is projected as the first area.

4. In Paragraph 3, The above at least one processor executes the above at least one instruction stored in the memory, A first image including pixel data is obtained from data corresponding to the projection surface obtained by the at least one sensor, and Based on the first image and the second image obtained based on the pixel data included in the first image, the intersection image is obtained, and If the number of corresponding to at least one cross pixel having a pixel value greater than or equal to a first threshold value in the above cross image is greater than or equal to a first threshold number, the test image is reduced to the reduced test image based on a preset first unit, and Control the projection unit to project the reduced test image onto an area on the projection surface based on at least one cross pixel, and Based on the above reduced test image, update the number corresponding to the at least one intersection pixel, and An electronic device that acquires the area on the projection surface on which the reduced test image is projected as the first area when the number corresponding to at least one updated cross pixel is less than the first threshold number.

5. In Paragraph 1, The above at least one processor executes the above at least one instruction stored in the memory, Identifying at least one valid pixel having a pixel value greater than or equal to a second threshold value in the acquired infrared image from the infrared data corresponding to the area where the above test image is projected, and An electronic device that acquires the second region on the projection plane based on at least one identified valid pixel.

6. In Paragraph 5, The above at least one processor executes the above at least one instruction stored in the memory, If the above at least one effective pixel is greater than or equal to the second threshold number, the projection unit is controlled to reduce the test image into a reduced test image based on a preset second unit, and Based on the above at least one valid pixel, the projection unit is controlled to project the reduced test image onto an area of ​​the projection surface, and Based on the above reduced test image, update the number corresponding to the at least one valid pixel, and An electronic device that, if the number corresponding to at least one updated valid pixel is less than the second threshold number, acquires the area on the projection surface on which the reduced test image is projected as the second area.

7. In Paragraph 1, The video above is a primary test video, and The above at least one processor executes the above at least one instruction stored in the memory, Before projecting the above basic test image, the projection unit is controlled to project an initial test image including an infrared pattern, and If the function related to correction based on the above infrared pattern is not performed, the projection unit is controlled to project the above basic test image including the RGB pattern, and An electronic device that acquires the second region based on the infrared data included in the data corresponding to the projection surface obtained through the projection surface on which the above initial test image is projected.

8. In Paragraph 1, The above at least one processor executes the above at least one instruction stored in the memory, An electronic device that acquires the small-sized area as the projection area if the small-sized area included in the first area and included in the second area is greater than or equal to a predetermined threshold size.

9. In Paragraph 1, Including a moving part; further The above at least one processor executes the above at least one instruction stored in the memory, An electronic device that controls the moving part so that the electronic device moves from a current position corresponding to the projection of the test image to a second position corresponding to the image projection in the projection area.

10. In Paragraph 1, The above at least one processor executes the above at least one instruction stored in the memory, Using an artificial intelligence model, at least one obstacle is identified based on data corresponding to the projection surface, and A third region on the projection surface that is included in the region excluding the region corresponding to the obstacle, and An electronic device that obtains the projection area based on the first area, the second area, and the third area.

11. A method for controlling an electronic device comprising a projection unit that projects an image onto a projection surface, and at least one sensor that acquires data corresponding to the projection surface, the sensor comprising infrared data acquired from infrared reflected from the projection surface and data not acquired from infrared reflection, wherein A step of controlling the projection unit to project a test image onto the projection surface; A step of acquiring a first region on the projection surface based on data not acquired by infrared reflection, which is included in data corresponding to the projection surface on which the test image is projected, acquired through the at least one sensor while the test image is projected onto the projection surface; A step of acquiring a second region corresponding to the region of the projection surface where the image is projected by the projection unit, based on infrared data included in the data corresponding to the projection surface acquired through the at least one sensor; and A control method comprising the step of obtaining a projection area on the projection plane based on the first area and the second area.

12. In Paragraph 11, The step of acquiring the first region based on data not acquired by infrared reflection, which is included in data corresponding to the projection surface on which the test image is projected, acquired through the at least one sensor while the test image is projected onto the projection surface, is: A control method comprising the step of obtaining a first area having a size smaller than the area on the projection surface, which is entirely covered by the projection of the test image, wherein the test image is projected.

13. In Paragraph 12, The step of obtaining the first region, which is smaller in size than the area on the projection surface and is entirely covered by the projection of the test image, wherein the test image is projected, is A step of obtaining an intersection image based on a plurality of images obtained from data corresponding to the projection surface; A step of reducing the test image into a reduced test image based on at least one intersection pixel of the above-mentioned intersection image; A step of controlling the projection unit to project the reduced test image onto an area on the projection surface based on at least one intersection pixel of the above-mentioned intersection image; and A control method comprising the step of acquiring the area on the projection surface on which the reduced test image is projected as the first area.

14. In Paragraph 13, The step of acquiring an intersection image based on a plurality of images acquired based on data corresponding to the projection surface above is: A first image including pixel data is obtained from data corresponding to the projection surface obtained by the at least one sensor, and The intersection image is obtained based on the first image and the second image obtained based on the pixel data included in the first image, and The step of reducing the test image into a reduced test image based on at least one intersection pixel of the above-mentioned intersection image is: If the number of corresponding to at least one intersection pixel having a pixel value greater than or equal to a first threshold value in the above intersection image is greater than or equal to a first threshold number, the step of reducing the test image into the reduced test image based on a preset first unit, and acquiring the area on the projection surface on which the reduced test image is projected as the first area, Based on the above reduced test image, update the number corresponding to the at least one intersection pixel, and A control method for acquiring the area on the projection surface on which the reduced test image is projected as the first area when the number of corresponding to at least one updated cross pixel is less than the first threshold number.

15. A non-transient computer-readable recording medium storing computer instructions that cause an electronic device to perform an operation when executed by a processor of an electronic device comprising: a projection unit for projecting an image onto a projection surface; and at least one sensor for acquiring data corresponding to said projection surface, the sensor including infrared data acquired from infrared reflected from said projection surface and data not acquired from infrared reflection, wherein the operation is: A step of controlling the projection unit to project a test image onto the projection surface; A step of acquiring a first region on the projection surface based on data not acquired by infrared reflection, which is included in data corresponding to the projection surface on which the test image is projected, acquired through the at least one sensor while the test image is projected onto the projection surface; A step of acquiring a second region corresponding to image projection among the regions of the projection surface where an image is projected by the projection unit, based on infrared data included in the data corresponding to the projection surface acquired through the at least one sensor; and A non-transient computer-readable recording medium comprising the step of obtaining a projection area on the projection surface based on the first area and the second area.

Images