Electronic device and control method therefor

Abstract

This electronic device comprises: a distance sensor for measuring a distance to a wall surface; a depth sensor; a driving device including a motor; a memory for storing at least one instruction; and at least one processor including an image processing circuit. The at least one processor individually and / or collectively executes the at least one instruction to cause the electronic device to: acquire first map data on the basis of information about the distance to the wall surface acquired by the distance sensor; control, on the basis of the first map data, the driving device to move to a position corresponding to a scan pose pertaining to a reference point corresponding to scan data collection; and acquire second map data on the basis of scan data acquired by the depth sensor at the position corresponding to the scan pose.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device for acquiring map data and a method for controlling the same.

[0002] Thanks to advancements in electronic technology, various types of electronic devices are being used in daily life. Among these devices may be those that acquire map data.

[0003] For example, there may be an electronic device that generates map data based on distance information or images acquired by a sensor while driving through space.

[0004] Map data refers to data that represents structural information about a specific space or environment.

[0005] An electronic device according to an exemplary embodiment of the present disclosure comprises a distance sensor for measuring a distance to a wall surface, a depth sensor, a driving device including a motor, a memory for storing at least one instruction, and at least one processor including a processing circuit, wherein the at least one processor executes the at least one instruction individually and / or collectively, thereby causing the electronic device to acquire first map data based on distance information to a wall surface acquired by the distance sensor, control the driving device to move to a position corresponding to a scan pose regarding a reference point corresponding to scan data collection based on the first map data, and acquire second map data based on scan data acquired by the depth sensor at the position corresponding to the scan pose.

[0006] The above at least one processor can execute the above at least one instruction individually and / or collectively, thereby enabling the electronic device to acquire data corresponding to a first map including distance information to a plurality of walls based on the first map data, and to acquire a position corresponding to a plurality of scan poses based on the distance information to the plurality of walls.

[0007] The above at least one processor can execute the above at least one instruction individually and / or collectively, thereby enabling the electronic device to acquire a plurality of positions corresponding to a specific value as positions corresponding to the plurality of scan poses when the distance from the plurality of walls exceeds a specific value.

[0008] The above at least one processor can execute the above at least one instruction individually and / or collectively, thereby enabling the electronic device to acquire data corresponding to a first map including a plurality of grids based on the first map data, and to acquire the center of each of the plurality of grids as a position corresponding to a scan pose.

[0009] The above at least one processor can individually and / or collectively execute the above at least one instruction to enable the electronic device to acquire a scan rotation angle based on the Field of View (FoV) of the depth sensor and to control the driving device to rotate based on the scan rotation angle so as to acquire scan data by the depth sensor.

[0010] The above at least one processor can individually and / or collectively execute the above at least one instruction to enable the electronic device to obtain spatial data from the second map data and to obtain data corresponding to the second map containing the spatial data.

[0011] The above at least one processor can execute the above at least one instruction individually and / or collectively, thereby enabling the electronic device to acquire spatial data from the second map data, and the spatial data may include at least one of environmental information of the space, zone information of the space, and object information within the space.

[0012] The electronic device further includes an input interface comprising a circuit, and the at least one processor can execute the at least one instruction individually and / or collectively, thereby enabling the electronic device to acquire spatial data from the second map data and, based on an input corresponding to the spatial data received through the input interface, perform an operation corresponding to the input.

[0013] The electronic device further includes a camera, and the at least one processor can enable the electronic device to acquire spatial data based on the captured image acquired through the camera and the second map data by executing the at least one instruction individually and / or collectively.

[0014] A control method for an electronic device according to an exemplary embodiment of the present disclosure includes the steps of: acquiring first map data based on distance information from a wall surface acquired by a distance sensor; moving to a position corresponding to a scan pose with respect to a reference point corresponding to scan data collection based on the first map data; and acquiring second map data based on scan data acquired by a depth sensor at a position corresponding to the scan pose.

[0015] The above control method may further include the step of acquiring data corresponding to a first map including distance information with a plurality of walls based on the first map data, and the step of acquiring a position corresponding to a plurality of scan poses based on the distance information with the plurality of walls.

[0016] The step of acquiring the plurality of scan poses may include, when the distance from the plurality of walls exceeds a specific value, the step of acquiring a plurality of positions corresponding to the specific value as positions corresponding to the plurality of scan poses.

[0017] The above control method may further include the step of acquiring data corresponding to a first map including a plurality of grids based on the first map data, and the step of acquiring the center of each of the plurality of grids as a position corresponding to a scan pose.

[0018] The method may further include the step of acquiring a scan rotation angle based on the Field of View (FoV) of the depth sensor and the step of acquiring scan data by the depth sensor by rotating based on the scan rotation angle.

[0019] The above control method may further include the step of obtaining spatial data from the second map data and the step of obtaining data corresponding to the second map containing the spatial data.

[0020] The above control method further includes the step of obtaining spatial data from the second map data, and the spatial data may include at least one of environmental information of the space, zone information of the space, and object information within the space.

[0021] The above control method may further include the steps of acquiring spatial data from the second map data, receiving an input corresponding to the spatial data, and performing an operation corresponding to the user input based on the input.

[0022] The above control method may further include the step of acquiring spatial data based on the captured image acquired through the camera and the second map data.

[0023] A non-transient readable recording medium comprising a program for executing a method of controlling an electronic device according to an exemplary embodiment of the present disclosure, wherein the method of controlling the electronic device comprises the steps of: acquiring first map data based on distance information from a wall surface acquired by a distance sensor; moving to a position corresponding to a scan pose with respect to a reference point corresponding to scan data collection based on the first map data; and acquiring second map data based on scan data acquired by a depth sensor at a position corresponding to the scan pose.

[0024] The above method may further include the step of acquiring data corresponding to a first map including distance information to a plurality of walls based on the first map data, and the step of acquiring a position corresponding to a plurality of scan poses based on the distance information to the plurality of walls.

[0025] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following detailed description taken together with the accompanying drawings:

[0026] FIG. 1 is a diagram for schematically illustrating the exemplary operation of an electronic device according to various embodiments.

[0027] FIG. 2 is a perspective view illustrating an exemplary form of an electronic device according to various embodiments.

[0028] FIG. 3 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0029] FIG. 4 is a detailed block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0030] FIG. 5 is a diagram illustrating an exemplary operation in which an electronic device according to various embodiments acquires second map data.

[0031] FIG. 6 is a drawing for illustrating an exemplary first map UI according to various embodiments.

[0032] FIG. 7 is a drawing for illustrating an exemplary first map UI according to various embodiments.

[0033] FIG. 8 is a diagram illustrating an exemplary operation in which an electronic device according to various embodiments acquires second map data.

[0034] FIG. 9 is a drawing illustrating an exemplary operation in which an electronic device according to various embodiments acquires a changed scan pose.

[0035] FIG. 10 is a flowchart illustrating an exemplary operation of an electronic device scanning space according to various embodiments.

[0036] FIG. 11 is a drawing illustrating an exemplary operation in which an electronic device according to various embodiments acquires a scan rotation angle.

[0037] FIG. 12 is a diagram illustrating an exemplary operation in which an electronic device according to various embodiments scans space by means of a distance sensor and a depth sensor.

[0038] FIG. 13 is a flowchart illustrating an exemplary operation in which an electronic device according to various embodiments acquires second map data.

[0039] The terms used in the various embodiments of this Disclosure have been selected to be as widely used and general as possible, taking into account their functions within this Disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant description section of this Disclosure. Therefore, terms used in this Disclosure should be defined not merely by their names, but based on their meanings and the overall content of this Disclosure.

[0040] The various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments.

[0041] In relation to the description of the drawings, similar reference numerals may be used for similar or related components.

[0042] The singular form of the noun corresponding to the item may include one or multiple items, unless the relevant context clearly indicates otherwise.

[0043] In the present disclosure, each of the phrases such as “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “at least one of A, B, or C” may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0044] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order).

[0045] Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that the component may be connected to the other component directly (e.g., via a wire), wirelessly, or through a third component.

[0046] Terms such as "include" or "have" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this disclosure, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0047] When it is said that a component is "connected," "combined," "supported," or "in contact" with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0048] When it is said that a component is located "on" another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0049] The term "and / or" includes a combination of multiple related described components or any of the multiple related described components.

[0050] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor, except for a "module" or "part" that needs to be implemented in specific hardware.

[0051] Various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present disclosure is not limited by the relative sizes or spacing depicted in the attached drawings.

[0052] In the present disclosure, the term "user" may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0053] Various embodiments are described in more detail below with reference to the attached drawings.

[0054] FIG. 1 is a drawing for illustrating the exemplary operation of an electronic device according to various embodiments.

[0055] Referring to FIG. 1, the electronic device (100) can acquire map data while driving through space. "Map data" may refer to data representing structural information about a specific space or environment.

[0056] For example, the electronic device (100) can acquire first map data based on distance information from a wall surface acquired by a distance sensor, and acquire second map data based on scan data acquired by a depth sensor.

[0057] A map may include a picture that represents the state of space on a plane by reducing it by a certain ratio.

[0058] According to embodiments of the present disclosure, the map may include a drawing in which the planar structure of the interior of a house is reduced by a certain ratio and represented by a predetermined symbol.

[0059] For example, the map may include a drawing that represents the floor plan of the interior of the house using lines. However, it is not limited to this, and the map may also include the locations of major objects inside the house.

[0060] The electronic device (100) can acquire a position corresponding to a scan pose with respect to a reference point corresponding to scan data collection based on the first map data, and can drive to a position corresponding to the scan pose to acquire scan data.

[0061] "Driving" may include the action of an object moving using power.

[0062] According to embodiments of the present disclosure, driving may include the operation of an object moving in any direction through power. Alternatively, driving may include the operation of an object moving along a preset track through power.

[0063] For example, driving may include the action of changing the position during the process in which the electronic device acquires a captured image of space.

[0064] Here, "Scan Pose" refers to a reference point for collecting scan data about a space, and can mean the optimal position and orientation that an electronic device must assume to scan a specific area or object.

[0065] The electronic device (100) may drive to a position corresponding to the scan pose, then rotate in place at the position corresponding to the scan pose, acquire scan data by a depth sensor, and acquire second map data based on the acquired scan data.

[0066] When the electronic device (100) acquires data corresponding to a first map that includes distance information to a plurality of walls based on the first map data, it can acquire a position corresponding to a plurality of scan poses based on the distance information to a plurality of walls.

[0067] For example, data corresponding to the first map may correspond to the first map UI (User Interface). The first map UI (and the second map UI), etc., will be explained in detail in the following section.

[0068] For example, the electronic device (100) can obtain a position corresponding to an optimal scan pose for scanning space by calculating the distance to a plurality of walls displayed on a first map UI (10) surrounding the electronic device (100).

[0069] When the electronic device (100) acquires second map data based on scan data acquired by a depth sensor at a position corresponding to a scan pose, it can acquire spatial data from the second map data.

[0070] "Spatial data" refers to data related to geographical information and may mean data for expressing locations and attributes in physical or virtual space. For example, spatial data may include at least one of environmental information of the space, zone information of the space, and object information within the space.

[0071] The electronic device (100) can acquire data corresponding to a second map containing spatial data based on scan data. For example, the data corresponding to the second map may correspond to a second map UI (20).

[0072] "The first map UI" may refer to a map UI that visually represents a two-dimensional space, and "the second map UI" may refer to a map UI that visually represents a three-dimensional space.

[0073] In FIG. 1, the electronic device (100) is shown in the form of a portable projector, but is not limited thereto, and the electronic device (100) can be implemented as various types of electronic devices such as a portable electronic device or an electronic device that can be carried by a user.

[0074] A projector refers to a device that displays images by projecting light containing images onto an external screen or wall, and a portable projector refers to a projector that can be moved using wheels, motors, etc.

[0075] Hereinafter, the operation of an electronic device (100) according to various embodiments of the present disclosure will be described.

[0076] FIG. 2 is a perspective view illustrating an exemplary form of an electronic device according to various embodiments.

[0077] Referring to FIG. 2, a movable electronic device (100) is illustrated. This is merely an example of a disclosed form and may have a shape different from the illustrated form.

[0078] The electronic device (100) can obtain data (e.g., distance information, images, etc.) obtained by scanning the space through various sensors.

[0079] An image according to one example may include a captured image acquired using a sensor (e.g., a camera) equipped in an electronic device, an input image received from an external device through a communication unit (e.g., a communication unit including a communication circuit), a graphic image generated by an electronic device, etc.

[0080] Images according to one example may include, depending on the aspect ratio, horizontal images where the width is longer than the height (e.g., landscape images, horizontal images), vertical images where the height is longer than the width (e.g., portrait images, vertical images), etc. For example, horizontal images may include images with a 16:9 aspect ratio, and vertical images may include images with a 9:16 aspect ratio. Specific numbers are examples for convenience of explanation and are not limited thereto.

[0081] An image according to one example may include various resolutions depending on the number of pixels constituting the image (the product of the number of pixels in the horizontal direction and the number of pixels in the vertical direction). For example, depending on the resolution, the image may include high-resolution images (FHD (1920Y1080), 8K (7680Y4320), etc.) and low-resolution images (640Y480, etc.).

[0082] A shooting according to one example of the present disclosure may include the operation of an electronic device that controls a camera (e.g., a camera including an image sensor and a lens) equipped in an electronic device to convert an optical image formed through a lens into an electrical signal and acquire an image.

[0083] The electronic device (100) can travel through space following a plurality of scan poses obtained through the driving device (130).

[0084] For example, the electronic device (100) can travel while scanning the space in order of the closest among the multiple locations to acquire data scanned in the space at multiple locations corresponding to multiple scan poses.

[0085] The electronic device (100) includes a driving device (130) for movement (e.g., a motor, a wheel) and can perform movement using the provided driving device. In the illustrated example, the electronic device (100) is shown and described as being moved using a wheel, but in implementation, other means in the form of a caterpillar may be used, and if the electronic device (100) is implemented as a drone, it may be possible to equip a propeller instead of a wheel.

[0086] In the illustrated example, the electronic device (100) is shown as having a driving device (130) directly, but the driving device (130) may be a separate device. For example, the electronic device (100) may be combined with a movable device such as a robot vacuum cleaner, and may be mounted on the robot vacuum cleaner to control the movement of the robot vacuum cleaner.

[0087] The form of the electronic device shown is merely an example and can be implemented in various forms. Specific configurations constituting the electronic device (100) will be described later with reference to FIG. 3.

[0088] FIG. 3 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0089] Referring to FIG. 3, the electronic device (100) includes a distance sensor (110), a depth sensor, a driving device (e.g., a driving device including a motor) (130), a memory (140), and a processor (e.g., a processor including a processing circuit) (150).

[0090] According to one embodiment, the distance sensor (110) is configured to measure the distance to various obstacles, such as a wall. For example, the electronic device (100) can measure the distance between the electronic device (100) and obstacles around the electronic device (100) based on the sensing value of the distance sensor (110).

[0091] The electronic device (100) may obtain distance information between multiple walls based on distance information between the wall and the electronic device (100). For example, when the electronic device (100) obtains distance information between multiple walls, it may generate data (e.g., a map UI) representing the distance information between the walls. This is explained in detail in FIG. 6.

[0092] The distance sensor (110) may include a 3D-ToF sensor, a ToF camera sensor, and a PIR sensor, but is not limited thereto, and may include various sensors capable of measuring the distance to an obstacle.

[0093] When the distance sensor (110) acquires a sensing value for an obstacle, at least one processor (150) can acquire distance information to the obstacle through the distance sensor (110).

[0094] The depth sensor (120) is configured to acquire data including distance information to various obstacles, such as walls. Here, the depth sensor (120) may include a 3D depth sensor.

[0095] For example, when the depth sensor (120) acquires a plurality of images containing depth information, at least one processor (150) can acquire three-dimensional data about space and obstacles through the depth sensor (120).

[0096] In this case, the electronic device (100) may rotate in place and rotate 360 ​​degrees to acquire three-dimensional data about space and obstacles.

[0097] According to one embodiment, the driving device (130) is configured to move the electronic device (100). To this end, the driving device (130) includes a motor, wheels, etc., and can move the electronic device (100) through the movement of the wheels.

[0098] The driving device (130) can adjust the scanning direction of the electronic device (100). For example, the camera may adjust the direction it looks by adjusting the body position of the electronic device (100) as shown in FIG. 2, or the scanning direction may be adjusted by adjusting the lens position within the camera.

[0099] Here, the scan direction refers to the direction in which obstacles are scanned, and may also be referred to as the direction the electronic device faces, the projection direction, etc.

[0100] According to an embodiment, the memory (140) may store various programs, data, instructions, etc. used in the electronic device (100). In addition, the memory (140) may store various information according to various embodiments of the present disclosure.

[0101] Depending on the purpose of data storage, the memory (140) may be implemented in the form of a memory embedded in the electronic device (100) or in the form of a memory that can be attached to and detached from the electronic device (100).

[0102] For example, data for driving the electronic device (100) may be stored in memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in memory that is detachable from the electronic device (100).

[0103] In the case of memory embedded in an electronic device (100), it may be implemented in the form of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash, etc.), hard drive, or solid state drive (SSD).

[0104] In the case of a memory that can be attached to and detached from an electronic device (100), it can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.) or an external memory that can be connected to a USB port (e.g., USB memory).

[0105] The memory (140) may include various instructions required for the operation of the processor (150). The instructions may include an instruction to obtain distance information from a wall surface by a distance sensor, an instruction to obtain a position corresponding to a scan pose based on first map data, an instruction to drive to a position corresponding to a scan pose, and an instruction to obtain second map data based on scan data obtained by a depth sensor at a position corresponding to a scan pose.

[0106] The memory (140) can store distance information from the wall surface obtained by the distance sensor (110) or scan data obtained by the depth sensor (120).

[0107] Here, "distance information from the wall surface" may refer to the distance value between the wall surface in the space where the electronic device (100) is located and the electronic device (100).

[0108] "Scan data" may refer to data obtained by scanning a wall or obstacle identified while the electronic device (100) rotates and moves at a position corresponding to the scan pose.

[0109] The memory (140) can store the first map data, the first map UI, the second map data, or the second map UI.

[0110] "First map data" may refer to two-dimensional data acquired by a distance sensor. For example, the first map data may include data expressing spatial information, such as the location, size, and shape of obstacles or walls, on the X and Y axes.

[0111] "Second map data" may refer to three-dimensional data acquired by a depth sensor. For example, the second map data may include data expressing spatial information, such as the location, size, and shape of obstacles or walls, along the X, Y, and Z axes.

[0112] According to an embodiment, at least one processor (150) includes various processing circuits and controls the overall operation of the electronic device (100). Specifically, at least one processor (150) is connected to each component of the electronic device (100) to control the overall operation of the electronic device (100).

[0113] At least one processor (150) can perform the operation of an electronic device (100) according to various embodiments by executing at least one instruction stored in memory.

[0114] According to an embodiment, at least one processor (150) may be implemented as a digital signal processor (DSP) that processes digital signals, a microprocessor, or a TCON (Timing controller). However, it is not limited thereto and may include or be defined by one or more of a central processing unit (CPU), a Micro Controller Unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, or an AI (Artificial Intelligence) processor. Additionally, at least one processor (150) may be implemented as a System on Chip (SoC) or Large Scale Integration (LSI) with a built-in processing algorithm, or may be implemented in the form of a Field Programmable Gate Array (FPGA). At least one processor (150) can perform various functions by executing computer executable instructions stored in memory.

[0115] At least one processor (150) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, and / or machine learning accelerator. Accordingly, the processor (150) may include various processing circuits and / or multiple processors. For example, as used in this specification and claims, the term “processor” may include various processing circuits including at least one processor, and one or more of the at least one processor may be configured to perform various functions described in this specification, either individually or collectively in a distributed manner. Where in this specification “processor,” “at least one processor,” or “one or more processors” is referred to as being configured to perform various functions, such terms include, by example and without limitation, cases where one processor performs some functions and another processor performs the remaining functions, as well as cases where a single processor performs all functions. Additionally, the at least one processor may be a combination of multiple processors that perform various functions in a distributed manner. The at least one processor may execute program instructions to perform or realize various functions.

[0116] At least one processor (150) can acquire first map data based on distance information from the wall surface acquired by the distance sensor (110).

[0117] For example, at least one processor (150) can receive a value in which a distance sensor (110) senses a wall or obstacle to obtain distance information to the wall or obstacle and obtain first map data including distance information to the wall or obstacle.

[0118] At least one processor (150) can control the driving device (130) to move to a position corresponding to a scan pose with respect to a reference point corresponding to scan data collection based on the first map data.

[0119] "Scan pose" refers to a reference point corresponding to the collection of scan data for a space, and may mean the optimal posture that an electronic device must assume to scan a specific area or object from that reference point.

[0120] The position corresponding to the scan pose may refer to the position of the reference point corresponding to the acquisition of scan data. This reference point may refer to the optimal point for acquiring scan data.

[0121] For example, to obtain map data for a specific area (e.g., a living room) among a plurality of areas, at least one processor (150) may scan the projection surface detected in the specific area and obtain a central point based on the plurality of projection surfaces as a reference point.

[0122] However, it is not limited thereto, and at least one processor (150) may acquire a scan pose (which may include a reference point or the location of a reference point corresponding to the scan data collection) based on a wall distance or a grid. This is explained in detail in FIGS. 5 and FIGS. 8.

[0123] At least one processor (150) can acquire second map data based on scan data acquired by the depth sensor (120) at a position corresponding to the scan pose.

[0124] For example, at least one processor (150) can control the driving device (130) to rotate in place at a position corresponding to the scan pose, thereby acquiring scan data for a plurality of projection surfaces through the depth sensor (120).

[0125] At least one processor (150) can acquire data corresponding to a first map including distance information to a plurality of walls based on the first map data. At least one processor (150) can also acquire a position corresponding to a plurality of scan poses based on distance information to a plurality of walls.

[0126] At least one processor (150) can travel through a plurality of scan poses as a travel path and acquire scan data for a plurality of projection surfaces through a depth sensor (120). This is explained in detail in FIG. 6.

[0127] At least one processor (150) can control one or any combination of other components of the electronic device and can perform operations or data processing related to communication. At least one processor (150) can execute one or more programs or instructions stored in memory. For example, at least one processor (150) can perform the method according to an embodiment of the present disclosure by executing one or more instructions stored in memory.

[0128] If the method according to the embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors.

[0129] For example, when the first operation, the second operation, and the third operation are performed by the method according to the embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence dedicated processor).

[0130] One or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory (140). If one or more processors are dedicated artificial intelligence processors, the dedicated artificial intelligence processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model. The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0131] Being created through learning means that a basic artificial intelligence model is trained using multiple training data by a learning algorithm, thereby creating a predefined rule of operation or an artificial intelligence model configured to perform a desired characteristic (or objective). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0132] An artificial intelligence model can be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated during the learning process so that the loss or cost values ​​obtained by the artificial intelligence model are reduced or minimized.

[0133] Artificial neural networks may include deep neural networks (DNNs), such as, but are not limited to, Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), Generative Adversarial Networks (GANs), or Deep Q-Networks.

[0134] At least one processor (150) may be implemented as a single core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore).

[0135] When at least one processor (150) is implemented as a multi-core processor, each of the multiple cores included in the multi-core processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by multiple cores may be included in the multi-core processor.

[0136] Each of the multiple cores (or some of the multiple cores) included in the multi-core processor may independently read and execute program instructions for implementing the method according to the embodiment of the present disclosure, or all (or some of) of the multiple cores may be linked together to read and execute program instructions for implementing the method according to the embodiment of the present disclosure.

[0137] FIG. 4 is a detailed block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0138] Referring to FIG. 4, an electronic device (100) according to one embodiment of the present disclosure may include a distance sensor (110), a depth sensor (120), a driving device (e.g., a driving device including a motor) (130), a memory (140), at least one processor (e.g., a processor including a processing circuit) (150), an input interface (e.g., an input interface including a circuit) (160), or a camera (170). In the following, redundant descriptions of the contents described above may be omitted or summarized.

[0139] The input interface (160) can receive various feedback from the user. For example, the electronic device (100) can receive input corresponding to spatial data (e.g., user input) through the input interface (160). The user input may correspond to an input containing spatial data. That is, the user input may include content instructing to perform a specific command in a specific space.

[0140] The electronic device (100) can perform an action corresponding to the user input based on the received user input.

[0141] For example, when the electronic device (100) receives user input for a setting that controls each configuration of the electronic device (100) through an input interface, it can operate each configuration in response to the user input.

[0142] For example, when the input interface (160) receives user input containing spatial data, at least one processor (150) can perform an operation corresponding to the user input.

[0143] For example, when a user says "come to the bedroom," at least one processor (150) can receive user input through a microphone and control the driving device (130) to move to the bedroom.

[0144] The input interface (160) may include a microphone, a touchscreen, etc., but is not limited thereto, and may include various input interfaces capable of receiving user input.

[0145] A microphone is a component for receiving sound input and converting it into an audio signal. The microphone is electrically connected to at least one processor (150) and can receive sound under the control of at least one processor (150).

[0146] For example, the microphone may be formed as an integrated unit on the upper side, front side, or other directions of the electronic device (100). Alternatively, the microphone may be provided in a remote control or the like, separate from the electronic device (100). In this case, the remote control may receive sound through the microphone and provide the received sound to the electronic device (100).

[0147] The microphone may include circuits included in various configurations, such as a microphone that collects analog sound, an amplifier circuit that amplifies the collected sound, an A / D converter circuit that samples the amplified sound and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0148] The microphone may be implemented in the form of a sound sensor, and any configuration capable of collecting sound is acceptable.

[0149] According to one embodiment, a camera (170) (e.g., an RGB camera) is configured to capture an object and obtain a color image.

[0150] "Color" may refer to a combination of red, green, and blue colors. "Image" may include static or dynamic images.

[0151] A camera (170) may be provided in at least one of the top area, bottom area, and side area of ​​the electronic device (100) to photograph a space or an obstacle. The camera (170) may be implemented as a single camera, and may also be implemented as a plurality of cameras depending on the embodiment.

[0152] The camera (170) can provide a captured image to at least one processor (150) to acquire map data within the image.

[0153] The camera (170) may be implemented as a wide-angle camera to capture a wide field of view, but is not limited thereto.

[0154] According to one example of the present disclosure, "shooting" may include the operation of an electronic device that controls a camera (e.g., a camera including an image sensor and a lens) equipped in an electronic device to convert an optical image formed through a lens into an electrical signal and acquire an image.

[0155] The camera (170) includes a lens, a shutter, an aperture, a solid-state image sensor, an AFE (Analog Front End), and a TG (Timing Generator). The shutter controls the time when light reflected from a subject enters the camera, and the aperture controls the amount of light incident on the lens by mechanically increasing or decreasing the size of the opening through which light enters. When light reflected from a subject accumulates as photocharges, the solid-state image sensor outputs an image based on the photocharges as an electrical signal. The TG outputs a timing signal for reading out pixel data from the solid-state image sensor, and the AFE samples and digitizes the electrical signal output from the solid-state image sensor.

[0156] When the camera (170) acquires a captured image, at least one processor (150) may acquire spatial data based on the acquired captured image and the second map data.

[0157] For example, at least one processor (150) can combine the second map data and the captured image obtained through the camera (170) to obtain more diverse spatial data (e.g., a 3D map including color information).

[0158] Although an electronic device (100) including various additional configurations is illustrated in FIG. 4, some of the illustrated configurations may be implemented in an omitted form. Additionally, other configurations not illustrated may be included.

[0159] For example, the electronic device (100) may further include a communication unit.

[0160] For example, the communication unit may include various communication circuits and can perform data communication between an external device and an electronic device using at least one of data communication methods including wired LAN, wireless LAN, Wi-Fi, Wi-Fi Direct, Bluetooth, ZigBee, WFD (Wi-Fi Direct), infrared communication (IrDA, infrared Data Association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), Wibro (Wireless Broadband Internet), WiMAX (World Interoperability for Microwave Access), SWAP (Shared Wireless Access Protocol), WiGig (Wireless Gigabit Alliances), and RF communication.

[0161] In addition to the communication method described above, the communication unit can perform communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

[0162] At least one processor (150) can receive a signal requesting data to obtain second map data from a server, etc. through a communication unit.

[0163] At least one processor (150) can transmit a signal requesting second map data, etc. to a server, etc. through a communication unit. For example, based on the received second map data, etc., at least one processor (150) can receive user input corresponding to a specific location among the second map data and perform an operation corresponding to the user input.

[0164] At least one processor (150) may receive a control signal to move to a position corresponding to a scan pose based on the first map data from a server device, etc., or a control signal to acquire a second projection area. However, it is not limited thereto.

[0165] The electronic device (100) may further include interfaces such as an HDMI port, DP, RGB, DVI, USB, and Thunderbolt for receiving video / audio signals connected to external devices or servers. HDMI, DP, and Thunderbolt are ports capable of simultaneously transmitting video and audio signals. The electronic device (100) can output distance information and second map data by performing various processing such as demuxing, decoding, and scaling on various signals received from external devices, servers communicating with external devices, etc., through the communication unit and these various interfaces.

[0166] The electronic device (100) may include a display.

[0167] The display is a component for displaying the operating status, notification messages, UI screens, etc. of an electronic device (100). The display can be implemented in various forms such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, or a PDP (Plasma Display Panel). The display may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as an a-si TFT (amorphous silicon thin film transistor), an LTPS (low temperature poly silicon) TFT, or an OTFT (organic TFT). The display can be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, or a three-dimensional display. The display may also be implemented with only one or more light-emitting elements.

[0168] The electronic device (100) can change the display state of the display according to various states, such as when the electronic device (100) is turned on, when it is operating normally, when there is insufficient power, or when there is an error, so that the user can intuitively understand the state of the electronic device (100).

[0169] For example, at least one processor (150) can control a display to display a UI corresponding to the acquired second map data. The UI corresponding to the second map data may correspond to a graphic element that visually represents the second map data in the form of a 2D or 3D map.

[0170] However, this is not limited thereto, and at least one processor (150) can control the display to display a UI corresponding to the first map data before acquiring the second map data.

[0171] However, the display configuration is merely one of the various embodiments, and the display configuration may be omitted. For example, the electronic device (100) may be a device that directly has a display or a device connected to an external device.

[0172] For example, if the electronic device (100) is implemented as a set-top box, one-connect box, projector, etc., the operations of the electronic device (100) described above may be performed in an electronic device that does not include a display.

[0173] In this case, at least one processor (150) may obtain a UI, etc. based on the second map data and provide the obtained UI to an external display device (a device equipped with a display) through a communication unit. The external display device may display the provided UI through a display equipped on the external display device.

[0174] The electronic device (100) may include a microphone. Here, the microphone may correspond to a configuration included in the input interface (160) described above.

[0175] The microphone can receive the user's voice when activated. For example, the microphone may be formed integrally on the upper side, front side, or side side of the electronic device (100).

[0176] The microphone may include various components such as a microphone that collects user voice in an analog form, an amplifier circuit that amplifies the collected user voice, an A / D conversion circuit that samples the amplified user voice and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0177] The microphone can transmit the received user voice to the electronic device (100). The electronic device (100) can input the received user voice into a speech recognition model to perform speech recognition. For example, the electronic device (100) can perform speech recognition on the user voice by performing STT (Speech to Text) on the user voice. The microphone receives the user voice and can transmit the received user voice to the electronic device (100). Subsequently, the electronic device (100) can perform speech recognition by inputting the received user voice into a speech recognition model. For example, the electronic device (100) can perform speech recognition on the user voice by performing STT (Speech to Text) on the user voice.

[0178] For example, at least one processor (150) can receive user input through a microphone. The user input may correspond to user input that corresponds to spatial data. Here, spatial data, etc. will be described in detail in the following section.

[0179] Hereinafter, the exemplary operation of the electronic device (100) is described in more detail through FIGS. 5 to 13. FIGS. 5 to 13 describe various embodiments for the convenience of explanation. However, the various embodiments of FIGS. 5 to 13 may be implemented in any combination.

[0180] FIG. 5 is a diagram illustrating an exemplary operation in which an electronic device according to various embodiments acquires second map data.

[0181] Referring to FIG. 5, the electronic device (100) can generate a 2D map UI (10) by acquiring 2D map data through mapping while driving through space.

[0182] "Mapping" can refer to the process of transforming or interconnecting a system, data, space, concept, etc., into a different format or structure. For example, mapping may include geographic mapping, which visually represents specific locations, routes, or spatial data.

[0183] Referring to FIG. 5, the electronic device (100) can obtain a position corresponding to a scan pose regarding a reference point (11) corresponding to scan data collection based on the first map data. For example, if the electronic device (100) extracts distance information with a plurality of walls through the first map data, it can obtain the most central point from the plurality of walls as the position of the reference point (11).

[0184] Referring to FIG. 5, the electronic device (100) can move to the position of a reference point (11) to acquire scan data for a plurality of walls.

[0185] The electronic device (100) can collect scan data for a plurality of projection surfaces (12) while rotating in place at the position of a reference point (11) (a reference point corresponding to the scan pose).

[0186] When the electronic device (100) completes the collection of scan data for multiple walls at a reference point (11), it can repeatedly perform the operation of collecting scan data for multiple walls while moving to the nearest position and rotating in place until the collection of data at all positions (positions corresponding to the scan pose) is completed.

[0187] FIG. 6 is a drawing for illustrating an exemplary first map UI according to various embodiments.

[0188] The electronic device (100) acquires first map data based on distance information with respect to a wall surface acquired by a distance sensor, and can acquire data corresponding to the first map including distance information with respect to a plurality of walls based on the first map data. The data corresponding to the first map may correspond to the first map UI.

[0189] For example, the electronic device (100) can generate a first map UI (10) that expresses distance information to a plurality of walls as relative distances within the map based on the first map data. For example, if the distance value to a plurality of walls obtained by a distance sensor is greater in area B than in area A, the map UI can be generated so that area B occupies a larger area than area A.

[0190] Additionally, the electronic device (100) may generate a distance map UI (Distance Map) (10') that expresses distance information to multiple walls by location within the map based on the first map data.

[0191] For example, the electronic device (100) may express the distance to multiple walls by displaying in white (0) the further the distance to an obstacle is from a location within a map, and in black (1) the closer the distance to an obstacle is.

[0192] However, this is not limited to this, and the degree of proximity to obstacles can be displayed on the map UI through various visual representation methods.

[0193] FIG. 7 is a drawing for illustrating an exemplary first map UI according to various embodiments.

[0194] Referring to FIG. 7, the electronic device (100) can acquire multiple locations corresponding to the scan pose, where the distance value to a plurality of walls is greater or equal to each zone within the first map UI (10).

[0195] For example, the electronic device (100) can obtain a position as a scan pose (11-1) in which the distance value to the nearest two-sided wall surface is the same and the distance value to the farthest two-sided wall surface is the greatest for a plurality of wall surfaces forming a specific area (10-1).

[0196] The electronic device (100) can obtain a position with the same distance value from the nearest multiple wall surface to a multiple wall surface forming a specific area (10-1) as a scan pose (11-2).

[0197] However, it is not limited to this, and a position formed by connecting a scan pose (11-1) and a scan pose (11-2) that form a specific area (10-1) with a straight line can also be obtained as a position corresponding to the scan pose.

[0198] In this case, the electronic device (100) may also acquire scan data by traveling along a straight-line connected scan pose and scanning the projection surface.

[0199] When the electronic device (100) completes a scan by the depth sensor at a position corresponding to a specific scan pose, the electronic device (100) can update the map by displaying a scan complete indicator on the existing first map UI so that the scanned area is not scanned again and / or the scanned area is reduced.

[0200] For example, when the electronic device (100) completes scanning of the projection surface in the scan pose (11-1), the existing first map UI (10) can be updated to a new first map UI (10') by displaying the scanned area as a gray area (14).

[0201] However, this is not limited to this, and indications of scanned areas may be displayed on the map UI through various visual representation methods.

[0202] When the distance from a plurality of walls exceeds a preset value, the electronic device (100) may acquire a position corresponding to a preset value as a position corresponding to a plurality of scan poses.

[0203] For example, the electronic device (100) may acquire a first map UI that displays an area (13) where the distance from a plurality of wall surfaces forming a specific area (10-2) exceeds a preset value, and may acquire an area that becomes the boundary line of the area (13) exceeding the preset value as a position corresponding to a plurality of scan poses.

[0204] In this case, the electronic device (100) may acquire scan data by scanning the projection surface while driving over the area that becomes the boundary line of the area (13) that exceeds a preset value.

[0205] FIG. 8 is a diagram illustrating an exemplary operation in which an electronic device according to various embodiments acquires second map data.

[0206] Referring to FIG. 8, the electronic device (100) generates data (e.g., first map UI (10)) corresponding to a first map including a plurality of grids, and can obtain a position corresponding to a scan pose based on each grid.

[0207] For example, the electronic device (100) can generate a first map UI including a plurality of grids based on first map data and acquire the center of each of the plurality of grids as a position corresponding to a scan pose.

[0208] For example, the electronic device (100) can acquire the center of the grid (15) displayed on the first map UI (10) as a position corresponding to the scan pose (position of the reference point (11)). The electronic device (100) can move to the reference point (11) acquired based on the grid (15), rotate in place at the reference point (11), and scan a plurality of projection surfaces (12).

[0209] When the electronic device (100) completes scanning of the projection surface at the located grid (15), it can move to an adjacent grid and repeat the above-described operation until scanning operations at all grids are completed.

[0210] As described above, the electronic device (100) can acquire a scan pose (or a position corresponding to the scan pose) based on a grid or wall distance.

[0211] The electronic device (100) can obtain the distance between the electronic device (100) and a plurality of walls from the map data obtained primarily (first map data). The map data may correspond to data obtained from the space surrounding the electronic device (100). A plurality of walls and obstacles may be located in the surrounding space.

[0212] When the electronic device (100) acquires a scan pose based on the distance to multiple walls in this manner, it can acquire a reference point (for scanning the surrounding space via a depth sensor) without moving to a location close to the wall (or a specific space (e.g., a room)). Accordingly, the electronic device (100) can acquire additional reference points by sensing other spaces without moving directly to the acquired reference point. Through this, the electronic device (100) can acquire a more optimal position by comparing multiple reference points.

[0213] Meanwhile, when a scan pose is acquired based on wall distance, the electronic device (100) can acquire map data from only the distances to multiple walls. Accordingly, if there are multiple obstacles in the surrounding space, map data may not be acquired based on information regarding the location and size of the obstacles. The obstacles may correspond to obstacles of low height (lower height compared to the wall height) for the electronic device (100).

[0214] In this case, since the electronic device (100) cannot generate map data by taking into account both walls and obstacles, it may not be able to accurately identify the optimal location (reference point) for acquiring scan data of the surrounding space.

[0215] The electronic device (100) can acquire the center of each of the acquired multiple grids as a position corresponding to the scan pose. In this case, unlike when acquiring the scan pose based on wall distance, obstacles around the electronic device (100) can be taken into account. This will be explained in detail in FIG. 9.

[0216] FIG. 9 is a drawing illustrating an exemplary operation in which an electronic device according to various embodiments acquires a changed scan pose.

[0217] If there is an obstacle (90) that hinders the movement of the electronic device (100) within the grid (15), the electronic device (100) can obtain a new reference point (11') where the obstacle is not located within a certain area(s) from the existing reference point (11) that is the center of the grid.

[0218] A certain area(s) may mean an area that includes all areas within a predetermined distance (d) from the electronic device (100).

[0219] If the grid area is moved out of the grid area within a pre-set distance (d) from the new reference point (11'), the grid can be excluded from the grid candidates. The grid candidates may correspond to the grids corresponding to each of the multiple candidate reference points that can become the final reference point.

[0220] Accordingly, if there are many low-height obstacles around the electronic device (100), the grid can be excluded from the candidates, thereby preventing or reducing the generation of multiple grid candidates for selecting a projection scan position more than necessary. Through this, when the electronic device (100) acquires a reference point based on a grid instead of a distance-based reference point, the entire process for acquiring the reference point can be shortened more efficiently.

[0221] FIG. 10 is a flowchart illustrating an exemplary operation of an electronic device scanning space according to various embodiments.

[0222] The electronic device (100) can start mapping and scanning while traveling through space to acquire map data (1010).

[0223] The electronic device (100) can acquire 2D map data by mapping and scanning through a distance sensor while driving through space (1020).

[0224] The electronic device (100) can acquire scan poses for scanning multiple projection planes based on 2D map data (1030). The scan poses may include scan poses acquired based on a grid and scan poses acquired based on wall distance.

[0225] Acquiring a scan pose may mean that the electronic device (100) acquires a position corresponding to the scan pose. The position may correspond to the position of a reference point corresponding to the acquisition of scan data. This has been explained in detail above, and the detailed explanation thereof may not be repeated here.

[0226] When the electronic device (100) acquires a position corresponding to a scan pose, it can move to the nearest position (the position corresponding to the scan pose) and then rotate at that position to scan multiple projection surfaces (1040). The electronic device (100) can acquire scan data by scanning multiple projection surfaces through a depth sensor.

[0227] The electronic device (100) can acquire 3D map data based on scan data and 2D map data acquired through a depth sensor.

[0228] The electronic device (100) can stop the operation to acquire map data when there is no more space to explore and no space to scan while traveling through space (1050).

[0229] Here, "areas to explore" may mean areas that the electronic device (100) has not yet explored or areas where data has not been collected.

[0230] However, it is not limited to this, and the electronic device (100) may repeat the above-described operation to acquire map data and update the map when a change in the environment is detected, such as a change in furniture or a change in space.

[0231] FIG. 11 is a drawing illustrating an exemplary operation in which an electronic device according to various embodiments acquires a scan rotation angle.

[0232] The electronic device (100) can change the rotation angle for scanning by taking into account the Field of View (FoV) of the depth sensor.

[0233] For example, the electronic device (100) can acquire a scan rotation angle based on the Field of View (FoV) of the depth sensor and control the driving device to rotate based on the scan rotation angle. The electronic device (100) can acquire scan data by the depth sensor while rotating based on the scan rotation angle.

[0234] For example, the electronic device (100) can change the scan rotation angle (102) from 360 degrees to an angle greater than 300 degrees when the FoV (101-1) of the depth sensor is 60 degrees.

[0235] On the other hand, the electronic device (100) may change the scan rotation angle from 360 degrees to an angle greater than 180 degrees when the FoV of the depth sensor is 180 degrees.

[0236] FIG. 12 is a diagram illustrating an exemplary operation in which an electronic device according to various embodiments scans space by means of a distance sensor and a depth sensor.

[0237] The electronic device (100) can obtain distance information or two-dimensional data from a wall surface by scanning the space (30) through a distance sensor. Additionally, the electronic device (100) can obtain scan data or three-dimensional data by scanning the space (30) through a depth sensor.

[0238] Meanwhile, distance sensors can scan the space based on a wider field of view than depth sensors.

[0239] For example, the electronic device (100) can acquire sensing values ​​for a range of about 360 degrees of space based on the field of view (101-2) of the distance sensor.

[0240] On the other hand, the electronic device (00) can acquire sensing values ​​for a space ranging from about 30 to 90 degrees based on the field of view (101-1) of the depth sensor.

[0241] However, it is not limited to this, and the space can be scanned based on various field of view of the sensor.

[0242] FIG. 13 is a flowchart illustrating the operation of an electronic device according to various embodiments acquiring second map data.

[0243] The electronic device can acquire first map data based on distance information from the wall surface (S1310). For example, the electronic device can acquire two-dimensional data based on distance information from the wall surface acquired through a distance sensor.

[0244] The electronic device can move to a location (a location corresponding to a scan pose) obtained based on the first map data (S1320). For example, the electronic device can obtain a location corresponding to a scan pose with respect to a reference point corresponding to scan data collection based on two-dimensional map data. The electronic device can move to the location closest among a plurality of scan poses to obtain scan data.

[0245] The electronic device can acquire second map data based on scan data acquired at a position corresponding to the scan pose (S1330). For example, the electronic device can acquire three-dimensional map data based on scan data acquired by a depth sensor at a position corresponding to the scan pose.

[0246] The electronic device can generate a 3D map UI containing various spatial data based on 2D map data and 3D map data.

[0247] The method of acquiring map data described in FIG. 13 can be performed by a device having various configurations such as FIG. 3 and FIG. 4 described above, but is not necessarily limited thereto and can be performed by a device having various configurations.

[0248] The various embodiments described above may be implemented as individual embodiments, or at least one embodiment may be combined with one another, either wholly or partially, to be implemented together in a single device.

[0249] According to the various embodiments described above, the electronic device can accurately and quickly generate a 3D map by acquiring first map data by a distance sensor, moving to a position corresponding to a scan pose acquired based on the first map data, and acquiring second map data by a depth sensor. Ultimately, the user experience can be improved.

[0250] Various embodiments of the present disclosure may be implemented as software stored on a machine-readable storage media (e.g., non-transient machine-readable storage media) that can be mounted on or connected to a smartphone, a user terminal device, and various other electronic devices (e.g., a computer).

[0251] For example, a non-transient readable storage medium may be provided that stores software for sequentially performing the steps of: acquiring first map data based on distance information from a wall surface acquired by a distance sensor; moving to a position corresponding to a scan pose regarding a reference point corresponding to scan data collection based on the first map data; and acquiring second map data based on scan data acquired by a depth sensor at a position corresponding to a scan pose.

[0252] A device equipped with such a non-transient readable medium can perform various operations, such as acquiring first map data based on distance information from a wall surface described in the various embodiments described above, moving to a position corresponding to a scan pose regarding a reference point corresponding to scan data collection, and acquiring second map data based on scan data acquired by a depth sensor.

[0253] In non-transitory readable storage media, 'non-transitory' simply means that the storage medium does not contain a signal and is tangible; it does not distinguish whether data is stored semi-permanently or temporarily on the storage medium.

[0254] A program for performing the method according to the various embodiments described above may be distributed online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0255] Each component (e.g., module or program) according to various embodiments may consist of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the same or similar functions as those performed by each of the respective components prior to integration.

[0256] Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0257] Although various preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure, including the claims and equivalents, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure. Furthermore, it should be understood that the embodiments described in this specification can be used in combination with one another.

Claims

1. In an electronic device, Distance sensor for measuring distance to a wall; Depth sensor; Driving device including a motor; Memory for storing at least one instruction; and at least one processor including a processing circuit; and The above at least one processor enables the electronic device to execute the above at least one instruction individually and / or collectively, Based on the distance information with respect to the wall surface obtained by the distance sensor above, first map data is obtained, and Based on the first map data above, the driving device is controlled to move to a position corresponding to a scan pose with respect to a reference point corresponding to scan data collection, and An electronic device that acquires second map data based on scan data acquired by the depth sensor at a position corresponding to the above scan pose.

2. In Paragraph 1, The above at least one processor enables the electronic device to execute the above at least one instruction individually and / or collectively, thereby, Based on the above first map data, data corresponding to the first map including distance information with a plurality of walls is obtained, and An electronic device that obtains a position corresponding to a plurality of scan poses based on distance information with the plurality of walls.

3. In Paragraph 2, The above at least one processor enables the electronic device to execute the above at least one instruction individually and / or collectively, thereby, An electronic device that, when the distance from the plurality of walls exceeds a specific value, acquires a plurality of positions corresponding to the specific value as positions corresponding to the plurality of scan poses.

4. In Paragraph 1, The above at least one processor enables the electronic device to execute the above at least one instruction individually and / or collectively, thereby, Based on the above first map data, data corresponding to a first map including a plurality of grids is obtained, and An electronic device that acquires the center of each of the above plurality of grids at a position corresponding to the scan pose.

5. In Paragraph 1, The above at least one processor enables the electronic device to execute the above at least one instruction individually and / or collectively, thereby, A scan rotation angle is obtained based on the Field of View (FoV) of the depth sensor, and An electronic device that controls the driving device to rotate based on the above scan rotation angle to acquire scan data by the depth sensor.

6. In Paragraph 1, The above at least one processor enables the electronic device to execute the above at least one instruction individually and / or collectively, thereby, Spatial data is obtained from the above second map data, and An electronic device that obtains data corresponding to a second map containing the above spatial data.

7. In Paragraph 1, The above at least one processor enables the electronic device to execute the above at least one instruction individually and / or collectively, thereby, Spatial data is obtained from the above second map data, and The above spatial data is, An electronic device comprising at least one of environmental information of the space, zone information of the space, and object information within the space.

8. In Paragraph 1, It further includes an input interface including a circuit, The above at least one processor enables the electronic device to execute the above at least one instruction individually and / or collectively, thereby, Spatial data is obtained from the above second map data, and An electronic device that performs an operation corresponding to an input based on an input corresponding to the spatial data received through the input interface.

9. In Paragraph 1, Including a camera; and The above at least one processor enables the electronic device to execute the above at least one instruction individually and / or collectively, thereby, An electronic device that acquires spatial data based on a captured image acquired through the camera and the second map data.

10. In a method for controlling an electronic device, A step of acquiring first map data based on distance information with respect to a wall surface acquired by a distance sensor; A step of moving to a position corresponding to a scan pose with respect to a reference point corresponding to scan data collection based on the first map data; and A control method comprising: a step of acquiring second map data based on scan data acquired by a depth sensor at a position corresponding to the above scan pose.

11. In Paragraph 10, A step of obtaining data corresponding to a first map including distance information with a plurality of walls based on the first map data; and A control method further comprising the step of obtaining a position corresponding to a plurality of scan poses based on distance information with the plurality of walls.

12. In Paragraph 11, The step of acquiring the above plurality of scan poses is, A control method comprising the step of, when the distance from the plurality of walls exceeds a specific value, acquiring a plurality of positions corresponding to the specific value as positions corresponding to the plurality of scan poses.

13. In Paragraph 10, A step of obtaining data corresponding to a first map including a plurality of grids based on the first map data; and A control method further comprising the step of acquiring the center of each of the plurality of grids as a position corresponding to the scan pose.

14. In Paragraph 10, A step of acquiring a scan rotation angle based on the Field of View (FoV) of the depth sensor; and A control method further comprising the step of acquiring scan data by the depth sensor by rotating based on the above scan rotation angle.

15. In a non-transient computer-readable recording medium on which a program is recorded, the program is executed individually and / or collectively by at least one processor comprising a processing circuit to enable the device to perform a method, and The above method is, A step of acquiring first map data based on distance information with respect to a wall surface acquired by a distance sensor; A step of moving to a position corresponding to a scan pose with respect to a reference point corresponding to scan data collection based on the first map data; and A non-transient computer-readable recording medium comprising: a step of acquiring second map data based on scan data acquired by a depth sensor at a position corresponding to the above scan pose.

Images