Electronic apparatus and control method thereof

Abstract

This electronic device comprises: a memory; at least one sensor; and at least one processor operatively connected to the memory and the at least one sensor and configured to execute instructions, wherein the instructions, when executed by the at least one processor, cause the electronic device to: obtain, when a request for a map corresponding to a target space is received, first surface data for a first surface corresponding to the driving direction of the electronic device on the basis of first sensor data obtained through the at least one sensor in a first driving for the target space; obtain second surface data for a second surface that is different from the first surface, on the basis of second sensor data obtained through the at least one sensor in a second driving for the target space; and obtain a map on the basis of the first surface data and the second surface data.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device that provides a map for a space in which the electronic device exists and a control method thereof.

[0002] Mobile devices can provide a variety of services based on maps of space.

[0003] For example, suppose the movable device is a projector. The projector can navigate using a map of the space.

[0004] For example, assume the mobile device is a robotic vacuum cleaner. The robotic vacuum cleaner can perform its cleaning function using a map of the space.

[0005] There are ways to obtain a map of a space, either by having the device itself analyze the space or by having the user directly input information about the space.

[0006] When the device itself performs the function of analyzing space, there are problems such as long processing time or low accuracy.

[0007] When users directly enter information about a space, there is a problem that accuracy is low and inconvenience is caused to the user.

[0008] The present disclosure is designed to improve the above-described problem, and an object of the present disclosure is to provide an electronic device and a control method thereof that provides a map based on sensor data collected by distinguishing a horizontal plane and a second plane.

[0009] According to one embodiment, an electronic device includes a memory, at least one sensor, and at least one processor operatively connected to the memory and the at least one sensor and configured to execute instructions, wherein the instructions, when executed by the at least one processor, cause the electronic device, when a request for a map corresponding to a target space is received, to obtain first surface data for a first surface corresponding to a driving direction of the electronic device based on first sensor data obtained through the at least one sensor in a first driving about the target space, to obtain second surface data for a second surface different from the first surface based on second sensor data obtained through the at least one sensor in a second driving about the target space, and to obtain the map based on the first surface data and the second surface data.

[0010] The first surface data may include first information corresponding to a horizontal plane including an x-axis and a y-axis based on the driving direction of the electronic device, and the second surface data may include second information corresponding to a vertical plane including a z-axis based on the driving direction of the electronic device.

[0011] The first surface data may include first spatial information of the first surface and first object information of the first surface.

[0012] The at least one sensor includes a first distance sensor and an acceleration sensor, the first sensor data includes first distance data acquired through the first distance sensor and first acceleration data acquired through the acceleration sensor, and the instructions, when executed by the at least one processor, can cause the electronic device to detect first edge information based on the first distance data, acquire first direction information of the electronic device based on the first acceleration data, and acquire first surface data based on the first edge information and the first direction information.

[0013] The second surface data may include second spatial information of the second surface and second object information of the second surface.

[0014] The at least one sensor includes a second distance sensor, the second sensor data includes second distance data acquired through the second distance sensor and second acceleration data acquired through the acceleration sensor, and the instructions, when executed by the at least one processor, may cause the electronic device to detect second edge information based on the second distance data, acquire second direction information of the electronic device based on the second acceleration data, and acquire second surface data based on the second edge information and the second direction information.

[0015] The at least one sensor further includes a vision sensor, the second sensor data further includes image data acquired through the vision sensor, and the instructions, when executed by the at least one processor, may cause the electronic device to update the second object information in the image data.

[0016] The at least one sensor further includes a tilt sensor, the second sensor data further includes tilt data acquired through the tilt sensor, and the instructions, when executed by the at least one processor, can cause the electronic device to acquire a first tilt angle in a roll direction, a second tilt angle in a pitch direction, and a third tilt angle in a yaw direction of the electronic device based on the tilt data, and to update the second spatial information based on the first tilt angle, the second tilt angle, and the third tilt angle.

[0017] The first distance sensor may be a Lidar sensor, the second distance sensor may be a ToF (Time of Flight) sensor, and the tilt sensor may be a gyro sensor.

[0018] The instructions, when executed by the at least one processor, may cause the electronic device to obtain third spatial information by combining the first spatial information and the second spatial information if the first spatial information and the second spatial information correspond to the same location, obtain third object information by combining the first object information and the second object information if the first spatial information and the second spatial information correspond to the same location, and obtain the map including the third spatial information and the third object information.

[0019] According to one embodiment, a method for controlling an electronic device includes, when a request for a map corresponding to a target space is received, obtaining first surface data for a first side corresponding to a driving direction of the electronic device based on first sensor data obtained in a first driving for the target space, obtaining second surface data for a second side different from the first side based on second sensor data obtained in a second driving for the target space, and obtaining the map based on the first surface data and the second surface data.

[0020] The first surface data may include first information corresponding to a horizontal plane including an x-axis and a y-axis based on the driving direction of the electronic device, and the second surface data may include second information corresponding to a vertical plane including a z-axis based on the driving direction of the electronic device.

[0021] The first surface data may include first spatial information of the first surface and first object information of the first surface.

[0022] The electronic device includes a first distance sensor and an acceleration sensor, the first sensor data includes first distance data acquired through the first distance sensor and first acceleration data acquired through the acceleration sensor, and the step of acquiring the first surface data may include detecting first edge information based on the first distance data, acquiring first direction information of the electronic device based on the first acceleration data, and acquiring the first surface data based on the first edge information and the first direction information.

[0023] The second surface data may include second spatial information of the second surface and second object information of the second surface.

[0024] The above and other aspects, features and advantages of the embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0025] FIG. 1 is a drawing for explaining a map according to one embodiment.

[0026] FIG. 2 is a block diagram illustrating an electronic device according to one embodiment.

[0027] FIG. 3 is a block diagram illustrating a specific configuration of the electronic device of FIG. 2, according to one embodiment.

[0028] FIG. 4 is a diagram for explaining an operation of acquiring horizontal plane data and vertical plane data through a sensor unit according to one embodiment.

[0029] FIG. 5 is a diagram for explaining an operation of acquiring horizontal plane data and vertical plane data through a sensor unit according to one embodiment.

[0030] FIG. 6 is a diagram for explaining an operation of generating a map according to one embodiment.

[0031] FIG. 7 is a diagram illustrating an operation of generating a map based on a preset event, according to one embodiment.

[0032] FIG. 8 is a diagram for explaining an operation of generating a map based on spatial information and object information, according to one embodiment.

[0033] FIG. 9 is a diagram illustrating an operation of generating a map by merging distance data according to one embodiment.

[0034] FIG. 10 is a drawing for explaining a horizontal slope according to one embodiment.

[0035] FIG. 11 is a drawing for explaining a vertical slope according to one embodiment.

[0036] Figure 12 is a drawing for explaining horizontal distortion according to one embodiment.

[0037] FIG. 13 is a drawing for explaining rotation information of an electronic device according to one embodiment.

[0038] FIG. 14 is a drawing for explaining rotation information of a projection surface according to one embodiment.

[0039] FIG. 15 is a drawing for explaining z-axis rotation information of a projection surface according to one embodiment.

[0040] FIG. 16 is a drawing for explaining y-axis rotation information of a projection surface according to one embodiment.

[0041] FIG. 17 is a drawing for explaining an operation of performing a keystone function by taking into account vertical inclination, according to one embodiment.

[0042] FIG. 18 is a drawing for explaining an operation of performing a keystone function while taking into account horizontal inclination, according to one embodiment.

[0043] FIG. 19 is a diagram illustrating a horizontal plane map, a vertical plane map, and a combined map according to one embodiment.

[0044] FIG. 20 is a drawing for explaining a map according to one embodiment.

[0045] FIG. 21 is a diagram illustrating an operation of generating a combined map using a horizontal plane map and a vertical plane map according to one embodiment.

[0046] FIG. 22 is a diagram for explaining an operation of obtaining a map through a server according to one embodiment.

[0047] FIG. 23 is a drawing for explaining an operation of outputting a projection image according to one embodiment.

[0048] FIG. 24 is a drawing for explaining a method of controlling an electronic device according to one embodiment.

[0049] The embodiments described in this specification and the configurations shown in the drawings are merely examples of embodiments, and various modifications may be made without departing from the scope and spirit of this specification.

[0050] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0051] The terms used in the embodiments of this disclosure are widely used today, selected with consideration to the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Some terms may have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the disclosure. Therefore, the terms used in this disclosure should not be defined simply as names, but rather based on the meanings of the terms and the overall content of this disclosure.

[0052] Expressions such as "has," "can have," "includes," or "may include" indicate the presence of a feature (e.g., a number, function, operation, or component such as a part) and do not exclude the presence of additional features.

[0053] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0054] Expressions such as "first," "second," "first," or "second," can describe various components, regardless of order and / or importance, and are used only to distinguish one component from another, but do not limit the components in question.

[0055] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0056] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a described feature, number, step, operation, component, part, or combination thereof, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0057] In the present disclosure, a "module" or "part" performs at least one function or operation, and may be implemented in hardware or software, or a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be associated with operations performed by at least one processor.

[0058] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0059] An embodiment of the present (S) disclosure is described in more detail with reference to the attached drawings below.

[0060] FIG. 1 is a drawing for explaining a map according to one embodiment.

[0061] Referring to embodiment (1) of FIG. 1, the electronic device (100) can generate a map. The map can represent a map of a space in which the electronic device (100) moves. The map can include information related to three-dimensional space. The map can include information regarding the driving of the electronic device (100). The map can include spatial information and object information.

[0062] Spatial information may include coordinate information for three-dimensional space.

[0063] Object information may include information related to an object existing in three-dimensional space.

[0064] An electronic device (100) can identify (or obtain) a driving route using a map. The electronic device (100) can identify information about objects existing in a space using the map. The electronic device (100) can identify a driving route by considering objects existing in the space.

[0065] As described above and in the description below, the 3D map may be described as a map or a 2D map.

[0066] Referring to embodiment (2) of FIG. 1, the electronic device (100) can be implemented as various devices (100-1, 100-2, 100-3, 100-4).

[0067] The first device (100-1) may be a device including a projection unit that projects an image. The first device (100-1) may be a projector.

[0068] The second device (100-2) may be a projector having a different appearance from the first device (100-1).

[0069] The third device (100-3) may be a device that performs a cleaning function. The third device (100-3) may be a robot vacuum cleaner.

[0070] The fourth device (100-4) may be a device that performs a service function. The fourth device (100-4) may be a mobile service robot. The service may include a service that provides information to the user in the form of images or audio.

[0071] FIG. 2 is a block diagram illustrating an electronic device (100) according to one embodiment.

[0072] Referring to FIG. 2, the electronic device (100) may include a memory (113), a sensor unit (121), and at least one processor (111).

[0073] The memory (113) can store various sensor data acquired from the sensor unit (121) and data acquired by the electronic device (100) (horizontal plane data, vertical plane data, 3D map, etc.). The memory (113) can store various intermediate data used in the process of the electronic device (100) acquiring a 3D map.

[0074] The sensing unit (121) can sense information about the space in which the electronic device (100) is placed. The sensing unit (121) can sense information related to the electronic device (100). Various analysis operations can be performed based on the sensor data collected by the sensing unit (121).

[0075] At least one processor (111) is connected to a memory (113) and a sensor unit (121) to perform various functions or commands of the electronic device (100).

[0076] At least one processor (111) can obtain first sensor data through the sensor unit (121) in a first drive for the target space when a request for a three-dimensional map corresponding to the target space is received.

[0077] The target space may represent a space in which the electronic device (100) drives. The target space may represent a space for generating a three-dimensional map.

[0078] For example, the target space can be identified based on the driving of the electronic device (100).

[0079] For example, the target space can be determined by user input.

[0080] A request for a 3D map may indicate a request to generate a 3D map. The request for a 3D map may be determined based on the occurrence of a preset event. When the preset event occurs, at least one processor (111) may identify that a request for a 3D map has been received.

[0081] The preset events may include at least one of an event in which a setup mode is executed after the first boot, an event in which preset user input is received, an event in which preset input is received from an external device, and a notification event.

[0082] The event that causes setup mode to run after the first boot can represent the event that causes setup mode to run when power is supplied from a factory reset state or a setup reset state.

[0083] An event in which a preset user input is received may represent an event in which a user input requesting a 3D map is received. The user input may be received via voice or manipulation.

[0084] An event in which a preset input is received from an external device may represent an event in which an input requesting a 3D map is received from an external device.

[0085] A notification event can indicate an event that triggers a preset notification. A preset notification can be a notification indicating a change in information related to a 3D map. For example, a notification event can include a notification indicating a change in the position of an object in space.

[0086] When a request for a 3D map corresponding to a target space is received, at least one processor (111) may perform a first drive (first drive) to analyze the target space. The electronic device (100) may include a movable member (122), and at least one processor (111) may control the movable member (122) to perform a driving function. At least one processor (111) may perform an analysis of the target space through the first drive.

[0087] At least one processor (111) can obtain horizontal plane data (first data) for a horizontal plane (first plane) parallel to the driving direction of the electronic device (100) based on the first sensor data. The horizontal plane data can be described as the first data or data for the first plane.

[0088] A horizontal plane parallel to the driving direction of the electronic device (100) may mean an xy plane in a three-dimensional space where the electronic device (100) exists.

[0089] Horizontal plane data may include information related to a horizontal plane. The horizontal plane data may include at least one of spatial information or object information regarding the horizontal plane. The horizontal plane data may be data representing information about a two-dimensional plane.

[0090] At least one processor (111) can determine whether all horizontal plane data has been acquired for the entire target space. At least one processor (111) can determine whether the collection of horizontal plane data has been completed for the entire target space. The movement of the electronic device (100) for collecting the completed horizontal plane data can be described as a first movement.

[0091] Upon completion of the first run, at least one processor (111) may initiate a new run (second run) to acquire vertical plane data (data for the second plane). The first plane and the second plane may be vertical. The vertical plane data may be described as the second data or data for the second plane.

[0092] According to various embodiments, at least one processor (111) may identify a driving path used for a second driving based on horizontal plane data acquired through a first driving. At least one processor (111) may collect vertical plane data using the horizontal plane data acquired through the first driving. At least one processor (111) may collect vertical plane information based on already acquired horizontal plane information rather than randomly collecting information about a vertical plane (second plane). For example, at least one processor (111) may acquire z-axis information corresponding to a horizontal plane structure as vertical plane data based on a horizontal plane structure.

[0093] When horizontal plane data is acquired, at least one processor (111) can acquire second sensor data through the sensor unit (121) in a second drive for the target space.

[0094] The horizontal plane data may only include information on the xy plane regarding the driving direction of the electronic device (100). Therefore, the horizontal plane data may only include two-dimensional information. At least one processor (111) may additionally obtain z-plane information (or z-axis information) not included in the horizontal plane data. At least one processor (111) may obtain second sensor data through a second drive. The second drive may be performed after the first drive is completed.

[0095] According to various embodiments, horizontal and vertical data can be acquired simultaneously in a single run. An example of this is described in FIG. 5. The embodiment of FIG. 5 discloses an operation in which horizontal and vertical data are acquired simultaneously, and other operations may overlap with those of other embodiments.

[0096] At least one processor (111) can obtain vertical plane data for a vertical plane perpendicular to the horizontal plane based on the second sensor data.

[0097] A vertical plane may refer to a plane perpendicular to a horizontal plane. The vertical plane may represent a plane perpendicular to the xy plane corresponding to the driving direction of the electronic device (100).

[0098] Vertical plane data may include information related to a vertical plane. The vertical plane data may include at least one of spatial information of the vertical plane or object information of the vertical plane.

[0099] For example, vertical plane data may include height information of a space (z-axis information) or height information of an object (z-axis information).

[0100] For example, vertical plane data may include information about a plane that is perpendicular to the plane corresponding to the horizontal plane data.

[0101] At least one processor (111) can determine whether all vertical plane data has been acquired for the entire target space. At least one processor (111) can determine whether the collection of vertical plane data has been completed for the target space. The driving of the electronic device (100) for collecting completed vertical plane data can be described as a second driving.

[0102] Upon completion of the second drive, at least one processor (111) can obtain a three-dimensional map of the target space. At least one processor (111) can obtain the three-dimensional map based on horizontal plane data obtained through the first drive and vertical plane data obtained through the second drive.

[0103] At least one processor (111) can obtain a three-dimensional map combining horizontal plane data and vertical plane data. The three-dimensional map can be generated by combining horizontal plane data and vertical plane data.

[0104] At least one processor (111) can obtain information about a three-dimensional space by combining horizontal plane data and vertical plane data based on position information (or coordinate information) for the same target space. The horizontal plane data may include information about the xy plane based on the driving direction of the electronic device (100), and the vertical plane data may include information about the z-axis based on the driving direction of the electronic device (100). When combining the horizontal plane data and the vertical plane data, a three-dimensional map representing information about a three-dimensional target space may be generated.

[0105] For example, at least one processor (111) can directly combine horizontal plane data and vertical plane data to generate a three-dimensional map.

[0106] For example, at least one processor (111) can transmit horizontal plane data and vertical plane data to the server (200) and receive a three-dimensional map through the server (200). Additional descriptions related to this are described in FIG. 22.

[0107] The horizontal plane data may include information about a horizontal plane including the x-axis and y-axis based on the driving direction of the electronic device (100).

[0108] A horizontal plane parallel to the driving direction of the electronic device (100) may mean an xy plane in a three-dimensional space where the electronic device (100) exists.

[0109] The vertical plane data may include information about a vertical plane including the z-axis based on the driving direction of the electronic device (100).

[0110] The vertical plane data may include information on a vertical plane that is perpendicular to the horizontal plane based on the driving direction of the electronic device (100).

[0111] Descriptions of the x-axis, y-axis, z-axis, xy plane, etc. are described in Fig. 13.

[0112] At least one processor (111) can obtain horizontal plane data including first spatial information of a horizontal plane and first object information of a horizontal plane based on first sensor data.

[0113] The first sensor data may include at least one piece of data acquired through the first drive. At least one processor (111) may analyze the first sensor data to acquire first spatial information and first object information. The first spatial information and the first object information may be information acquired based on the xy plane.

[0114] The first spatial information may be information representing a spatial structure for the xy plane of the target space. The first object information may be information representing an object existing in the xy plane of the target space.

[0115] The sensor unit (121) may include a first distance sensor and an acceleration sensor. The first sensor data may include first distance data acquired through the first distance sensor and first acceleration data acquired through the acceleration sensor.

[0116] At least one processor (111) may detect first edge information based on first distance data. The first edge information may represent information about edge(s) in the xy plane identified in the target space. The first edge information may include the shape of the edge in the xy plane and the position of the edge. The structure of the edge may be described in terms encompassing the shape of the edge and the position of the edge.

[0117] At least one processor (111) can detect an edge through a first distance sensor. At least one processor (111) can obtain a distance between the electronic device (100) and the detected edge based on first distance data obtained through the first distance sensor. At least one processor (111) can obtain a position (relative position) of the edge. At least one processor (111) can synthesize a plurality of distance data to include first edge information including the shape and position of the edge.

[0118] At least one processor (111) can obtain first direction information of the electronic device (100) based on the first acceleration data. At least one processor (111) can obtain information about movement and direction of the at least one processor (111) based on the first acceleration data obtained through the acceleration sensor. At least one processor (111) can identify in which direction the electronic device (100) is moving based on the acceleration data. At least one processor (111) can identify at what speed the electronic device (100) is moving based on the acceleration data.

[0119] At least one processor (111) can identify the location of the electronic device (100) by analyzing acceleration data indicating the movement and direction of the electronic device (100). At least one processor (111) can determine the location of the electronic device (100) in the target space based on the acceleration data. Direction information can be described as location information.

[0120] At least one processor (111) can obtain horizontal plane data including first spatial information and first object information based on first edge information and first direction information.

[0121] The first edge information may indicate the relative positions of edges detected based on the first distance data. The first direction information may indicate the position where the first distance data was detected in the target space. At least one processor (111) may use both the first edge information and the first direction information to determine the absolute position of the edge in the target space.

[0122] At least one processor (111) can analyze the shape of the detected edge to determine whether the edge represents a space or an object. The edge representing a space may be linear. The length of the edge representing a space may exceed a first threshold. The length of the edge representing an object may be less than a second threshold. The first threshold may be greater than the second threshold.

[0123] The shape of an edge representing an object may be pre-stored. At least one processor (111) can identify an edge as corresponding to an object if the shape of the edge is a preset shape.

[0124] At least one processor (111) can obtain at least one of first spatial information or first object information based on first distance data and acceleration data acquired at the same point in time. At least one processor (111) can obtain first distance data at the first point in time and first acceleration data at the first point in time. At least one processor (111) can determine a first sensing position of the electronic device (100) based on the first acceleration data. At least one processor (111) can identify a position of an edge based on the first distance data acquired at the first sensing position.

[0125] At least one processor (111) can analyze the positions of edges acquired from multiple sensing locations to acquire first spatial information and first object information. The multiple sensing locations can be included in a target space.

[0126] At least one processor (111) can obtain vertical plane data including second spatial information of a vertical plane and second object information of a vertical plane based on second sensor data.

[0127] The second sensor data may include at least one piece of data acquired through the second driving. At least one processor (111) may analyze the second sensor data to acquire second spatial information and second object information. The second spatial information and second object information may be information acquired based on the z-axis.

[0128] The second spatial information may be information indicating a spatial structure along the z-axis of the target space. The second object information may be information indicating an object existing along the z-axis of the target space.

[0129] The sensor unit (121) may include a second distance sensor and an acceleration sensor. The second sensor data may include second distance data acquired through the second distance sensor and second acceleration data acquired through the acceleration sensor.

[0130] At least one processor (111) can detect second edge information based on second distance data.

[0131] Second edge information may represent information about edge(s) along the z-axis identified in the target space. The second edge information may include the shape and position of the edge along the z-axis. The structure of the edge may be described in terms encompassing the shape and position of the edge.

[0132] At least one processor (111) can detect an edge through a second distance sensor. At least one processor (111) can obtain a distance between the electronic device (100) and the detected edge based on second distance data obtained through the second distance sensor. At least one processor (111) can obtain a position (relative position) of the edge. At least one processor (111) can synthesize a plurality of distance data to include second edge information including the shape and position of the edge.

[0133] At least one processor (111) can obtain second direction information of the electronic device (100) based on the second acceleration data. At least one processor (111) can obtain information on movement and direction of the at least one processor (111) based on the second acceleration data obtained through the acceleration sensor. At least one processor (111) can identify the location of the electronic device (100) by analyzing acceleration data indicating the movement and direction of the electronic device (100).

[0134] At least one processor (111) can obtain vertical plane data including second spatial information and second object information based on second edge information and second direction information.

[0135] The second edge information may indicate the relative positions of edges detected based on the second distance data. The second direction information may indicate the position where the second distance data was detected in the target space. At least one processor (111) may use both the second edge information and the second direction information to determine the absolute position of the edge in the target space.

[0136] At least one processor (111) can analyze the shape of the detected edge to determine whether the edge represents a space or an object. The edge representing a space may be linear. The length of the edge representing a space may exceed a third threshold. The length of the edge representing an object may be less than a fourth threshold. The third threshold may be greater than the fourth threshold.

[0137] The shape of an edge representing an object may be pre-stored. At least one processor (111) can identify an edge as corresponding to an object if the shape of the edge is a preset shape.

[0138] At least one processor (111) can obtain at least one of second spatial information or second object information based on second distance data and acceleration data acquired at the same point in time. At least one processor (111) can obtain second distance data at a third point in time and second acceleration data at a third point in time. At least one processor (111) can determine a second sensing location of the electronic device (100) based on the second acceleration data. At least one processor (111) can identify the location of the edge based on the second distance data acquired at the second sensing location.

[0139] At least one processor (111) can analyze the positions of edges acquired from multiple sensing locations to acquire second spatial information and second object information. The multiple sensing locations can be included in a target space.

[0140] The sensor unit (121) may include a vision sensor. The vision sensor may include a camera. The second sensor data may include second distance data, second acceleration data, and image data acquired through the vision sensor.

[0141] At least one processor (111) can update second object information in the image data. At least one processor (111) can analyze the image data to identify an object. At least one processor (111) can analyze the image data to identify at least one of the type of object and the location of the object.

[0142] Analyzing objects solely based on edges detected by a distance sensor can result in relatively low accuracy. At least one processor (111) can enhance the accuracy of object analysis using a vision sensor. At least one processor (111) can analyze image data to identify at least one of the object type and object location.

[0143] At least one processor (111) can update second object information acquired based on second distance data and second acceleration data by analyzing image data. As a result of the update, the accuracy of the second object information can be improved.

[0144] The sensor unit (121) may include a tilt sensor. The second sensor data may include second distance data, second acceleration data, and tilt data acquired through the tilt sensor.

[0145] At least one processor (111) can obtain a first tilt angle in the roll direction, a second tilt angle in the pitch direction, and a third tilt angle in the yaw direction of the electronic device (100) based on the tilt data. A description of the roll direction, the pitch direction, and the yaw direction is described in FIG. 13.

[0146] At least one processor (111) can update second spatial information based on the first tilt angle, the second tilt angle, and the third tilt angle.

[0147] At least one processor (111) can identify the posture (or posture information) of the electronic device (100) based on inclination data acquired through the inclination sensor. For example, the electronic device (100) can determine whether the electronic device (100) was tilted to a certain degree when acquiring sensor data based on the inclination data.

[0148] At least one processor (111) can update second spatial information based on the tilt data. At least one processor (111) can correct a sensing error in the second sensor data by analyzing the tilt data. The sensing error may be an error caused by the electronic device (100) being tilted.

[0149] For example, the first distance sensor may be a Lidar sensor.

[0150] For example, the second distance sensor may be a Time of Flight (ToF) sensor.

[0151] For example, the tilt sensor may be a gyro sensor.

[0152] At least one processor (111) can obtain third spatial information by combining first spatial information and second spatial information based on the same location. At least one processor (111) can obtain third object information by combining first object information and second object information based on the same location. At least one processor (111) can obtain a three-dimensional map including third spatial information and third object information.

[0153] At least one processor (111) can merge first sensor data and second sensor data to obtain a single, integrated data, a three-dimensional map. The merge reference may be the same location. For example, horizontal plane data corresponding to a first location in the target space and vertical plane data corresponding to the first location in the target space may be merged.

[0154] In various embodiments, the acceleration sensor may be replaced with a position sensor, acceleration data may be replaced with position data, and direction information may be replaced with position information.

[0155] In various embodiments, object information may be excluded in relation to the 3D map.

[0156] In the above description, it is described that first spatial information and first object information are acquired from first sensor data, and second spatial information and second object information are acquired from second sensor data. According to various embodiments, spatial information and object information may not be acquired from each sensor data separately. The electronic device (100) may acquire one spatial information and one object information by merging the first sensor data and the second sensor data.

[0157] The electronic device (100) first acquires horizontal plane data and then vertical plane data. If horizontal plane data and vertical plane data are randomly collected, the processing time for generating a 3D map may be long. If horizontal plane data is acquired first and vertical plane data is acquired based on the horizontal plane data, the processing time can be shortened because the basic structure or processing algorithm is simplified.

[0158] FIG. 3 is a block diagram for explaining a specific configuration of the electronic device (100) of FIG. 2, according to one embodiment.

[0159] Referring to FIG. 3, the electronic device (100) may include at least one of a processor (111), a projection unit (112), a memory (113), a communication interface (114), an operation interface (115), an input / output interface (116), a speaker (117), a microphone (118), a power supply unit (119), a driving unit (120), a sensor unit (121), or a moving member (122).

[0160] The configuration illustrated in Fig. 3 is merely an example of various embodiments, and new configurations may be added.

[0161] At least one processor (111) may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes a digital signal. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics-processing unit (GPU), a communication processor (CP), or an advanced reduced instruction set computer (RISC) machines (ARM) processor, or may be defined by the relevant terminology. At least one processor (111) may be implemented as a system on chip (SoC) having a built-in processing algorithm, a large scale integration (LSI), or may be implemented in the form of a field programmable gate array (FPGA). At least one processor (111) may perform various functions by executing computer executable instructions stored in a memory (113).

[0162] The projection unit (112) is a component that projects an image to the outside. According to various embodiments of the present disclosure, the projection unit (112) can be implemented with various projection methods (e.g., CRT (cathode-ray tube) method, LCD (Liquid Crystal Display) method, DLP (Digital Light Processing) method, laser method, etc.). For example, the CRT method has the same principle as a CRT monitor. The CRT method magnifies the image with a lens in front of the cathode-ray tube (CRT) and displays the image on the screen. Depending on the number of cathode-ray tubes, it is divided into a single-tube type and a three-tube type, and in the case of a three-tube type, the red, green, and blue cathode-ray tubes can be implemented separately.

[0163] Another example is the LCD method, which displays images by passing light from a light source through liquid crystals. LCD methods are divided into single-panel and three-panel types. In the case of the three-panel type, light from a light source is separated into red, green, and blue by a dichroic mirror (a mirror that reflects only the colored light and transmits the rest). After passing through the liquid crystals, the light can be refocused into a single point.

[0164] Another example is the DLP method, which displays images using a DMD (Digital Micromirror Device) chip. The DLP projection unit may include a light source, a color wheel, a DMD chip, a projection lens, etc. The light output from the light source can be colored as it passes through the rotating color wheel. The light passing through the color wheel is input to the DMD chip. The DMD chip contains numerous micromirrors and reflects the light input to the DMD chip. The projection lens can play a role in magnifying the light reflected from the DMD chip to the image size.

[0165] Another example is a laser system that uses a Diode Pumped Solid State (DPSS) laser and a galvanometer. A multi-color laser uses three DPSS lasers, one for each RGB color, with their optical axes overlapped by a special mirror. The galvanometer includes a mirror and a high-power motor that moves the mirror at high speeds. For example, the galvanometer can rotate the mirror at up to 40 kHz / sec. The galvanometer is mounted according to the scan direction, but since projectors typically scan in a planar manner, the galvanometer can also be positioned separately along the x and y axes.

[0166] The projection unit (112) may include various types of light sources. For example, the projection unit (112) may include at least one light source among a lamp, an LED, and a laser.

[0167] The projection unit (112) can output images in a 4:3 screen ratio, a 5:4 screen ratio, or a 16:9 wide screen ratio depending on the purpose of the electronic device (100) or the user's settings, and can output images in various resolutions such as WVGA (854*480), SVGA (800*600), XGA (1024*768), WXGA (1280*720), WXGA (1280*800), SXGA (1280*1024), UXGA (1600*1200), and Full HD (1920*1080) depending on the screen ratio.

[0168] The projection unit (112) can perform various functions for adjusting the output image under the control of at least one processor (111). For example, the projection unit (112) can perform functions such as zoom, keystone, quick corner (4 corner) keystone, and lens shift.

[0169] The projection unit (112) can enlarge or reduce the image depending on the distance from the screen (projection distance). That is, the zoom function can be performed depending on the distance from the screen. At this time, the zoom function may include a hardware method that adjusts the screen size by moving the lens and a software method that adjusts the screen size by cropping the image, etc. When the zoom function is performed, the focus of the image can be adjusted. For example, the method of adjusting the focus includes a manual focus method, an electric focus method, etc. The manual focus method refers to a method of focusing manually, and the electric focus method refers to a method of automatically focusing using a motor built into the projector when the zoom function is performed. When performing the zoom function, the projection unit (112) may provide a digital zoom function through software, and may provide an optical zoom function that performs the zoom function by moving the lens through the driving unit (120).

[0170] The projection unit (112) can perform a keystone correction function. If the height is not right for front projection, the screen may be distorted upwards or downwards. The keystone correction function refers to a function that corrects a distorted screen. For example, if distortion occurs in the left and right directions of the screen, it can be corrected using horizontal keystone, and if distortion occurs in the up and down directions, it can be corrected using vertical keystone. The quick corner (4 corner) keystone correction function is a function that corrects the screen when the center area of ​​the screen is normal but the corner areas are not balanced. The lens shift function is a function that moves the screen as it is when the screen is off the screen.

[0171] The projection unit (112) can automatically analyze the surrounding environment and projection environment without user input to provide zoom / keystone / focus functions. The projection unit (112) can automatically provide zoom / keystone / focus functions based on the distance between the electronic device (100) and the screen detected by a sensor (depth camera, distance sensor, infrared sensor, light sensor, etc.), information about the space where the electronic device (100) is currently located, information about the amount of ambient light, etc.

[0172] The projection unit (112) can provide a lighting function using a light source. The projection unit (112) can provide a lighting function by outputting a light source using an LED. According to various embodiments, the projection unit (112) can include one LED, and according to other embodiments, the electronic device (100) can include a plurality of LEDs. The projection unit (112) can output a light source using a surface-emitting LED according to an implementation example. The surface-emitting LED can refer to an LED having a structure in which an optical sheet is arranged on the upper side of the LED so that the light source is evenly distributed and output. When a light source is output through the LED, the light source can be evenly distributed through the optical sheet, and the light source distributed through the optical sheet can be incident on the display panel.

[0173] The projection unit (112) can provide the user with a dimming function for adjusting the intensity of the light source. When a user input for adjusting the intensity of the light source is received from the user through the operation interface (115) (e.g., a touch display button or dial), the projection unit (112) can control the LED to output the intensity of the light source corresponding to the received user input.

[0174] The projection unit (112) can provide a dimming function based on content analyzed by at least one processor (111) without user input. The projection unit (112) can control the LED to output the intensity of the light source based on information about the currently provided content (e.g., content type, content brightness, etc.).

[0175] The projection unit (112) can control the color temperature under the control of at least one processor (111). The at least one processor (111) can control the color temperature based on the content. When the content is identified to be output, the at least one processor (111) can obtain frame-by-frame color information of the content whose output has been determined. Then, the at least one processor (111) can control the color temperature based on the obtained frame-by-frame color information. The at least one processor (111) can obtain at least one primary color of the frame based on the frame-by-frame color information. Then, the at least one processor (111) can adjust the color temperature based on the obtained at least one primary color. For example, the color temperature that the at least one processor (111) can adjust can be classified into a warm type or a cold type. It is assumed that a frame to be output (hereinafter, referred to as an output frame) includes a scene in which a fire has occurred. At least one processor (111) can identify (or obtain) that the primary color is red based on color information included in the current output frame. In addition, at least one processor (111) can identify a color temperature corresponding to the identified primary color (red). The color temperature corresponding to red may be a warm type. At least one processor (111) can use an artificial intelligence model to obtain the color information or primary color of the frame. According to various embodiments, the artificial intelligence model can be stored in the electronic device (100) (e.g., the memory (113)). According to another embodiment, the artificial intelligence model can be stored in an external server that can communicate with the electronic device (100).

[0176] The memory (113) may be implemented as an internal memory such as a ROM (e.g., an electrically erasable programmable read-only memory (EEPROM)) or RAM included in at least one processor (111), or may be implemented as a separate memory from at least one processor (111). In this case, the memory (113) may be implemented as a memory embedded in the electronic device (100) or as a memory detachable from the electronic device (100) depending on the purpose of data storage. For example, data for driving the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding functions of the electronic device (100) may be stored in a memory detachable from the electronic device (100).

[0177] In the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD), and in the case of memory that can be attached or detached to the electronic device (100), it may be implemented in the form of a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc.

[0178] The memory (113) may store at least one command regarding the electronic device (100). In addition, the memory (113) may store an O / S (Operating System) for driving the electronic device (100). The memory (113) may also store various software programs or applications for operating the electronic device (100) according to various embodiments of the present disclosure. In addition, the memory (113) may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

[0179] The memory (113) may store various software modules for operating the electronic device (100) according to various embodiments of the present disclosure, and at least one processor (111) may control the operation of the electronic device (100) by executing the various software modules stored in the memory (113). That is, the memory (113) is accessed by at least one processor (111), and data reading / recording / modifying / deleting / updating, etc. may be performed by at least one processor (111).

[0180] In the present disclosure, the term memory (113) may be used to mean a storage unit, a ROM, a RAM within at least one processor (111), or a memory card (e.g., a micro SD card, a memory stick) mounted on an electronic device (100).

[0181] The communication interface (114) is a configuration that performs communication with various types of external devices according to various types of communication methods. The communication interface (114) may include a wireless communication module or a wired communication module. Each communication module may be implemented in the form of at least one hardware chip.

[0182] A wireless communication module may be a module that communicates wirelessly with an external device. For example, the wireless communication module may include at least one of a Wi-Fi module, a Bluetooth module, an infrared communication module, or other communication modules.

[0183] Wi-Fi and Bluetooth modules can communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, various connection information, such as the service set identifier (SSID) and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received.

[0184] Infrared communication modules perform communication based on infrared communication (IrDA, infrared Data Association) technology, which transmits data wirelessly over short distances using infrared light, which is between visible light and millimeter waves.

[0185] In addition to the above-described communication method, other communication modules may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0186] A wired communication module may be a module that communicates with an external device via a wire. For example, the wired communication module may include at least one of a Local Area Network (LAN) module, an Ethernet module, a paired cable, a coaxial cable, a fiber optic cable, or an Ultra Wide-Band (UWB) module.

[0187] The manipulation interface (115) may include various types of input devices. For example, the manipulation interface (115) may include a physical button. In this case, the physical button may include a function key, a directional key (e.g., a four-way key), or a dial button. According to various embodiments, the physical button may be implemented as multiple keys. According to another embodiment, the physical button may be implemented as one key. When the physical button is implemented as one key, the electronic device (100) may receive a user input in which one key is pressed for a threshold time or longer. When a user input in which one key is pressed for a threshold time or longer is received, at least one processor (111) may perform a function corresponding to the user input. For example, at least one processor (111) may provide a lighting function based on the user input.

[0188] The manipulation interface (115) can receive user input using a non-contact method. When receiving user input using a contact method, physical force must be transmitted to the electronic device (100). Therefore, a method for controlling the electronic device (100) regardless of physical force may be required. The manipulation interface (115) can receive user gestures and perform operations corresponding to the received user gestures. The manipulation interface (115) can receive user gestures through a sensor (e.g., an image sensor or an infrared sensor).

[0189] The manipulation interface (115) can receive user input using a touch method. For example, the manipulation interface (115) can receive user input via a touch sensor. According to various embodiments, the touch method can be implemented in a non-contact manner. For example, the touch sensor can determine whether the user's body has approached within a threshold distance. The touch sensor can identify user input even when the user does not touch the touch sensor. According to another implementation example, the touch sensor can identify user input when the user touches the touch sensor.

[0190] The electronic device (100) can receive user input in various ways other than the above-described operation interface (115). In various embodiments, the electronic device (100) can receive user input through an external remote control device. The external remote control device can be a remote control device corresponding to the electronic device (100) (e.g., a dedicated control device of the electronic device (100)) or a user's portable communication device (e.g., a smartphone or wearable device). The user's portable communication device can store an application for controlling the electronic device (100). The portable communication device can obtain user input through the stored application and transmit the obtained user input to the electronic device (100). The electronic device (100) can receive user input from the portable communication device and perform an operation corresponding to the user's control command.

[0191] The electronic device (100) can receive user input using voice recognition. According to various embodiments, the electronic device (100) can receive the user's voice through a microphone included in the electronic device (100). According to other embodiments, the electronic device (100) can receive the user's voice from a microphone or an external device. The external device can acquire the user's voice through the microphone of the external device and transmit the acquired user's voice to the electronic device (100). The user's voice transmitted from the external device can be audio data or digital data converted from audio data (e.g., audio data converted into a frequency domain, etc.). The electronic device (100) can perform an operation corresponding to the received user's voice. The electronic device (100) can receive audio data corresponding to the user's voice through the microphone. In addition, the electronic device (100) can convert the received audio data into digital data. In addition, the electronic device (100) can convert the converted digital data into text data using the STT (Speech To Text) function. According to various embodiments, the STT (Speech To Text) function can be performed directly in the electronic device (100).

[0192] According to another embodiment, the STT (Speech To Text) function may be performed by an external server. The electronic device (100) may transmit digital data to the external server. The external server may convert the digital data into text data and obtain control command data based on the converted text data. The external server may transmit the control command data (which may also include text data) to the electronic device (100). The electronic device (100) may perform an operation corresponding to the user's voice based on the obtained control command data.

[0193] The electronic device (100) may provide a voice recognition function using a single assistant (or artificial intelligence assistant, e.g., Bixby™), but this is merely an example and the electronic device (100) may provide a voice recognition function using multiple assistants. In this case, the electronic device (100) may provide a voice recognition function by selecting one of the multiple assistants based on a trigger word corresponding to the assistant or a key present on a remote control.

[0194] The electronic device (100) can receive user input using screen interaction. Screen interaction may refer to a function of identifying whether a predetermined event occurs through an image projected by the electronic device (100) on the screen (or projection surface) and acquiring user input based on the predetermined event. The predetermined event may refer to an event in which a predetermined object is identified at a location (e.g., a location where a UI for receiving user input is projected). The predetermined object may include at least one of a part of the user's body (e.g., a finger), a pointer, or a laser point. If the predetermined object is identified at a location corresponding to the projected UI, the electronic device (100) may identify that a user input for selecting the projected UI has been received. For example, the electronic device (100) may project a guide image to display the UI on the screen. In addition, the electronic device (100) may identify whether the user selects the projected UI. The electronic device (100) can identify that the user has selected the projected UI when a predetermined object is identified at the location of the projected UI. The projected UI can include at least one item. The electronic device (100) can perform spatial analysis to identify whether the predetermined object is at the location of the projected UI. The electronic device (100) can perform spatial analysis through a sensor (e.g., an image sensor, an infrared sensor, a depth camera, a distance sensor, etc.). By performing the spatial analysis, the electronic device (100) can identify whether a predetermined event occurs at the location (the location where the UI is projected). In addition, when it is identified that the predetermined event occurs at the location (the location where the UI is projected), the electronic device (100) can identify that a user input for selecting a UI corresponding to the location has been received.

[0195] The input / output interface (116) is configured to input / output at least one of an audio signal and an image signal. The input / output interface (116) can receive at least one of an audio signal and an image signal from an external device, and can output a control command to the external device.

[0196] Depending on the implementation example, the input / output interface (116) may be implemented as an interface that inputs / outputs only audio signals and an interface that inputs / outputs only image signals, or may be implemented as one interface that inputs / outputs both audio signals and image signals.

[0197] In various embodiments of the present disclosure, the input / output interface (116) may be implemented as at least one wired input / output interface among HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), USB C-type, DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (Dsubminiature), and DVI (Digital Visual Interface). According to various embodiments, the wired input / output interface may be implemented as an interface that inputs / outputs only audio signals and an interface that inputs / outputs only image signals, or may be implemented as one interface that inputs / outputs both audio signals and image signals.

[0198] The electronic device (100) can receive data via a wired input / output interface, but this is merely an example of various embodiments, and can also receive power via the wired input / output interface. For example, the electronic device (100) can receive power from an external battery via a USB C-type or from an outlet via a power adapter. As another example, the electronic device (100) can receive power from an external device (e.g., a laptop or monitor) via a DP.

[0199] The audio signal may be implemented to be input through a wired input / output interface, and the image signal may be implemented to be input through a wireless input / output interface (or communication interface). Alternatively, the audio signal may be implemented to be input through a wireless input / output interface (or communication interface), and the image signal may be implemented to be input through a wired input / output interface.

[0200] The speaker (117) is a component that outputs an audio signal. The speaker (117) may include an audio output mixer, an audio signal processor, and an audio output module. The audio output mixer may synthesize a plurality of audio signals to be output into at least one audio signal. For example, the audio output mixer may synthesize an analog audio signal and another analog audio signal (e.g., an analog audio signal received from the outside) into at least one analog audio signal. The audio output module may include a speaker or an output terminal. According to various embodiments, the audio output module may include a plurality of speakers, and in this case, the audio output module may be arranged inside the main body, and sound emitted by covering at least a portion of the diaphragm of the audio output module may pass through a waveguide and be transmitted to the outside of the main body. The audio output module may include a plurality of audio output units, and the plurality of audio output units may be arranged symmetrically on the exterior of the main body, so as to radiate sound in all directions, that is, in a 360-degree omnidirectional manner.

[0201] The microphone (118) is a component for receiving a user's voice or other sounds and converting them into audio data. The microphone (118) can receive the user's voice in an activated state. For example, the microphone (118) can be formed integrally on the upper side, the front side, the side side, etc. of the electronic device (100). The microphone (118) can include various components such as a microphone for collecting the user's voice in analog form, an amplifier circuit for amplifying the collected user's voice, an A / D conversion circuit for sampling the amplified user's voice and converting it into a digital signal, and a filter circuit for removing noise components from the converted digital signal.

[0202] The power supply unit (119) can receive power from an external source and supply power to various components of the electronic device (100). The power supply unit (119) according to various embodiments of the present disclosure can receive power through various methods. In various embodiments, the power supply unit (119) can receive power using a connector (130) as illustrated in FIG. 1. The power supply unit (119) can receive power using a 220 V DC power cord. However, the present invention is not limited thereto, and the electronic device (100) can receive power using a USB power cord or a wireless charging method.

[0203] The power supply unit (119) can be supplied with power using an internal battery or an external battery. The power supply unit (119) according to various embodiments of the present disclosure can be supplied with power through the internal battery. For example, the power supply unit (119) can charge the power of the internal battery using at least one of a 220V DC power cord, a USB power cord, and a USB C-Type power cord, and can be supplied with power through the charged internal battery. The power supply unit (119) according to various embodiments of the present disclosure can be supplied with power through an external battery. For example, when the electronic device (100) is connected to the external battery through various wired communication methods such as a USB power cord, a USB C-Type power cord, and a socket home, the power supply unit (119) can be supplied with power through the external battery. That is, the power supply unit (119) can be supplied with power directly from the external battery, or can charge the internal battery through the external battery and be supplied with power from the charged internal battery.

[0204] The power supply unit (119) according to the present disclosure can receive power using at least one of the multiple power supply methods described above.

[0205] With respect to power consumption, the electronic device (100) may have a power consumption value (e.g., 43 W) or lower due to socket type and other standards. In this case, the electronic device (100) may vary its power consumption to reduce power consumption when using a battery. That is, the electronic device (100) may vary its power consumption based on the power supply method, power usage, etc.

[0206] The driving unit (120) can drive at least one hardware component included in the electronic device (100). The driving unit (120) can generate a physical force and transmit it to at least one hardware component included in the electronic device (100).

[0207] The driving unit (120) can generate driving power for movement of a hardware component included in the electronic device (100) (e.g., movement of the electronic device (100)) or rotation of the component (e.g., rotation of a projection lens).

[0208] The driving unit (120) can adjust the projection angle of the projection unit (112). The driving unit (120) can move the position of the electronic device (100). The driving unit (120) can control a moving member to move the electronic device (100). For example, the driving unit (120) can control the moving member using a motor.

[0209] The sensor unit (121) may include at least one sensor. The sensor unit (121) may include at least one of a tilt sensor for detecting the tilt of the electronic device (100) and an image sensor for capturing an image. The tilt sensor may be an acceleration sensor or a gyro sensor, and the image sensor may mean a camera or a depth camera. The tilt sensor may be described as a motion sensor. The sensor unit (121) may include various sensors in addition to the tilt sensor or the image sensor. For example, the sensor unit (121) may include an illuminance sensor and a distance sensor. The distance sensor may be a ToF (Time of Flight). The sensor unit (121) may include a lidar sensor.

[0210] The electronic device (100) can control a lighting function by linking with an external device. The electronic device (100) can receive lighting information from the external device. The lighting information can include at least one of brightness information or color temperature information set in the external device. The external device can mean a device connected to the same network as the electronic device (100) (e.g., an IoT device included in the same home / work network) or a device that is not in the same network as the electronic device (100) but can communicate with the electronic device (100) (e.g., a remote control server). For example, assume that an external lighting device (IoT device) included in the same network as the electronic device (100) is outputting red light at a brightness of 50. The external lighting device (IoT device) can directly or indirectly transmit lighting information (e.g., information indicating that it is outputting red light at a brightness of 50) to the electronic device (100). The electronic device (100) can control the output of a light source based on the lighting information received from the external lighting device. For example, if the lighting information received from the external lighting device includes information to output red light at a brightness of 50, the electronic device (100) can output red light at a brightness of 50.

[0211] The electronic device (100) can control a lighting function based on biometric information. At least one processor (111) can obtain the user's biometric information. The biometric information can include at least one of the user's body temperature, heart rate, blood pressure, respiration, and electrocardiogram. The biometric information can include various types of information in addition to the information described above. For example, the electronic device (100) can include a sensor for measuring biometric information. The at least one processor (111) can obtain the user's biometric information through the sensor and control the output of a light source based on the obtained biometric information. As another example, the at least one processor (111) can receive the biometric information from an external device through an input / output interface (116). The external device can refer to the user's portable communication device (e.g., a smartphone or a wearable device). The at least one processor (111) can obtain the user's biometric information from the external device and control the output of the light source based on the obtained biometric information. According to an implementation example, the electronic device (100) can identify whether the user is sleeping, and if the user is identified as sleeping (or preparing to sleep), at least one processor (111) can control the output of the light source based on the user's biometric information.

[0212] An electronic device (100) according to various embodiments of the present disclosure can provide various smart functions.

[0213] The electronic device (100) is connected to a portable terminal device for controlling the electronic device (100), and a screen output from the electronic device (100) can be controlled through user input input from the portable terminal device. For example, the portable terminal device can be implemented as a smartphone including a touch display, and the electronic device (100) receives screen data provided by the portable terminal device from the portable terminal device and outputs it, and a screen output from the electronic device (100) can be controlled according to user input input from the portable terminal device.

[0214] The electronic device (100) can share content or music provided by the mobile terminal device by connecting to the mobile terminal device through various communication methods such as Miracast, Airplay, wireless DEX, and Remote PC.

[0215] In addition, the mobile terminal device and the electronic device (100) can be connected in various connection methods. In various embodiments, the mobile terminal device can search for the electronic device (100) to perform a wireless connection, or the electronic device (100) can search for the mobile terminal device to perform a wireless connection. In addition, the electronic device (100) can output content provided by the mobile terminal device.

[0216] In various embodiments, when a mobile terminal device is placed near an electronic device (100) while content or music is being output from the mobile terminal device, and a preset gesture is detected through the display of the mobile terminal device (e.g., motion tap view), the electronic device (100) can output the content or music being output from the mobile terminal device.

[0217] In various embodiments, when the mobile terminal device is outputting content or music and the mobile terminal device comes closer to the electronic device (100) to a preset distance or less (e.g., non-contact tap view) or the mobile terminal device comes into contact with the electronic device (100) twice at a short interval (e.g., contact tap view), the electronic device (100) can output the content or music being output by the mobile terminal device.

[0218] In the above-described embodiment, it has been described that the same screen as the screen provided by the mobile terminal device is provided by the electronic device (100), but the present disclosure is not limited thereto. That is, when a connection is established between the mobile terminal device and the electronic device (100), the mobile terminal device may output a first screen provided by the mobile terminal device, and the electronic device (100) may output a second screen provided by the mobile terminal device that is different from the first screen. For example, the first screen may be a screen provided by a first application installed on the mobile terminal device, and the second screen may be a screen provided by a second application installed on the mobile terminal device. For example, the first screen and the second screen may be different screens provided by a single application installed on the mobile terminal device. For example, the first screen may be a screen including a remote control-type UI for controlling the second screen.

[0219] An electronic device (100) according to the present disclosure can output a standby screen. For example, if the electronic device (100) is not connected to an external device or if no input is received from the external device for a preset period of time, the electronic device (100) can output a standby screen. Conditions for the electronic device (100) to output a standby screen are not limited to the examples described above, and the standby screen can be output under various conditions.

[0220] The electronic device (100) may output a standby screen in the form of a blue screen, but the present disclosure is not limited thereto. For example, the electronic device (100) may extract only the shape of an object from data received from an external device, acquire an amorphous object, and output a standby screen including the acquired amorphous object.

[0221] The electronic device (100) may further include a display.

[0222] The display may be implemented in various forms, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), etc. The display may also include a driving circuit, a backlight unit, etc., which may be implemented in forms, such as an amorphous silicon thin film transistor (a-si TFT), a low temperature poly silicon (LTPS) TFT, and an organic TFT (OTFT). The display may be implemented in a touch screen combined with a touch sensor, a flexible display, a three-dimensional display (3D display), etc. According to various embodiments of the present disclosure, the display may include a display panel that outputs an image, as well as a bezel that houses the display panel. According to various embodiments of the present disclosure, the bezel may include a touch sensor for detecting user interaction.

[0223] The electronic device (100) may further include a shutter unit.

[0224] The shutter portion may include at least one of a shutter, a fixing member, a rail, or a body.

[0225] The shutter can block light output from the projection unit (112). The fixing member can fix the position of the shutter. The rail can be a path for moving the shutter and the fixing member. The body can be configured to include the shutter and the fixing member.

[0226] The movable member (122) may refer to a member for moving from a first position to a second position in a space where the electronic device (100) is placed. The electronic device (100) may control the movable member (122) to move the electronic device (100) using a force generated by the driving unit (120). The electronic device (100) may generate a force to be transmitted to the movable member (122) using a motor included in the driving unit (120).

[0227] The movable member (122) may include at least one wheel (e.g., a circular wheel). The electronic device (100) may move to a target location (or target position) through the movable member. When a user input or a control command is received, the electronic device (100) may rotate the movable member by transmitting a force generated through a motor to the movable member. The electronic device (100) may control the movable member to adjust the rotation speed, rotation direction, etc. The electronic device (100) may perform a movement operation (or movement function) by controlling the movable member based on the target location or the direction of movement, etc.

[0228] FIG. 4 is a drawing for explaining an operation of acquiring horizontal plane data and vertical plane data through a sensor unit (121) according to one embodiment.

[0229] Referring to FIG. 4, the sensor unit (121) may include at least one of a first distance sensor, an acceleration sensor, a gyro sensor, a second distance sensor, a position sensor, a tilt sensor, and a vision sensor.

[0230] Referring to the embodiment (410) of FIG. 4, the sensor unit (121) may include a first distance sensor and an acceleration sensor.

[0231] The electronic device (100) can obtain distance data through a first distance sensor. The electronic device (100) can detect an edge based on the distance data. The electronic device (100) can obtain the distance between the electronic device (100) and the edge based on the distance data. The distance between the electronic device (100) and the edge can be described as the distance between the first distance sensor and the edge.

[0232] For example, the first distance sensor may be a Lidar sensor.

[0233] The electronic device (100) can obtain acceleration data through an acceleration sensor. The electronic device (100) can recognize a direction based on the acceleration data.

[0234] Recognizing a direction may refer to recognizing in which direction the electronic device (100) is positioned. For example, the electronic device (100) may identify that it is positioned toward a first location using acceleration data.

[0235] For example, an acceleration sensor can be implemented as an inertial measurement unit (IMU).

[0236] According to various embodiments, the electronic device (100) may recognize a direction using a gyro sensor instead of an acceleration sensor. The electronic device (100) may obtain gyro data through the gyro sensor. The electronic device (100) may recognize a direction based on the gyro data.

[0237] The electronic device (100) can obtain horizontal plane data based on at least one of an edge and a direction. The horizontal plane can refer to a plane perpendicular to the z-axis direction (see FIG. 13) based on the driving direction of the electronic device (100). The electronic device (100) can obtain at least one of the distance between the electronic device (100) and the edge, and the position (direction) of the electronic device (100) that recognized the edge. The electronic device (100) can obtain (or identify or generate) horizontal plane data of space based on at least one of the obtained distance and position (direction).

[0238] Horizontal plane data may include the structure of a horizontal plane of a space (or target space). The horizontal plane structure may include the spatial structure of at least one region.

[0239] Referring to the embodiment (420) of FIG. 4, the sensor unit (121) may include at least one of a second distance sensor, an acceleration sensor, a position sensor, a tilt sensor, and a vision sensor.

[0240] The second distance sensor may correspond to the first distance sensor of embodiment (410). According to various embodiments, the second distance sensor may be implemented as a ToF (Time of Flight) sensor.

[0241] The acceleration sensor may correspond to the acceleration sensor of embodiment (420).

[0242] A location sensor can collect data to identify the location of an electronic device (100). The electronic device (100) can obtain location data through the location sensor. The location sensor can be implemented as a sensor that transmits and / or receives beacon signals, a GPS sensor, a lidar sensor, or the like.

[0243] According to various embodiments, the electronic device (100) may determine the current location of the electronic device (100) without a location sensor. For example, the electronic device (100) may determine the current location of the electronic device (100) based on data obtained from at least one of a first distance sensor and a second distance sensor.

[0244] The tilt sensor can collect tilt data indicating the tilt of the electronic device (100). The tilt data may be data indicating the degree to which the electronic device (100) is tilted in a direction. The direction may represent at least one of the three-dimensional axes.

[0245] The electronic device (100) can recognize the inclination of the electronic device (100) based on inclination data collected from the inclination sensor. Recognizing the inclination may refer to recognizing whether the electronic device (100) is rotated about a certain axis in three-dimensional space. A description related to this is provided in FIG. 13.

[0246] For example, a tilt sensor can be implemented as a gyro sensor.

[0247] A vision sensor can collect data to acquire an image. An electronic device (100) can acquire image data through the vision sensor. The electronic device (100) can detect an object based on the image data.

[0248] For example, a vision sensor can be implemented as a camera.

[0249] The electronic device (100) can acquire vertical plane data based on at least one of an edge, a direction, a position, a slope, and an object. The vertical plane can refer to a plane perpendicular to the x-axis direction (see FIG. 13) or a plane perpendicular to the y-axis direction based on the driving direction of the electronic device (100). The vertical plane can refer to a plane perpendicular to a horizontal plane.

[0250] The electronic device (100) can obtain at least one of the following: the distance between the electronic device (100) and an edge, the position (direction) of the electronic device (100) that recognized the edge, the inclination of the electronic device (100), the distance between the electronic device (100) and an object, and the position (direction) of the electronic device (100) that recognized the object. The electronic device (100) can obtain (or identify or generate) vertical plane data of space based on the obtained information.

[0251] Vertical plane data may include the structure of a vertical plane of a space (or target space). The vertical plane structure may include the spatial structure of at least one region.

[0252] According to various embodiments, the electronic device (100) may first perform a sensing operation used to obtain horizontal plane data of embodiment (410), and then perform a sensing operation used to obtain vertical plane data of embodiment (420).

[0253] FIG. 5 is a drawing for explaining an operation of acquiring horizontal plane data and vertical plane data through a sensor unit (121) according to one embodiment.

[0254] Referring to the embodiment (510) of FIG. 5, the sensor unit (121) may include at least one of a first distance sensor, a second distance sensor, an acceleration sensor, a position sensor, a tilt sensor, and a vision sensor. A description thereof is provided in FIG. 4.

[0255] The electronic device (100) can simultaneously perform sensing operations used to acquire horizontal plane data and vertical plane data. The electronic device (100) can receive data from at least one of a first distance sensor, a second distance sensor, an acceleration sensor, a position sensor, a tilt sensor, and a vision sensor. The electronic device (100) can identify horizontal plane data and vertical plane data based on the received data.

[0256] For example, the electronic device (100) can detect an edge based on distance data acquired through a first distance sensor. The electronic device (100) can recognize the direction of the electronic device (100) based on acceleration data acquired through an acceleration sensor. The electronic device (100) can acquire horizontal plane data based on the detected edge and recognized direction.

[0257] For example, the electronic device (100) can detect an edge based on distance data acquired through a second distance sensor. The electronic device (100) can recognize a direction of the electronic device (100) based on acceleration data acquired through an acceleration sensor. The electronic device (100) can recognize a location of the electronic device (100) based on location data acquired through a position sensor. The electronic device (100) can recognize an inclination of the electronic device (100) based on inclination data acquired through a tilt sensor. The electronic device (100) can detect an object based on image data acquired through a vision sensor.

[0258] The electronic device (100) can obtain vertical plane data based on at least one of a detected edge, a recognized direction, a recognized position, a recognized inclination, and a detected object.

[0259] When location data is used, the electronic device (100) can accurately identify the location of an object or an edge location, etc. In various embodiments, location data may not be used when acquiring vertical plane data.

[0260] FIG. 6 is a diagram for explaining an operation of generating a map according to one embodiment.

[0261] Referring to FIG. 6, the electronic device (100) can acquire horizontal plane (first plane) data through the first drive (S610). The electronic device (100) can determine whether analysis of the horizontal plane (first plane) of all spaces has been completed (S620).

[0262] If the analysis is not completed (S620-N), the electronic device (100) may repeat step S610.

[0263] Once the analysis is completed (S620-Y), the electronic device (100) can acquire vertical plane (second plane) data through a second drive (S630). The electronic device (100) can determine whether the analysis of the vertical plane (second plane) of all spaces has been completed (S640).

[0264] If the analysis is not completed (S640-N), the electronic device (100) may repeat step S630.

[0265] Once the analysis is complete (S640-Y), the electronic device (100) can generate a map based on the horizontal plane (first plane) data and the vertical plane (second plane) data (S650).

[0266] FIG. 7 is a diagram illustrating an operation of generating a map based on a preset event, according to one embodiment.

[0267] Steps S710, S720, S730, and S740 of FIG. 7 may correspond to steps S610, S620, S630, and S640 of FIG. 6.

[0268] The electronic device (100) can determine whether a preset event has occurred (S705). The preset event may include at least one of an event in which a setup mode is executed after the first boot, an event in which preset user input is received, an event in which preset input is received from an external device, and a notification event.

[0269] The event that causes setup mode to run after the first boot can represent the event that causes setup mode to run when power is supplied from a factory reset state or a setup reset state.

[0270] An event in which a preset user input is received may indicate an event in which user input requesting a map is received. The user input may be received via voice or manipulation.

[0271] An event in which a preset input is received from an external device may represent an event in which an input requesting a map is received from an external device.

[0272] A notification event can indicate an event that triggers a preset notification. A preset notification can be a notification indicating a change in map-related information. For example, a notification event can include a notification indicating a change in the location of an object in space.

[0273] When a preset event occurs (S705-Y), the electronic device (100) can perform steps S710, S720, S730, and S740.

[0274] When the analysis of the vertical plane (second plane) of all spaces is completed (S740-Y), the electronic device (100) can obtain spatial information based on the horizontal plane (first plane) data and the vertical plane (second plane) data (S751).

[0275] The electronic device (100) can obtain object information based on horizontal plane (first plane) data and vertical plane (second plane) data (S752).

[0276] The electronic device (100) can generate a map including spatial information and object information (S753).

[0277] FIG. 8 is a diagram for explaining an operation of generating a map based on spatial information and object information, according to one embodiment.

[0278] Referring to FIG. 8, the electronic device (100) can obtain horizontal plane (first plane) data and vertical plane (second plane) data (S810). The electronic device (100) can obtain edge information based on the horizontal plane (first plane) data and the vertical plane (second plane) data (S821). The electronic device (100) can identify data representing edges detected in the horizontal plane (first plane) data and the vertical plane (second plane) data. The electronic device (100) can obtain edge information based on the identified data.

[0279] Edge information may include various information related to a detected edge. Edge information may include at least one of edge shape, edge position, and edge distance.

[0280] The electronic device (100) can identify at least one area based on edge information (S822). The electronic device (100) can identify at least one area in the space being analyzed. The electronic device (100) can identify an edge structure using edge information. The electronic device (100) can identify an edge structure through an edge location.

[0281] For example, the electronic device (100) can identify a candidate region in which three edges are connected to each other within a critical angle in an edge structure. The electronic device (100) can identify a region among the candidate regions that has an open structure in a direction without edges as a target region. The electronic device (100) can identify a region in which the length of the region having an open structure is within a critical length as a target region.

[0282] The electronic device (100) can identify at least one target area that satisfies a preset condition in space. The electronic device (100) can obtain spatial information indicating the coordinates of the at least one target area (S823).

[0283] The electronic device (100) can detect an object based on horizontal plane (first plane) data and vertical plane (second plane) data. The electronic device (100) can identify the object type (S831). The electronic device (100) can identify the object location (S832). The electronic device (100) can obtain object information including the object type and object location (S833).

[0284] The electronic device (100) can generate a map including spatial information and object information (S840).

[0285] FIG. 9 is a diagram illustrating an operation of generating a map by merging distance data according to one embodiment.

[0286] Referring to FIG. 9, the electronic device (100) can obtain horizontal plane (first plane) data and vertical plane (second plane) data (S905).

[0287] The electronic device (100) can detect edges based on horizontal plane (first plane) data and vertical plane (second plane) data (S910). The electronic device (100) can identify a target area based on the detected edge (S915). The target area may refer to an area that is closed by a threshold ratio or more among the outlines of the target area.

[0288] The electronic device (100) can detect an object based on horizontal plane (first plane) data and vertical plane (second plane) data (S920).

[0289] After the target area and object are identified, the electronic device (100) can merge the distance data of the target area and the distance data of the object (S925). The electronic device (100) can identify the relative position of the target area with respect to the electronic device (100) using the distance between the electronic device (100) and the target area. The electronic device (100) can identify the relative position of the object with respect to the electronic device (100) using the distance between the electronic device (100) and the object.

[0290] The electronic device (100) can merge distance data of a target area with distance data of an object. The distance data can be merged based on the direction in which the electronic device (100) detected the distance data.

[0291] For example, the electronic device (100) can obtain distance data of a target area detected at a first location and in a first direction. The electronic device (100) can obtain distance data of an object detected at the first location and in the first direction. The electronic device (100) can merge the distance data of the target area detected at the first location and in the first direction and the distance data of the object. The first location can indicate a location (or coordinate) in space. The first direction can indicate a direction in which the electronic device (100) senses data or a driving direction.

[0292] The electronic device (100) can detect plane information based on the merged distance data (S930). The plane information can include at least one of the structure of the plane and the slope of the plane.

[0293] The electronic device (100) can obtain the structure of a plane by using the distance deviation of edges. The electronic device (100) can obtain the structure of a plane by using the distance deviation of a plurality of detected edges.

[0294] For example, the electronic device (100) can obtain a first distance from a first location to a first edge and a second distance from a second location to the first edge. The electronic device (100) can obtain a structure of a plane based on the deviation between the first distance and the second distance.

[0295] The electronic device (100) can obtain the inclination of a plane by using the distance deviation of edges. The electronic device (100) can obtain the inclination of a plane by using the distance deviation of a plurality of detected edges.

[0296] For example, the electronic device (100) can obtain a first distance from a first position to a first edge detected with a first slope and a second distance from a second position to a first edge detected with a second slope. The electronic device (100) can obtain a slope of a plane based on a deviation (first deviation) between the first slope and the second slope and a deviation (second deviation) between the first distance and the second distance.

[0297] The electronic device (100) can identify (or specify) the location of an object based on the merged distance data and plane information (S935).

[0298] The electronic device (100) can generate (or update) a map based on plane information and the location of an object (S940).

[0299] FIG. 10 is a drawing for explaining horizontal inclination (yaw direction inclination) according to one embodiment.

[0300] Referring to FIG. 10, according to an embodiment (1010), an electronic device (100) can output a projection image (1011) in a horizontal projection direction onto a projection surface (10). It is assumed that the horizontal inclination is 0. The horizontal inclination may refer to the degree to which the electronic device (100) is tilted to the left or right toward the front.

[0301] According to an embodiment (1020), an electronic device (100) can output a projection image (1021) in a horizontal projection direction to a projection surface (10). It is assumed that the horizontal inclination (1022) is 30 degrees. When the horizontal inclination is 30 degrees to the right, the electronic device (100) can output the projection image (1021) to the right by 30 degrees to the right on the projection surface (10).

[0302] The electronic device (100) can obtain horizontal plane data and vertical plane data based on the horizontal inclination. The electronic device (100) can obtain sensor data based on the horizontal inclination. The electronic device (100) can obtain at least one of the horizontal plane data and the vertical plane data based on the horizontal inclination and the sensor data.

[0303] The horizontal inclination can represent the rotation angle (yaw) with respect to the z-axis in Fig. 13. The horizontal inclination can be described as the yaw angle.

[0304] FIG. 11 is a drawing for explaining vertical inclination (pitch direction inclination) according to one embodiment.

[0305] Referring to FIG. 11, according to an embodiment (1110), an electronic device (100) can output a projection image on a projection surface in a horizontal projection direction. It is assumed that the vertical inclination is 0. The vertical inclination may refer to the degree to which the electronic device (100) is tilted upward or downward toward the front. If the vertical inclination is 0, it may be a situation in which the projection image is output horizontally. A virtual line (1111) representing the floor and a virtual line (1111) along which the electronic device (100) faces the front may be the same (or parallel).

[0306] According to an embodiment (1120), the electronic device (100) can output a projection image on a projection surface in a horizontal projection direction. It is assumed that the vertical inclination (1122) is 30 degrees. If the vertical inclination is 30 degrees upward, the electronic device (100) can output the projection image upward by 30 degrees upward on the projection surface. The virtual line (1111) representing the floor surface and the virtual line (1121) along which the electronic device (100) faces the front may not be parallel. The vertical inclination (1122) may represent the angle between the virtual line (1111) representing the floor surface and the virtual line (1121) along which the electronic device (100) faces the front.

[0307] The electronic device (100) can obtain horizontal plane data and vertical plane data based on the vertical inclination. The electronic device (100) can obtain sensor data based on the vertical inclination. The electronic device (100) can obtain at least one of the horizontal plane data and the vertical plane data based on the vertical inclination and the sensor data.

[0308] The vertical slope can represent the rotation angle (pitch) relative to the y-axis in Fig. 13. The vertical slope can be described as a pitch angle.

[0309] FIG. 12 is a drawing for explaining horizontal distortion (roll direction inclination) according to one embodiment.

[0310] Referring to FIG. 12, according to an embodiment (1210), an electronic device (100) can output a projection image without horizontal distortion. A reference for no horizontal distortion is indicated by a horizontal line (1211). According to an embodiment (1210), the reference horizontal line and the horizontal line of the electronic device (100) can be aligned.

[0311] According to an embodiment (1220), the electronic device (100) may have a horizontal deviation (1222) of 30 degrees to the right. The reference horizontal line (1211) and the horizontal line (1221) of the electronic device may differ by the horizontal deviation (1222).

[0312] The electronic device (100) can obtain horizontal plane data and vertical plane data based on horizontal distortion. The electronic device (100) can obtain sensor data based on horizontal distortion. The electronic device (100) can obtain at least one of horizontal plane data and vertical plane data based on horizontal distortion and sensor data.

[0313] Horizontal distortion can represent the rotation angle (roll) with respect to the x-axis in Fig. 13. Horizontal distortion can be described as a roll angle.

[0314] FIG. 13 is a drawing for explaining rotation information of an electronic device (100) according to one embodiment.

[0315] FIG. 13 is a drawing for explaining the horizontal distortion, horizontal inclination, and vertical inclination of the electronic device (100).

[0316] Example 13 (1310) of FIG. 13 is a graph defining rotation directions along the x, y, and z axes. Rotation around the x-axis can be defined as roll, rotation around the y-axis can be defined as pitch, and rotation around the z-axis can be defined as yaw.

[0317] Embodiment (1320) of Fig. 13 can explain the rotation direction of the projection surface (10) as the rotation direction defined in embodiment (1310). The x-axis rotation information of the projection surface (10) may correspond to a roll that rotates based on the x-axis of the projection surface (10). The y-axis rotation information of the projection surface (10) may correspond to a pitch that rotates based on the y-axis of the projection surface (10). The z-axis rotation information of the projection surface (10) may correspond to a yaw that rotates based on the z-axis of the projection surface (10).

[0318] The x-axis rotation information can be described as first-axis rotation information, first-axis tilt information, or horizontal distortion information.

[0319] The y-axis rotation information can be described as second-axis rotation information, second-axis tilt information, or vertical tilt information.

[0320] The z-axis rotation information can be described as third-axis rotation information, third-axis tilt information, or horizontal tilt information.

[0321] The sensor unit (121) can obtain status information of the electronic device (100). The status information of the electronic device (100) may refer to the rotational status of the electronic device (100). The sensor unit (121) may include at least one of a gravity sensor, an acceleration sensor, or a gyro sensor. The x-axis rotation information of the electronic device (100) and the y-axis rotation information of the electronic device (100) may be determined based on sensor data obtained through the sensor unit (121). However, it may be difficult to set a standard for the z-axis rotation information of the electronic device (100) unless it is based on east, west, south, or north. Therefore, the electronic device (100) may consider the status information of the projection surface (10) without separately considering the z-axis rotation information of the electronic device (100). The electronic device (100) may perform an image correction operation by considering the z-axis rotation information of the projection surface (10).

[0322] FIG. 14 is a drawing for explaining rotation information of a projection surface according to one embodiment.

[0323] Example 14 (1410) of FIG. 14 is a graph defining rotation directions along the x, y, and z axes. Rotation around the x-axis can be defined as roll, rotation around the y-axis can be defined as pitch, and rotation around the z-axis can be defined as yaw.

[0324] Embodiment (1420) of Fig. 14 can explain the rotation direction of the projection surface (10) as the rotation direction defined in embodiment (1410). The x-axis rotation information of the projection surface (10) may correspond to a roll that rotates based on the x-axis of the projection surface (10). The y-axis rotation information of the projection surface (10) may correspond to a pitch that rotates based on the y-axis of the projection surface (10). The z-axis rotation information of the projection surface (10) may correspond to a yaw that rotates based on the z-axis of the projection surface (10).

[0325] The x-axis rotation information can be described as the first-axis rotation information. The y-axis rotation information can be described as the second-axis rotation information. The z-axis rotation information can be described as the third-axis rotation information.

[0326] FIG. 15 is a drawing for explaining z-axis rotation information of a projection surface according to one embodiment.

[0327] Example 15 (1510) of FIG. 15 is a drawing of an electronic device (100) viewed from above, showing a situation in which the electronic device (100) outputs a projection image while the projection surface (10) is not rotated along the z-axis. It is assumed that the electronic device (100) is placed on a table (20).

[0328] Example 15 (1520) of FIG. 15 is a drawing of an electronic device (100) viewed from above, showing a situation in which the electronic device (100) outputs a projection image while the projection surface (10) is rotated counterclockwise by a certain angle (θ1) with respect to the z-axis. It is assumed that the electronic device (100) is placed on a table (20).

[0329] The electronic device (100) can identify that the plane (10) is tilted by a certain angle (θ1). The electronic device (100) can identify that the plane (10) is tilted by a certain angle (θ1) in the yaw direction.

[0330] FIG. 16 is a drawing for explaining y-axis rotation information of a projection surface according to one embodiment.

[0331] Referring to the embodiment (1610) of Fig. 16, the projection surface (10) is shown in a state where it is not rotated around the y-axis.

[0332] Referring to embodiment (1620) of Fig. 16, a state in which the projection surface (10) is rotated around the y-axis is shown. It is assumed that the projection surface (10) is rotated by a certain angle (θ2) around the y-axis.

[0333] The electronic device (100) can identify that the plane (10) is tilted by a certain angle (θ2). The electronic device (100) can identify that the plane (10) is tilted by a certain angle (θ2) in the pitch direction.

[0334] FIG. 17 is a drawing for explaining an operation of performing a keystone function by taking into account vertical inclination, according to one embodiment.

[0335] Referring to the embodiment (1710) of FIG. 17, the electronic device (100) can output a projection image in a state where a vertical tilt exists.

[0336] Referring to Example (1720), the electronic device (100) can output a projection image (1721) in a state where a vertical inclination exists, and due to the vertical inclination, the projection image (1721) can be output in a trapezoidal shape rather than a rectangular shape, which is the original image shape. To solve a problem caused by the presence of the vertical inclination, the electronic device (100) can perform a keystone function.

[0337] Referring to Example (1730), the electronic device (100) can perform a keystone function to transform the original image so that the final output projection image (1731) becomes rectangular in shape.

[0338] The electronic device (100) can acquire sensor data while tilted in the pitch direction. The electronic device (100) can acquire horizontal plane data and vertical plane data based on the sensor data acquired while tilted in the pitch direction.

[0339] FIG. 18 is a drawing for explaining an operation of performing a keystone function while taking into account horizontal inclination, according to one embodiment.

[0340] Referring to the embodiment (1810) of FIG. 18, the electronic device (100) can output a projection image in a state where a horizontal inclination exists.

[0341] Referring to Example (1820), the electronic device (100) can output a projection image (1821) in a state where a horizontal inclination exists, and due to the horizontal inclination, the projection image (1821) can be output in a trapezoidal shape rather than a rectangular shape, which is the original image shape. To solve a problem caused by the presence of the horizontal inclination, the electronic device (100) can perform a keystone function.

[0342] Referring to embodiment (1830), the electronic device (100) can perform a keystone function to transform the original image so that the final output projection image (1831) becomes rectangular in shape.

[0343] The electronic device (100) can acquire sensor data while tilted in the yaw direction. The electronic device (100) can acquire horizontal plane data and vertical plane data based on the sensor data acquired while tilted in the yaw direction.

[0344] FIG. 19 is a drawing for explaining a horizontal plane map, a vertical plane map, and a three-dimensional map according to one embodiment.

[0345] Referring to an embodiment (1910) of FIG. 19, the electronic device (100) can obtain a horizontal plane map (1911) based on horizontal plane data. The horizontal plane map (1911) can include information on a two-dimensional horizontal plane with respect to a target space analyzed by the electronic device (100). The horizontal plane map (1911) can refer to a map of a plane viewed from the z-axis direction in a space (see FIG. 13) in which the electronic device (100) is placed. The horizontal plane map (1911) can refer to a map of an xy plane in a space (see FIG. 13) in which the electronic device (100) is placed.

[0346] Referring to an embodiment (1920) of FIG. 19, the electronic device (100) may obtain vertical plane maps (1921, 1922) based on vertical plane data. The vertical plane map (1922) may include information on a vertical plane with respect to a target space analyzed by the electronic device (100). The vertical plane maps (1921, 1922) may refer to a map including information on the z-axis in a space (see FIG. 13) in which the electronic device (100) is placed. The vertical plane maps (1922, 1922) may refer to a map of a plane perpendicular to the xy plane in the space (see FIG. 13) in which the electronic device (100) is placed.

[0347] According to various embodiments, the electronic device (100) may acquire vertical plane data after acquiring a horizontal plane map (1911). The electronic device (100) may reflect the vertical plane data on the horizontal plane map (1911) to acquire vertical plane maps (1921, 1922). The electronic device (100) may identify an edge included in the horizontal plane map (1911) and identify z-axis information of the identified edge. The electronic device (100) may identify the vertical plane maps (1921, 1922) using the z-axis information of the identified edge.

[0348] Referring to the embodiment (1930) of FIG. 19, the electronic device (100) can obtain a three-dimensional map (1931, 1932) based on a horizontal plane map (1911) and a vertical plane map (1921, 1922). The three-dimensional map (1931, 1932) may be a map that implements a target space to be analyzed in a three-dimensional form.

[0349] According to various embodiments, the 3D map (1932) may include information related to an object. The information related to the object may include at least one of the object type (e.g., a standing light, a sofa, a table, a chair, etc.) or the object's location. The 3D map (1932) may include objects arranged in spatial structures and locations. Through the 3D map (1932), a user can recognize the location of an object in a 3D form.

[0350] FIG. 20 is a drawing for explaining a three-dimensional map according to one embodiment.

[0351] Referring to FIG. 20, the electronic device (100) can obtain a first type of three-dimensional map (2010). The first type of three-dimensional map (2010) may be a three-dimensional map that does not include object information. The first type of three-dimensional map (2010) may be a map obtained by combining a horizontal plane map and a vertical plane map.

[0352] The electronic device (100) can acquire a second type of three-dimensional map (2020) by considering object information in the first type of three-dimensional map (2010). The electronic device (100) can reflect information indicating the type of object and location information of the object in the first type of three-dimensional map (2010). The electronic device (100) can generate a second type of three-dimensional map (2020) in which a UI (2021, 2022, 2023, 2024, 2025, 2026, 2027, 2028, 2029) indicating an object at a location in the first type of three-dimensional map (2010) is displayed.

[0353] Information indicating the object type may include at least one of text indicating the object type or an image (e.g., an icon) indicating the object type.

[0354] The electronic device (100) can provide a second type of three-dimensional map (2020) to the user.

[0355] For example, the electronic device (100) can display a second type of three-dimensional map (2020) through a display included in the electronic device (100).

[0356] For example, the electronic device (100) can output a second type of three-dimensional map (2020) through a projection unit (112) included in the electronic device (100).

[0357] The electronic device (100) can obtain a third type of three-dimensional map (2030) expressed in a different perspective from the second type of three-dimensional map (2020). The electronic device (100) can provide the third type of three-dimensional map (2030).

[0358] FIG. 21 is a diagram for explaining an operation of generating a map (3D map or combined map) using a horizontal plane map and a vertical plane map according to one embodiment.

[0359] Referring to FIG. 21, the electronic device (100) can obtain horizontal plane (first plane) data (S2105). The electronic device (100) can determine whether a horizontal plane (first plane) map has been generated (S2110).

[0360] If the horizontal plane (first plane) map is not generated (S2110-N), the electronic device (100) can repeat step S2105.

[0361] Once the horizontal plane (first plane) map has been generated (S2110-Y), the electronic device (100) can identify whether a preset event has occurred (S2115).

[0362] The preset event may include at least one of an event in which a closed space with three sides blocked is identified based on horizontal plane (first plane) data, an event in which a preset space (e.g., a room, a living room, etc.) is recognized based on image data, and an event in which a preset user input is received (e.g., an operation input for a button, a voice input including a word).

[0363] An event in which a closed space with three sides is identified may include an event in which a space is identified in which three edges within a critical angle form a closed region except for one of the four sides.

[0364] When a preset event is identified (S2115-Y), the electronic device (100) can acquire vertical plane (second plane) data (S2120). The electronic device (100) can determine whether a vertical plane (second plane) map has been generated (S2125).

[0365] If the vertical plane (second plane) map is not generated (S2125-N), the electronic device (100) may repeat step S2120.

[0366] Once the vertical plane (second plane) map is generated (S2125-Y), the electronic device (100) can generate a map (3D map or combined map) based on the horizontal plane (first plane) map and the vertical plane (second plane) map (S2130).

[0367] FIG. 22 is a diagram for explaining an operation of obtaining a map through a server (200) according to one embodiment.

[0368] Steps S2205, S2210, S2230, S2251, S2252, and S2253 of FIG. 22 may correspond to steps S705, S710, S730, S751, S752, and S753 of FIG. 7.

[0369] However, unlike in Fig. 7, the subject of some actions may be a server (200).

[0370] The electronic device (100) may be connected to a server (200). The server (200) may be a server that is connected to the electronic device (100) and provides a map to the electronic device (100). For example, the server (200) may be a server that manages the electronic device (100). For example, the server (200) may be an IoT (Internet of Things) server.

[0371] According to a preset event, the electronic device (100) can obtain horizontal plane (first plane) data through a first drive (S2210). The electronic device (100) can obtain vertical plane (second plane) data through a second drive (S2230). The electronic device (100) can transmit the horizontal plane (first plane) data and the vertical plane (second plane) data to the server (200) (S2235).

[0372] According to various embodiments, the electronic device (100) may first transmit horizontal plane (first plane) data, and then transmit vertical plane (second plane) data after transmitting the horizontal plane (first plane) data.

[0373] The server (200) can receive horizontal plane (first plane) data and vertical plane (second plane) data from the electronic device (100). The server (200) can obtain spatial information based on the horizontal plane (first plane) data and the vertical plane (second plane) data (S2251). The server (200) can obtain object information based on the horizontal plane (first plane) data and the vertical plane (second plane) data (S2252).

[0374] The server (200) can generate a map including spatial information and object information (S2253). The server (200) can transmit the map to the electronic device (100) (S2260).

[0375] The electronic device (100) can receive a map from the server (200) (S2260). The electronic device (100) can display the map (S2265).

[0376] According to various embodiments, in addition to the operations described in FIG. 22, the operations described in FIGS. 6 to 9 may be optionally performed on the server (200).

[0377] FIG. 23 is a drawing for explaining an operation of outputting a projection image according to one embodiment.

[0378] Referring to FIG. 23, the electronic device (100) can acquire a map (S2350). After acquiring the map, the electronic device (100) can identify a projection area based on the map (S2360). The electronic device (100) can identify a plane larger than a threshold size on the map. The electronic device (100) can identify the plane larger than the threshold size identified on the map as a projection area.

[0379] According to various embodiments, the electronic device (100) may provide a map to a user and receive user input for selecting a projection area through the provided map. The electronic device (100) may determine the projection area based on the received user input.

[0380] The electronic device (100) can move to a position corresponding to the projection area (S2370). The position corresponding to the projection area may indicate a position to which the electronic device (100) must move in order to output a projection image to the projection area.

[0381] After the electronic device (100) moves to a position corresponding to the projection area, the electronic device (100) can output a projection image to the projection area (S2380).

[0382] FIG. 24 is a drawing for explaining a control method of an electronic device (100) according to one embodiment.

[0383] Referring to FIG. 24, a control method of an electronic device includes, when a request for a map corresponding to a target space is received, a step of obtaining first sensor data in a first driving for the target space (S2410), a step of obtaining horizontal plane (first plane) data for a horizontal plane (first plane) parallel to a driving direction of the electronic device based on the first sensor data (S2420), a step of obtaining second sensor data in a second driving for the target space (S2430), a step of obtaining vertical plane (second plane) data for a vertical plane (second plane) perpendicular to the horizontal plane (first plane) based on the second sensor data (S2440), and a step of obtaining a map in which the horizontal plane (first plane) data and the vertical plane (second plane) data are combined (S2450).

[0384] The horizontal plane (first plane) data may include information about a horizontal plane (first plane) including the x-axis and y-axis based on the driving direction of the electronic device, and the vertical plane (second plane) data may include information about a vertical plane (second plane) including the z-axis based on the driving direction of the electronic device.

[0385] The step of acquiring horizontal plane (first plane) data (S2320) can acquire horizontal plane (first plane) data including first spatial information of the horizontal plane (first plane) and first object information of the horizontal plane (first plane) based on first sensor data.

[0386] The electronic device includes a first distance sensor and an acceleration sensor, and the first sensor data includes first distance data acquired through the first distance sensor and first acceleration data acquired through the acceleration sensor, and the step of acquiring horizontal plane (first plane) data (S2320) may include detecting first edge information based on the first distance data, acquiring first direction information of the electronic device based on the first acceleration data, and acquiring horizontal plane (first plane) data including first spatial information and first object information based on the first edge information and the first direction information.

[0387] The step of acquiring vertical plane (second plane) data (S2340) can acquire vertical plane (second plane) data including second spatial information of the vertical plane (second plane) and second object information of the vertical plane (second plane) based on second sensor data.

[0388] The electronic device includes a second distance sensor and an acceleration sensor, and the second sensor data includes second distance data acquired through the second distance sensor and second acceleration data acquired through the acceleration sensor, and the step of acquiring vertical plane (second plane) data (S2340) may include detecting second edge information based on the second distance data, acquiring second direction information of the electronic device based on the second acceleration data, and acquiring vertical plane (second plane) data including second spatial information and second object information based on the second edge information and the second direction information.

[0389] The electronic device may include a vision sensor, the second sensor data may include second distance data, second acceleration data, and image data acquired through the vision sensor, and the control method may include a step of updating second object information in the image data.

[0390] The electronic device includes a tilt sensor, the second sensor data includes second distance data, second acceleration data, and tilt data acquired through the tilt sensor, and the control method may include a step of acquiring a first tilt angle in a roll direction, a second tilt angle in a pitch direction, and a third tilt angle in a yaw direction of the electronic device based on the tilt data, and a step of updating second spatial information based on the first tilt angle, the second tilt angle, and the third tilt angle.

[0391] The first distance sensor may be a Lidar sensor, the second distance sensor may be a ToF (Time of Flight) sensor, and the tilt sensor may be a gyro sensor.

[0392] The step of obtaining a map (S2350) may obtain third spatial information by combining first spatial information and second spatial information based on the same location, obtain third object information by combining first object information and second object information based on the same location, and obtain a map including the third spatial information and the third object information.

[0393] The methods according to the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing electronic device.

[0394] The methods according to the various embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device.

[0395] The various embodiments of the present disclosure described above may also be performed through an embedded server provided in an electronic device, or an external server of at least one of the electronic device and the display device.

[0396] According to an example embodiment of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device according to the disclosed embodiments, which is a device that can call instructions stored in the storage medium and operate according to the called instructions. When the instructions are executed by a processor, the processor may directly or under the control of the processor use other components to perform a function corresponding to the instructions. The instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain signals and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0397] According to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0398] Each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and other sub-components among the aforementioned sub-components may be further included in the various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration. Operations performed by modules, programs or other components according to the various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order or other operations may be added.

[0399] Although the embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea of ​​the present disclosure.

Claims

1. In electronic devices, memory; at least one sensor; and At least one processor operatively connected to the memory and the at least one sensor, the processor executing instructions; The above instructions, when executed by the at least one processor, cause the electronic device to: When a request for a map corresponding to a target space is received, first surface data for a first surface corresponding to a driving direction of the electronic device is obtained based on first sensor data acquired through the at least one sensor in a first drive for the target space, In the second drive for the target space, second surface data for a second surface different from the first surface is acquired based on second sensor data acquired through the at least one sensor, and An electronic device that obtains the map based on the first surface data and the second surface data.

2. In paragraph 1, The above first surface data is, Contains first information corresponding to a horizontal plane including the x-axis and the y-axis based on the driving direction of the electronic device, The above second surface data is, An electronic device comprising second information corresponding to a vertical plane including the z-axis based on the driving direction of the electronic device.

3. In paragraph 1, The above first surface data is, An electronic device comprising first spatial information of the first surface and first object information of the first surface.

4. In paragraph 3, At least one sensor above, Including a first distance sensor and an acceleration sensor, The above first sensor data is, Including first distance data acquired through the first distance sensor and first acceleration data acquired through the acceleration sensor, The above instructions, when executed by the at least one processor, cause the electronic device to: Detecting first edge information based on the first distance data, Obtaining first direction information of the electronic device based on the first acceleration data, An electronic device that obtains the first surface data based on the first edge information and the first direction information.

5. In paragraph 4, The above second surface data is, An electronic device including second spatial information of the second surface and second object information of the second surface.

6. In paragraph 5, At least one sensor above, Including a second distance sensor, The above second sensor data is, Including second distance data acquired through the second distance sensor and second acceleration data acquired through the acceleration sensor, The above instructions, when executed by the at least one processor, cause the electronic device to: Detecting second edge information based on the second distance data, Obtain second direction information of the electronic device based on the second acceleration data, An electronic device that obtains the second surface data based on the second edge information and the second direction information.

7. In paragraph 6, At least one sensor above, Including more vision sensors, The above second sensor data is, Further including image data acquired through the above vision sensor, The above instructions, when executed by the at least one processor, cause the electronic device to: An electronic device that updates the second object information in the image data.

8. In paragraph 6, At least one sensor above, Including an inclination sensor, The above second sensor data is, Further including the inclination data acquired through the above inclination sensor, The above instructions, when executed by the at least one processor, cause the electronic device to: Based on the above inclination data, a first inclination angle in the roll direction, a second inclination angle in the pitch direction, and a third inclination angle in the yaw direction of the electronic device are acquired, An electronic device configured to update the second spatial information based on the first tilt angle, the second tilt angle, and the third tilt angle.

9. In paragraph 8, The above first distance sensor, It is a Lidar sensor, The above second distance sensor, It is a ToF (Time of Flight) sensor, The above inclination sensor, An electronic device that is a gyro sensor.

10. In paragraph 6, The above instructions, when executed by the at least one processor, cause the electronic device to: If the first spatial information and the second spatial information correspond to the same location, the first spatial information and the second spatial information are combined to obtain third spatial information, If the first spatial information and the second spatial information correspond to the same location, the first object information and the second object information are combined to obtain third object information, An electronic device that obtains the map including the third spatial information and the third object information.

11. In a method for controlling an electronic device, When a request for a map corresponding to a target space is received, a step of obtaining first surface data for a first surface corresponding to a driving direction of the electronic device based on first sensor data obtained in a first drive for the target space; A step of acquiring second surface data for a second surface different from the first surface based on second sensor data acquired in a second drive for the target space; and A control method, comprising: a step of obtaining the map based on the first surface data and the second surface data.

12. In paragraph 11, The above first surface data is, Contains first information corresponding to a horizontal plane including the x-axis and the y-axis based on the driving direction of the electronic device, The above second surface data is, A control method, comprising second information corresponding to a vertical plane including the z-axis based on the driving direction of the electronic device.

13. In paragraph 11, The above first surface data is, A control method including first space information of the first side and first object information of the first side.

14. In paragraph 13, The above electronic device Including a first distance sensor and an acceleration sensor, The above first sensor data is, Including first distance data acquired through the first distance sensor and first acceleration data acquired through the acceleration sensor, The step of acquiring the first surface data is: Detecting first edge information based on the first distance data, Obtaining first direction information of the electronic device based on the first acceleration data, A control method for obtaining the first surface data based on the first edge information and the first direction information.

15. In paragraph 14, The above second surface data is, A control method including second spatial information of the second surface and second object information of the second surface.

Images