Electronic device for recognizing structure of space by using camera and control method thereof

Abstract

An electronic device is disclosed. The electronic device may comprise a camera, a display, and at least one processor operatively connected to the camera and the display, wherein the at least one processor is configured to: acquire an image of a space through the camera; on the basis of the image, detect at least one of a horizontal line that is a boundary between a wall and a floor included in the space and a vertical line that is a boundary between walls included in the space; when a vertex of the space is covered by an object, predict a position of the vertex on the basis of the at least one of the horizontal line and the vertical line; and generate an indoor map of the space on the basis of the predicted position of the vertex and the at least one of the horizontal line and the vertical line.

Description

Electronic device for recognizing the structure of space using a camera and method for controlling the same

[0001] Embodiments of the present disclosure relate to an electronic device that recognizes the structure of a space using a camera and a method for controlling the same.

[0002] The variety of services and additional features offered through electronic devices, such as smartphones, is steadily increasing. To enhance the utility of these devices and satisfy the diverse needs of users, telecommunications service providers and electronic device manufacturers are competitively developing electronic devices to offer a variety of features and differentiate themselves from competitors. Consequently, the various functions offered through electronic devices are also becoming increasingly sophisticated.

[0003] As wireless communication technology advances, devices utilizing artificial intelligence (AI) are becoming more widely adopted. For example, home appliances connected to the Internet of Things (IoT) can utilize AI. IoT technology can collect and analyze data generated by these devices to provide intelligent Internet technology services that create new value in human lives. Through the convergence and integration of existing Internet technologies with various industries, IoT technology can be applied to areas such as smart homes, smart buildings, smart cities, smart cars, and smart appliances.

[0004] Meanwhile, homes are equipped with a variety of home appliances for the convenience of users. Services leveraging IoT technology are being proposed to make the operation and control of these appliances more convenient. Home network technology can provide services to users within the home via the home network.

[0005] For example, home network technology can provide users with a map based on the structure of an indoor space, displaying objects such as appliances and / or furniture in a manner similar to the actual environment. The map may be generated based on recognition of the indoor space.

[0006] According to one embodiment, an electronic device may include a camera, a display, and at least one processor operatively connected to the camera and the display.

[0007] According to one embodiment, the at least one processor can obtain an image of a space captured through the camera.

[0008] According to one embodiment, the at least one processor may detect at least one of a horizontal line, which is a boundary between a wall and a floor included in the space, or a vertical line, which is a boundary between walls included in the space, based on the image.

[0009] In one embodiment, the at least one processor may be configured to predict the location of the vertex based on at least one of the horizontal line or the vertical line, if the vertex of the space is occluded by an object.

[0010] In one embodiment, the at least one processor may generate an indoor map of the space based on at least one of the horizontal line or the vertical line and the location of the predicted vertex.

[0011] According to one embodiment, a method for controlling an electronic device may include an operation of acquiring an image of a space through a camera of the electronic device.

[0012] According to one embodiment, a method of controlling an electronic device may include an operation of detecting at least one of a horizontal line, which is a boundary between a wall and a floor included in the space, or a vertical line, which is a boundary between walls included in the space, based on the image.

[0013] According to one embodiment, a method of controlling an electronic device may include an operation of predicting a position of a vertex of the space based on at least one of the horizontal line or the vertical line when the vertex is obscured by an object.

[0014] According to one embodiment, a method of controlling an electronic device may include generating an indoor map of the space based on at least one of the horizontal line or the vertical line and the location of the predicted vertex.

[0015] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs, wherein the one or more programs may include instructions for causing an electronic device, according to one embodiment, the at least one processor, to obtain an image of a space captured through a camera of the electronic device.

[0016] According to one embodiment, the one or more programs may include instructions for the electronic device to detect at least one of a horizontal line that is a boundary between a wall and a floor included in the space or a vertical line that is a boundary between walls included in the space based on the image.

[0017] In one embodiment, the one or more programs may include instructions for the electronic device to predict the position of a vertex of the space based on at least one of the horizontal line or the vertical line, when the vertex is obscured by an object.

[0018] In one embodiment, the one or more programs may include instructions for the electronic device to generate an indoor map of the space based on at least one of the horizontal line or the vertical line and the location of the predicted vertex.

[0019] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0020] FIG. 2 is a drawing for explaining a brief configuration of an electronic device according to one embodiment.

[0021] FIG. 3 is a flowchart illustrating an indoor map generation operation of an electronic device according to one embodiment.

[0022] FIG. 4 is a diagram illustrating an operation of generating an indoor map using a camera of an electronic device according to one embodiment.

[0023] FIG. 5 is a diagram for explaining a camera initialization operation for spatial recognition of an electronic device according to one embodiment.

[0024] FIG. 6A is a diagram illustrating an operation of providing a spatial recognition result of an electronic device and generating an indoor map based on received user input according to one embodiment.

[0025] FIG. 6b is a diagram illustrating an operation of providing a spatial recognition result of an electronic device and generating an indoor map based on received user input according to one embodiment.

[0026] FIG. 7 is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0027] FIG. 8A is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0028] FIG. 8b is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0029] FIG. 9A is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0030] FIG. 9b is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0031] FIG. 10 is a drawing for explaining an operation of displaying a spatial recognition result of an electronic device according to one embodiment.

[0032] FIG. 11 is a diagram illustrating an operation of displaying an indoor map generated based on spatial recognition of an electronic device, according to one embodiment.

[0033] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one embodiment. Referring to FIG. 1 , in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0034] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0035] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0036] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0037] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0038] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0039] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0040] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0041] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0042] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0043] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0044] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0045] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0046] The camera module (180) can capture still images and moving images. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0047] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0048] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0049] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0050] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0051] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0052] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0053] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0054] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0055] FIG. 2 is a drawing for explaining a brief configuration of an electronic device according to one embodiment.

[0056] Referring to FIG. 2, an electronic device (101) (e.g., the electronic device (101) of FIG. 1 may include a camera (180) (e.g., the camera module (180) of FIG. 1), a display (160) (e.g., the display module (160) of FIG. 1), and a processor (120) (e.g., the processor (120) of FIG. 1).

[0057] According to one embodiment, the processor (120) can display an image captured by a camera (180) of an actual space on a display (160).

[0058] According to one embodiment, the camera (180) may be a stereo camera including two or more cameras capable of measuring the distance between the electronic device and an object. According to one embodiment, the processor (120) may arrange a grid on the floor area of ​​a space included in the image based on the stereo camera. The grid is formed by arranging a plurality of horizontal straight lines and a plurality of vertical straight lines at set equal intervals, and each of the plurality of horizontal straight lines and the plurality of vertical straight lines may be perpendicular.

[0059] According to one embodiment, the processor (120) may set a reference for position and / or a reference for distance by arranging a grid in a floor area included in an image. For example, even if the area of ​​the actual space being captured changes due to changes in the position and orientation of the electronic device (101), the processor (120) may identify the relative position of the actual space included in each image based on the grid. According to one embodiment, the grid may or may not be displayed in the image.

[0060] According to one embodiment, the processor (120) may detect (or, extract, recognize) at least one of a horizontal line or a vertical line included in an image. For example, a horizontal line may include a boundary between a floor and a wall of a space. For example, a vertical line may include a boundary between two walls of a space.

[0061] According to one embodiment, when the processor (120) detects a plurality of horizontal lines in an image, the processor (120) may display the plurality of horizontal lines on the display (160). According to one embodiment, the processor (120) may highlight a horizontal line estimated to be a boundary between a floor and a wall among the plurality of horizontal lines.

[0062] In one embodiment, when a user input is received to select a horizontal line other than a highlighted horizontal line among a plurality of horizontal lines, the processor (120) may generate an indoor map based on the selected horizontal line rather than the existing highlighted horizontal line. In one embodiment, the processor (120) may remove the highlight effect of the existing highlighted horizontal line and display a highlight on the selected horizontal line. In one embodiment, the operation of displaying a highlight on one of the plurality of lines and the operation of changing the highlighted line based on the user input may also be applied to vertical lines.

[0063] According to one embodiment, the processor (120) can predict (or obtain, calculate, estimate, identify, determine) the horizontal line of the occluded area based on at least one of the horizontal line or vertical line of the unoccluded area when the horizontal line and / or vertex of the space is occluded by an object.

[0064] According to one embodiment, the processor (120) may acquire information on the length of a horizontal line based on an image of a space captured by a stereo camera. For example, the processor (120) may acquire information on the length of a horizontal line included in an actual space based on a set minimum interval between a plurality of horizontal or vertical straight lines included in a grid and the number of intervals included in a horizontal line.

[0065] According to one embodiment, the processor (120) may generate an indoor map of a space based on the detected horizontal lines and the predicted plurality of horizontal lines, based on the detection of a closed curve including the detected horizontal lines and the predicted plurality of horizontal lines. For example, the indoor map of the space may include information related to the structure and area of ​​the space (e.g., the width and height of the floor).

[0066] According to one embodiment, the operation of an electronic device (101) that recognizes the structure of a space using a camera (180) will be described in more detail with reference to FIGS. 3 to 11 below.

[0067] FIG. 3 is a flowchart illustrating an indoor map generation operation of an electronic device according to one embodiment.

[0068] Referring to FIG. 3, an electronic device (e.g., an electronic device (101) of FIG. 1, a processor (120) of FIG. 1, an electronic device (101) of FIG. 2, or a processor (120) of FIG. 2) can obtain an image capturing a space through a camera (e.g., a camera module (180) of FIG. 1 or a camera (180) of FIG. 2) in operation 310.

[0069] According to one embodiment, the electronic device may display a user interface (UI) on a display (e.g., the display module (160) of FIG. 1 or the display (160) of FIG. 2) for receiving user input for selecting a shape of the space before capturing the space through a camera. For example, the electronic device may display a user interface for selecting whether the space has an orthogonal, curved, or diagonal structure for generating an indoor map (e.g., an indoor floor plan), as illustrated in FIG. 4 below.

[0070] In this way, based on receiving the rough structure of the space through the user interface, the electronic device can generate a more accurate indoor map.

[0071] In one embodiment, the electronic device may perform a camera initialization operation to recognize a space. For example, the camera may be a stereo camera including two or more cameras capable of measuring the distance between the electronic device and an object.

[0072] In one embodiment, the camera initialization operation may include arranging a grid-shaped reference line in an area corresponding to the floor within an image captured by a stereo camera. The grid is composed of a plurality of horizontal straight lines and a plurality of vertical straight lines arranged at set equal intervals, and each of the plurality of horizontal straight lines and the plurality of vertical straight lines may be perpendicular.

[0073] In one embodiment, the electronic device can establish a reference for position and / or a reference for distance by arranging a grid on a floor area included in an image. For example, even if the position and orientation of the electronic device change, thereby changing the area of ​​the real space being captured, the electronic device can identify the relative position of the real space included in each image based on the grid. In one embodiment, the grid may or may not be displayed within the image.

[0074] According to one embodiment, the camera initialization operation will be described in more detail with reference to FIG. 5 below.

[0075] According to one embodiment, the electronic device may display an image of an indoor space captured by a camera on a display (e.g., a display module (160) of FIG. 1).

[0076] In one embodiment, in operation 320, the electronic device may detect (or extract, recognize) at least one of a horizontal line, which is a boundary between a wall and a floor included in a space, or a vertical line, which is a boundary between walls, based on an image.

[0077] According to one embodiment, the electronic device may display at least one of a horizontal line or a vertical line.

[0078] In one embodiment, when a plurality of horizontal lines are detected in a boundary area between a floor and a wall, the electronic device may display the detected plurality of horizontal lines within the image. In one embodiment, the electronic device may highlight a horizontal line among the plurality of horizontal lines that is estimated to be the boundary between the floor and the wall.

[0079] For example, if multiple horizontal lines are detected, the electronic device may select one of the multiple lines based on the boundary of the area where the grid is arranged.

[0080] For example, if a plurality of horizontal lines are detected in an image capturing an area of ​​an indoor space, the electronic device may select one horizontal line from among the plurality of lines based on at least one line selected from another area of ​​the indoor space.

[0081] In one embodiment, when a user input is received to select a horizontal line other than a highlighted horizontal line among a plurality of horizontal lines, the electronic device may generate an indoor map based on the selected horizontal line rather than the existing highlighted horizontal line. In one embodiment, the electronic device may remove the highlight effect of the existing highlighted horizontal line and display a highlight on the selected horizontal line. In one embodiment, the operations of displaying a highlight on one of the plurality of lines, the operations of changing the highlighted line based on the user input, and the operations of generating an indoor map based on the line selected by the user may also be applied to vertical lines.

[0082] According to one embodiment, the operation of generating an indoor map based on the display operation of the detected plurality of lines and the selected lines based on the user input will be described below with reference to FIGS. 6A and 6B.

[0083] In one embodiment, in operation 330, the electronic device may predict (or obtain, calculate, estimate, identify, determine) the location of a vertex based on at least one of a horizontal line or a vertical line when the vertex of the space is occluded by an object.

[0084] According to one embodiment, the electronic device can predict an extension of a detected horizontal line or an extension of a detected vertical line. According to one embodiment, the electronic device can predict the location of an intersection of two or more of the extensions of the horizontal line or the extensions of the vertical line as the location of a vertex.

[0085] For example, if a portion of a vertex of a space is obscured by an object, but a portion of each of multiple horizontal lines that are boundaries between the floor and the wall is detected, the location of the intersection can be predicted based on the extension lines of the detected portions.

[0086] For example, an electronic device may predict information related to the location of an intersection (e.g., coordinate information of the intersection) based on a grid arranged on a floor surface. In one embodiment, the electronic device may obtain the distance between two adjacent vertices (or the length of a horizontal line connecting two vertices) based on a grid arranged on a floor surface.

[0087] According to one embodiment, the operation of predicting the extension of a line and predicting the vertex of an occluded part will be described in more detail with reference to FIG. 7 below.

[0088] In one embodiment, when one of a plurality of horizontal lines, which are boundaries between a floor and a wall, is entirely obscured by an object, the electronic device may generate a virtual line based on two or more of the detected horizontal lines or vertical lines. In one embodiment, the electronic device may predict an extension of the horizontal line or an extension of the vertical line. In one embodiment, the location of an intersection of one of the extensions of the horizontal line or the extensions of the vertical line and the virtual line may be predicted as the location of a vertex.

[0089] According to one embodiment, the operations of predicting the extension of a line and predicting the vertices of an occluded portion are described in more detail with reference to FIGS. 8a and 8b below.

[0090] In one embodiment, if one of a plurality of horizontal lines, which are boundaries between a floor and a wall, is entirely obscured by an object, the electronic device may detect a floor area included in the image. For example, the electronic device may detect the floor area based on a change in the distance between the electronic device and an object (e.g., a floor or a wall) using a stereo camera. In one embodiment, the electronic device may predict the location of an intersection between the floor area and two or more of the vertical lines and the floor area as the location of a vertex.

[0091] According to one embodiment, an embodiment of predicting the intersection of a floor area and a vertical line is described in more detail with reference to FIGS. 9a and 9b below.

[0092] In one embodiment, after receiving a user input selecting one or more lines, the electronic device may display a message prompting movement to capture another area of ​​the space. For example, if the electronic device detects both horizontal and vertical lines associated with a specific area of ​​the indoor space and does not move for a set period of time, the electronic device may display a message including text prompting movement to another area, an arrow pointing in the direction of movement prompting, and / or a highlight on an edge area of ​​the display corresponding to the direction of movement prompting.

[0093] According to one embodiment, the electronic device may acquire length information of a horizontal line based on an image of the space captured by a stereo camera. For example, the electronic device may acquire length information of a horizontal line included in an actual space based on a set minimum interval between a plurality of horizontal or vertical lines included in a grid and the number of intervals included in a horizontal line.

[0094] In one embodiment, in operation 340, the electronic device may generate an indoor map of the space based on at least one of a horizontal line or a vertical line and the location of the predicted vertex.

[0095] In one embodiment, the electronic device can predict a plurality of horizontal lines obscured by an object based on the positions of the predicted vertices.

[0096] According to one embodiment, the electronic device may generate an indoor map of a space based on the detected horizontal line and the predicted plurality of horizontal lines, based on the detected horizontal line and the predicted plurality of horizontal lines. For example, the indoor space may be defined as a closed curve that is a polygon or a closed curve that includes a curve, and when a closed curve that includes the detected horizontal line and the predicted plurality of horizontal lines is detected, the electronic device may generate an indoor map of the space in the form of the detected closed curve.

[0097] In this way, the structure of a space can be predicted by capturing only a portion of the space's boundaries, rather than the entire space, through a camera, and a more accurate indoor map can be created by receiving user input.

[0098] FIG. 4 is a diagram illustrating an operation of generating an indoor map using a camera of an electronic device according to one embodiment.

[0099] Referring to FIG. 4, an electronic device (e.g., the electronic device (101) of FIG. 1, the processor (120) of FIG. 1, the electronic device (101) of FIG. 2, or the processor (120) of FIG. 2) may display a user interface (410) for selecting a space shape on a display (e.g., the display module (160) of FIG. 1 or the display (160) of FIG. 2).

[0100] For example, a user interface (410) for selecting a space shape may select whether the space is an orthogonal (e.g., rectangle) structure or a polygonal (e.g., polygon) structure including curves or diagonals.

[0101] In this way, based on receiving the rough structure of the space through the user interface (410), the electronic device can generate a more accurate indoor map.

[0102] In one embodiment, the electronic device may omit the action of displaying a user interface (410) for selecting a spatial shape.

[0103] According to one embodiment, the electronic device may perform an initialization operation to recognize a space through a camera (e.g., the camera module (180) of FIG. 1 or the camera (180) of FIG. 2). According to one embodiment, the electronic device may display a user interface (420) indicating that the initialization operation is being performed when the initialization operation is performed. According to one embodiment, the camera may be a stereo camera including two or more cameras capable of measuring the distance between the electronic device and an object.

[0104] For example, a camera initialization action may include prompting a user to move the electronic device left and right, and arranging a grid-like reference line in an area corresponding to the floor within an image captured by a stereo camera while the user moves. The grid may be composed of a plurality of horizontal lines and a plurality of vertical lines arranged at set equal intervals, and each of the plurality of horizontal lines and the plurality of vertical lines may be perpendicular to each other.

[0105] In one embodiment, the electronic device can establish a reference for position and / or a reference for distance by arranging a grid on a floor area included in an image. For example, even if the position and orientation of the electronic device change, thereby changing the area of ​​the real space being captured, the electronic device can identify the relative position of the real space included in each image based on the grid. In one embodiment, the grid may or may not be displayed within the image.

[0106] According to one embodiment, the camera initialization operation will be described in more detail with reference to FIG. 5 below.

[0107] According to one embodiment, the electronic device may detect (or, extract, recognize) at least one horizontal line (430, 431) and / or at least one vertical line included in a space based on an image captured by a camera. For example, the at least one horizontal line (430, 431) may include a horizontal line (430) that is a boundary between a wall and a floor included in the space. According to one embodiment, the at least one vertical line may include a vertical line that is a boundary between walls.

[0108] According to one embodiment, the electronic device may display at least one horizontal line (430, 431) and / or at least one vertical line detected within an image captured by the camera.

[0109] According to one embodiment, the electronic device may highlight a horizontal line (430) among a plurality of horizontal lines (430, 431) that is presumed to be the boundary between the floor and the wall.

[0110] For example, if multiple horizontal lines are detected, the electronic device may select one of the multiple lines based on the boundary of the area where the grid is arranged.

[0111] For example, when a plurality of horizontal lines (430, 431) are detected in an image capturing an area of ​​an indoor space, the electronic device may select one horizontal line (430) from among the plurality of lines based on at least one line selected from another area of ​​the indoor space.

[0112] In one embodiment, when a user input is received to select a horizontal line other than the highlighted horizontal line (430) among a plurality of horizontal lines (430, 431), the electronic device may generate an indoor map based on the selected horizontal line rather than the existing highlighted horizontal line. In one embodiment, the electronic device may remove the highlight effect of the existing highlighted horizontal line and display a highlight on the selected horizontal line. In one embodiment, the operations of displaying a highlight on one of the plurality of lines, the operations of changing the highlighted line based on the user input, and the operations of generating an indoor map based on the line selected by the user may also be applied to vertical lines.

[0113] According to one embodiment, the operation of generating an indoor map based on the display operation of the detected plurality of lines and the selected lines based on the user input will be described below with reference to FIGS. 6A and 6B.

[0114] In one embodiment, the electronic device may determine one of a plurality of horizontal lines included in an area captured by the camera, and store the height of the determined horizontal line. For example, the height of the horizontal line may be the distance between the electronic device and the horizontal line in the plane direction of the floor area including the horizontal line (e.g., in a direction perpendicular to all straight lines included in the floor area).

[0115] In one embodiment, the electronic device may highlight a defined horizontal line. This allows the user to move the electronic device to capture a different area after confirming the highlighted horizontal line.

[0116] In one embodiment, if the electronic device does not move for a set period of time after detecting both horizontal and vertical lines associated with a particular area of ​​an indoor space, the electronic device may display a message including text guiding movement to another area, an arrow pointing in the direction of movement guidance, and / or a highlight on an edge area of ​​the display corresponding to the direction of movement guidance.

[0117] In one embodiment, if a plurality of detected horizontal lines are not connected and are partially obscured by objects positioned at vertices of the space, the electronic device can predict the positions of vertices of the obscured portions based on the detected horizontal lines and / or vertical lines.

[0118] According to one embodiment, the electronic device can predict an extension of a detected horizontal line or an extension of a detected vertical line. According to one embodiment, the electronic device can predict the location of an intersection of two or more of the extensions of the horizontal line or the extensions of the vertical line as the location of a vertex.

[0119] For example, if a portion of a vertex of a space is obscured by an object, but a portion of each of multiple horizontal lines that are boundaries between the floor and the wall is detected (e.g., four or more horizontal lines are detected), the location of the intersection can be predicted based on the extension lines of the detected portions.

[0120] For example, an electronic device may predict information related to the location of an intersection (e.g., coordinate information of the intersection) based on a grid arranged on a floor surface. In one embodiment, the electronic device may obtain the distance between two adjacent vertices (or the length of a horizontal line connecting two vertices) based on a grid arranged on a floor surface.

[0121] According to one embodiment, the operation of predicting the extension of a line and predicting the vertex of an occluded part will be described in more detail with reference to FIG. 7 below.

[0122] In one embodiment, when one of a plurality of horizontal lines, which are boundaries between a floor and a wall, is entirely obscured by an object (e.g., four or more horizontal lines or vertical lines are detected), the electronic device may generate a virtual line based on two or more of the detected horizontal lines or vertical lines. In one embodiment, the electronic device may predict an extension of the horizontal line or an extension of the vertical line. In one embodiment, the location of an intersection of one of the extensions of the horizontal line or the extensions of the vertical line and the virtual line may be predicted as the location of a vertex.

[0123] According to one embodiment, the operations of predicting the extension of a line and predicting the vertices of an occluded portion are described in more detail with reference to FIGS. 8a and 8b below.

[0124] In one embodiment, when one of a plurality of horizontal lines, which are boundaries between a floor and a wall, is entirely obscured by an object (e.g., four or more vertical lines are detected), the electronic device may detect a floor area included in the image. For example, the electronic device may use a stereo camera to detect the floor area based on a change in the distance between the electronic device and the object (e.g., the floor or the wall). In one embodiment, the electronic device may predict the location of an intersection between the floor area and two or more of the vertical lines and the floor area as the location of a vertex.

[0125] According to one embodiment, an embodiment of predicting the intersection of a floor area and a vertical line is described in more detail with reference to FIGS. 9a and 9b below.

[0126] According to one embodiment, when the floor area includes a height difference in the floor area, such as a staircase or a ramp, the electronic device may detect horizontal lines of different heights in the step area. According to one embodiment, the electronic device may project the remaining horizontal lines at the height of one of a plurality of horizontal lines of different heights, or may project the plurality of horizontal lines at a set height and correct them into a single horizontal line. For example, in the case of stairs, correction may be performed in the form of adding two horizontal lines of different heights based on the step. For example, in the case of a ramp, the boundary between the ramp and the wall may be vertically projected at a specific height to obtain a horizontal line corresponding to the ramp. According to one embodiment, correction may be performed in the form of adding a horizontal line corresponding to the ramp between two horizontal lines that have the ramp in between.

[0127] In one embodiment, the electronic device may generate an indoor map (e.g., a floor plan) based on a calibrated horizontal line.

[0128] FIG. 5 is a diagram for explaining a camera initialization operation for spatial recognition of an electronic device according to one embodiment.

[0129] Referring to FIG. 5, an electronic device (101) (e.g., the electronic device (101) of FIG. 1, the processor (120) of FIG. 1, the electronic device (101) of FIG. 2, or the processor (120) of FIG. 2) may perform an initialization operation to recognize a space (510) through a camera (e.g., the camera module (180) of FIG. 1 or the camera (180) of FIG. 2). According to one embodiment, the camera may be a stereo camera including two or more cameras capable of measuring a distance between the electronic device and an object.

[0130] According to one embodiment, the camera initialization operation may induce the user to move the electronic device (101) left and right, and acquire the distance to an object (e.g., a floor or a wall) within the space (510) through an image captured by the stereo camera while moving.

[0131] According to one embodiment, the electronic device (101) may obtain a distance to an object within a space (510) using a proximity sensor. According to one embodiment, if the electronic device (101) does not include a proximity sensor for distance measurement, the electronic device (101) may obtain a distance to an object within a space (510) using only a camera.

[0132] According to one embodiment, the electronic device (101) may distinguish between a floor area and a wall area within a space (510) based on a distance from an object. According to one embodiment, the electronic device (101) may include an operation of arranging reference lines in the form of a grid (520) in an area corresponding to the floor. The grid (520) is formed by arranging a plurality of horizontal straight lines and a plurality of vertical straight lines at set equal intervals, and each of the plurality of horizontal straight lines and the plurality of vertical straight lines may be perpendicular.

[0133] According to one embodiment, the electronic device (101) can set a reference for position and / or a reference for distance by arranging a grid (520) in a floor area included in an image. For example, even if the position and orientation of the electronic device (101) change and the area of ​​the actual space being captured changes, the electronic device can identify the relative position of the actual space included in each image based on the grid. According to one embodiment, the grid may or may not be displayed in the image.

[0134] In FIG. 5, the floor area in the space (510) is depicted as having the same height, but this is not limited thereto, and the floor area may include stairs and / or a ramp.

[0135] According to one embodiment, the electronic device may display a user interface (e.g., interface (420) of FIG. 4) on a display (e.g., display module (160) of FIG. 1 or display (160) of FIG. 2) indicating that the initialization operation is being performed when the initialization operation is performed.

[0136] FIG. 6A is a diagram illustrating an operation of providing a spatial recognition result of an electronic device and generating an indoor map based on received user input according to one embodiment.

[0137] Referring to FIG. 6A, an electronic device (e.g., an electronic device (101) of FIG. 1, a processor (120) of FIG. 1, an electronic device (101) of FIG. 2, or a processor (120) of FIG. 2) may display a plurality of horizontal lines (610, 620) together with an image captured by a camera (e.g., a camera module (180) of FIG. 1 or a camera (180) of FIG. 2) on a display (e.g., a display module (160) of FIG. 1 or a display (160) of FIG. 2). In FIG. 6A, only a plurality of horizontal lines (610, 620) are displayed, but if a vertical line is detected, a vertical line may also be displayed.

[0138] In one embodiment, the electronic device may highlight a horizontal line (610) among a plurality of horizontal lines (610, 620) that is presumed to be the boundary between the floor and the wall.

[0139] For example, if multiple horizontal lines (610, 620) are detected, the electronic device may select one of the multiple horizontal lines (610) based on the boundary of the area in which the grid is arranged.

[0140] For example, when a plurality of horizontal lines (610, 620) are detected in an image capturing an area of ​​an indoor space, the electronic device may select one horizontal line (610) from among the plurality of lines based on at least one line selected from another area of ​​the indoor space.

[0141] According to one embodiment, the operation of selecting one line from among a plurality of lines based on a confirmed line in another area of ​​space is described in more detail below with reference to FIG. 6b.

[0142] In one embodiment, if a highlighted horizontal line (610) among a plurality of horizontal lines (610, 620) is not the boundary between the floor and the wall, the user may select another horizontal line (620).

[0143] In one embodiment, when a user input is received to select a horizontal line (620) other than a highlighted horizontal line (610) among a plurality of horizontal lines (610, 620), the electronic device may generate an indoor map (e.g., a floor plan) based on the selected horizontal line (620) rather than the existing highlighted horizontal line (610).

[0144] According to one embodiment, the electronic device may remove the highlight effect of an existing highlighted horizontal line (610) and display a highlight on a horizontal line (620) selected by user input.

[0145] In one embodiment, the actions of highlighting one of a plurality of lines, changing which line is highlighted based on user input, and generating an indoor map based on a line selected by the user may also be applied to vertical lines.

[0146] FIG. 6b is a diagram illustrating an operation of providing a spatial recognition result of an electronic device and generating an indoor map based on received user input according to one embodiment.

[0147] Referring to FIG. 6B, the electronic device can detect a plurality of lines (e.g., horizontal lines) (630, 631, 632) included in a first image capturing an area of ​​space and display them in the first image.

[0148] In one embodiment, the electronic device may determine the first line (631) as the boundary between the floor and the wall based on receiving a user input selecting a first line (631) from among a plurality of lines (630, 631, 632).

[0149] In one embodiment, the electronic device may store the height of the first line (631). For example, the height of the line may be the distance between the electronic device and the line in a plane direction of a floor area containing the line (e.g., a direction perpendicular to all straight lines contained in the floor area).

[0150] According to one embodiment, when the first line (631) is determined, the electronic device can display a plurality of lines (e.g., horizontal lines) (640, 641, 642) included in a second image capturing another area within the space on the second image.

[0151] According to one embodiment, the electronic device may automatically select a second line (641) from among a plurality of lines (640, 641, 642) included in the second image as the boundary between the floor and the wall based on the height of the first line (631). For example, the electronic device may select a second line (641) from among a plurality of lines (640, 641, 642) included in the second image that has a height most similar to the first line (631).

[0152] In one embodiment, the electronic device may highlight and display the second line (641). In one embodiment, after highlighting and displaying the second line (641), if no user input is received for a set period of time or the electronic device is moved to capture another area, the electronic device may determine the second line (641) as the boundary between the floor and the wall.

[0153] In one embodiment, after highlighting and displaying the second line (641), if the user selects a line other than the highlighted second line (641) among the plurality of lines (640, 641, 642), the electronic device may determine the line selected by the user input as the boundary between the floor and the wall. In one embodiment, the electronic device may delete the highlighting of the second line (641) and highlight the line selected by the user input.

[0154] FIG. 7 is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0155] Referring to FIG. 7, an electronic device (e.g., an electronic device (101) of FIG. 1, a processor (120) of FIG. 1, an electronic device (101) of FIG. 2, or a processor (120) of FIG. 2) can capture areas of space through a camera (e.g., a camera module (180) of FIG. 1 or a camera (180) of FIG. 2). For example, each vertex of the space may be covered by a plurality of objects (710, 711, 712, 713).

[0156] According to one embodiment, the electronic device can detect a plurality of horizontal lines (720, 721, 722, 723) through captured images. For example, the plurality of horizontal lines (720, 721, 722, 723) may be selected by the electronic device or selected by user input.

[0157] According to one embodiment, when a plurality of detected horizontal lines (720, 721, 722, 723) are not connected and are partially obscured by objects (710, 711, 712, 713) positioned at vertices of a space, the electronic device can predict the location of the vertices of the obscured portions based on the detected horizontal lines (720, 721, 722, 723).

[0158] According to one embodiment, the electronic device can predict an extension of a detected horizontal line (720, 721, 722, 723). According to one embodiment, the electronic device can predict the location of an intersection of two or more extensions of the horizontal lines (720, 721, 722, 723) as the location of a vertex.

[0159] For example, an electronic device can predict information related to the location of an intersection (e.g., coordinate information of the intersection) based on a grid arranged on a floor surface.

[0160] For example, the electronic device can predict the location of the intersection of the extension of the first horizontal line (720) and the extension of the second horizontal line (721) as the location of the first vertex covered by the first object (710).

[0161] For example, the electronic device can predict the location of the intersection of the extension of the second horizontal line (721) and the extension of the third horizontal line (722) as the location of the second vertex covered by the second object (711).

[0162] For example, the electronic device can predict the location of the intersection of the extension of the third horizontal line (722) and the extension of the fourth horizontal line (723) as the location of the third vertex covered by the third object (712).

[0163] For example, the electronic device can predict the location of the intersection of the extension of the fourth horizontal line (723) and the extension of the first horizontal line (720) as the location of the fourth vertex covered by the fourth object (713).

[0164] In one embodiment, the electronic device can predict a plurality of horizontal lines obscured by an object based on the positions of the predicted vertices.

[0165] In one embodiment, the electronic device may generate an indoor map (e.g., a floor plan) of a space based on the detected horizontal line and the predicted horizontal line, based on the detected closed curve including the detected horizontal line and the predicted horizontal line. For example, an indoor space may be defined by a polygonal closed curve or a closed curve including a curve, and the electronic device may generate an indoor map of the space in the form of the detected closed curve when a closed curve including the detected horizontal line and the predicted horizontal line is detected.

[0166] According to one embodiment, the electronic device can obtain a distance between adjacent vertices based on the position coordinates of the predicted vertices.

[0167] In one embodiment, the electronic device can obtain the distance between two adjacent vertices (or the length of a horizontal line connecting two vertices) based on a grid arranged on a floor surface.

[0168] FIG. 8A is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0169] FIG. 8b is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0170] Referring to FIG. 8A, an electronic device (e.g., the electronic device (101) of FIG. 1, the processor (120) of FIG. 1, the electronic device (101) of FIG. 2, or the processor (120) of FIG. 2) can capture areas of a space through a camera (e.g., the camera module (180) of FIG. 1 or the camera (180) of FIG. 2). For example, each vertex of the space, and one entire horizontal line among a plurality of horizontal lines that are boundaries between a floor and a wall, may be covered by a plurality of objects (810, 811, 812).

[0171] According to one embodiment, when the electronic device cannot detect a horizontal line of an area due to an object (812), the electronic device can detect a plurality of horizontal lines (820, 821, 822) and a vertical line (823) through the captured images. For example, the plurality of horizontal lines (820, 821, 822) and the vertical line (823) may be selected by the electronic device or selected by a user input. For example, the vertical line (823) may be at least one of the vertical lines orthogonal to the undetectable horizontal line.

[0172] According to one embodiment, when a plurality of detected horizontal lines (820, 821, 822) and vertical lines (823) are not connected and are partially obscured by objects (810, 811, 812) positioned at vertices of a space, the electronic device can predict the location of the vertices of the obscured portions based on the detected horizontal lines (820, 821, 822) and / or vertical lines (823).

[0173] According to one embodiment, as illustrated in FIG. 8B, the electronic device can predict an extension of a detected horizontal line (820, 821, 822) or an extension of a detected vertical line (823). According to one embodiment, the electronic device can predict the location of an intersection of two or more of the extensions of the horizontal lines (820, 821, 822) or the extensions of the vertical lines (823) as the location of a vertex.

[0174] For example, an electronic device can predict information related to the location of an intersection (e.g., coordinate information of the intersection) based on a grid arranged on a floor surface.

[0175] For example, the electronic device can predict the location of the intersection of the extension of the first horizontal line (820) and the extension of the second horizontal line (821) as the location of the first vertex (840) covered by the first object (810).

[0176] For example, the electronic device can predict the location of the intersection of the extension of the second horizontal line (821) and the extension of the third horizontal line (822) as the location of the second vertex (841) covered by the second object (811).

[0177] For example, the electronic device can predict the location of the intersection of the extension of the first horizontal line (820) and the extension of the first vertical line (823) as the location of the third vertex (842) covered by the third object (812).

[0178] According to one embodiment, the electronic device may generate a virtual line (830) based on two or more of the detected horizontal lines (820, 821, 822) or vertical lines (823), as illustrated in FIG. 8B. For example, the electronic device may generate a virtual line (830) that is parallel to the second horizontal line (821). For example, the virtual line (830) may have the same height as at least one of the first horizontal line (820) to the third horizontal line (822).

[0179] In one embodiment, the virtual line (830) may pass through the third vertex (842).

[0180] According to one embodiment, the location of the intersection of the virtual line (830) with one of the extensions of the horizontal lines (820, 821, 822) or the extensions of the vertical lines (823) can be predicted as the location of the fourth vertex (843).

[0181] In one embodiment, the electronic device can predict a plurality of horizontal lines obscured by an object based on the positions of the predicted vertices (840, 841, 842, 843).

[0182] In one embodiment, the electronic device may generate an indoor map (e.g., a floor plan) of a space based on the detected horizontal line and the predicted horizontal line, based on the detected closed curve including the detected horizontal line and the predicted horizontal line. For example, an indoor space may be defined by a polygonal closed curve or a closed curve including a curve, and the electronic device may generate an indoor map of the space in the form of the detected closed curve when a closed curve including the detected horizontal line and the predicted horizontal line is detected.

[0183] According to one embodiment, the electronic device can obtain the distance between adjacent vertices based on the position coordinates of the predicted vertices (840, 841, 842, 843).

[0184] In one embodiment, the electronic device can obtain the distance between two adjacent vertices (or the length of a horizontal line connecting two vertices) based on a grid arranged on a floor surface.

[0185] FIG. 9A is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0186] FIG. 9b is a diagram for explaining a spatial recognition operation of an electronic device according to one embodiment.

[0187] Referring to FIG. 9A, an electronic device (e.g., the electronic device (101) of FIG. 1, the processor (120) of FIG. 1, the electronic device (101) of FIG. 2, or the processor (120) of FIG. 2) can capture areas of a space through a camera (e.g., the camera module (180) of FIG. 1 or the camera (180) of FIG. 2). For example, each vertex of the space, and one entire horizontal line among a plurality of horizontal lines that are boundaries between a floor and a wall, may be covered by a plurality of objects (910, 911, 912).

[0188] In one embodiment, when the electronic device cannot detect a horizontal line of an area due to an object (912), the electronic device can detect a plurality of vertical lines (920, 921, 922, 923) through the captured images. For example, the plurality of vertical lines (920, 921, 922, 923) may be selected by the electronic device or selected by user input.

[0189] According to one embodiment, when the detected plurality of vertical lines (920, 921, 922, 923) are not connected and partially obscured by an object (910, 911, 912) positioned at the vertex of the space, as illustrated in FIG. 9B, the electronic device may detect a floor area (930) included in the image. For example, the electronic device may detect the floor area (930) based on a change in the distance between the electronic device and the object (e.g., the floor or a wall) using a stereo camera. According to one embodiment, the electronic device may predict the location of the intersection of the floor area (930) and the plurality of vertical lines (920, 921, 922, 923) as the location of the vertex (940, 941, 942, 943).

[0190] For example, an electronic device can predict information related to the location of an intersection (e.g., coordinate information of the intersection) based on a grid arranged on a floor surface.

[0191] For example, the electronic device can predict the location of the intersection of the extension of the first vertical line (920) and the bottom area (930) as the location of the first vertex (940) covered by the first object (910).

[0192] For example, the electronic device can predict the location of the intersection of the extension of the second vertical line (921) and the bottom area (930) as the location of the second vertex (941) covered by the second object (911).

[0193] For example, the electronic device can predict the location of the intersection of the extension of the third vertical line (922) and the floor area (930) as the location of the third vertex (942) covered by the third object (912).

[0194] For example, the electronic device can predict the location of the intersection of the extension of the fourth vertical line (923) and the floor area (930) as the location of the fourth vertex (943) covered by the third object (912).

[0195] In one embodiment, the electronic device can predict a plurality of horizontal lines obscured by an object based on the positions of the predicted vertices (910, 941, 942, 943).

[0196] In one embodiment, the electronic device may generate an indoor map (e.g., a floor plan) of a space based on the detected horizontal line and the predicted horizontal line, based on the detected horizontal line and the predicted horizontal line. For example, an indoor space may be defined by a polygonal closed curve or a closed curve including a curve, and the electronic device may generate an indoor map of the space in the form of the detected closed curve when a closed curve including the detected horizontal line and the predicted horizontal line is detected.

[0197] According to one embodiment, the electronic device can obtain the distance between adjacent vertices based on the position coordinates of the predicted vertices (910, 941, 942, 943).

[0198] In one embodiment, the electronic device can obtain the distance between two adjacent vertices (or the length of a horizontal line connecting two vertices) based on a grid arranged on a floor surface.

[0199] FIG. 10 is a drawing for explaining an operation of displaying a spatial recognition result of an electronic device according to one embodiment.

[0200] Referring to FIG. 10, an electronic device (e.g., an electronic device (101) of FIG. 1, a processor (120) of FIG. 1, an electronic device (101) of FIG. 2, or a processor (120) of FIG. 2) can display an image (1010) of a space captured by a camera (e.g., a camera module (180) of FIG. 1 or a camera (180) of FIG. 2) on a display (e.g., a display module (160) of FIG. 1 or a display (160) of FIG. 2).

[0201] According to one embodiment, the electronic device may display at least one horizontal line (1020, 1021) detected in the image (1010).

[0202] According to one embodiment, the electronic device may display a predicted intersection (1030) based on an extension of at least one detected horizontal line (1020, 1021). For example, the intersection (1030) may be obtained by an extension of a plurality of horizontal lines (1020, 1021), as illustrated in FIG. 7 .

[0203] According to one embodiment, the electronic device may display a predicted virtual vertical line (1031) based on the detected plurality of horizontal lines (1020, 1021) and the predicted intersection point (1030). According to one embodiment, the vertical line (1031) may be obtained so as to pass through the predicted intersection point (1030) and be perpendicular to the horizontal lines (1020, 1021), respectively.

[0204] According to one embodiment, the vertical line (1031) may be obtained so as to pass through the intersection point (1030) and be perpendicular to the floor area.

[0205] In one embodiment, the vertical line (1031) may be obtained to pass through the intersection point (1030) and be parallel to a determined vertical line included in another area.

[0206] In one embodiment, the electronic device may display only an image (1011) acquired through a camera in a portion of the display without displaying lines.

[0207] FIG. 11 is a diagram illustrating an operation of displaying an indoor map generated based on spatial recognition of an electronic device, according to one embodiment.

[0208] Referring to FIG. 11, an electronic device (e.g., the electronic device (101) of FIG. 1, the processor (120) of FIG. 1, the electronic device (101) of FIG. 2, or the processor (120) of FIG. 2) may generate an indoor map (1110) of a space (e.g., a floor plan) based on the detection of a closed curve including a detected horizontal line and a predicted horizontal line. For example, an indoor space may be defined as a closed curve that is a polygon or a closed curve including a curve, and the electronic device may generate an indoor map (1110) of a space in the form of a detected closed curve when a closed curve including a detected horizontal line and a predicted horizontal line is detected.

[0209] According to one embodiment, the electronic device can obtain a distance between adjacent vertices based on the position coordinates of the predicted vertices.

[0210] In one embodiment, the electronic device can obtain the distance between two adjacent vertices (or the length of a horizontal line connecting two vertices) based on a grid arranged on a floor surface.

[0211] According to one embodiment, the electronic device may display an indoor map (1110) including information related to the shape of the structure of the indoor space and the area of ​​the structure (e.g., length and width) on a display (e.g., display module (160) of FIG. 1 or display (160) of FIG. 2).

[0212] In this way, the structure of a space can be predicted by capturing only a portion of the space's boundaries, rather than the entire space, through a camera, and a more accurate indoor map can be created by receiving user input.

[0213] According to one embodiment, an electronic device may include a camera, a display, and at least one processor operatively connected to the camera and the display.

[0214] According to one embodiment, the at least one processor can obtain an image of a space captured through the camera.

[0215] According to one embodiment, the at least one processor may detect at least one of a horizontal line, which is a boundary between a wall and a floor included in the space, or a vertical line, which is a boundary between walls included in the space, based on the image.

[0216] In one embodiment, the at least one processor may be configured to predict the location of the vertex based on at least one of the horizontal line or the vertical line, if the vertex of the space is occluded by an object.

[0217] In one embodiment, the at least one processor may generate an indoor map of the space based on at least one of the horizontal line or the vertical line and the location of the predicted vertex.

[0218] In one embodiment, the at least one processor may display at least one of the horizontal line or the vertical line.

[0219] In one embodiment, the at least one processor may receive a user input selecting one of the plurality of horizontal lines based on the plurality of horizontal lines being included in the image.

[0220] In one embodiment, the at least one processor may generate the indoor map of the space based on the selected first horizontal line.

[0221] In one embodiment, the at least one processor may select a second horizontal line based on at least one line detected in another area of ​​the space among the plurality of horizontal lines.

[0222] In one embodiment, the at least one processor may highlight the second horizontal line.

[0223] In one embodiment, the at least one processor may delete a highlight displayed on the second horizontal line and highlight the first horizontal line based on whether the second horizontal line and the first horizontal line are different.

[0224] In one embodiment, the at least one processor may, after receiving the user input, display a message prompting movement to image another area of ​​the space.

[0225] According to one embodiment, the at least one processor can predict an extension of the horizontal line or an extension of the vertical line.

[0226] According to one embodiment, the at least one processor can predict the location of an intersection of two or more of the extensions of the horizontal line or the extensions of the vertical line as the location of the vertex.

[0227] In one embodiment, the at least one processor may generate a virtual line based on two or more of the horizontal lines or the vertical lines.

[0228] According to one embodiment, the at least one processor can predict an extension of the horizontal line or an extension of the vertical line.

[0229] According to one embodiment, the at least one processor can predict the location of the intersection of the virtual line with one of the extension of the horizontal line or the extension of the vertical line as the location of the vertex.

[0230] In one embodiment, the at least one processor may detect a floor area included in the image.

[0231] In one embodiment, the at least one processor can predict the location of the intersection of the floor area and the vertical line as the location of the vertex.

[0232] In one embodiment, the at least one processor can predict a plurality of horizontal lines obscured by the object based on the positions of the predicted vertices.

[0233] In one embodiment, the at least one processor may generate an indoor map of the space based on the detected horizontal line and the predicted plurality of horizontal lines, based on the detection of a closed curve including the detected horizontal line and the predicted plurality of horizontal lines.

[0234] In one embodiment, the at least one processor may receive user input selecting a shape of the space prior to imaging the space through the camera.

[0235] In one embodiment, the camera may include a stereo camera.

[0236] According to one embodiment, the at least one processor may obtain length information of the horizontal line based on the image of the space obtained through the stereo camera.

[0237] According to one embodiment, a method for controlling an electronic device may include an operation of acquiring an image of a space through a camera of the electronic device.

[0238] According to one embodiment, a method of controlling an electronic device may include an operation of detecting at least one of a horizontal line, which is a boundary between a wall and a floor included in the space, or a vertical line, which is a boundary between walls included in the space, based on the image.

[0239] According to one embodiment, a method of controlling an electronic device may include an operation of predicting a position of a vertex of the space based on at least one of the horizontal line and the vertical line when the vertex is obscured by an object.

[0240] According to one embodiment, a method of controlling an electronic device may include generating an indoor map of the space based on at least one of the horizontal line or the vertical line and the location of the predicted vertex.

[0241] According to one embodiment, the method for controlling an electronic device may further include an operation of displaying at least one of the horizontal line or the vertical line.

[0242] According to one embodiment, a method of controlling an electronic device may further include receiving a user input for selecting one of a plurality of horizontal lines based on whether the image includes a plurality of horizontal lines.

[0243] According to one embodiment, the operation of generating an indoor map of the space may generate the indoor map of the space based on the selected first horizontal line.

[0244] In one embodiment, the act of displaying at least one of the horizontal line or the vertical line may select a second horizontal line based on at least one line detected in another area of ​​the space among the plurality of horizontal lines.

[0245] In one embodiment, the act of displaying at least one of the horizontal line or the vertical line may display a highlight on the second horizontal line.

[0246] According to one embodiment, a control method of an electronic device may further include an operation of deleting a highlight displayed on the second horizontal line and highlighting the first horizontal line based on whether the second horizontal line and the first horizontal line are different.

[0247] According to one embodiment, the method for controlling an electronic device may further include, after receiving the user input, displaying a message inducing movement to capture another area of ​​the space.

[0248] According to one embodiment, the operation of predicting the position of the vertex may predict an extension of the horizontal line or an extension of the vertical line.

[0249] According to one embodiment, the operation of predicting the position of the vertex may predict the position of two or more intersections of the extension of the horizontal line or the extension of the vertical line as the position of the vertex.

[0250] According to one embodiment, the operation of predicting the position of the vertex may generate a virtual line based on two or more of the horizontal line or the vertical line.

[0251] According to one embodiment, the operation of predicting the position of the vertex may predict an extension of the horizontal line or an extension of the vertical line.

[0252] According to one embodiment, the operation of predicting the position of the vertex may predict the position of the intersection of the virtual line with one of the extension of the horizontal line or the extension of the vertical line as the position of the vertex.

[0253] According to one embodiment, the operation of predicting the position of the vertex may detect a floor area included in the image.

[0254] According to one embodiment, the operation of predicting the position of the vertex may predict the position of the intersection of the floor area and the vertical line as the position of the vertex.

[0255] In one embodiment, the operation of generating an indoor map of the space may predict a plurality of horizontal lines obscured by the object based on the positions of the predicted vertices.

[0256] In one embodiment, the operation of generating an indoor map of the space may include generating an indoor map of the space based on a closed curve including the detected horizontal line and the predicted plurality of horizontal lines, based on the detected horizontal line and the predicted plurality of horizontal lines.

[0257] According to one embodiment, a method of controlling an electronic device may further include receiving a user input for selecting a shape of a space before capturing the space through the camera.

[0258] In one embodiment, the camera may include a stereo camera.

[0259] According to one embodiment, a control method of an electronic device may further include an operation of obtaining length information of the horizontal line based on the image of the space obtained through the stereo camera.

[0260] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs, wherein the one or more programs may include instructions for causing an electronic device, according to one embodiment, the at least one processor, to obtain an image of a space captured through a camera of the electronic device.

[0261] According to one embodiment, the one or more programs may include instructions for the electronic device to detect at least one of a horizontal line that is a boundary between a wall and a floor included in the space or a vertical line that is a boundary between walls included in the space based on the image.

[0262] In one embodiment, the one or more programs may include instructions for the electronic device to predict the position of a vertex of the space based on at least one of the horizontal line or the vertical line, when the vertex is obscured by an object.

[0263] In one embodiment, the one or more programs may include instructions for the electronic device to generate an indoor map of the space based on at least one of the horizontal line or the vertical line and the location of the predicted vertex.

[0264] In one embodiment, the one or more programs may include instructions that cause the electronic device to display at least one of the horizontal line or the vertical line.

[0265] In one embodiment, the one or more programs may include instructions for the electronic device to receive a user input selecting one of the plurality of horizontal lines, if there are multiple horizontal lines.

[0266] In one embodiment, the one or more programs may include instructions for the electronic device to generate the indoor map of the space based on the selected first horizontal line.

[0267] In one embodiment, the one or more programs may include instructions for the electronic device to select a second horizontal line based on at least one line detected in another area of ​​the space among the plurality of horizontal lines.

[0268] In one embodiment, the one or more programs may include instructions that cause the electronic device to highlight the second horizontal line.

[0269] In one embodiment, the one or more programs may include instructions for the electronic device to delete a highlight displayed on the second horizontal line and highlight the first horizontal line based on whether the second horizontal line and the first horizontal line are different.

[0270] In one embodiment, the one or more programs may include instructions that cause the electronic device, after receiving the user input, to display a message that induces movement to image another area of ​​the space.

[0271] In one embodiment, the one or more programs may include instructions for the electronic device to predict an extension of the horizontal line or an extension of the vertical line.

[0272] According to one embodiment, the one or more programs may include instructions for the electronic device to predict the location of two or more intersections of the extensions of the horizontal line or the extensions of the vertical line as the location of the vertex.

[0273] In one embodiment, the one or more programs may include instructions for the electronic device to generate a virtual line based on two or more of the horizontal lines or the vertical lines.

[0274] In one embodiment, the one or more programs may include instructions for the electronic device to predict an extension of the horizontal line or an extension of the vertical line.

[0275] In one embodiment, the one or more programs may include instructions for the electronic device to predict a location of an intersection of the virtual line with one of the extension of the horizontal line or the extension of the vertical line as a location of the vertex.

[0276] According to one embodiment, the one or more programs may include instructions for the electronic device to detect a floor area included in the image.

[0277] In one embodiment, the one or more programs may include instructions for the electronic device to predict the location of the intersection of the floor area and the vertical line as the location of the vertex.

[0278] In one embodiment, the one or more programs may include instructions for the electronic device to predict a plurality of horizontal lines obscured by the object based on the locations of the predicted vertices.

[0279] In one embodiment, the one or more programs may include instructions for the electronic device to generate an indoor map of the space based on the detected horizontal line and the predicted plurality of horizontal lines, based on detection of a closed curve including the detected horizontal line and the predicted plurality of horizontal lines.

[0280] In one embodiment, the one or more programs may include instructions for the electronic device to receive user input selecting a shape of a space before imaging the space through the camera.

[0281] In one embodiment, the camera may include a stereo camera.

[0282] According to one embodiment, the one or more programs may include commands for the electronic device to obtain length information of the horizontal line based on the image of the space obtained through the stereo camera.

[0283] Electronic devices according to the embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0284] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0285] The term "module" used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0286] One embodiment of the present document may be implemented as software (e.g., a program (240)) including one or more instructions stored in a storage medium (e.g., an internal memory (236) or an external memory (238)) readable by a machine (e.g., an electronic device (201)). For example, a processor (e.g., a processor (220)) of the machine (e.g., an electronic device (201)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0287] According to one embodiment, the method according to one embodiment disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0288] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In electronic devices, camera; display; and At least one processor operatively connected to the camera and the display; At least one processor of the above, Obtain images of the space through the above camera, Based on the above image, at least one of a horizontal line, which is a boundary between a wall and a floor included in the space, or a vertical line, which is a boundary between walls included in the space, is detected, If a vertex of the above space is occluded by an object, the position of the vertex is predicted based on at least one of the horizontal line or the vertical line, An electronic device configured to generate an indoor map of the space based on at least one of the horizontal line or the vertical line and the location of the predicted vertex.

2. In paragraph 1, At least one processor of the above, Displaying at least one of the horizontal line or the vertical line through the display, Based on the fact that the above image contains a plurality of horizontal lines, receiving a user input for selecting one of the plurality of horizontal lines, An electronic device for generating an indoor map of said space based on said first horizontal line selected above.

3. In paragraph 2, At least one processor of the above, Among the plurality of horizontal lines, a second horizontal line is selected based on at least one line detected in another area of ​​the space, Highlight the second horizontal line above, An electronic device that deletes a highlight displayed on the second horizontal line and highlights the first horizontal line based on the difference between the second horizontal line and the first horizontal line.

4. In paragraph 2 or 3, At least one processor of the above, An electronic device that, after receiving said user input, displays a message through said display that induces movement to capture another area of ​​said space.

5. In any one of paragraphs 1 to 4, At least one processor of the above, Predicting an extension of the above horizontal line or an extension of the above vertical line, An electronic device that predicts the location of two or more intersections of the extensions of the horizontal line or the extensions of the vertical line as the location of the vertex.

6. In any one of paragraphs 1 to 4, At least one processor of the above, Generate a virtual line based on two or more of the above horizontal lines or the above vertical lines, Predicting an extension of the above horizontal line or an extension of the above vertical line, An electronic device that predicts the location of the intersection of the virtual line with one of the extensions of the horizontal line or the extensions of the vertical line as the location of the vertex.

7. In any one of paragraphs 1 to 4, At least one processor of the above, Detect the floor area included in the above image, An electronic device that predicts the location of the intersection of the above floor area and the above vertical line as the location of the above vertex.

8. In any one of paragraphs 1 to 7, At least one processor of the above, Predict multiple horizontal lines obscured by the object based on the positions of the predicted vertices, An electronic device for generating an indoor map of the space based on the detected horizontal line and the predicted plurality of horizontal lines, wherein the closed curve including the detected horizontal line and the predicted plurality of horizontal lines is detected.

9. In any one of paragraphs 1 to 8, At least one processor of the above, An electronic device that receives user input for selecting a shape of a space prior to imaging said space through said camera.

10. In any one of paragraphs 1 to 9, The above camera, Includes a stereo camera, At least one processor of the above, An electronic device that obtains length information of the horizontal line based on an image of the space obtained through the stereo camera.

11. In a method for controlling an electronic device, An action of obtaining an image of a space through a camera of the above electronic device; An operation of detecting at least one of a horizontal line, which is a boundary between a wall and a floor included in the space, or a vertical line, which is a boundary between walls included in the space, based on the above image; An operation of predicting the position of a vertex based on at least one of the horizontal line or the vertical line when the vertex of the space is occluded by an object; and A method of controlling an electronic device, comprising: generating an indoor map of the space based on at least one of the horizontal line or the vertical line and the location of the predicted vertex.

12. In paragraph 11, An action of displaying at least one of the horizontal line or the vertical line; and further comprising an action of receiving a user input for selecting one of the plurality of horizontal lines based on the plurality of horizontal lines being included in the above image; The action of generating an indoor map of the above space is: A control method of an electronic device for generating an indoor map of the space based on the first horizontal line selected above.

13. In paragraph 12, An action of displaying at least one of the horizontal line or the vertical line, Among the plurality of horizontal lines, a second horizontal line is selected based on at least one line detected in another area of ​​the space, Highlight the second horizontal line above, A control method of an electronic device further comprising: an operation of deleting a highlight displayed on the second horizontal line and highlighting the first horizontal line based on the difference between the second horizontal line and the first horizontal line; 14. In paragraph 12 or 13, A method for controlling an electronic device, further comprising: displaying a message that induces movement to capture another area of ​​the space after receiving the user input.

15. In any one of paragraphs 11 to 14, The operation of predicting the position of the above vertex is, Predicting an extension of the above horizontal line or an extension of the above vertical line, A control method of an electronic device for predicting the position of two or more intersections of the extensions of the horizontal line or the extensions of the vertical line as the position of the vertex.

Images