Electronic device and control method therefor

Abstract

An electronic device is disclosed. The electronic device comprises a communication interface, a camera, a driving device, a memory and one or more processors, wherein the one or more processors control that: when at least one IOT device is identified through the camera while traveling the space in which the at least one IOT device is positioned, state information of the at least one IOT device is acquired on the basis of an image corresponding to the IOT device acquired through the camera and a plurality of images received through the communication interface; and, when the at least one IOT device is not identified through the camera while traveling the space in which the at least one IOT device is positioned, the state information of the at least one IOT device is acquired on the basis of the strength of a signal received from the at least one IOT device through the communication interface; and a function corresponding to the acquired state information of the at least one IOT device is performed.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device for acquiring state information of at least one IoT device and a method for controlling the electronic device for managing at least one IoT device.

[0002] Thanks to advancements in electronic technology, various types of electronic devices are being used in daily life. Among these electronic devices, there may be devices that identify and manage at least one IoT device.

[0003] For example, there may be an electronic device that identifies at least one IoT device through a camera while driving in user space.

[0004] IoT (Internet of Things) devices can refer to electronic devices that are connected to the Internet and can transmit and receive data or interact with other devices.

[0005] According to at least one embodiment of the present disclosure, the electronic device comprises a communication interface, a camera, a driving device, a memory, and one or more processors. The one or more processors can be controlled to obtain state information of the at least one IoT device based on an image corresponding to the IoT device obtained through the camera and a plurality of images received through the communication interface when the at least one IoT device is identified through the camera while driving in a space where the at least one IoT device is located, and when the at least one IoT device is not identified through the camera while driving in a space where the at least one IoT device is located, to obtain state information of the at least one IoT device based on the strength of a signal received from the at least one IoT device through the communication interface, and to perform a function corresponding to the obtained state information of the at least one IoT device.

[0006] Additionally, according to at least one embodiment of the present disclosure, a control method for an electronic device managing at least one IoT device may include: a step of obtaining state information of the at least one IoT device based on an image corresponding to the IoT device obtained by the electronic device and a plurality of images received from a server when the at least one IoT device is identified while driving in a space where the at least one IoT device is located; a step of obtaining state information of the at least one IoT device based on the strength of a signal received from the at least one IoT device when the at least one IoT device is not identified while driving in a space where the at least one IoT device is located; and a step of controlling to perform a function corresponding to the state of the at least one IoT device obtained.

[0007] Additionally, a non-transient readable recording medium according to at least one embodiment of the present disclosure stores a program for performing a control method of an electronic device for managing at least one IoT device, comprising the steps of: acquiring state information of the at least one IoT device based on an image corresponding to the at least one IoT device acquired by the electronic device and a plurality of images received from a server when the at least one IoT device is identified while driving in a space where the at least one IoT device is located; acquiring state information of the at least one IoT device based on the strength of a signal received from the at least one IoT device when the at least one IoT device is not identified while driving in a space where the at least one IoT device is located; and controlling to perform a function corresponding to the acquired state of the at least one IoT device.

[0008] FIG. 1 is a drawing for explaining the operation of an electronic device according to at least one embodiment of the present disclosure.

[0009] FIG. 2 is a drawing for explaining the form of an electronic device according to at least one embodiment of the present disclosure.

[0010] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to at least one embodiment of the present disclosure.

[0011] FIG. 4 is a detailed block diagram for illustrating an electronic device according to at least one embodiment of the present disclosure.

[0012] FIG. 5 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure transmitting and receiving data with an external device.

[0013] FIG. 6 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure transmitting and receiving data with an external device.

[0014] FIG. 7 is a sequence diagram showing the order in which an electronic device, an IoT device, and a server operate according to at least one embodiment of the present disclosure.

[0015] FIG. 8 is a drawing for illustrating a map of a user space where an IoT device is located according to at least one embodiment of the present disclosure.

[0016] FIG. 9 is a flowchart illustrating how an electronic device according to at least one embodiment of the present disclosure performs an operation.

[0017] FIG. 10 is a flowchart illustrating how an electronic device according to at least one embodiment of the present disclosure performs an operation.

[0018] The terms used in the various embodiments of this Disclosure have been selected to be as widely used and general as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant description section of this disclosure. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0019] The various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments.

[0020] In relation to the description of the drawings, similar reference numerals may be used for similar or related components.

[0021] The singular form of the noun corresponding to the item may include one or multiple items, unless the relevant context clearly indicates otherwise.

[0022] In the present disclosure, each of the phrases such as “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “at least one of A, B, or C” may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0023] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order).

[0024] Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that the component may be connected to the other component directly (e.g., via a wire), wirelessly, or through a third component.

[0025] Terms such as "include" or "have" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this disclosure, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0026] When it is said that a component is "connected," "combined," "supported," or "in contact" with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0027] When it is said that a component is located "on" another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0028] The term "and / or" includes a combination of multiple related described components or any of the multiple related described components.

[0029] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor, except for a "module" or "part" that needs to be implemented in specific hardware.

[0030] Meanwhile, the various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0031] In the present disclosure, the term "user" may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0032] An embodiment of the present disclosure will be described in more detail below with reference to the attached drawings.

[0033] FIG. 1 is a drawing for explaining the operation of an electronic device according to at least one embodiment of the present disclosure.

[0034] Referring to FIG. 1, the electronic device (100) can identify the status of at least one IoT device (200-1, 200-2) while driving through a space where at least one IoT device (200-1, 200-2) is located.

[0035] An electronic device (100) according to one embodiment receives a signal from at least one IoT device (200-1) and can identify the state of at least one IoT device (200-1) based on the strength of the signal.

[0036] Here, "at least one IoT device" may mean an electronic device that is connected to the Internet and can transmit and receive data or interact with other devices.

[0037] Here, "space" may mean a space where at least one IoT device is located, and may also mean a space where the electronic device (100) can travel.

[0038] Here, "state of at least one IoT device" may mean a state including at least one of a registration state, identification information, ownership state, location state, or operation state. Here, the ownership state may mean a state including information on at least one of a user ownership state, an unregistered state, or a third-party ownership state.

[0039] The electronic device (100) according to the present disclosure can acquire status information of at least one IoT device and perform an operation corresponding to the acquired status information of at least one IoT device, so that the user can conveniently manage IoT devices within the user space.

[0040] For example, if the ownership status of at least one IoT device is owned by another person, the electronic device (100) may change the registration status of the device without user intervention by performing an operation to initialize the registration status of at least one IoT device and proceed with registration as a user's device.

[0041] In FIG. 1, the electronic device (100) is shown in the form of a drivable projector, but is not limited thereto, and the electronic device (100) can be implemented as various types of electronic devices, such as a drivable electronic device or an electronic device that can be carried by a user.

[0042] FIG. 2 is a drawing illustrating the form of an electronic device according to at least one embodiment of the present disclosure.

[0043] Referring to FIG. 2, a movable electronic device (100) is illustrated. This is merely an example of a disclosed form and may have a shape different from the illustrated form.

[0044] The electronic device (100) may include a camera (120) on the front.

[0045] In the illustrated example, the electronic device (100) can identify the status of at least one IoT device located in space by photographing at least one IoT device through a camera (120) while driving.

[0046] Although FIG. 2 illustrates that one camera (120) is positioned on the front of the electronic device (100), the electronic device (100) may have multiple cameras positioned thereon during implementation.

[0047] The electronic device (100) includes a driving device (130) for movement (e.g., a motor, a wheel) and can perform movement using the provided driving device. Meanwhile, although the illustrated example is illustrated and described as moving the electronic device (100) using a wheel, it is also possible to use other means in the form of a caterpillar when implementing it, and if the electronic device (100) is implemented as a drone, it is also possible to equip a propeller instead of a wheel.

[0048] Meanwhile, in the illustrated example, the electronic device (100) is shown as having a driving device (130) directly, but the driving device (130) may be a separate device. For example, the electronic device (100) may be combined with a movable device such as a robot vacuum cleaner, and may be mounted on the robot vacuum cleaner to control the movement of the robot vacuum cleaner.

[0049] Meanwhile, the form of the illustrated electronic device is merely an example and can be implemented in various forms. Specific configurations constituting the electronic device (100) will be described later with reference to FIG. 3.

[0050] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to at least one embodiment of the present disclosure.

[0051] The electronic device (100) includes a communication interface (110), a camera (120), a driving device (130), a memory (140), and one or more processors (150).

[0052] According to one embodiment, the communication interface (110) can communicate with various types of external devices, such as a server or an IoT device. For example, the communication interface (110) can receive information about an IoT device or information about a user from an external server. The operation of communicating with an external device will be described in detail in FIG. 5.

[0053] The communication interface (110) may include wired or wireless input / output interfaces (or input / output terminals) according to various standards. For example, one or more connection interfaces may include various interfaces such as HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), DVI (Digital Visual Interface), AP-based Wi-Fi (Wi-Fi, Wireless LAN Network), Bluetooth, Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, AES / EBU (Audio Engineering Society / European Broadcasting Union), Optical, Coaxial, etc.

[0054] According to one embodiment, the camera (120) is configured to capture an IoT device and generate a captured image, wherein the captured image may include both video and still images.

[0055] In particular, the camera (120) can photograph various IoT devices located within the space. The camera (120) may be provided in at least one of the top area, bottom area, and side area of ​​the electronic device (100) to photograph IoT devices. The camera (120) may be implemented as a single camera, and may also be implemented as a plurality of cameras depending on the embodiment.

[0056] Additionally, the camera (120) can provide images of the IoT devices to the processor (150) to manage the IoT devices within the space.

[0057] Additionally, the camera (120) may be implemented as a wide-angle camera to capture a wide field of view, but is not limited thereto.

[0058] According to one embodiment, the driving device (130) is configured to move the electronic device (100). To this end, the driving device (130) includes a motor, wheels, etc., and can move the electronic device (100) through the movement of the wheels.

[0059] Meanwhile, the driving device (130) can adjust the shooting direction of the electronic device (100). For example, the direction in which the camera (120) looks can be adjusted by adjusting the body position of the electronic device (100) as shown in FIG. 2, or the shooting direction can be adjusted by adjusting the lens position within the camera (120).

[0060] Here, the shooting direction refers to the direction in which the IoT device is being filmed, and may also be referred to as the direction the electronic device is facing, the projection direction, etc.

[0061] According to an embodiment, the memory (140) may store data necessary for various embodiments of the present disclosure. Depending on the purpose of data storage, the memory (140) may be implemented in the form of a memory embedded in the electronic device (100) or in the form of a memory that can be attached to and detached from the electronic device (100).

[0062] For example, data for driving the electronic device (100) may be stored in memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in memory that is detachable from the electronic device (100).

[0063] In the case of memory embedded in an electronic device (100), it may be implemented in the form of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD).

[0064] In the case of a memory that can be attached to and detached from an electronic device (100), it can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.) or an external memory that can be connected to a USB port (e.g., USB memory).

[0065] The memory (140) may include various instructions required for the operation of the processor (150). Here, the instructions may include instructions for identifying the state of at least one IoT, instructions for receiving and processing a signal, instructions for performing an operation corresponding to the state of at least one IoT device, instructions for controlling the movement of the electronic device (100), instructions for generating map data for space, etc.

[0066] The memory (140) can store map information. Here, the map information is information about the space where the electronic device (100) is located, and may be information about the space divided into multiple regions. For example, if the electronic device (100) is located in a house, it may be generated based on a floor plan of the house. This map information can be generated directly by the electronic device (100), or it can be received and used from another electronic device (e.g., a robot vacuum cleaner).

[0067] At this time, the memory (140) can store additional information about each space by using information within the map information. Also, the memory (140) can store information about the state of an IoT device, such as the location, direction, or operating status of at least one IoT device.

[0068] According to an embodiment, the processor (150) controls the overall operation of the electronic device (100). Specifically, the processor (150) is connected to each component of the electronic device (100) and can control the overall operation of the electronic device (100).

[0069] The processor (150) can control the driving device so that the electronic device drives in a space where at least one IoT device is located.

[0070] When the processor (150) identifies at least one IoT device through a camera while driving in a space where at least one IoT device is located, it can obtain status information of at least one IoT device based on an image corresponding to the IoT device obtained through the camera and a plurality of images received through a communication interface.

[0071] For example, when an IoT device is recognized through a camera while driving, the processor (150) can obtain status information of the IoT device, such as the location or direction of the IoT device, by comparing an image of the recognized device and its surroundings with a plurality of images transmitted from an external device.

[0072] If the processor (150) is driving through a space where at least one IoT device is located and at least one IoT device is not identified through a camera, it may obtain status information of at least one IoT device based on the strength of a signal received from at least one IoT device through a communication interface.

[0073] For example, if the processor (150) does not recognize the IoT device through the camera while driving, it may obtain status information of the IoT device, such as the location of the IoT device, based on the signal strength of the IoT device or the strength and direction, without information about the captured image.

[0074] The processor (150) can be controlled to perform a function corresponding to the state information of at least one IoT device acquired.

[0075] For example, if the ownership status of at least one IoT device is owned by another person, the processor (150) may initialize the ownership status of at least one IoT device and proceed with registration of at least one IoT device based on at least one of unique identification information or user information for at least one IoT device received through a communication interface. That is, the processor (150) may perform an operation to change the registered name of the IoT device from another person to a user.

[0076] Here, "user information" may mean information including at least one of information about the space or user account information.

[0077] According to an embodiment, the processor (150) can control at least one IoT device (200), server (300), or user terminal device (400).

[0078] The processor (150) can perform the operation of the electronic device (100) according to various embodiments by executing at least one instruction stored in memory.

[0079] According to an embodiment, the processor (150) may be implemented as a digital signal processor (DSP) that processes digital signals, a microprocessor, or a TCON (Timing controller). However, it is not limited thereto and may include or be defined by one or more of a central processing unit (CPU), a Micro Controller Unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, or an AI (Artificial Intelligence) processor. Additionally, the processor (150) may be implemented as a System on Chip (SoC) or Large Scale Integration (LSI) with a built-in processing algorithm, or may be implemented in the form of a Field Programmable Gate Array (FPGA). The processor (150) can perform various functions by executing computer executable instructions stored in memory.

[0080] The processor (150) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator.

[0081] The processor (150) can control one or any combination of other components of the electronic device and can perform operations or data processing related to communication. The processor (150) can execute one or more programs or instructions stored in memory. For example, the processor (150) can perform the method according to an embodiment of the present disclosure by executing one or more instructions stored in memory.

[0082] If the method according to the embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors.

[0083] For example, when the first operation, the second operation, and the third operation are performed by the method according to the embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence dedicated processor).

[0084] The processor (150) may be implemented as a single-core processor including one core, or as one or more multi-core processors including multiple cores (e.g., homogeneous multi-core or heterogeneous multi-core).

[0085] When the processor (150) is implemented as a multi-core processor, each of the multiple cores included in the multi-core processor may include internal processor memory such as cache memory and on-chip memory, and a common cache shared by multiple cores may be included in the multi-core processor.

[0086] Each of the multiple cores (or some of the multiple cores) included in the multi-core processor may independently read and execute program instructions for implementing the method according to the embodiment of the present disclosure, or all (or some of) of the multiple cores may be linked together to read and execute program instructions for implementing the method according to the embodiment of the present disclosure.

[0087] FIG. 4 is a detailed block diagram for illustrating an electronic device according to at least one embodiment of the present disclosure.

[0088] Referring to FIG. 4, an electronic device (100) according to one embodiment of the present disclosure may include a communication interface (110), a camera (120), a driving device (130), a memory (140), a processor (150), an output interface (160), and an input interface (170). Parts that overlap with the description above will be omitted or abbreviated below.

[0089] The output interface (160) can provide various feedback or device notification information to the user. In particular, the output interface (160) may be equipped with a speaker (161), a display (162), etc., but this is merely one embodiment and may further include other output devices (e.g., a haptic device, etc.).

[0090] The processor (150) can control the output of notification information for the IoT device through the output interface (160).

[0091] For example, the processor (150) may generate notification information including at least one of the registration status, identification information, ownership status, location status, operation status, usage history information, or type information of the IoT device, and may provide the generated notification information to a user through an output interface (160), such as a speaker (161) or a display (162).

[0092] At this time, the speaker (161) is provided in at least one of the top area, bottom area, and side area of ​​the electronic device (100) and can provide various auditory feedback or notification information through audio.

[0093] The display (162) may be located in at least one of the top area, bottom area, and side area of ​​the electronic device (100) to provide various visual feedback or notification information to the user. The display (162) may be implemented as a single display, and may also be implemented as a plurality of displays depending on the embodiment.

[0094] For example, if the presence of an IoT device is confirmed through the camera (120) but the status of the IoT device is not identified, such as when no signal is detected, the speaker (161) may provide location information about the IoT device to the user and provide notification information requesting confirmation of the IoT device.

[0095] Additionally, if the ownership status of an IoT device in the space is identified as being owned by another person, the electronic device (100) may provide information about the operation history of the device during the period corresponding to the ownership status to the user through an output interface (160), such as a speaker (161) or a display (162).

[0096] However, it is not limited to this, and the electronic device (100) may also provide notification information requesting confirmation of the IoT device by displaying a captured image on the display (162).

[0097] The display (162) can be implemented as various types of displays, such as a large format display (LFD) or a projector display that projects and displays images. For example, if the electronic device (100) includes a display in the form of a projector display, it may display notification information about the device by projecting images onto a wall or floor surface.

[0098] The input interface (170) can receive various feedback from the user. For example, if the electronic device (100) provides a notification to the user requesting confirmation of the status of the IoT device, the user's feedback can be received by recognizing the user's voice through the microphone (171).

[0099] However, it is not limited to this, and the electronic device (100) may also receive user feedback through a touch screen (172).

[0100] FIG. 5 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure transmitting and receiving data with an external device.

[0101] The electronic device (100) can communicate with IoT devices (200), servers (300), and user terminal devices (400) connected via a network. Devices connected to the electronic device (100) may be collectively referred to as external electronic devices or external devices. The electronic device (100) may control at least some of the external electronic devices or external devices.

[0102] In FIG. 5, for convenience, the IoT device is depicted as an air purifier, but it may include various electronic devices that can interact with other devices through a network, such as a robot vacuum cleaner, air conditioner, refrigerator, TV, camera, lighting, and smart device. Also, the user terminal device (400) is depicted as a smartphone, but it may include various electronic devices such as a tablet PC, laptop, and wearable device.

[0103] In FIG. 5, the server (300) can be implemented as various computing devices such as a workstation, cloud, data drive, and data station. Additionally, the network may include both wired and wireless networks. The wired network may include a cable network or a telephone network, and the wireless network may include any network that transmits and receives signals via radio waves. The wired network and the wireless network may be connected to each other.

[0104] Specifically, networks may include wide area networks (WANs) such as the Internet, local area networks (LANs) formed around access points (APs), and short-range wireless networks that do not use access points (APs). Short-range wireless networks may include Bluetooth (IEEE 802.15.1), Zigbee (IEEE 802.15.4), Wi-Fi Direct, Near Field Communication (NFC), Z-Wave, etc., but are not limited thereto.

[0105] According to FIG. 5, the server (300) can obtain data regarding the status or user information of an IoT device (200) in space based on various data received from various electronic devices connected through a network.

[0106] For example, the server (300) can receive information about the status of the IoT device (200) from the electronic device (100) and store it in a database (DB).

[0107] The database (DB) may be managed as multiple databases depending on the ownership status of the IoT device (200). For example, the server (300) may manage a user-owned DB, an unregistered DB, and a third-party-owned DB in correspondence with the ownership status of the IoT device. Here, the unregistered DB may include data regarding an unconfirmed state where the physical device is verified but the ownership status is not verified.

[0108] However, this is not limited to this, and information regarding the status of IoT devices may be integrated and managed in a single database.

[0109] Additionally, the server (300) may obtain and store information about the user (50) from the user terminal device (400). Here, the information about the user may include at least one of information about the space or user account information.

[0110] For example, the server (300) may generate map data for the space where the IoT device is located based on information about the space obtained from various electronic devices (100, 200, 400) and status information of the IoT device.

[0111] However, this is not limited thereto, and the operation of generating map data for the space may also be performed by an electronic device (100). The operation of generating and updating a map for the space is described in detail in FIG. 8.

[0112] The processor (150) of the electronic device (100) can identify the state of at least one IoT device based on the state of at least one IoT device or user information obtained from at least one external device (200, 300, 400).

[0113] The processor (150) of the electronic device (100) may transmit the state of at least one IoT device to at least one external device (200, 300, 400) so that the external device (200, 300, 400) updates a database that stores information about the state of at least one IoT device.

[0114] FIG. 6 is a diagram illustrating the operation of an electronic device according to at least one embodiment of the present disclosure transmitting and receiving data with an external device.

[0115] The electronic device (100) can receive device status information, such as the operation history and location status of the device, from the IoT device (200). Additionally, the electronic device (100) may receive device status information or user information from the server (300).

[0116] If the IoT device (200) is in an unregistered state or is owned by another person, the device information can be entered into a server to register the device under the owner's name.

[0117] For example, if the ownership status of at least one IoT device is unregistered, the processor (150) of the electronic device (100) may proceed with the registration of at least one IoT device based on at least one of unique identification information or user information for at least one IoT device received through a communication interface.

[0118] Here, the operation of "proceeding with the registration of an IoT device" may mean that the electronic device (100) performs operations necessary for the registration of an IoT device, such as transmitting IoT device information and user information to the server (300), and the server (300) proceeds with the device registration. Here, the user information may include at least one of information about a space or user account information.

[0119] For example, if the operation required for IoT device registration is an operation of generating and transmitting a specific signal, the processor (150) of the electronic device (100) can generate the signal required for device registration and transmit it to the IoT device (200). In this case, registration of the IoT device can be performed solely by the operation of the electronic device (100) without physical intervention by the user.

[0120] For example, the electronic device (100) can obtain information regarding the ownership status of the device among the status information for the IoT device (200) based on an image obtained through a camera and a plurality of images received from the IoT device. If the ownership status of the IoT device (200) is in a state of being owned by another person, the electronic device (100) can proceed with the initialization of the ownership status by receiving information regarding a signal that operates the IoT device (200) (for example, information regarding a signal corresponding to the windless button of a remote control) and transmitting it to the IoT device (200).

[0121] That is, the operation to initialize the ownership status of the IoT device (200) can be performed without physical access by the user. However, if the operation required for device registration is an operation that requires physical action, such as pressing a physical button, user intervention may be required.

[0122] Additionally, the processor (150) of the electronic device (100) can proceed with the registration of at least one IoT device based on the purchase information for at least one IoT device when the ownership status of at least one IoT device is unregistered and purchase information for at least one IoT device is received through a communication interface.

[0123] On the other hand, the processor (150) of the electronic device (100) may proceed with the registration of at least one IoT device by performing an action necessary for the registration of at least one IoT device when the ownership status of at least one IoT device is unregistered and purchase information for at least one IoT device is not received through a communication interface.

[0124] That is, when purchase information for an IoT device is received, the server (300) can proceed with device registration based on the purchase information without needing to perform operations necessary for the registration of the IoT device, such as transmitting device information and user information to the server (300).

[0125] Here, "actions required for registration of an IoT device" may include at least one of transmitting device information, setting up a device, creating an account, or connecting a device.

[0126] When the device is registered in the name of the owner, the server (300) can transmit the updated status of the IoT device to the electronic device (100).

[0127] FIG. 7 is a sequence diagram showing the order in which an electronic device, an IoT device, and a server operate according to at least one embodiment of the present disclosure. FIG. 7 is illustrated as the electronic device (100) and the IoT device (200) directly communicate, but is not necessarily limited thereto, and as illustrated in FIG. 6, a server (300) may relay communication between the electronic device (100) and the IoT device (200).

[0128] Referring to FIG. 7, the electronic device (100) can patrol the space where the IoT device (200) is located to identify the IoT device (200) (S710). The IoT device (200) can store information about the IoT device (200) and generate an electrical signal (S720) to transmit data about the device information or signal to the electronic device (100) or the server (300).

[0129] The electronic device (100) can identify the IoT device based on data received from the IoT device (200) while patrolling the space (S731). The server (300) can proceed with registration for the IoT device (200) based on data received from the IoT device (200) (S732). The server (300) can store device information received from the IoT device, and can also store user information by obtaining user input or user information from an external device (S733).

[0130] The electronic device (100) can obtain status information of the IoT device (200) based on device information and user information received from the server (300) (S740). The electronic device (100) can also update the database (DB) by transmitting information about the status of the IoT device (200) to the server (300) (S750).

[0131] Meanwhile, although the above description was based on the case where electronic devices, IoT devices, and servers communicate, the same operation can also be performed by communicating with various electronic devices, such as user terminal devices, as shown in Fig. 5.

[0132] FIG. 8 is a drawing for illustrating a map of a space where an IoT device is located according to at least one embodiment of the present disclosure.

[0133] When the processor (150) of the electronic device (100) identifies the location status of at least one IoT device, it can generate map data for the space where at least one IoT device is located based on the location status of at least one IoT device.

[0134] For example, the electronic device (100) can generate map data regarding the location and space of IoT devices (200-1 to 200-4) while driving through space via a driving device. At least one IoT device (200-1 to 200-4) may include various types of electronic devices used in space, such as air conditioners, TVs, smart lighting devices, and IP cameras.

[0135] In this case, the electronic device (100) may obtain a sensor-based map by determining the distance to surrounding obstacles through a sensor, or may obtain a sensor-based map based on the distance to surrounding obstacles determined by another external device. The processor (150) of the electronic device (100) may update the map of the space at preset times based on the generated map data.

[0136] Specifically, the electronic device (100) identifies the distance to the wall as s1 through a sensing value, and if the identified distance becomes shorter than s1, it may identify the presence of an obstacle other than the wall, such as an electronic device. At this time, whether the obstacle other than the wall is an IoT device may be identified based on an image captured by a camera or the signal strength of the IoT device.

[0137] The electronic device (100) can transmit the aforementioned sensing data or map data to a server (300) or a user terminal device (400) through a communication interface. The server (300) can estimate the locations of IoT devices (200-1, 200-2, 200-3, 200-4) based on the aforementioned sensing data and map data, and can generate and update a map of the space.

[0138] Meanwhile, although the above description is based on the case where the electronic device (100) maps the space, it is not necessarily limited to the electronic device (100), and the same operation can be performed by various external devices equipped with driving functions. For example, in the case of a robot vacuum cleaner or other driving robots, it can freely drive within the space while equipped with a camera. For these types of external devices as well, whenever they pass through the user's space, they can photograph or sense the space to map it and then transmit the data to the user terminal device (400), etc.

[0139] FIG. 9 is a flowchart illustrating how an electronic device according to at least one embodiment of the present disclosure performs an operation.

[0140] Referring to FIG. 9, when an IoT device is identified while driving, the electronic device can obtain status information of at least one IoT device based on an image corresponding to the IoT device captured by the electronic device and a plurality of images received from a server (S910).

[0141] For example, status information of IoT devices in a space, such as the type or form of the IoT device, can be obtained by analyzing the similarity between an image captured by an electronic device and multiple captured images stored on a server. However, this is not limited to this, and various image analysis techniques may be applied.

[0142] And, if at least one IoT device is not identified while driving, the electronic device may obtain status information of at least one IoT device based on the strength of a signal received from at least one IoT device (S920).

[0143] For example, an electronic device can estimate the location and direction of an IoT device by detecting the point where the signal strength increases and driving in that direction.

[0144] And, the electronic device can perform an operation corresponding to the status information of at least one IoT device acquired (S930).

[0145] For example, if the ownership status of an IoT device is identified as unregistered, initial registration as user ownership can be performed. Additionally, if the ownership status of an IoT device is identified as belonging to a third party, the electronic device may perform re-registration as user ownership. This is explained in detail in FIG. 10.

[0146] FIG. 10 is a flowchart illustrating how an electronic device according to at least one embodiment of the present disclosure performs an operation.

[0147] The electronic device can reset the ownership status of at least one IoT device if the ownership status of at least one IoT device is owned by another person (S1010). For example, the electronic device can reset the ownership status of the IoT device by communicating with a server and deleting the registered name of the IoT device registered with the server.

[0148] And the electronic device may proceed with the registration of at least one IoT device based on at least one of unique identification information or user information for at least one IoT device (S1020). In this case, the user information may include at least one of information about a space or user account information.

[0149] For example, an electronic device can perform device registration by communicating with a server to create and store user account information corresponding to unique identification information for the IoT device.

[0150] The control method of the electronic device described in FIGS. 9 and FIGS. 10 may be performed by a device having various configurations such as FIGS. 3 and FIGS. 4 described above, but is not necessarily limited thereto and may be performed by a device having various configurations.

[0151] The various embodiments described above may be implemented as individual embodiments, or at least one embodiment may be combined with one another, either wholly or partially, to be implemented together in a single device.

[0152] According to the various embodiments described above, an electronic device capable of traveling through space can manage IoT devices within a user space without direct user intervention by identifying the location and ownership status of IoT devices installed in the space. Ultimately, this can enhance the user experience.

[0153] Various embodiments of the present disclosure may be implemented as software stored on a machine-readable storage media that can be mounted on or connected to a smartphone, a user terminal device, and various other electronic devices (e.g., a computer).

[0154] Specifically, a non-transient readable recording medium may be provided that stores software for sequentially performing the steps of: driving an electronic device in a space where at least one IoT device is located; when at least one IoT device is identified during driving, photographing at least one IoT device and comparing the photographed image with a plurality of photographed images to obtain state information of at least one IoT device; when at least one IoT device is not identified during driving, receiving a signal from at least one IoT device and obtaining state information of at least one IoT device based on the strength of the signal; and performing an operation corresponding to the state information of at least one IoT device.

[0155] A device equipped with such a non-transient readable medium can perform various operations, such as driving in space as described in the various embodiments above, obtaining status information of an IoT device, and performing operations corresponding to the status information of an IoT device.

[0156] In non-transitory readable recording media, 'non-transitory' merely means that the recording medium does not contain a signal and is tangible; it does not distinguish whether data is stored semi-permanently or temporarily on the storage medium.

[0157] Alternatively, a program for performing the methods according to the various embodiments described above may be distributed online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a recording medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0158] Each component (e.g., module or program) according to various embodiments may consist of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the same or similar functions as those performed by each of the respective components prior to integration.

[0159] Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0160] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, Communication interface; camera; A driving device for moving the above electronic device; Memory for storing at least one instruction; and It includes one or more processors that execute at least one of the above instructions; The above one or more processors, When at least one IoT device is identified through the camera while driving in a space where at least one IoT device is located, state information of the at least one IoT device is obtained based on an image corresponding to the IoT device acquired through the camera and a plurality of images received through the communication interface. If the at least one IoT device is not identified through the camera while driving in the space where the at least one IoT device is located, status information of the at least one IoT device is obtained based on the strength of the signal received from the at least one IoT device through the communication interface, and An electronic device that controls to perform a function corresponding to the status information of at least one IoT device obtained above.

2. In Paragraph 1, The status information of the above-mentioned at least one IoT device is, It includes at least one of a registration status, identification information, ownership status, location status, or operation status, and The above ownership status is, An electronic device comprising information on at least one of a user ownership status, an unregistered status, or a third party ownership status.

3. In Paragraph 1, The above one or more processors, Based on information or user information corresponding to the at least one IoT device obtained from the server, state information of the at least one IoT device is obtained, and An electronic device that controls the server to update a database (DB) storing information corresponding to the at least one IoT device, and transmits the acquired status information of the at least one IoT device to the server through the communication interface.

4. In Paragraph 1, The above one or more processors, When location information of at least one IoT device is obtained based on the above state information, map data corresponding to the space where the at least one IoT device is located is obtained based on the obtained location information, and An electronic device that updates a map corresponding to the space at predetermined time points based on the map data above.

5. In Paragraph 1, The above one or more processors, If the ownership status of the at least one IoT device included in the above status information is a third-party ownership status, control is made to initialize the ownership status of the at least one IoT device. Registration of the at least one IoT device is performed based on at least one of unique identification information or user information for the at least one IoT device received through the communication interface, and The above user information is, An electronic device comprising information including at least one of information about the space or user account information.

6. In Paragraph 1, Including an output interface; further The above one or more processors, An electronic device that controls the generation of notification information including at least one of the registration status, identification information, ownership status, location status, operation status, usage history information, or type information of at least one IoT device, and outputs it through the output interface.

7. In Paragraph 1, The above one or more processors, If the ownership status of the at least one IoT device included in the above status information is unregistered, the registration of the at least one IoT device is performed based on at least one of the unique identification information or user information for the at least one IoT device received through the communication interface. The above user information is, An electronic device comprising at least one of information about the space or user account information.

8. In Paragraph 1, The above one or more processors, If the ownership status of the at least one IoT device included in the above status information is unregistered, and purchase information for the at least one IoT device is received through the communication interface, registration of the at least one IoT device is performed based on the purchase information for the at least one IoT device. An electronic device that, if the ownership status of the at least one IoT device included in the above status information is unregistered and purchase information for the at least one IoT device is not received through the communication interface, performs a function corresponding to the registration of the at least one IoT device to proceed with the registration of the at least one IoT device.

9. In Paragraph 8, The above one or more processors, An electronic device that performs at least one function among device information transmission, device configuration, account creation, or device connection to proceed with the registration of the at least one IoT device.

10. A method for controlling an electronic device that manages at least one IoT device, When the at least one IoT device is identified while driving in a space where the at least one IoT device is located, the step of obtaining status information of the at least one IoT device based on an image corresponding to the IoT device captured by the electronic device and a plurality of images received from a server; If the at least one IoT device is not identified while driving in a space where the at least one IoT device is located, the step of obtaining status information of the at least one IoT device based on the strength of a signal received from the at least one IoT device; and A control method comprising: a step of controlling to perform a function corresponding to the state information of at least one IoT device obtained above.

11. In Paragraph 10, The status information of the above-mentioned at least one IoT device is, It includes at least one of a registration status, identification information, ownership status, location status, or operation status, and The above ownership status is, A control method comprising information on at least one of a user-owned status, an unregistered status, or a third-party-owned status.

12. In Paragraph 10, The method further includes the step of controlling the server to transmit the acquired status information of at least one IoT device to the server so that the server updates a database that stores information corresponding to at least one IoT device. The step of acquiring state information of at least one IoT above is, A control method comprising: a step of obtaining state information of at least one IoT device based on information or user information corresponding to at least one IoT device obtained from the server.

13. In Paragraph 10, When location information of at least one IoT device is obtained based on the above state information, a step of obtaining map data corresponding to the space where the at least one IoT device is located based on the obtained location information; and A control method further comprising the step of updating a map corresponding to the space at each preset time point based on the map data.

14. In Paragraph 10, The above-mentioned controlling step is, If the ownership status of the at least one IoT device included in the above status information is a third-party ownership status, a step of controlling the ownership status of the at least one IoT device to be initialized; and The method includes the step of registering the at least one IoT device based on at least one of unique identification information or user information for the at least one IoT device received from the server. The above user information is, A control method comprising information including at least one of information about the space or user account information.

15. A non-transient readable recording medium storing a program for executing a control method of an electronic device managing at least one IoT device, The above control method is, When the at least one IoT device is identified while driving in a space where the at least one IoT device is located, the step of obtaining status information of the at least one IoT device based on an image corresponding to the at least one IoT device captured by the electronic device and a plurality of images received from a server; If the at least one IoT device is not identified while driving in a space where the at least one IoT device is located, the step of obtaining status information of the at least one IoT device based on the strength of a signal received from the at least one IoT device; and A non-transient readable recording medium comprising: a step of controlling to perform a function corresponding to the state information of at least one IoT device obtained above.

Images