Method and system for detecting abnormal operation of robotic vacuum cleaner

Abstract

An electronic device is disclosed. The electronic device includes: a communication interface including circuitry; a sensor; a memory storing data including a plurality of movement modes when the robotic device is normally operated; and at least one processor electrically coupled to the memory, and the at least one processor configured to: obtain a movement pattern of the robotic device based on at least one of a signal strength received from the robotic device through the communication interface or sensing data of the sensor, and identifying an operation state of the robotic device as a normal operation or an abnormal operation according to the movement pattern obtained based on the data.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and a method for controlling the electronic device, and more particularly to an electronic device for identifying an operation state of a robot device according to a movement pattern of the robot device and a control method for controlling the electronic device. Background Art

[0002] With the development of electronic technology, various types of electronic devices are being developed and released to perform various operations. For example, the electronic devices may include various forms of robotic devices that perform specific operations by being driven within a space, such as but not limited to being a service robot that replaces people in stores, cafes, restaurants, etc., a cleaning robot that automatically cleans an area that needs to be cleaned by sucking up foreign matter without requiring a separate operation by the user, etc.

[0003] However, for a robotic device located in a private space such as a home, surrounding environment data sensed by the robotic device may be data that requires a high level of security and may become data that violates privacy when leaked to the public or accessed by hackers.

[0004] Accordingly, there is a need for a method of sensing abnormal operation of a robotic device and providing feedback by a nearby electronic device when abnormal access to the robotic device (eg, a hacker attack) occurs and the robotic device operates abnormally. Summary of the invention

[0005] According to one aspect of the present disclosure, there is provided an electronic device comprising: a communication interface; a sensor; a memory storing data including a plurality of stored movement patterns corresponding to normal operation of a robot device; and at least one processor electrically coupled to the memory and configured to: obtain a movement pattern of the robot device based on at least one of a signal strength received from the robot device through the communication interface or sensing data from the sensor, and identify an operating state of the robot device as normal operation or abnormal operation based on a comparison between the obtained movement pattern and a plurality of stored movement patterns.

[0006] According to another aspect of the present disclosure, a control method for an electronic device is provided, which includes a memory storing data including multiple movement patterns corresponding to normal operation of a robot device, the method comprising: obtaining the movement pattern of the robot device based on at least one of the signal strength received from the robot device or the sensing data of a sensor; and identifying the operation state of the robot device as normal operation or abnormal operation based on a comparison between the obtained movement pattern and multiple stored movement patterns. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The above and other aspects, features and advantages of certain embodiments will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0008] Figure 1 is a diagram showing an electronic device and a robot apparatus according to one or more embodiments;

[0009] Figure 2 is a block diagram showing a configuration of an electronic device according to one or more embodiments;

[0010] Figure 3 is a diagram showing an electronic device measuring signal strength according to one or more embodiments;

[0011] Figure 4 is a diagram illustrating a change in an electronic device recognition position according to one or more embodiments;

[0012] Figure 5 is a diagram showing an electronic device identifying a normal operation and an abnormal operation according to one or more embodiments;

[0013] Figure 6 is a diagram showing a method of obtaining a plurality of clusters according to one or more embodiments;

[0014] Figure 7 is a diagram illustrating an electronic device using a plurality of clusters received from an external electronic device according to one or more embodiments;

[0015] Figure 8 is a diagram illustrating a neural network model according to one or more embodiments;

[0016] Fig. 9 is a diagram illustrating normal operation and abnormal operation according to one or more embodiments;

[0017] Fig.10 is a diagram showing an electronic device including a camera according to one or more embodiments;

[0018] Fig.11 is a diagram showing an electronic device using a camera to identify an operation state of the electronic device according to one or more embodiments;

[0019] Fig.12 is a diagram showing an operation state in which an electronic device recognizes a robot apparatus avoiding an object according to one or more embodiments;

[0020] Fig.13 is a diagram showing a user terminal device providing a notification according to one or more embodiments; and

[0021] Fig.14is a flowchart illustrating a control method of an electronic device according to one or more embodiments. DETAILED DESCRIPTION

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings.

[0023] The terms used to describe one or more embodiments of the present disclosure are currently widely used general terms selected in consideration of their functions in this article. However, these terms may change according to the intentions of technicians in the relevant fields, legal or technical interpretations, the emergence of new technologies, etc. In addition, in some cases, terms may be arbitrarily selected, in which case the meaning of the terms will be disclosed in more detail in the corresponding description. Accordingly, the terms used in this article should not be simply understood as their names, but should be understood based on the meaning of the terms and the overall context of the present disclosure.

[0024] In the present disclosure, expressions such as “having”, “may have”, “including”, “may include”, etc. are used to specify the existence of corresponding features (for example, elements such as values, functions, operations or components), but do not exclude the existence or possibility of additional features.

[0025] The expression “at least one of A and / or B” should be understood to mean any one of “A” or “B” or “A and B”.

[0026] As used herein, expressions such as “first”, “second”, “1st”, “2nd”, etc. may be used to refer to various elements regardless of their order and / or importance, and it should be noted that these expressions are only used to distinguish one element from another rather than to limit the relevant elements.

[0027] When an element (eg, a first element) is indicated as being “(operably or communicatively) coupled to” or “connected to” another element (eg, a second element), it can be understood that the element is directly coupled to the other element or coupled through other elements (eg, a third element).

[0028] Unless otherwise specified, singular expressions include plural expressions. It should be understood that terms such as "formed" or "comprising" are used herein to indicate the presence of features, quantities, steps, operations, elements, components or combinations thereof, without excluding the possibility of the presence or addition of one or more other features, quantities, steps, operations, elements, components or combinations thereof.

[0029] The term "module" or "component" used in the embodiments of this document performs at least one function or operation, and can be implemented by hardware or software, or by a combination of hardware and software. In addition, in addition to the "module" or "component" that needs to be implemented on specific hardware, multiple "modules" or multiple "components" can be integrated into at least one module and implemented on at least one processor (not shown).

[0030] In the present disclosure, the term “user” may refer to a person using an electronic device or a device (eg, an artificial intelligence electronic device) using an electronic device.

[0031] One or more embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0032] Figure 1 is a diagram illustrating an electronic device and a robot apparatus according to one or more embodiments.

[0033] Reference Figure 1 , a plurality of electronic devices 100-1, 100-2, and 100-3 may be set in an area such as a home. For example, the plurality of electronic devices may be various types of home appliances that are being developed and supplied to consumers for use.

[0034] For example, each of the electronic devices 100 - 1 , 100 - 2 , and 100 - 3 may be implemented as an Internet of Things (IoT) device and may perform communication with each other.

[0035] For example, each of the electronic devices 100 - 1 , 100 - 2 , and 100 - 3 may include an air conditioner 100 - 1 , an air purifier 100 - 2 , a display apparatus 100 - 3 , and the like.

[0036] However, this is just an example, and the electronic device 100 can be implemented as various types of household appliances. For example, in addition to the display device 100-3 including various types of displays, the electronic device 100 can also include at least one of a user terminal device, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a notebook PC, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a medical device, a camera, a virtual reality (VR) implementation device, or a wearable device. The wearable device may include at least one of the following: an ornament type (e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD)), a fabric or clothing embedded type (e.g., electronic clothing), a skin attachment type (e.g., a skin pad or a tattoo), or a biological implant circuit. In one or more embodiments, the electronic device 100 may include at least one of the following: a television, a digital video disc (DVD) player, an audio, a refrigerator, an air conditioner, a cleaner, an oven, a microwave, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync TM 、AppleTV TM or GoogleTV TM ), game consoles (e.g., Xbox TM , PlayStation TM ), electronic dictionary, electronic key, camera or electronic photo frame.

[0037] In other embodiments, the electronic device 100 may include at least one of the following: various medical devices (e.g., various portable medical measuring devices (glucose measuring devices, heart rate measuring devices, blood pressure measuring devices, temperature measuring devices, etc.), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), computed tomography (CT), imaging devices, ultrasonic devices, etc.), navigation devices, global navigation satellite systems (GNSS)), event data recorders (EDRs), flight data recorders (FDRs), vehicle infotainment devices, marine electronic devices (e.g., marine navigation devices, gyrocompasses, etc.), avionics devices, security devices, vehicle head units, industrial or household robots, drones, automatic teller machines (ATMs) of financial institutions, points of sale (POS) of stores, or Internet of Things devices (e.g., light bulbs, various sensors, sprinklers, fire alarms, thermostats, street lights, toasters, fitness equipment, hot water tanks, heaters, boilers, etc.).

[0038] For the convenience of the following description, the electronic device 100 may be assumed to be an air conditioner 100-1, the external electronic device may be assumed to be at least one of an air purifier 100-2 or a display device 100-3, and various examples of the present disclosure have been described assuming an embodiment of the air conditioner 100-1, but are not limited thereto, and various examples of the present disclosure may be implemented in the above-mentioned various types of household appliances. For example, various examples may be implemented in the air purifier 100-2 and the display device 100-3, respectively. In addition, the electronic device 100 may be referred to as a first electronic device (e.g., air conditioner 100-1), and the external device may be referred to as a second electronic device different from the first electronic device (e.g., air purifier 100-2 or display device 100-3).

[0039] The electronic apparatus 100 according to one or more embodiments may recognize a movement pattern of the robot device 200, and recognize an operation state of the robot device 200 as a normal operation or an abnormal operation according to the recognized movement pattern.

[0040] The robot device 200 may refer to various types of devices that have the ability to perform one or more functions. For example, the robot device 200 may be a device that performs one or more functions by itself. In an example, the robot device 200 may refer to an intelligent device that, in addition to simple repetitive functions, can also operate autonomously by sensing the surrounding environment of the robot device 200 in real time and collecting data based on sensor data of sensors (e.g., light detection and ranging (LiDAR) sensors, cameras, etc.).

[0041] The robot device 200 according to one or more embodiments may include a driver including an actuator or a motor. The driver according to one or more embodiments may include wheels, brakes, etc., and the robot device 200 may move itself within a specific space using the wheels, brakes, etc. included in the driver.

[0042] In addition, the robot device 200 may include an articulated robot. Here, the articulated robot may refer to an element of the robot device 200 to replace the function of a human arm or hand.

[0043] The robot device 200 can be divided into industrial use, medical use, home use, military use, exploration use, etc. according to the field or executable function. The industrial use robot device according to one or more embodiments can be divided into a robot device used in a product manufacturing process in a factory, a robot device that performs customer reception, order reception, service, etc. in a store or restaurant, etc. For example, the robot device 200 can be implemented as a service robot device, which can deliver a service product to a user's desired location or to a specific location in various places, such as but not limited to restaurants, hotels, supermarkets, hospitals, clothing stores, etc.

[0044] However, the above is only an example, and the robot device 200 may be variously classified according to application fields, functions, and usage purposes, without being limited to the above example.

[0045] For example, Figure 1 As shown, the robot device 200 can be implemented as a robot cleaner located at home. Here, the robot cleaner may refer to a device driven by electricity for automatically sucking foreign matter. For the convenience of the following description, it is assumed that the robot device 200 is a robot cleaner, and it is assumed that the robot cleaner is implemented in a flat form that is in close contact with the floor to suck foreign matter on the floor, but this is only one embodiment, and the robot device 200 can be implemented in various forms as described above.

[0046] For example, because the robot device 200 is configured to sense the surrounding environment of the robot device 200 by being placed in a home including a private area (e.g., a bathroom, a bedroom, a dressing room, etc.), the sensed data (obtained by the robot device 200 sensing the surrounding environment) may be data that requires a high level of security and may be data that violates privacy when leaked to the outside. For example, the sensed data may be a captured image from a camera provided in the robot device 200, and the captured image may include an image from a private area in the house.

[0047] For example, problems may arise if sensing data of the robot device 200 is leaked to the outside due to abnormal access to the robot 200, if the robot device 200 is moved to a private area (e.g., a bathroom, bedroom, dressing room, etc.) without authorization, or if the robot device is unable to avoid objects in the home (e.g., people, pets, walls, furniture, household appliances, etc.), thereby causing inconvenience, anxiety, and other concerns to the user.

[0048] Accordingly, the electronic apparatus 100 according to one or more embodiments may recognize a movement pattern of the robot device 200, and recognize an operation state of the robot device 200 as a normal operation or an abnormal operation according to the recognized movement pattern.

[0049] In addition, instead of having multiple electronic devices 100-1, 100-2 and 100-3 placed in the home operate independently and individually to identify the operating status of the robot device 200, the operating status of the robot device 200 can be identified by the multiple electronic devices 100-1, 100-2 and 100-3 operating in organic cooperation with each other.

[0050] Figure 2 is a block diagram illustrating a configuration of an electronic device according to one or more embodiments.

[0051] Reference Figure 2 , the electronic device 100 according to one or more embodiments may include a communication interface 110, a sensor 120, a memory 130, and at least one processor 140. However, the present disclosure is not limited thereto, and thus, other components may be provided in the electronic device 100.

[0052] According to one or more embodiments, the communication interface 110 can receive inputs of various types of data and information by communicating with external devices. For example, the communication interface 110 can receive inputs of various types of data, information, etc. from, for example, but not limited to, home appliances (for example, but not limited to display devices, air conditioners, air purifiers, etc.), external storage media (for example, USB memory), external servers (for example, WEBHARD), etc., through communication methods such as but not limited to AP-based Wi-Fi (for example, Wi-Fi, wireless LAN network), Bluetooth, ZigBee, wired / wireless local area network (LAN), wide area network (WAN), Ethernet, IEEE 1394, High Definition Multimedia Interface (HDMI), Universal Serial Bus (USB), Mobile High Definition Link (MHL), Audio Engineering Society / European Broadcasting Union (AES / EBU), optical fiber, coaxial cable, etc., such as but not limited to.

[0053] Specifically, the communication interface 110 according to one or more embodiments may perform communication with the robot device 200. For example, the communication interface 110 may perform direct communication (e.g., device-to-device communication) with the robot device 200 according to the control of at least one processor 140, and identify the signal strength received from the robot device 200.

[0054] For example, the communication interface 110 may identify the signal strength received from the robot device 200 by performing direct communication with the robot device 200 using a beacon (a Wi-Fi or Bluetooth Low Energy (BLE) based beacon). Here, the signal strength may include a received signal strength indication (RSSI) value.

[0055] For example, the RSSI value may be expressed as a negative number (dBm, mW unit), and may be expressed as a value approaching 0 as the signal strength increases, as the distance between the electronic device 100 and the robot device 200 becomes shorter, the RSSI value may be approaching 0, and as the distance between the electronic device 100 and the robot device 200 becomes farther, the RSSI value may be farther from 0. According to an example, the RSSI value may be expressed as a value from 0 (dBm) to -100 (dBm).

[0056] As described below, at least one processor 140 may identify the distance between the electronic device 100 and the robot device 200 based on the signal strength. According to an embodiment, at least one processor 140 may obtain a movement pattern of the robot device 200 based on the identified distance. For example, at least one processor 140 receives a plurality of distances between the electronic device 100 and the robot device 200, each of which may correspond to a signal obtained at one of a plurality of time intervals. The plurality of time intervals may be periodic or non-periodic. Accordingly, at least one processor 140 may obtain a movement pattern of the robot device 200 based on the identified plurality of distances.

[0057] In addition, the communication interface 110 according to one or more embodiments may perform communication with at least one external electronic device (eg, the air purifier 100 - 2 , the display apparatus 100 - 3 ) provided at home.

[0058] For example, the communication interface 110 may receive a signal strength (e.g., an RSSI value) obtained by an external electronic device (at least one of the air purifier 100-2 or the display device 100-3), and receive a distance between the external electronic device and the robot device 200. In addition, the at least one processor 140 may obtain a movement pattern of the robot device 200 based on the distance between the electronic device 100 and the robot device 200 received through the communication interface 110 and the distance between the external electronic device and the robot device 200.

[0059] According to one or more embodiments, the sensor 120 may obtain sensing data by sensing the surrounding environment of the electronic device 100. According to an embodiment, the sensor 120 may include one or more sensors. Here, the sensing data may include the distance of the robot device 200 adjacent to the electronic device 100, the direction in which the robot device 200 is located, the distance of an object adjacent to the electronic device 100, and the characteristics of the object (e.g., geometric characteristics, photometric characteristics, etc.).

[0060] For example, the sensor 120 may include a camera, and the camera may obtain sensing data (e.g., image data) by capturing the surrounding environment of the robot device 200 according to the control of the at least one processor 140. The at least one processor 140 may recognize the distance between the electronic device 100 and the robot device 200 by analyzing the image data. The at least one processor 140 may obtain a movement pattern of the robot device 200 based on the recognized distance.

[0061] In addition, the communication interface 110 may receive sensing data (e.g., image data) obtained by an external electronic device from one or more external electronic devices (the air purifier 100-2 or the display device 100-3), and receive the distance between the external electronic device and the robot device 200. The at least one processor 140 may obtain a movement pattern of the robot device 200 based on the distance between the electronic device 100 and the robot device 200 received through the communication interface 110, and the distance between each of the one or more external electronic devices and the robot device 200.

[0062] According to the example, the electronic device 100 may: i) obtain a movement mode of the robot device 200 based on a distance between the electronic device and the robot device 200, or ii) when the distance between the external electronic device and the robot device 200 is received from an external electronic device, obtain the movement mode of the robot device 200 based on the distance between the electronic device 100 and the robot device 200 and the distance between the external electronic device and the robot device 200 received from the external electronic device.

[0063] In addition, the electronic device 100 may obtain the movement pattern of the robot device 200 based on signal strength, based on sensing data, or based on signal strength and sensing data. However, the present disclosure is not limited thereto, and therefore, according to another embodiment, the electronic device 100 may obtain the movement pattern by other methods.

[0064] The at least one processor 140 may identify the operation state of the robot device 200 as a normal operation or an abnormal operation according to the obtained movement pattern based on the data including a plurality of movement patterns corresponding to the normal operation of the robot device 200 stored in the memory 130 .

[0065] According to an example, the memory 130 may store data required by various embodiments of the present disclosure. The memory 130 may be implemented in a memory form embedded in the electronic device 100 according to data storage purposes, or in a memory form attachable to or detachable from the electronic device 100.

[0066] For example, operation data of the electronic device 100 may be stored in a memory embedded in the electronic device 100, and extended function data of the electronic device 100 may be stored in a memory that is attachable to or detachable from the electronic device 100. The memory embedded in the electronic device 100 may be implemented as at least one of the following: a volatile memory (e.g., a dynamic random access memory (DRAM), a static RAM (SRAM), or a synchronous dynamic RAM (SDRAM)) or a non-volatile memory (e.g., a one-time programmable read-only memory (OTPROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a mask ROM, a flash ROM, a flash memory (e.g., a NAND flash memory or a NOR flash memory), a hard disk drive (HDD), or a solid-state drive (SSD)). In addition, the memory that can be attached to or detached from the electronic device 100 can be implemented in the form of a memory card (e.g., Compact Flash (CF), Secure Digital (SD), Micro Secure Digital (micro-SD), Mini Secure Digital (mini-SD), Extreme Digital (xD), Multi Media Card (MMC)), etc.), an external memory that can be connected to a USB port (e.g., a USB memory), etc.

[0067] According to an example, the memory 130 may store at least one instruction or a computer program including instructions for controlling the electronic device 100 .

[0068] At least one processor 140 according to one or more embodiments may perform overall control operations of the electronic device 100 .

[0069] At least one processor 140 may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a majority integrated core (MIC), a digital signal processor (DSP), a neural processing unit (NPU), a hardware accelerator, or a machine learning accelerator. At least one processor 140 may control one or any combination of other elements of the electronic device and perform operations associated with communication or data processing. At least one processor 140 may execute at least one program or instruction stored in a memory. For example, at least one processor 140 may execute a method according to one or more embodiments by executing at least one instruction stored in a memory.

[0070] If the method according to one or more embodiments of the present disclosure includes multiple operations, the multiple operations may be performed by one processor, or by multiple processors. For example, when the method according to one or more embodiments performs the first operation, the second operation, and the third operation, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor), and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0071] At least one processor 140 may be implemented as a single-core processor including one core, or may be implemented as at least one multi-core processor including multiple cores (e.g., homogeneous multi-cores or heterogeneous multi-cores). If at least one processor 140 is implemented as a multi-core processor, the multiple cores included in the multi-core processor may respectively include processor internal memories, such as cache memories and on-chip memories, and a common cache shared by multiple cores may be included in the multi-core processor. In addition, each core (or a portion of the multiple cores) included in the multi-core processor may independently read and execute program instructions for implementing the method according to one or more embodiments of the present disclosure, or read and execute program instructions for implementing the method according to one or more embodiments of the present disclosure because all (or a portion) of the multiple cores are connected thereto.

[0072] If the method according to one or more embodiments includes multiple operations, the multiple operations may be performed by one of the multiple cores included in the multi-core processor, or by multiple cores. For example, when the first operation, the second operation, and the third operation are performed by the method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by the first core included in the multi-core processor, or the first operation and the second operation may be performed by the first core included in the multi-core processor, and the third operation may be performed by the second core included in the multi-core processor.

[0073] In an embodiment of the present disclosure, a processor may refer to a system on chip (SoC), a single-core processor, a multi-core processor, or a core integrated with at least one processor and other electronic components included in a single-core processor or a multi-core processor, and the core described herein may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator, machine learning accelerator, etc., but the embodiments of the present disclosure are not limited thereto.

[0074] Specifically, the at least one processor 140 may obtain a movement pattern, and identify the operation state of the robot device 200 as a normal operation or an abnormal operation according to the obtained movement pattern based on the data stored in the memory 130 .

[0075] Figure 3is a diagram illustrating an electronic device measuring signal strength according to one or more embodiments.

[0076] Reference Figure 3 , an example in which the at least one processor 140 obtains a movement pattern of the robotic device based on the signal strength will be described.

[0077] According to one or more embodiments, the air conditioner 100 - 1 in the home may perform communication with the robot device 200 , and identify a first distance between the air conditioner 100 - 1 and the robot device 200 based on a signal strength received from the robot device 200 .

[0078] For example, if the distance between the air conditioner 100-1 and the robot device 200 is 1 kilometer, the RSSI value may be distributed between -49 and -56 dBm, and if the distance between the air conditioner 100-1 and the robot device 200 is 2 meters, the RSSI value may be distributed between -59 and -67 dBm. The specific numerical values ​​are only examples and are not limited thereto. Accordingly, at least one processor 140 provided in the air conditioner 100-1 may identify the first distance between the air conditioner 100-1 and the robot device 200 based on the signal strength.

[0079] In addition, the air purifier 100-2 in the home can also identify the second distance between the air purifier 100-2 and the robot device 200 based on the signal strength received from the robot device 200, and the display device 100-3 can also identify the third distance between the display device 100-3 and the robot device 200 based on the signal strength received from the robot device 200.

[0080] Then, the air conditioner 100 - 1 may receive the second distance from the air purifier 100 - 2 and receive the third distance from the display device 100 - 3 .

[0081] For example, multiple electronic devices 100-1, 100-2, and 100-3 located at home can respectively perform communication with each other. In the example, the electronic device 100 can perform communication with an external electronic device (e.g., air purifier 100-2) through a network device, a server, etc. Here, the network device can be implemented as various types of devices that perform a bridging role, such as an access point (AP). For example, the network device can be implemented as a router, a switch, an IP router, and a wireless network card.

[0082] However, the present embodiment is not limited thereto, and the electronic device 100 may perform peer-to-peer (P2P) communication with an external electronic device. Here, the P2P communication may include Bluetooth, Wi-Fi P2P (or Wi-Fi Direct), and the like.

[0083] According to an embodiment, the air conditioner 100-1 may identify the position of the robot device 200 based on a first distance identified by the air conditioner 100-1, a second distance received from the air purifier 100-2, and a third distance received from the display device 100-3.

[0084] For example, at least one processor 140 provided in the air conditioner 100-1 may identify the position of the robot device 200 based on the first distance, the second distance, and the third distance according to the trilateration (true range multilateration) method.

[0085] For example, at least one processor 140 may identify the position of the robot device 200 based on the position of the air conditioner 100-1 (first reference point), the position of the air purifier 100-2 (second reference point), the position of the display device 100-3 (third reference point), and the first distance to the third distance according to the trilateration (true range multilateration) method.

[0086] At least one processor 140 provided in the air conditioner 100-1 may also identify the position of the robot device 200 based on the position of the air conditioner 100-1 (first reference point) and the first distance identified by the air conditioner 100-1.

[0087] In addition, in addition to being able to be based on the position of the electronic device 100 (first reference point) and the first distance identified by the electronic device 100, at least one processor 140 may also identify the position of the robot device 200 based on the position of an external electronic device (for example, the air purifier 100-2; second reference point) and the second distance identified by the external electronic device.

[0088] According to an embodiment, at least one processor 140 may identify a change in the position of the robot device 200 and obtain a movement pattern of the robot device 200 based on the identified change in position.

[0089] Figure 4 It is a diagram showing an electronic device identifying a change in position according to one or more embodiments.

[0090] <Time point t>

[0091] As Figure 3 shown, at least one processor 140 according to one or more embodiments may perform communication with the robot device 200 and identify a first distance 1 between the air conditioner 100-1 and the robot device 200 based on the signal strength received from the robot device 200.

[0092] In addition, the air purifier 100-2 at home can also identify the second distance 2 between the air purifier 100-2 and the robot device 200 based on the signal strength received from the robot device 200, and the display device 100-3 can also identify the third distance 3 between the display device 100-3 and the robot device 200 based on the signal strength received from the robot device 200.

[0093] According to an embodiment, at least one processor 140 can identify the position of the robot device 200 according to the trilateration method (true distance multilateration) based on the position of the air conditioner 100-1 (the first reference point), the position of the air purifier 100-2 (the second reference point), the position of the display device 100-3 (the third reference point), and the first distance 1, the second distance 2, and the third distance 3.

[0094] <Time point t+1>

[0095] According to one or more embodiments, at least one processor 140 can start communicating with the robot device 200' starting from the time point t+1 after a certain time has elapsed from the time point t, and identify the first distance 1' between the air conditioner 100-1 and the robot device 200' based on the signal strength received from the robot device 200'.

[0096] In addition, the air purifier 100-2 at home can also identify the second distance 2' between the air purifier 100-2 and the robot device 200' based on the signal strength received from the robot device 200', and the display device 100-3 can also identify the third distance 3' between the display device 100-3 and the robot device 200' based on the signal strength received from the robot device 200'.

[0097] According to an embodiment, at least one processor 140 can identify the position of the robot device 200 according to the trilateration method (true distance multilateration) based on the position of the air conditioner 100-1 (the first reference point), the position of the air purifier 100-2 (the second reference point), the position of the display device 100-3 (the third reference point), and the first distance 1', the second distance 2', and the third distance 3'.

[0098] Wherein, a certain time interval between the time point t and the time point t+1 can be a preset time interval, and various settings can be made for the interval of sending signals by the robot device 200, etc.

[0099] At least one processor 140 according to one or more embodiments may identify a change in the position of the robot device 200 based on the position of the robot device 200 identified at time point t and the position of the robot device 200' identified at time point t+1, and obtain a movement pattern of the robot device 200 according to the change in the position. The change in the position of the robot device 200 may include a change in the coordinates of the robot device 200 in a space (e.g., home) where the robot device 200 is located (e.g., 2D coordinate change, 3D coordinate change).

[0100] According to an embodiment, the at least one processor 140 may identify the operation state of the robot device 200 as a normal operation or an abnormal operation based on the movement pattern of the robot device 200 .

[0101] Figure 5 is a diagram showing an electronic device identifying a normal operation and an abnormal operation according to one or more embodiments, and Figure 6 is a diagram illustrating a method of obtaining a plurality of clusters according to one or more embodiments.

[0102] First, refer to Figure 6 , the at least one processor 140 may classify a plurality of movement patterns corresponding to the normal operation of the robot device 200 stored in the memory 130 and obtain a plurality of clusters. For example, the at least one processor 140 may group a plurality of movement patterns corresponding to the normal operation of the robot device 200 stored in the memory 130 and obtain a plurality of groups. Here, the plurality of movement patterns stored in the memory 130 may include a movement pattern based on a driving history when the robot device 200 is normally operated, a movement pattern set at a manufacturer of the robot device 200, a movement pattern set by a user of the robot device 200, and the like.

[0103] For example, the first cluster among the k clusters obtained by classifying the plurality of movement modes may include the first movement mode of the robot device 200, the second cluster may include the second movement mode of the robot device, and the third cluster may include the third movement mode (e.g., a rotation movement mode) of the robot device 200. The first movement mode may be a movement mode for moving near or along a stationary object such as a wall, furniture, household appliances, etc. The second movement mode may be a movement mode for avoiding a moving object (e.g., a person, a pet). However, the above is merely an example for ease of description and is not limited thereto.

[0104] The at least one processor 140 according to one or more embodiments may identify a cluster corresponding to the obtained movement pattern from among a plurality of clusters.

[0105] Reference Figure 5, the at least one processor 140 may identify the operation state according to the obtained movement pattern as a normal operation based on the cluster corresponding to the obtained movement pattern identified from among the plurality of clusters.

[0106] In another example, the at least one processor 140 may identify the operation state according to the obtained movement pattern as an abnormal operation based on a cluster corresponding to the obtained movement pattern not being identified from among the plurality of clusters.

[0107] For example, at least one processor 140 may compare the cluster corresponding to the mobile pattern 500 with one or more clusters in the cluster data 501, and determine whether the cluster corresponding to the mobile pattern 500 matches any one of the one or more clusters in the cluster data 501. If at least one processor 140 determines a match, at least one processor 140 may determine the operating state according to the obtained mobile pattern as a normal operation. However, if at least one processor 140 determines a mismatch, at least one processor 140 may determine the operating state according to the obtained mobile pattern as an abnormal operation.

[0108] For example, the at least one processor 140 may respectively output the obtained possibilities of the movement patterns being included in the plurality of clusters.

[0109] In an example, at least one processor 140 may identify an operating state according to the mobile pattern as normal operation based on a probability that the obtained mobile pattern is included in any one of the multiple clusters being greater than or equal to a threshold value (e.g., 0.5) because the cluster corresponding to the obtained mobile pattern is identified from the multiple clusters.

[0110] For example, at least one processor 140 may identify the operating state of the robot device 200 according to the obtained movement pattern as normal operation based on the probability that the obtained movement pattern is included in the first cluster is 0.7, the probability that the obtained movement pattern is included in the second cluster is 0.1, and the probability that the obtained movement pattern is included in the second cluster is 0.2, because the obtained movement pattern corresponds to the first cluster (or is included in the first cluster).

[0111] In an example, at least one processor 140 may identify an operation state according to an obtained movement pattern as an abnormal operation based on the possibility that the obtained movement patterns are respectively included in a plurality of clusters being less than a threshold value (e.g., 0.5) because a cluster corresponding to the obtained movement pattern is not identified from the plurality of clusters.

[0112] For example, at least one processor 140 may identify the operating state of the robot device 200 according to the obtained movement pattern as an abnormal operation based on the probability that the obtained movement pattern is included in the first cluster is 0.2, the probability that the obtained movement pattern is included in the second cluster is 0.1, and the probability that the obtained movement pattern is included in the third cluster is 0.3, because the obtained movement patterns do not correspond to multiple clusters.

[0113] For convenience of description, the possibility of the obtained motion pattern being included in any one of the three clusters (first to third clusters), the number of clusters, and the distribution of clusters are examples for convenience of description. However, the present invention is not limited to the above examples.

[0114] Figure 7 is a diagram illustrating an electronic device using a plurality of clusters received from an external electronic device according to one or more embodiments.

[0115] According to one or more embodiments, at least one processor 140 may receive a plurality of movement patterns from an external electronic device, and obtain a plurality of clusters corresponding to normal operation of the robot apparatus 200 by coordinating i) the plurality of movement patterns stored in the memory 130 and ii) the plurality of movement patterns received from the external electronic device.

[0116] For example, at least one processor 140 may coordinate (or organize) i) a plurality of mobility patterns obtained by the electronic device 100 and ii) a plurality of mobility patterns obtained by an external electronic device and sent by the external electronic device to the electronic device 100, and obtain a plurality of clusters corresponding to normal operations by classifying the coordinated plurality of mobility patterns.

[0117] In an example, the at least one processor 140 may classify the coordinated multiple movement patterns for each movement pattern. For example, the at least one processor 140 may classify the coordinated multiple movement patterns into multiple clusters using K-means clustering (or K-means algorithm).

[0118] According to an embodiment, at least one processor 140 may identify cohesion C of corresponding clusters, and identify silhouette scores S between a plurality of clusters.

[0119] According to an embodiment, the at least one processor 140 may identify, based on the cohesion C and the silhouette score S, whether the corresponding cluster corresponds to a movement pattern when the robot device 200 normally operates.

[0120] Here, the number of clusters k can be proportional to the number n of multiple electronic devices 100-1, 100-2 and 100-3, ..., 100-n located in the home, and can be inversely proportional to the number p of electronic devices (e.g., air purifier 100-2, display device 100-3, etc.) located in the same space as the electronic device (e.g., air conditioner 100-1) among the multiple electronic devices 100-1, 100-2 and 100-3, ..., 100-n located in the home.

[0121] In an example, when the at least one processor 140 classifies the coordinated plurality of movement patterns into a plurality of clusters, the number k of optimal clusters may be expressed by Equation 1 below.

[0122] [Equation 1]

[0123] k=N / P×α

[0124] Here, α represents a correction factor.

[0125] In an example, an optimal clustering level indicating whether a plurality of clusters obtained by classifying a plurality of coordinated movement patterns are appropriately classified for each movement pattern may be expressed by Equation 2 below.

[0126] [Equation 2]

[0127] I = C × S × β

[0128] Here, C represents cohesion, S represents silhouette score, and β represents a correction factor.

[0129] Accordingly, the greater the cohesion, the higher the silhouette score, and the higher the optimal clustering level can be. In the example, based on k and I being respectively greater than or equal to the threshold, the at least one processor 140 can recognize that the plurality of clusters obtained by classifying the coordinated plurality of movement patterns have been properly classified for each movement pattern, and that a plurality of clusters corresponding to normal operation have been obtained. Here, each time a movement pattern based on the driving history of the robot device 200 is obtained, the threshold can be updated.

[0130] According to an embodiment, at least one processor 140 may identify the operating state according to the obtained mobile pattern as normal operation based on identifying a cluster corresponding to the obtained mobile pattern from among multiple clusters corresponding to normal operation, and identify the operating state according to the obtained mobile pattern as abnormal operation based on not identifying the cluster corresponding to the obtained mobile pattern from among multiple clusters corresponding to normal operation.

[0131] Figure 8 is a diagram illustrating a neural network model according to one or more embodiments.

[0132] Reference Figure 8, the at least one processor 140 may identify the operation state of the robot device 200 as a normal operation or an abnormal operation by inputting the obtained movement pattern into the neural network model.

[0133] In an example, the neural network model may be a trained model that identifies the operating state of the robot device 200 as normal operation or abnormal operation when the obtained movement pattern is input based on a plurality of movement patterns corresponding to normal operation stored in the memory 130.

[0134] In an example, the neural network model may be a trained model that is coordinated based on i) a plurality of movement patterns stored in a memory 130 of the electronic device 100, and ii) a plurality of movement patterns obtained by an external electronic device and a plurality of movement patterns sent by the external electronic device to the electronic device 100, and identifies the operating state of the robot device 200 as normal operation or abnormal operation when the obtained movement pattern is input.

[0135] In an example, the neural network model may be a trained model that identifies the operating state of the robot device 200 as normal operation or abnormal operation when the obtained movement pattern is input based on k according to Equation 1 and I according to Equation 2 being greater than or equal to a threshold value, respectively, and based on a movement pattern included in a corresponding cluster corresponding to normal operation.

[0136] Functions associated with artificial intelligence according to the present disclosure will be operated by at least one processor 140 and memory 130 of the electronic device 100 .

[0137] The at least one processor 140 may be formed of one or more processors. At this time, the one or more processors may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), and a neural processing unit (NPU), but are not limited to the examples of the above processors.

[0138] The CPU can be used as a general-purpose processor to effectively execute complex programs through a multi-layer cache structure, and the general-purpose processor can perform not only general operations but also artificial intelligence operations. The CPU can have an advantage in a serial processing method that allows the previous calculation results and the subsequent calculation results to be organically connected through sequential calculations. The general-purpose processor is not limited to the above examples unless specified as the above-mentioned CPU.

[0139] The GPU can be used as a processor for a large number of operations (such as floating-point operations, etc.) used in graphics processing, and a large number of operations can be performed in parallel by integrating a large number of cores. Specifically, compared with the CPU, the GPU can have advantages in parallel processing methods such as convolution operations. In addition, the GPU can be used as a coprocessor to supplement the functions of the CPU. The large-capacity operation processor is not limited to the above examples except for the case specified as the above-mentioned GPU.

[0140] NPU can be a dedicated processor that uses an artificial neural network to perform artificial intelligence operations, and the layers that make up the artificial neural network can be implemented as hardware (such as silicon). At this time, since the NPU is specially designed according to the specifications required by the company, the degree of freedom may be lower than that of the CPU or GPU, but it can effectively handle the artificial intelligence operations required by the company. As a processor dedicated to artificial intelligence operations, the NPU can be implemented in various forms, such as but not limited to a tensor processing unit (TPU), an intelligent processing unit (IPU), a visual processing unit (VPU), etc. The artificial intelligence processor is not limited to the above examples except for the case where it is specified as the above-mentioned NPU.

[0141] In addition, one or more processors may be implemented as a system on chip (SoC). In this case, the SoC may include, in addition to the one or more processors, a memory and a network interface such as a bus for data communication between the processor and the memory.

[0142] If a plurality of processors are included in the SoC included in the electronic device 100, the electronic device 100 may use a portion of the plurality of processors to perform operations associated with artificial intelligence (e.g., operations associated with learning or reasoning of an artificial intelligence model). For example, the electronic device 100 may use at least one of the GPU, NPU, VPU, TPU, and hardware accelerators specifically used for artificial intelligence operations such as convolution operations and matrix multiplication operations among the plurality of processors to perform operations associated with artificial intelligence. However, the above is only one embodiment, and operations associated with artificial intelligence may be processed using a general-purpose processor (such as a CPU).

[0143] In addition, the electronic device 100 can perform operations of functions associated with artificial intelligence by using multiple cores (e.g., dual cores, quad cores, etc.) included in one processor. Specifically, the electronic device 100 can perform artificial intelligence operations such as convolution operations and matrix multiplication operations in parallel by using multiple cores included in at least one processor 140.

[0144] One or more processors may control the processing of input data according to predefined operating rules or artificial intelligence models stored in the memory 130. The features of the predefined operating rules or artificial intelligence models may be created through learning.

[0145] As used herein, creation by learning refers to creating predefined operating rules or artificial intelligence models of desired features by applying a learning algorithm to a plurality of learning data. Learning can be performed in the machine itself that executes artificial intelligence according to the present disclosure, or by a separate server and / or system.

[0146] The artificial intelligence model may be composed of multiple neural network layers. At least one layer may include at least one weight value, and the operation of one layer is performed by the operation result of the previous layer and at least one predefined operation. Examples of neural networks may include convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), deep Q networks, converters, etc., and unless otherwise specified, the neural networks of the present disclosure are not limited to the above examples.

[0147] A learning algorithm may be a method for training a predetermined target machine (e.g., a robot) to make decisions or predictions on its own using a plurality of learning data. Examples of learning algorithms may include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, and unless otherwise specified, the learning algorithms of the present disclosure are not limited to the above examples.

[0148] Fig. 9 is a diagram illustrating normal operation and abnormal operation according to one or more embodiments.

[0149] In the normal operation of the robot device 200 according to various embodiments of the present disclosure, the robot device 200 may move according to a driving algorithm based on sensing data of sensors provided in the robot device 200, a user command for the robot device 200, etc. That is, based on the movement mode of the robot device 200 corresponding to the driving algorithm of the robot device 200, the operation state of the robot device 200 may be a normal operation.

[0150] In another example, the operation state of the robot device 200 may be an abnormal operation based on the movement pattern of the robot device 200 not corresponding to the driving algorithm of the robot device 200. For example, if the robot device 200 moves according to abnormal access to the robot device 200 (e.g., a network attack, a hacker attack, etc. on the robot device 200) instead of the driving algorithm, the operation state of the robot device may be an abnormal operation because the movement pattern of the robot device 200 does not correspond to the driving algorithm of the robot device 200.

[0151] like Fig. 9As shown, when the movement mode of the robot device 200 in normal operation is wall-traveling as shown by arrow 10, it can be considered that the robot device 200 performing straight-line travel as shown by arrow 20 is operating under abnormal operation.

[0152] According to an example embodiment, in a case where the electronic device 100 is unable to identify whether abnormal access to the robot device 200 has occurred, the driving algorithm of the robot device 200, the sensing data of the robot device 200, the user commands directed to the robot device 200, etc. can identify whether the operating state of the robot device 200 is normal operation or abnormal operation based on the movement pattern obtained based on the signal strength, sensing data, etc.

[0153] Fig.10 is a diagram illustrating an electronic device including a camera according to one or more embodiments.

[0154] Reference Fig.10 , the sensor 120 of the electronic device 100 may include a camera.

[0155] According to one or more embodiments, the at least one processor 140 may obtain a movement pattern of the robot device 200 based on an image (e.g., image data) received from a camera. For example, the at least one processor 140 may obtain a movement pattern of the robot device 200 based on a position of the robot device 200 according to an image received at time point t and a position of the robot device 200′ according to an image received at time point t+1.

[0156] According to an embodiment, when a movement pattern is input to a neural network model, at least one processor 140 may identify normal operation or abnormal operation according to whether a cluster corresponding to the obtained movement pattern is identified from among a plurality of clusters, or an output of the neural network model.

[0157] Fig.11 is a diagram illustrating that an electronic device uses a camera to recognize an operation state of the electronic device according to one or more embodiments.

[0158] Reference Fig.11 The robot device 200 may include a display 210 for providing an operating status of the robot device 200 .

[0159] For example, when the operation state of the robot device 200 is abnormal operation (for example, the robot device 200 cannot be driven, the battery is insufficient, a fault occurs, etc.), the display 210 may display a first color (for example, red). In another example, when the operation state of the robot device 200 is normal operation, the display 210 may display a second color (for example, green).

[0160] Here, the display 210 may be implemented as an LED or the like (eg, an operation status indicator light).

[0161] According to an example, the at least one processor 140 may identify the operation state as an abnormal operation based on that a display provided in the robot device 200 is identified as displaying a first color based on an image.

[0162] In another example, the at least one processor 140 may identify the operation state as the normal operation based on that a display provided in the robot device 200 is identified as displaying the second color based on the image.

[0163] Fig.12 is a diagram illustrating an operation state in which an electronic device recognizes a robot apparatus avoiding an object according to one or more embodiments.

[0164] According to one or more embodiments, the at least one processor 140 may recognize an object included in an image received from a camera.

[0165] Here, the object may include various forms of obstacles associated with the movement pattern of the robot device 200, such as walls, furniture, home appliances, humans, pets, and the like.

[0166] According to an embodiment, at least one processor 140 may identify a movement pattern corresponding to an object. For example, when a wall, furniture, household appliances, etc. are identified based on a plurality of clusters, at least one processor 140 may identify a movement pattern along the object, and when a person, pet, book, small appliance, toy, clothing, etc. are identified, at least one processor 140 may identify a movement pattern avoiding the object.

[0167] According to an embodiment, the at least one processor 140 may identify the operation state as an abnormal operation based on that the obtained movement pattern is different from the movement pattern corresponding to the object.

[0168] Return to reference Figure 2 , the sensor 120 according to one or more embodiments may include a microphone.

[0169] According to one or more embodiments, at least one processor 140 may determine a first distance between the air conditioner 100-1 and the robot device 200 based on a magnitude of a sound generated according to an operation of the robot device 200 received through a microphone (or, a sound output by the robot device 200: e.g., sensing data).

[0170] In addition, the air purifier 100-2 in the home can determine the second distance between the air purifier 100-2 and the robot device 200 based on the amplitude of the sound received from the robot device 200, and the display device 100-3 can also determine the third distance between the display device 100-3 and the robot device 200 based on the amplitude of the sound received from the robot device 200.

[0171] According to an embodiment, the air conditioner 100-1 may receive the second distance from the air purifier 100-2, and may receive the third distance from the display device 100-3. Then, the air conditioner 100-1 may determine the position of the robot device 200 based on the first distance determined by the air conditioner 100-1, the second distance received from the air purifier 100-2, and the third distance received from the display device 100-3.

[0172] According to an embodiment, at least one processor 140 can identify the position of the robot device 200 at time point t according to trilateration (true distance multilateral measurement) based on the position of the air conditioner 100-1 (first reference point), the position of the air purifier 100-2 (second reference point), the position of the display device 100-3 (third reference point), and the first distance 1, the second distance 2 and the third distance 3.

[0173] According to an embodiment, the at least one processor 140 may identify the location of the robotic device 200 ′ at time t+1.

[0174] According to an embodiment, the at least one processor 140 may obtain a movement pattern of the robot device 200 based on a position of the robot device 200 at time t and a position of the robot device 200 ′ at time t+1.

[0175] Fig.13 is a diagram illustrating a user terminal device providing a notification according to one or more embodiments.

[0176] Reference Fig.13 , the electronic apparatus 100 according to one or more embodiments may provide a notification to the user terminal device based on the operation state of the robot device 200 being identified as an abnormal operation.

[0177] In addition, the electronic device 100 may provide at least one of sound feedback or visual feedback based on the operation state of the robot device 200 being identified as an abnormal operation. For example, the electronic device 100 may output a sound indicating that the robot device 200 is in an abnormal operation through a speaker (not shown), or output a screen indicating that the robot device 200 is in an abnormal operation through a display (not shown).

[0178] Fig.14is a flowchart illustrating a control method of an electronic device according to one or more embodiments. The electronic device may include a memory storing data including a plurality of movement patterns corresponding to normal operations of a robot apparatus.

[0179] According to an embodiment, the control method may include obtaining a movement pattern of the robot device based on a signal strength received from the robot device or sensing data from a sensor in operation S1410. However, the present disclosure is not limited thereto, and thus, the movement pattern of the robot device may be obtained in a different manner.

[0180] According to an embodiment, the method may include identifying an operation state of the robot apparatus as a normal operation or an abnormal operation based on the obtained movement pattern and data including a plurality of movement patterns corresponding to normal operations of the robot apparatus in operation S1420.

[0181] Among them, the control method may also include: classifying multiple mobile patterns to obtain multiple clusters, and the identification operation S1420 may include: based on identifying a cluster corresponding to the obtained mobile pattern from multiple clusters, identifying the operation state according to the obtained mobile pattern as normal operation; and based on not identifying a cluster corresponding to the obtained mobile pattern from multiple clusters, identifying the operation state according to the obtained mobile pattern as abnormal operation.

[0182] Here, obtaining multiple clusters may include: based on receiving multiple mobile patterns from an external electronic device, obtaining multiple clusters corresponding to normal operation by coordinating the received multiple mobile patterns and the multiple mobile patterns included in the electronic device, and the identification operation S1420 may include: based on identifying the cluster corresponding to the obtained mobile pattern from the multiple clusters corresponding to normal operation, identifying the operation state according to the obtained mobile pattern as normal operation; and based on not identifying the cluster corresponding to the obtained mobile pattern from the multiple clusters corresponding to normal operation, identifying the operation state according to the obtained mobile pattern as abnormal operation.

[0183] According to the example of the present disclosure, the recognition operation S1420 may include: identifying the operation state as normal operation or abnormal operation by inputting the obtained movement pattern into a trained neural network model, and the trained neural network model may be the following model: the model is trained to identify the operation state according to the obtained movement pattern as normal operation or abnormal operation based on multiple movement patterns corresponding to normal operation included in the data.

[0184] The obtaining operation S1410 according to the example of the present disclosure may include: identifying a change in a first distance between an electronic device and the robot device based on signal strength, identifying a change in a position of the robot device based on a change in the first distance and a change in a second distance between the external electronic device and the robot device received from an external electronic device, and obtaining a movement mode according to the change in the position, and the change in the position of the robot device may include a change in the coordinates of the robot device within the space where the robot device is located.

[0185] The obtaining operation S1410 according to the example of the present disclosure may include: identifying a change in a first distance between an electronic device and a robot device based on first sensing data received from a sensor, identifying a change in a second distance between an external electronic device and the robot device based on second sensing data received from an external electronic device, identifying a change in a position of the robot device based on a change in the first distance and a change in the second distance, and obtaining a movement pattern according to the change in the position.

[0186] Here, the second sensing data may include a microphone, the first sensing data may include the amplitude of the sound output by the robot apparatus received through the microphone, and the second sensing data may include the amplitude of the sound output by the robot apparatus received through the microphone provided in the external electronic device.

[0187] The sensor according to an example may include a camera, and the obtaining operation S1410 may include obtaining a movement pattern of the robot device based on an image received from the camera.

[0188] According to the example, obtaining operation S1410 may include: identifying an object included in an image and identifying a movement pattern corresponding to the object, and identifying operation S1420 may include: identifying the operation state as an abnormal operation based on that the movement pattern of the robot device is different from the movement pattern corresponding to the object, wherein the movement pattern corresponding to the object may include a movement pattern of the robot device avoiding the object.

[0189] However, various embodiments of the present disclosure may be applied not only to electronic devices but also to various types of home appliances.

[0190] The various embodiments described above can be implemented in a recordable medium readable by a computer or a computer-like device using software, hardware, or a combination of software and hardware. In some cases, the embodiments described herein can be implemented separately with a processor. According to software implementation, embodiments such as the processes and functions described herein can be implemented with separate software modules. Each software module can perform at least one function and operation described herein.

[0191] The computer instructions for executing the processing operations in the electronic device of the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in the non-transitory computer-readable medium may cause the specific device to execute the processing operations in the electronic device 100 of the various embodiments described above when the computer instructions are executed by the processor of the specific device.

[0192] A non-transitory computer-readable medium may refer to a medium (e.g., register, cache, memory, etc.) that stores data semi-permanently rather than very briefly, and can be read by a device. Specific examples of non-transitory computer-readable media may include, for example but not limited to, a compact disc (CD), a digital versatile disc (DVD), a hard disk, a Blu-ray disc, a USB, a memory card, a ROM, etc.

[0193] Although one or more exemplary embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and those skilled in the art will understand that various changes in form and details may be made therein without departing from the true spirit and full scope of the present disclosure, including the appended claims and their equivalents.

Claims

1. An electronic device, comprising: Communication interface; sensor; a memory storing data including a plurality of stored movement patterns corresponding to normal operation of the robotic device; as well as at least one processor electrically coupled to the memory and configured to: obtaining a movement pattern of the robotic device based on at least one of a signal strength received from the robotic device through the communication interface or sensing data from the sensor, and Based on a comparison between the obtained movement pattern and the plurality of stored movement patterns, an operation state of the robot device is identified as the normal operation or the abnormal operation.

2. The electronic device according to claim 1, wherein The at least one processor is further configured to: obtaining a plurality of clusters by classifying the plurality of stored movement patterns, identifying the operation state of the robot device as the normal operation based on a match between a cluster corresponding to the obtained movement pattern and one of the plurality of clusters, and The operation state of the robot device is identified as the abnormal operation based on a mismatch between the cluster corresponding to the obtained movement pattern and one of the plurality of clusters.

3. The electronic device according to claim 2, wherein The at least one processor is configured to: Based on receiving a plurality of received movement patterns from an external electronic device through the communication interface, a plurality of clusters corresponding to a normal operation are obtained by coordinating the plurality of received movement patterns with the plurality of stored movement patterns stored in the memory.

4. The electronic device according to claim 1, in, The at least one processor is configured to: By inputting the obtained movement pattern into the trained neural network model, the operation state is identified as the normal operation or the abnormal operation, and Among them, the trained neural network model is the following model: the model is trained to identify the operating state according to the obtained movement pattern as the normal operation or the abnormal operation based on the multiple stored movement patterns corresponding to the normal operation in the data.

5. The electronic device according to claim 1, in, The at least one processor is configured to: obtaining a plurality of first distances between the electronic device and the robotic apparatus based on the signal strengths measured at a plurality of intervals, obtaining a plurality of second distances between the external electronic device and the robotic apparatus from the external electronic device through the communication interface, and obtaining the movement pattern according to the plurality of first distances or the plurality of second distances, and The multiple first distances or the multiple second distances indicate coordinate changes of the robot device in the space where the robot device is located.

6. The electronic device according to claim 1, wherein The at least one processor is configured to: identifying a change in a first distance between the electronic device and the robotic apparatus based on first sensing data received from the sensor, identifying a change in a second distance between the external electronic device and the robotic apparatus based on second sensing data received from the external electronic device through the communication interface, identifying a change in the position of the robotic device based on a change in the first distance and a change in the second distance, and The movement pattern is obtained according to the change of the position.

7. The electronic device according to claim 6, in, The sensor includes a microphone, wherein the first sensing data comprises the amplitude of the sound output by the robot device and received by the microphone, and The second sensing data includes the amplitude of the sound output by the robot device received by other microphones arranged in the external electronic device.

8. The electronic device according to claim 1, in, The sensor includes a camera, and Wherein, the at least one processor is configured to obtain the movement pattern of the robotic device based on the image received from the camera.

9. The electronic device according to claim 8, in, The at least one processor is configured to: identifying an object from the image, identifying a movement pattern corresponding to the object, and identifying the operation state as the abnormal operation based on the movement pattern of the robot device being different from the movement pattern corresponding to the object, and The movement mode corresponding to the object includes a movement mode in which the robot device avoids the object.

10. The electronic device according to claim 8, wherein The at least one processor is configured to identify the operation state as the abnormal operation based on a display in the robot device being identified as displaying a preset color based on the image.

11. The electronic device according to claim 1, wherein The at least one processor is configured to provide a notification indicating the operating state of the robot device to a user terminal device through the communication interface based on the operating state being identified as the abnormal operation.

12. A control method of an electronic device, the electronic device comprising a memory storing data including a plurality of stored movement patterns corresponding to normal operation of a robotic device, the method comprising: obtaining a movement pattern of the robotic device based on at least one of a signal strength or sensing data of a sensor received from the robotic device; as well as Based on a comparison between the obtained movement pattern and the plurality of stored movement patterns, an operation state of the robot device is identified as a normal operation or an abnormal operation.

13. The control method according to claim 12, further comprising: A plurality of clusters are obtained by classifying the plurality of stored movement patterns, and Wherein, the identification includes: identifying the operation state of the robot device as the normal operation based on a match between a cluster corresponding to the obtained movement pattern and one of the plurality of clusters, and The operation state of the robot device is identified as the abnormal operation based on a mismatch between the cluster corresponding to the obtained movement pattern and one of the plurality of clusters.

14. The control method according to claim 13, wherein: The obtaining of multiple clusters comprises: Based on receiving a plurality of received movement patterns from an external electronic device, a plurality of clusters corresponding to a normal operation are obtained by coordinating the plurality of received movement patterns with a plurality of stored movement patterns included in the electronic device.

15. The control method according to claim 12, wherein: The identification includes: By inputting the obtained movement pattern into the trained neural network model, the operation state is identified as the normal operation or the abnormal operation, and Among them, the trained neural network model is the following model: the model is trained to identify the operating state according to the obtained movement pattern as the normal operation or the abnormal operation based on the multiple stored movement patterns corresponding to the normal operation included in the data.

Images