Electronic device and control method thereof

Abstract

An electronic device is disclosed. The electronic device includes: a camera; a memory configured to store attribute information and environment information; and a processor configured to recognize multiple objects based on an image obtained by the camera, recognize a first context of a first object from among the multiple objects based on a relationship between the attribute information of the multiple objects and the environment information, and control a travel state of the electronic device based on the first context.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly to an electronic device capable of autonomously traveling and a control method thereof. Background Art

[0002] As electronic technology continues to develop, new electronic devices (such as robots) have been developed and popularized, especially for use in various places such as homes, offices, public places, etc.

[0003] However, these devices may not be able to recognize the relationship between objects. If a person sits on a chair, the chair may be moved by the person separately from the person's movement. In this case, the movement of the chair and the collision between the chair and the electronic device may not be predicted. Summary of the invention

[0004] [Technical issues]

[0005] As such, a current situation may not be accurately analyzed or a future situation may not be accurately predicted because the relationship between a plurality of objects is not considered.

[0006] [Solution to the problem]

[0007] The embodiments of the present disclosure overcome the above disadvantages and other disadvantages not described above. In addition, the present disclosure is not required to overcome the above disadvantages, and the embodiments of the present disclosure may not overcome any of the above problems.

[0008] One or more embodiments provide an electronic device and a control method thereof for determining a control operation in consideration of a relationship between attribute information of a plurality of objects.

[0009] According to an embodiment of the present disclosure, an electronic device includes: a camera; a memory configured to store attribute information and environmental information; and a processor configured to identify multiple objects based on an image obtained by the camera, identify a first context of a first object from among the multiple objects based on the relationship between the attribute information of the multiple objects and the environmental information, and control a travel state of the electronic device based on the first context.

[0010] The processor may also be configured to identify a first danger level of the first object based on the first context and control a traveling state of the electronic device based on the first danger level.

[0011] The processor may also be configured to identify a first predicted context of the first object based on the relationship and the environmental information, and control a travel state of the electronic device based on the first predicted context.

[0012] One of the multiple objects may be a static object with static properties, another of the multiple objects may be a dynamic object with dynamic properties, and the processor may also be configured to predict at least one of a moving direction or a moving distance of at least one of the multiple objects based on a relationship between the static object and the dynamic object and environmental information, and control the travel state of the electronic device based on at least one of the predicted moving direction or the predicted moving distance.

[0013] The processor may be further configured to identify a context in which the static object may be moved by the dynamic object based on the dynamic object being identified as being within a threshold distance of the static object.

[0014] The electronic device may be configured to operate in a protection mode, a low noise mode, a monitoring mode, and a private mode, and the processor may be further configured to identify a current mode of the electronic device from among the protection mode, the low noise mode, the monitoring mode, and the private mode based on the first context.

[0015] The processor can also be configured to control the current mode of the electronic device to be at least one of a private mode or a low noise mode based on the private space information being obtained from the first object, the second object from the multiple objects being a dynamic object, and the relationship between the multiple objects and the private space information.

[0016] The processor may also be configured to control the current mode of the electronic device to be at least one of a monitoring mode and a protection mode based on the dangerous object from among the multiple objects being identified as having a dangerous attribute and the relationship between the multiple objects.

[0017] Each of the multiple objects may correspond to multiple users respectively, the attribute information may include profile information of each of the multiple users, and the processor may also be configured to identify the context of the multiple users based on the relationship between the multiple users and environmental information, and control the travel state of the electronic device based on the context.

[0018] At least one of the multiple objects may be a user, the attributes of each of the multiple objects as a user may include corresponding profile information of the user, and the processor may also be configured to identify the context of the user based on the relationship between the user and other objects in the multiple objects, environmental information and the user's behavior history, and control the travel state of the electronic device based on the context of the user.

[0019] The environmental information may include at least one of spatial information, time information, or position information related to the current space.

[0020] The processor may be further configured to identify a first context of the first object based on a relationship between attribute information of each of the plurality of objects and the environmental information.

[0021] According to an embodiment of the present disclosure, a control method for an electronic device storing attribute information and environmental information of an object is provided. The control method includes: recognizing multiple objects based on an image; recognizing a first context of a first object from among the multiple objects based on the relationship between the attribute information of the multiple objects and the environmental information; and controlling the travel state of the electronic device based on the first context.

[0022] The control method may further include identifying a first danger level of the first object based on the first context, and controlling the travel state of the electronic device may be based on the first danger level.

[0023] Identifying the first context may include identifying a first predicted context of the first object based on the relationship and the environmental information, and controlling the travel state of the electronic device may be based on the first predicted context.

[0024] One of the multiple objects may be a static object with static properties, another of the multiple objects may be a dynamic object with dynamic properties, the control method may further include identifying at least one of a moving direction or a moving distance of at least one of the multiple objects based on a relationship between the static object and the dynamic object and environmental information, and the travel state of the control electronic device may be based on at least one of a predicted moving direction or a predicted moving distance.

[0025] The control method may further include, based on the dynamic object being identified as being within a threshold distance of the static object, identifying a context in which the static object may be moved by the dynamic object.

[0026] Controlling the travel state of the electronic device may include identifying a current mode of the electronic device based on the first context, and the current mode may be selected from a protection mode, a low noise mode, a monitoring mode, or a privacy mode.

[0027] Identifying the first context may include identifying whether private space information is obtained from the first object and whether a second object from among multiple objects is a dynamic object, and controlling the travel state of the electronic device may include selecting a current mode as one of a private mode or a low noise mode based on the first context.

[0028] According to an embodiment of the present disclosure, a non-temporary computer-readable medium stores computer instructions, which, when executed by a processor of an electronic device storing attribute information and environmental information of an object, causes the electronic device to perform operations, the operations including: identifying multiple objects based on an image; identifying a first context of a first object from among multiple objects based on the relationship between the attribute information of the multiple objects and the environmental information; and controlling the travel state of the electronic device based on the first context.

[0029] According to an embodiment of the present disclosure, an electronic device includes: a camera; a motor; and a processor, configured to: identify a first path traveled by the electronic device; control the motor to move the electronic device along the first path; control the camera to capture an image; identify a first object and a second object in the image; identify whether the first path conflicts with a first position of the first object or a second position of the second object; based on the first path conflicting with at least one of the first position or the second position, identify a second path traveled by the electronic device; and control the motor to move the electronic device along the second path.

[0030] The processor can also be configured to: identify the relationship between the first object and the second object based on a first attribute of the first object and a second attribute of the second object; identify the context between the first object and the second object based on the relationship between the first object and the second object and environmental information; identify a third path for the electronic device to travel based on the context; and control the motor to move the electronic device along the third path.

[0031] The processor can also be configured to: identify the first object as a child and the second object as a bed; identify the context based on whether the third object identified as a protector is identified; based on the third object not being identified, identify the context as requiring protection, control the motor to approach the first object and the second object, and control the electronic device to monitor the child.

[0032] The processor may also be configured to recognize the first object and the second object by inputting the image into the neural network.

[0033] The electronic device may further include a microphone, and the processor may be further configured to recognize a relationship between the first object and the second object based on a voice corresponding to audio captured by the microphone.

[0034] Additional and / or other aspects and advantages of the present disclosure will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and / or other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0036] Figure 1 is a block diagram showing an electronic device according to an embodiment;

[0037] Figure 2 Is used to describe Figure 1 A block diagram of a specific configuration of an electronic device;

[0038] Figure 3 is a view for describing a route along which an electronic device moves in an indoor space according to an embodiment;

[0039] Figure 4 is used to describe Figure 3 a view of a context recognized by an electronic device at a specific location in the image;

[0040] Figure 5 is a view for describing an operation of changing a travel state of an electronic device on a specific moving route;

[0041] Figure 6 is a diagram for describing an embodiment in which an electronic device changes a moving route;

[0042] Figure 7 is a view for describing an operation of an electronic device monitoring an object at a specific position on a moving route;

[0043] Figure 8 is a view for describing an embodiment of changing the property of an object and changing an existing moving route;

[0044] Fig. 9 is a view for describing an embodiment in which an existing moving route is changed depending on a property of an object;

[0045] Fig.10 is a view for describing an operation of changing a travel state of an electronic device on a specific moving route;

[0046] Fig.11 is a view for describing an operation of an electronic device storing information about a member;

[0047] Fig.12 is a view for describing an embodiment in which an electronic device recognizes the presence or absence of a member and performs a control operation corresponding to the recognition result;

[0048] Fig.13 is a view for describing another embodiment in which an electronic device recognizes the presence or absence of a member and performs a control operation corresponding to the recognition result;

[0049] Fig.14 is a view for describing an operation of an electronic device to set a movement route based on information on movement of a member;

[0050] Fig.15 is a view for describing an operation of an electronic device setting a moving route to avoid collision with a member;

[0051] Fig.16 is a diagram for describing a plurality of operations performed by an electronic device to identify a travel state corresponding to a context;

[0052] Fig.17is a diagram for describing an operation of an electronic device to recognize a context based on information of an object or environmental information;

[0053] Fig.18 is a diagram for describing an operation of an electronic device analyzing and updating a context;

[0054] Fig.19 is a flowchart for describing an operation of an electronic device identifying a traveling state corresponding to a context; and

[0055] Fig. 20 is a flowchart for describing a control method of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0056] The terms used in the specification and claims have been selected in consideration of the functions in various embodiments. However, these terms may change with the intentions of engineers working in the corresponding fields, legal or technical interpretations, the emergence of new technologies, etc. In addition, some terms may be terms arbitrarily selected by the applicant. These terms may be interpreted as having the meanings defined in the specification, and in the absence of specific definitions of the terms, these terms may be interpreted based on the general content of the specification and the common technical knowledge in the corresponding technical field.

[0057] In addition, throughout the drawings in the specification, similar reference numerals or symbols represent parts or components that perform substantially the same function. For ease of description and understanding, the same reference numerals or symbols will be used in different embodiments. That is, even if components with the same reference numerals are shown in all of the multiple drawings, it does not mean that the multiple drawings show the same embodiment.

[0058] In addition, in the specification and claims, terms including ordinal numbers such as "first", "second", etc. can use corresponding components regardless of importance or order, and are used to distinguish components from each other without limiting the components. As an example, in the case where a component is represented by a term in combination with an ordinal number, the use or setting order of the component should not be limited by the ordinal number. If necessary, the ordinal numbers can be replaced with each other.

[0059] In the specification, the singular form is intended to include the plural form, unless the context clearly indicates otherwise. It should be further understood that the terms "comprising" or "formed by" used in this specification specify the presence of the features, numbers, steps, operations, components, parts or combinations thereof mentioned in this specification, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0060] Because various modifications can be made to the present disclosure, specific embodiments will be shown in the drawings and described in detail in the detailed description. However, it should be understood that the present disclosure is not limited to specific embodiments, but includes all modifications, equivalents and substitutions without departing from the scope and spirit of the present disclosure.

[0061] The expression “at least one of a, b, and c” should be understood to include: only a; only b; only c; both a and b; both a and c; both b and c; or all of a, b, and c.

[0062] In the embodiments, a "module", "unit" or "-operator / -device" refers to a component that performs at least one function or operation, and can be implemented by hardware or software or by a combination of hardware and software. In addition, in addition to the "module", "unit" or "-operator / -device" that needs to be implemented by specific hardware, multiple "modules", multiple "units" or multiple "-operators / -devices" can be integrated in at least one module or chip and implemented by at least one processor.

[0063] In the embodiment, the connection between one part and another part includes not only direct connection but also indirect connection via a medium. In addition, unless otherwise explicitly stated, "comprising" any component will be understood to imply the inclusion of other components but not the exclusion of any other components.

[0064] In the present disclosure, calculation (recognition) may be performed by a machine learning-based recognition system, and a deep learning-based recognition system is described as an example of a classification system that uses a series of machine learning algorithms based on a neural network.

[0065] The deep learning-based recognition system may include at least one classifier, and the classifier may correspond to one or more processors. The processor may be implemented by an array of multiple logic gates, or may be implemented by a combination of a general-purpose microprocessor and a memory storing a program that can be executed by the microprocessor.

[0066] The classifier can be implemented by a neural network-based classifier, a support vector machine (SVM), an AdaBoost classifier, a Bayesian classifier, a perceptron classifier, etc. In the following, an embodiment of implementing the classifier by a convolutional neural network (CNN)-based classifier will be described. A neural network-based classifier is a computational model implemented to simulate the computability of a biological system by using a plurality of artificial neurons connected by synapses, and performs human cognitive functions or learning processes through synapses with synaptic strengths (synaptic weights). However, the classifier is not limited thereto, but can be implemented by the various classifiers described above.

[0067] A general neural network includes an input layer, a hidden layer and an output layer, and the hidden layer can be composed of one or more layers as needed. As an algorithm for training such a neural network, a back propagation algorithm can be used.

[0068] The classifier can train a neural network so that once any data is input to the input layer of the neural network, the output data of the input learning data is output to the output layer of the neural network. Once the feature information extracted from the input data is input, the pattern of the feature information can be classified into any one of multiple classes, and the classification result can be output by using the neural network.

[0069] The processor is a classification system based on neural networks using a series of machine learning algorithms, and can use a recognition system based on deep learning.

[0070] Figure 1 is a block diagram illustrating an electronic device according to an embodiment.

[0071] refer to Figure 1 , the electronic device 100 may include a memory 110 , a camera 120 , and a processor 130 .

[0072] The electronic device 100 may be implemented by a device that automatically processes a given task or operation by using its own capabilities. For example, the electronic device 100 may be implemented by a robot. As an example, the electronic device 100 may be implemented by a service robot, a traveling robot, a transport robot, a cleaning robot, or a pet robot. Alternatively, for example, the electronic device 100 may be implemented by a household service robot or an industrial robot. The above examples are only used to describe the electronic device, and the electronic device is not necessarily limited thereto.

[0073] The memory 110 may be implemented by an internal memory included in the processor 130, such as a read-only memory (ROM) (e.g., an electrically erasable programmable read-only memory (EEPROM)) or a random access memory (RAM), or by a memory separate from the processor 130. In this case, the memory 110 may be implemented in the form of a memory embedded in the electronic device 100 or in the form of a memory attachable to and detachable from the electronic device, depending on the purpose of data storage. For example, data for driving the electronic device 100 may be stored in a memory embedded in the electronic device 100, and data for extended functions of the electronic device 100 may be stored in a memory attachable to and detachable from the electronic device 100. The memory embedded in the electronic device 100 can be implemented by at least one of a volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM) or synchronous dynamic RAM (SDRAM)), a non-volatile memory, such as a one-time programmable ROM (OTPROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an EEPROM, a mask ROM, a flash ROM, a flash memory (e.g., NAND flash or NOR flash), a hard disk drive or a solid-state drive (SSD), and the memory attachable to and detachable from the electronic device 100 can be implemented by a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multimedia card (MMC)), an external memory connectable to a USB port (e.g., a USB memory), etc.

[0074] According to an embodiment, the memory 110 may store attribute information and environment information of a plurality of objects related to a predetermined space.

[0075] Here, the attribute information may include various information related to the object. Specifically, the attribute information of the object may include the name, characteristic information, feature information and relationship information of the object.

[0076] Here, the attribute information may indicate a unique attribute of the object. For example, the attribute information may be information about whether the object has a static attribute or a dynamic attribute. In the case where the object is a person, the object may have a dynamic attribute, and in the case where the object is a thing, the object may have a static attribute. In addition, the property information may indicate the chemical properties of the material. In the case where the object is water, the object may have liquid properties, and in the case where the object is smoke, the object may have gas properties.

[0077] In addition, feature information may include various features that form the basis for contextual analysis. Feature information may include information about the structure, configuration, function, etc. of an object. For example, feature information of a bed object may include information about the general structure of the bed, information that the bed is composed of a mattress and a bed frame, and information that the bed is a place for people to sleep or lie down to rest. In addition, feature information may include the appearance and outline of a particular object. Specifically, feature information of a person object may include unique features of each person, such as face, voice, or fingerprint.

[0078] Relationship information may indicate information about the relationship between a specific object and another object. Relationship information may include various relationships between the attribute information of the object and the attribute information of another object. As an example, relationship information may include content indicating a relationship in which a first object uses a second object. For example, in a case where the first object is a person and the second object is a bed, the first object and the second object may have a relationship in which the first object uses the second object. As another example, the first object and the second object may be in a dangerous relationship. For example, in a case where the first object is a child and the second object is a fire, the first object and the second object may be in a dangerous relationship. As another example, relationship information may indicate a relationship between static objects and static objects, a relationship between static objects and dynamic objects, and a relationship between dynamic objects and dynamic objects.

[0079] The environmental information may include at least one of time information, space information or position information related to a predetermined space. In addition, the environmental information may change depending on a change in time, space or position.

[0080] The time information may include time information corresponding to a specific space. Here, the specific space may indicate the entire space in which the electronic device 100 is operated or a subspace in the entire space. The time information corresponding to the specific space may indicate the time when the specific space is used or the time predetermined by the user for the specific space. For example, the time for using the bedroom (11 pm to 8 am) may be obtained by using the usage time information of a general user, user behavior history information, time information directly set by the user, etc. The time information may include at least one of the start time, end time, or duration corresponding to the specific information.

[0081] In addition, spatial information may include information related to the use of a specific space. For example, the content that a bathroom is a space for showering and washing, a kitchen is a space for dining and cooking, and a bedroom is a space for resting and sleeping may correspond to spatial information. Such spatial information may be stored by default or obtained from an external device (e.g., an external server) based on the attributes of each space. Alternatively, usage information set by the user may be included in the spatial information. For example, in the case where a user adds a bedroom as a reading area, the corresponding information may be added to the spatial information. In addition, spatial information may include map information about a predetermined space.

[0082] The location information may include relative or absolute location information of a specific location or space. As an example, the location information may include absolute coordinate information of a specific location. For example, the location information may include coordinate information of a specific location where an object that the electronic device 100 is to avoid during travel is located. As another example, the location information may include relative coordinate information of each subspace included in the entire space. For example, the location information may include location information such as information that the kitchen is located in front of the living room and the main room is located to the right of the living room.

[0083] The environmental information may be obtained based on the input data by the electronic device 100. For example, the electronic device 100 may recognize that the space where the electronic device 100 is currently located is a bathroom based on the space information stored in the memory 110.

[0084] According to another example, environmental information may be obtained based on the recognized object. For example, in the case of recognizing a shower object, the electronic device 100 may recognize that the space where the electronic device 100 is currently located is a bathroom.

[0085] Although some embodiments of the attribute information and the environment information of the object have been described, the attribute information and the environment information of the object are not necessarily limited thereto and may include various information in addition to the above.

[0086] The camera 120 may be implemented by a front camera provided on the front surface of the electronic device 100, and may capture an image of an object in front of the electronic device 100 to obtain a captured image. The camera 120 may capture an image of a subject in front of the camera 120, such as a user's face, to obtain a user face image. Once the image of the subject is captured by the camera 120, the image of the subject is converted into an electrical image signal by a charge coupled device (CCD), and the image signal obtained by the conversion is amplified and converted into a digital signal. The digital signal obtained by the conversion as described above may be subjected to digital signal processing to be converted into a captured image.

[0087] The processor 130 controls the general operation of the electronic device 100. The processor 130 can be implemented by a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes digital signals. However, the present disclosure is not limited thereto, and the processor 130 may include one or more of a central processing unit (CPU), a microcontroller unit (MCU), a microprocessing unit (MPU), a controller, an application processor (AP), a graphics processing unit (GPU), a communication processor (CP), or an ARM processor, or may be defined by these terms. In addition, the processor 130 may be implemented by a system on chip (SoC) or a large-scale integration (LSI) in which a processing algorithm is embedded, or may be implemented in the form of a field programmable gate array (FPGA). The processor 130 can perform various functions by running computer executable instructions stored in the memory 110.

[0088] According to an embodiment, the processor 130 may recognize (or determine) a plurality of objects based on an image obtained by the camera 120, recognize a context of at least one of the plurality of objects based on a relationship between attribute information of the plurality of correspondingly recognized objects and environmental information, and control a traveling state (or an operating state or an operating mode) of the electronic device based on the recognized context. However, an image captured by a camera provided in an external device may also be received.

[0089] According to another embodiment, the processor 130 may identify multiple objects based on an audio signal obtained by the microphone 170 of the electronic device 100 or a microphone of an external device. In addition, the processor 130 may identify multiple objects based on various types of input data. Here, the input data may be obtained by a method in which the electronic device 100 directly obtains (or performs a pre-scan) the input data, a method in which the user directly inputs the input data, or a method in which the input data is received from an external server (e.g., a cloud server). Here, the electronic device 100 may use various sensors such as an image sensor, a sound sensor, an acceleration sensor, a gyroscope sensor, an ultrasonic sensor, an infrared sensor, and a radio frequency sensor to directly obtain the input data.

[0090] In addition, the processor 130 can provide the above-mentioned various data, such as the captured image, to the artificial intelligence learning model, and receive information about the recognized object from the artificial intelligence learning model. The processor 130 can recognize multiple objects separately at the same time point. For example, the processor 130 can simultaneously recognize a person object and a bed object currently located in front of the electronic device based on the captured image.

[0091] The relationship between the attribute information of multiple corresponding objects (or the relationship between objects) can indicate the following state or operation: it can be caused between multiple objects by the characteristics or attributes of each of the multiple objects. For example, the relationship between the attribute information of multiple corresponding objects may include a use relationship, a danger relationship, a protection relationship, a mobile relationship, etc. As an example, a person object and a bed object may be in a relationship where a person uses the bed. As another example, a fire object and a child object may be in a danger relationship. As another example, an adult object and a child object may be in a protection relationship. As another example, a chair object and a person object may be in a mobile relationship. Here, a mobile relationship may indicate a relationship in which a first object (person) can move a second object (chair). Here, the use relationship is a comprehensive concept described for ease of explanation, and various specific relationships may exist. When describing various embodiments, the relationship between the attribute information of a first object and the attribute information of a second object may be described as a relationship between the first object and the second object.

[0092] In addition, the processor 130 may determine the relationship between objects based on the presence or absence of the objects. The processor 130 may determine that the relationship is different depending on the presence or absence of a specific object. For example, in the case where a child object is identified but an adult object is not identified, the processor 130 may determine that there is no protector, identify the context in which the child object needs protection by additionally considering the environmental information, and perform appropriate control operations.

[0093] According to an embodiment, the processor 130 may identify various contexts based on the relationship between the attribute information of multiple objects (or the relationship between objects) and the environmental information. The context may be a concept including a situation, a state, a background, a thread, etc. Further, the context may include at least one of a past context corresponding to a past situation, a current context corresponding to a current situation, or a predicted context corresponding to a possible situation. Figure 5 Describe the specific embodiments of this context, and refer to Figure 6 Describe a specific example of a prediction context.

[0094] According to an embodiment, the processor 130 may identify a context based on the relationship between multiple objects and environmental information. For example, assume that information about the sleeping time from 11 pm to 8 am is stored as environmental information. When the electronic device 100 determines that the current time is 6 am, the electronic device 100 may identify the current context of "a person is sleeping in bed" taking into account the relationship (usage relationship) between the person object and the bed object and the environmental information (sleeping time).

[0095] According to another embodiment, the processor 130 may identify the context based on the relationship between the attribute information of a single object and the environmental information. For example, assume that information about the sleep time (from 11 pm to 8 am) corresponding to a specific object (e.g., an adult object) is stored as environmental information. In this case, the processor 130 may consider the sleep time (from 11 pm to 8 am) stored for the adult object to identify the context. However, for a child object, the processor 130 may identify the context without considering the stored sleep time.

[0096] According to another embodiment, the processor 130 may identify the context by using at least one of the attribute information of the object, the relationship between multiple objects, environmental information, or the relationship between the attribute information of a specific object and the environmental information. For example, the processor 130 may identify the context by considering the relationship between multiple objects. In the case where the context is not identified, the processor 130 may additionally consider the environmental information, and in the case where the context is not identified even if the environmental information is considered, the processor 130 may identify the context by additionally considering the relationship between the attribute information of the specific object and the environmental information. That is, the processor 130 may identify the context by sequentially considering various information, thereby improving the accuracy of context recognition.

[0097] The processor 130 may perform a control operation corresponding to the identified context. Specifically, the processor 130 may control the traveling state of the electronic device 100 based on the identified context, or perform control so that the electronic device 100 performs a specific function. As an example, the processor 130 may perform an operation of setting the basic route of travel of the electronic device 100 and moving along the basic route of travel, an operation of partially changing the basic route of travel and moving along a new route of travel, an operation of stopping on the basic route of travel, an operation of partially changing the basic route of travel and observing (or monitoring) a specific object at a specific location, an operation of stopping on the basic route of travel and outputting an alarm sound (or warning sound), and the like. The above operations are illustrative and the present disclosure is not necessarily limited thereto. For example, the processor 130 may identify the context of a person watching television (TV) based on a person object and a TV object, and change the basic route of travel when the basic route of travel is set in the space between the TV object and the person object so as not to hinder the person from watching TV.

[0098] The processor 130 may identify a danger level (or degree of danger) corresponding to the identified context based on the identified context and control the traveling state of the electronic device 100 based on the identified danger level.

[0099] Here, the danger level (or degree of danger) can indicate the degree of danger of the identified context. For example, depending on the user setting, the danger level can be classified as 0 to 100, and the larger the number, the more dangerous the situation. Here, a danger level of 0 can indicate a situation that no electronic device 100 needs to pay attention to. The danger level can be determined based on at least one of each object, the relationship between multiple objects, or absolute environmental information. Here, the relationship between multiple objects can indicate the relationship between a first object (a specific object) and a second object (another object different from a specific object), or can indicate a generation relationship between a first object and a second object. For example, the danger level can indicate the degree of danger of a dynamic object (a person), which is determined based on the relative context of the dynamic object relative to a static object (thing).

[0100] The processor 130 may predict a possible context of at least one of the plurality of objects based on the relationship between the plurality of objects and the environmental information, and control a traveling state of the electronic device based on the predicted context.

[0101] Here, the possible context may indicate various situations that may occur in the future based on the current context. The processor 130 may predict various situations that may occur in the future, rather than the current situation, based on the relationship between multiple objects and environmental information. The predicted context may be obtained by analyzing the current context and selecting a context that may occur in the future. As an example, in the case where a person is lying in bed, the predicted context may be a situation where the person leaves the bed. As another example, in the case where a person is sitting on a chair, the predicted context may be a situation where the person may move the chair backwards to leave the chair.

[0102] In addition to the current context, the processor 130 also identifies the predicted context, so the electronic device 100 can perform control operations corresponding to various contexts. As a result, the electronic device 100 can provide appropriate services or control operations to the user. In particular, the electronic device 100 can consider dangerous situations or important situations in advance so as to be able to prepare for the predicted situation.

[0103] One of the multiple objects may be a static object with static properties, and another of the multiple objects may be a dynamic object with dynamic properties. Here, the processor 130 may predict at least one of a moving direction or a moving distance of at least one of the multiple objects based on the relationship between the static object and the dynamic object and the environmental information, and control the traveling state of the electronic device based on at least one of the predicted moving direction or the predicted moving distance.

[0104] A static object may refer to an object that does not move by itself. For example, a static object may be an object such as a bed, a sofa, a chair, a ceramic, or a shower. A dynamic object may refer to an object that moves by itself. For example, a dynamic object may be an object such as a person, an adult, a child, or a pet.

[0105] Here, it is assumed that the processor 130 identifies both static objects and dynamic objects. In this case, the processor 130 may consider the relationship between the static object and the dynamic object to identify the context. As an example, the static object may be a relationship in which the static object can be moved by a dynamic object. For example, the chair object may be a relationship in which the chair object can be moved by a human object in a specific direction. However, not all static objects may be in a relationship in which the static object can be moved by a dynamic object. For example, the refrigerator object generally does not move after installation. Therefore, although the refrigerator object is a static object, the refrigerator object may not be in a relationship in which the refrigerator object is moved by a dynamic object. In addition, the chair object may be in a relationship in which the chair object can be moved by an adult object, but the chair object may not be in a relationship in which the chair object is moved by a child object (e.g., a baby). Depending on the attribute information of the static object or the relationship with another object, the static object may be in a relationship in which the static object can be moved.

[0106] The electronic device 100 can predict at least one of the moving direction, moving distance or moving speed of a specific object. Here, the specific object can be a static object or a dynamic object. The moving direction, moving distance or moving speed of the object can be determined based on the attribute information of the object. As an example, assume a situation where a person sits on a chair. The processor 130 can recognize that the chair object is in a relationship that the chair object can be moved by the person, and can predict the direction, moving distance, etc. in which the chair object can be moved by the person. For example, in the case where a person sits on the chair, the processor 130 can recognize that the person can push the chair back and can move the chair by approximately 0.5m to 1m. Here, the moving direction and moving distance can each have a predetermined value corresponding to the object. According to another example, the moving direction and moving distance can be obtained by analyzing the behavior of the identified human object.

[0107] The processor 130 may control the traveling state of the electronic device 100 based on at least one of the predicted moving direction or the predicted moving distance of the specific object. The processor 130 may change the existing moving route based on at least one of the predicted moving direction or the predicted moving distance of the specific object.

[0108] If it is determined that multiple objects are in contact with each other, the processor 130 may identify the context considering the relationship between the attribute information of the multiple objects. As an example, in the case where a static object and a dynamic object are in contact with each other, the processor 130 may identify the context considering the relationship between the static object and the dynamic object.

[0109] Specifically, in case of recognizing that the dynamic object is in at least one of a continuous contact state with the static object or a contact state with the static object, the processor 130 may recognize a context in which the static object may be moved by the dynamic object.

[0110] The continuous contact state or contact state may indicate that the first object and the second object are in physical contact with each other. For example, in the case where a human object sits on a chair object, the human object and the chair object may be in a continuous contact state.

[0111] According to another example, in a case where it is identified that the first object and the second object are located within a predetermined range, the processor 130 may determine that the first object and the second object are in a continuous contact state.

[0112] When the static object and the dynamic object are identified to be in a relationship in which the static object can be moved by the dynamic object and the dynamic object and the static object are identified to be in a continuous contact state or a contact state, the processor 130 can identify a predicted context in which the static object may be moved by the dynamic object. Here, the predicted context may include information about the predicted moving direction or predicted moving distance of the static object.

[0113] Furthermore, the processor 130 may identify a mode of the electronic device based on the identified context, and the mode of the electronic device may include at least one of a protection mode, a low noise mode, a monitoring mode, or a privacy mode.

[0114] The processor 130 may identify a traveling state of the electronic device 100 corresponding to the recognized context, and identify a mode (or traveling mode) in which the electronic device 100 may be controlled according to the recognized traveling state. The low noise mode may be a mode in which an operation of reducing the performance of the engine or reducing the revolutions per minute (RPM) of the motor is performed to reduce the noise generated by the electronic device 100. In particular, in the low noise mode, the speed of the electronic device 100 may be reduced and the moving angle of the electronic device 100 may be changed.

[0115] According to another example, in the case where the electronic device 100 includes a display, the electronic device 100 may include a low brightness mode. In the case where the processor 130 recognizes that the human subject is sleeping, the operation of reducing the noise of the electronic device 100 may be performed together with the operation of reducing the screen brightness of the display of the electronic device 100.

[0116] The monitoring mode may be a mode for performing an operation of observing a specific object. The electronic device 100 may stop on a basic travel route or change the basic travel route to observe a specific object. The monitoring mode may be determined based on the context of the specific object.

[0117] The protection mode may be a mode for protecting a specific object. According to an example, in order to protect a specific object, the electronic device 100 may change the basic route of travel to be a predetermined distance away from the object. The protection mode may correspond to a mode for preventing a collision between the electronic device 100 and a specific object. For example, in the case of identifying a person sitting on a chair, in the protection mode, the electronic device 100 may identify the possibility of a collision with the chair as a predicted context and change the basic route of travel to be away from the chair. According to another example, the electronic device 100 may approach a specific object to protect the specific object. Here, the protection mode may be a mode for performing operations for assisting in protecting a specific object. In the protection mode, the electronic device 100 may approach or move away from a specific object depending on the context.

[0118] The private mode may be a mode for stopping the collection of data about a specific space. In the private mode, the processor 130 may stop the image capture operation of the camera 120 within the specific space. For example, in a case where a person is taking a shower in the bathroom, the processor 130 may stop the image capture of the camera 120 in the private mode. In addition, the processor 130 may store in advance in the memory 110 the distance and direction in which an image of a specific space may be captured in the private mode, and the processor 130 may control the camera 120 of the electronic device 100 to prohibit image capture of the specific space. According to another example, the processor 130 may control the microphone 170 to stop receiving audio data in the private mode.

[0119] Furthermore, in the private mode, the processor 130 may perform control such that the electronic device 100 switches to the standby mode while maintaining a certain distance from the private space.

[0120] In the private mode, the processor 130 may request the user to confirm separately to perform the above control operation. As an example, the processor 130 may not automatically capture an image of a specific space and may request the user to confirm the execution of image capture. As another example, the processor 130 may request the user to confirm to stop capturing an image of a specific space.

[0121] In addition, when private space information is obtained from one of a plurality of objects and another of the plurality of objects is a dynamic object, the processor 130 can identify the context of the dynamic object based on the relationship between the plurality of objects and the private space information, and can perform control based on the identified context so that the electronic device is in at least one of a private mode or a low noise mode.

[0122] The environmental information may include spatial information, and the spatial information may include private space information. The private space may indicate an unopened space. The processor 130 may set an unopened space (such as a bathroom or a bedroom) as a private space, and store information about the private space in the memory 110. In addition, the processor 130 may obtain private space information as environmental information based on the identified object. The processor 130 may identify a private context based on the relationship between multiple objects and the private space information, and may perform control according to the private context so that the electronic device 100 travels in a private mode.

[0123] For example, in the case where the electronic device 100 recognizes a shower object, the electronic device 100 can recognize bathroom space information (private space information) based on the shower object. Here, in the case where the electronic device 100 recognizes a shower object and a person object, the electronic device 100 can recognize that the shower object and the person object are in a use relationship or an available relationship. In addition, the processor 130 can recognize a context in which a person takes a shower by using a shower based on the relationship between the shower object and the person object and the private space information, and the processor 130 can determine that the context corresponds to a private context. In addition, the processor 130 can control the travel state of the electronic device 100 according to the private context. Specifically, the processor 130 can perform control to cause the electronic device 100 to travel in a private mode.

[0124] Furthermore, in the context of identifying that one of the plurality of objects is dangerous to another of the plurality of objects based on the relationship between the plurality of objects, the processor 130 may perform control so that the electronic device is in at least one of a monitoring mode or a protection mode.

[0125] According to an embodiment, in the case where a predetermined first object and a second object are identified at the same time, the processor 130 may determine that the relationship between the first object and the second object is a dangerous relationship. In the case of identifying that the relationship between multiple objects is a dangerous relationship, the processor 130 may identify whether the context (or current context) of one of the multiple objects is a dangerous context based on the relationship between the multiple objects. Even in the case where the relationship between multiple objects is a dangerous relationship, the context of one of the multiple objects may not be a dangerous context. For example, a child object and a sofa object may be in a dangerous relationship. In the case where a child sits on the sofa, the processor 130 may identify that the relationship between the child and the sofa is a dangerous relationship and the context of the child object is a dangerous context. The processor 130 may identify a situation where a child may fall from the sofa as a dangerous context. However, even in the case where the child and the sofa are identified (or recognized) at the same time, the context of the child object may not be a dangerous context. For example, in the case where a child sits next to the sofa, although the relationship between the sofa and the child is a dangerous relationship, the context of the child object may not be a dangerous context.

[0126] Each of the multiple objects may be a predetermined user (or member), and the attributes of each of the multiple objects may include profile information of the predetermined user (or member). Here, the processor 130 may identify the context of the predetermined user (or member) based on the relationship between the predetermined user (or member) and the environmental information, and control the travel state of the electronic device based on the identified context.

[0127] The processor 130 may store profile information of members in a predetermined space (e.g., a house) in the memory 110. The profile information may include unique feature information of each member, such as each member's name, age, gender, height, face, fingerprint, appearance, voice, etc. The processor 130 may identify whether a member exists in a predetermined space and where the member is located in the predetermined space based on the profile information. In addition, the processor 130 may identify a context based on the relationship between member objects identified in the predetermined space and environmental information. Here, the relationship between members not only indicates the relationship between multiple members present in the predetermined space. The presence or absence of a member in a predetermined space may correspond to an object. For example, in the case where a first object does not exist in a predetermined space and a second object exists in the predetermined space, the processor 130 may identify a context based on the relationship between the first object and the second object.

[0128] In addition, the processor 130 may identify the context based on a combination of the presence or absence of members. For example, the processor 130 may identify the context based on the fact that the first to third members exist in a predetermined space and the fourth and fifth members do not exist in the predetermined space.

[0129] The relationship between member objects may include various relationships depending on the members. As an example, the relationship between the first member and the second member may be a relationship in which the first member protects the second member. Conversely, the relationship between the second member and the first member may be a relationship in which the second member is protected by the first member. For example, in a case where the first member does not exist in a predetermined space and only the second member exists in the predetermined space, the processor 130 may identify a context in which the second member needs to be protected. In addition, the processor 130 may control the travel state of the electronic device 100 to protect the second member.

[0130] In addition, at least one of the multiple objects may be a predetermined user (or member), and the attributes of each of the multiple objects may include profile information of the predetermined user (or member). Here, the processor 130 may identify the context of the predetermined user (or member) based on the relationship between the predetermined user (or member) and another object of the multiple objects, environmental information, and the behavior history of the predetermined user (or member), and control the travel state of the electronic device based on the identified context.

[0131] The behavior history may indicate the past behavior patterns of the user (or member). For example, the behavior history may indicate the past behaviors of a particular member, such as moving routes, watching TV, sleeping, reading, and cooking. The processor 130 may identify whether a particular member has a repetitive behavior pattern by analyzing the behavior history of a particular member, and may identify a context based on the repetitive pattern. For example, suppose that the first member has a behavior pattern in which the first member watches TV on the sofa at 8 a.m. every morning. The processor 130 may analyze the behavior history of the first member, and perform control to prevent the electronic device 100 from traveling in the space between the sofa and the TV at 8 a.m., so as not to hinder the first member from watching TV. As another example, in the case where a child sits on a chair and sways back and forth repeatedly, the processor 130 may identify that the child is in a dangerous context based on the behavior history. As another example, in the case where a child moves his / her arm to eat, puts down the silverware, and then stops moving his / her arm, the processor 130 may identify that the child leaves the dangerous context based on the behavior history.

[0132] Here, as an example, the relationship between a predetermined user (or member) and another one of the plurality of objects may indicate a relationship between members, or a relationship between a specific member and a separate object. Figures 11 to 15 A specific embodiment is described in which profile information of members is used.

[0133] Furthermore, the processor 130 may determine a dangerousness level of the recognized context based on the recognized context, and control the traveling state of the electronic device differently based on the dangerousness level.

[0134] The processor 130 can identify multiple contexts. As an example, the processor 130 can identify the current context and the predicted context based on the input image captured by the camera. In addition, the processor 130 can determine the degree of danger of at least one obtained context. The processor 130 can identify the degree of danger by considering the possibility of occurrence of the current context, the number of collisions, the frequency of occurrence, and the expected degree of damage when it occurs. In addition, the content of each item can be divided into multiple parts and stored in the memory 110, and the processor 130 can apply the data of each item stored differently for each context.

[0135] In addition, according to an embodiment, the processor 130 can control the travel state of the electronic device 100 based on the danger level of each context. For example, when the danger level of the current context is 60 and the danger level of the predicted context is 90, the processor 130 can perform a first control operation corresponding to the current context (danger level: 60), and a second control operation corresponding to the predicted context (danger level: 90). Here, the order of executing multiple control operations corresponding to multiple contexts can be determined by using a predetermined method. For example, the predetermined method may include a method of giving priority to the current context, a method of giving priority to a context with a higher degree of danger, a method of giving priority to control operations with a shorter execution time, a method of giving priority to control operations for specific users, and the like.

[0136] According to another embodiment, the processor 130 can determine the context with the highest degree of danger by comparing the degrees of danger corresponding to the plurality of contexts, and perform a control operation corresponding to the context with the highest degree of danger. In addition to the degree of danger (danger level), the processor 130 can also obtain the safety level (safety level), the protection necessity level, and the observation necessity level of each context.

[0137] Furthermore, the processor 130 may identify a context of at least one of the plurality of objects based on a relationship between attribute information and environmental information of each of the plurality of recognized objects.

[0138] As an example, the relationship may be a relationship between objects. As another example, the relationship may be a relationship between an object and environmental information. The processor 130 may identify the context by using a relationship between objects, a relationship between objects and environmental information, and at least one of the environmental information. For example, the relationship between objects may indicate a relationship between a person and a bed. In addition, the relationship between an object and environmental information may indicate a relationship between a person and a sleep time (from 11 p.m. to 8 a.m.). Here, the sleep time may be different for each person object.

[0139] According to another embodiment, the electronic device 100 may identify the context based only on the relationship between the multiple objects without the need for environmental information. In the case where the electronic device 100 cannot identify the context based on the relationship between the multiple objects or there is environmental information, the electronic device 100 may consider both the relationship between the multiple objects and the environmental information to identify the context.

[0140] The processor 130 may recognize at least two or more objects and may recognize a context in consideration of a relationship between attribute information of the plurality of objects. Although a case of recognizing two objects has been described in the present disclosure, the electronic device 100 may recognize a context in consideration of a relationship between three or more objects.

[0141] The electronic device 100 according to the present disclosure can recognize the context by considering the relationship between objects, so the electronic device 100 can recognize the context with high accuracy and perform appropriate control operations corresponding to various situations. In addition, the electronic device 100 can control specific operations by considering environmental information. Even in the case of recognizing the same object, the context can also change depending on another object or environmental information, so the recognition capability of the electronic device 100 for recognizing the context can be improved.

[0142] In addition, according to another embodiment, the electronic device 100 divides the context into the current context and the predicted context, so that in addition to the current situation, the electronic device 100 can also prepare for various situations that may occur. The electronic device 100 can quickly respond to various situations to improve the user's service satisfaction.

[0143] In addition, according to another embodiment, the electronic device 100 can identify the context by using the profile information of the member, so that the electronic device 100 can provide a personal service suitable for a predetermined member (or user) using a predetermined space. Specifically, the electronic device 100 can identify the context by considering the relationship between the members, so that a service suitable for the relationship between specific members can be provided.

[0144] Figure 2 Is used to describe Figure 1 A block diagram of a specific configuration of an electronic device.

[0145] refer to Figure 2 , the electronic device 100 according to an embodiment may include a memory 110 , a camera 120 , a processor 130 , a communication interface 140 , a user interface 150 , an input and output (I / O) interface 155 , a display 160 , and a microphone 170 .

[0146] Overlapping descriptions of the same parts of the memory 110 , the camera 120 , and the processor 130 as those of the above-described operations will be omitted.

[0147] The memory 110 may indicate a storage device implemented by a random access memory (RAM) 111, a read only memory (ROM) 112, a flash memory 113, and a disk 114. The memory 110 may indicate a memory device or a storage device that temporarily or permanently stores data. The memory 110 may store various software such as an operating system (O / S) or applications of the electronic device 100.

[0148] The RAM 111 may indicate a device that is readable and writable by accessing any area. The RAM 111 may be implemented by any one of a dynamic RAM (DRAM), a static RAM (SRAM), a synchronous dynamic RAM (SDRAM), and a ferroelectric random access memory (FeRAM).

[0149] ROM 112 may be a nonvolatile memory device in which data is retained even when power is turned off. ROM 112 may be implemented by a one-time programmable ROM (OTPROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a mask ROM, a flash ROM, or a flash memory.

[0150] The flash memory 113 may be a nonvolatile memory device from which data may be electrically erased and to which data may be electrically rewritten. The flash memory 113 may be implemented by a NAND flash memory or a NOR flash memory depending on an internal method.

[0151] The disk 114 may be implemented by a magnetic disk or an optical disk. The magnetic disk may be a storage device using a magnetic field and may be implemented by a floppy disk or a hard disk drive (HDD). In addition, the optical disk may be a storage device using light reflection and may be implemented in the form of an optical disk drive.

[0152] The memory 110 may include additional storage that may be implemented in various other schemes. For example, the memory 110 may include storage implemented in the form of a solid state device or solid state disk (SDD).

[0153] The memory embedded in the electronic device 100 may be implemented by a volatile memory (e.g., RAM 111), a nonvolatile memory (e.g., ROM 112 and flash memory 113), or at least one of a hard disk drive (HDD) or a solid state drive (SDD). The memory attachable to and detachable from the electronic device 100 may be implemented by a memory card (e.g., Compact Flash (CF), Secure Digital (SD), Micro Secure Digital (Micro-SD), Mini Secure Digital (Mini-SD), Extreme Digital (xD), or Multimedia Card (MMC)), an external memory connectable to a USB port (e.g., a Universal Serial Bus (USB) memory), or the like.

[0154] An implementation example of the memory 110 has been described above, but the memory 110 is not necessarily limited thereto.

[0155] The communication interface 140 is a transceiver (transmitter and receiver) that performs communication with various types of external devices in various types of communication modes. The communication interface 140 includes a Wi-Fi module 141, a Bluetooth module 142, an infrared communication module 143, a wireless communication module 144, etc. The processor 130 can use the communication interface 140 to perform communication with various external devices. Here, the external device may include an electronic device such as a TV, an image processing device such as a set-top box, an external server, a control device such as a remote controller, a sound output device such as a Bluetooth speaker, a lighting device, a home appliance such as a smart vacuum cleaner or a smart refrigerator, a server such as an Internet of Things (IoT) home manager, etc. In addition, the communication interface 140 may include a circuit for performing the above operations.

[0156] The Wi-Fi module 141 and the Bluetooth module 142 perform communication in a Wi-Fi mode and a Bluetooth mode, respectively. When the Wi-Fi module 141 or the Bluetooth module 142 is used, various connection information such as a service set identifier (SSID) and a session key are first sent and received, a communication connection is established using the connection information, and then various information can be sent and received.

[0157] The infrared communication module 143 performs communication by using an Infrared Data Association (IrDA) technology which uses infrared light between visible light and millimeter waves for short-range wireless data transmission.

[0158] The wireless communication module 144 may indicate a module that performs communication according to various communication protocols other than the above-mentioned Wi-Fi module 141 and the Bluetooth module 142, such as ZigBee, third generation (3G), third generation partnership project (3GPP), long term evolution (LTE), advanced LTE (LTE-A), fourth generation (4G) and fifth generation (5G).

[0159] In addition, the communication interface 140 may include at least one of a local area network (LAN) module, an Ethernet module, or a wired communication module that performs communication by using a twisted pair cable, a coaxial cable, an optical fiber cable, or the like.

[0160] According to an example, the communication interface 140 can use a communication module (e.g., a Wi-Fi module) to perform communication with an external device (remote controller) and an external server. The communication interface 140 can use a Wi-Fi module to perform communication with an external device (remote controller), and can use a Wi-Fi module to perform communication with an external server.

[0161] According to another example, the communication interface 140 may use different communication modules to perform communication with an external device (remote controller) and an external server. For example, the communication interface 140 may use at least one of an Ethernet module or a Wi-Fi module to perform communication with an external server, and may use a Bluetooth (BT) module to perform communication with an external device such as a remote controller. However, this is only an example, and in the case of performing communication with multiple external devices or external servers, the communication interface 140 may use at least one of various communication modules.

[0162] According to an implementation example, the communication interface 140 may additionally include a tuner and a demodulator.

[0163] The tuner may tune a user-selected channel or all pre-stored channels to receive a Radio Frequency (RF) broadcast signal through an antenna.

[0164] The demodulator may receive and demodulate a digital intermediate frequency (IF) (DIF) signal converted in the tuner, and may perform channel decoding and the like.

[0165] The processor 130 generally controls the operation of the electronic device 100 using various programs stored in the memory 110. The processor 130 according to an embodiment may be implemented by at least one of a main CPU 131, a graphics processing unit (GPU) 132, or a neural processing unit (NPU) 133. In addition, at least one of the main CPU 131, the GPU 132, or the NPU 133 may be connected to the RAM 111, the ROM 112, etc. corresponding to the memory 110 through a bus.

[0166] The main CPU 131 can control the O / S of the electronic device 100 by using a command set for system booting stored in the ROM 112, etc. Specifically, once a power-on command is input to supply power to the main CPU 131, the main CPU 131 copies the O / S stored in the memory 110 to the RAM 111 according to the instructions stored in the ROM 122, and runs the O / S to boot the system. Once the booting is completed, the main CPU 131 can copy various application programs stored in the memory 110 to the RAM 111, and run the application programs copied to the RAM 111 to perform various operations.

[0167] The main CPU 131 accesses the memory 110 to perform booting using the O / S stored in the memory 110. In addition, the main CPU 131 may perform various operations using various programs, content data, and the like stored in the memory 110.

[0168] The GPU 132 may correspond to a high-performance processing device for graphics processing, and may be a specialized electronic circuit designed to quickly process and change memory and accelerate the generation of images in a frame buffer for output on a screen. In addition, the GPU 132 may indicate a visual processing unit (VPU).

[0169] NPU 133 may correspond to an artificial intelligence (AI) chipset (or AI processor) and may be an AI accelerator. NPU 133 may correspond to a processor chip optimized to execute a deep neural network. NPU 133 may correspond to a processing device that executes a deep learning model instead of GPU 132, and NPU 133 may also correspond to a processing device that executes a deep learning model together with GPU 132.

[0170] Although the main CPU 131, GPU 132 and NPU 133 are all in Figure 2 , but the processor 130 may be implemented by at least one of the main CPU 131, the GPU 132, or the NPU 133.

[0171] The processor 130 can perform a graphics processing function (video processing function). For example, the processor 130 can render a screen including various objects such as icons, images, and texts by using a calculator and a renderer. Here, the calculator can calculate attribute values ​​such as the coordinate values ​​at which the corresponding object will be displayed, the form, size, and color of the corresponding object based on the received control command depending on the layout of the screen. In addition, the renderer can render screens including various layouts of objects based on the attribute values ​​calculated in the calculator. In addition, the processor 130 can perform various image processing on video data, such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion.

[0172] The processor 130 may perform processing on the audio data. Specifically, the processor 130 may perform various processing on the audio data, such as decoding, amplification, and noise filtering.

[0173] The user interface 150 may be implemented by a device such as a button, a touch panel, a mouse, or a keyboard, or may be implemented by a touch screen that can perform the above-mentioned display function and operation input function. Here, the button may be various types of buttons such as a mechanical button, a touch panel, and a scroll wheel formed in any area such as the front surface portion, the side surface portion, or the rear surface portion of the body appearance of the electronic device 100.

[0174] The I / O interface 155 may be one of High-Definition Multimedia Interface (HDMI), Mobile High-Definition Link (MHL), USB, DisplayPort (DP), Thunderbolt, Video Graphics Array (VGA) port, RGB port, D-subminiature (D-SUB), or Digital Video Interface (DVI).

[0175] HDMI is an interface capable of performing high-performance data transmission for audio / video (AV) devices that input and output audio signals and video signals. DP is an interface capable of realizing a full HD screen, an ultra-high-definition screen such as 2560×1600 or 3840×2160, and stereoscopic 3D images, and transmitting digital voice. Thunderbolt is an I / O interface for high-speed data transmission and connection, and personal computers (PCs), monitors, storage devices, etc. can all be connected in parallel through one port.

[0176] The I / O interface 155 may input and output at least one of an audio signal or a video signal.

[0177] According to an implementation example, the I / O interface 155 may include a port for inputting and outputting only audio signals and a separate port for inputting and outputting only video signals, or may include one port for inputting and outputting both audio signals and video signals.

[0178] The electronic device 100 may be implemented by a device that does not include a display and transmit a video signal to a separate display device.

[0179] The electronic apparatus 100 may transmit a voice signal received from an external device to an external server for voice recognition.

[0180] In this case, the communication module for communicating with the external device and the external server may be implemented as one module. Both of the communication modules for communicating with the external device and the external server may be, for example, Wi-Fi modules.

[0181] The communication module for communicating with an external device and an external server may be implemented as a separate module. For example, communication with an external device may be performed using a Bluetooth module, and communication with an external server may be performed using an Ethernet modem or a Wi-Fi module.

[0182] The electronic device 100 may further include a microphone 170. For example, the microphone 170 may be integrally formed with the electronic device 100 at the upper side, the front surface, the side surface, etc. of the electronic device 100. The microphone 170 may receive sound in an activation mode. The microphone 170 may include various components, such as a microphone that collects sounds such as user voices in an analog form, an amplifier circuit that amplifies the collected user voices, an A / D conversion circuit that samples the amplified user voices and converts the user voices into digital signals, and a filter circuit that removes noise components from the digital signals obtained by the conversion.

[0183] The microphone 170 is a component for receiving a user's voice or other sounds and converting the user's voice or other sounds into audio data. In this case, the microphone 170 may convert the received analog user voice signal into a digital voice signal and transmit the digital voice signal to the electronic device 100.

[0184] According to an embodiment, the electronic device 100 may send the received digital voice signal to a voice recognition server. In this case, the voice recognition server may convert the digital voice signal into text information by using a speech-to-text (STT) function. In this case, the voice recognition server may send the text information to another server or another electronic device to perform a search based on the text information, or may directly perform the search as needed.

[0185] According to another embodiment, the electronic device 100 may convert the digital voice signal by directly applying the STT function to the digital voice signal, and transmit the converted text information to an external server.

[0186] The speaker 180 may be a component that outputs various notification sounds, audio messages, etc. as well as various audio data processed by the I / O interface 155 .

[0187] The electronic device 100 may further include a mobile interface 190 to control the movement of the electronic device 100. For example, the mobile interface 190 may include a motor controller 192 and a motor 194. The motor 194 may be an electric motor, such as a DC motor. The motor controller 192 may control the motor 194. For example, the motor controller 192 may control the speed, direction, and torque of the motor 194. The motor 194 may include a plurality of motors. The motor controller 192 may independently control the speed, direction, and torque of each of the plurality of motors 194. The speed, direction, and torque may be controlled according to the travel state recognized by the electronic device.

[0188] The electronic device 100 does not necessarily include Figure 2 All components shown in the figure may be composed of only some components according to the implementation examples.

[0189] Figure 3 is a view for describing a route along which an electronic device moves in an indoor space according to an embodiment.

[0190] refer to Figure 3 , the electronic device 100 may analyze an image of the surrounding space captured by using the camera 120 and set a moving route 305 based on the image. The electronic device 100 may travel in the surrounding space based on the moving route 305. The electronic device 100 may recognize various contexts while traveling based on the moving route 305. The electronic device 100 may recognize a context 310 of a person lying on a bed and sleeping, a context 315 of a child sitting on a sofa, a context 320 of a person sitting on a chair, a context 325 of ceramics placed on a table, and a context 330 of a person taking a shower.

[0191] When the electronic device 100 recognizes a person object and a bed object, the electronic device 100 can obtain information that the attribute information of the person object is dynamic and the attribute information of the bed object is static. The electronic device 100 can recognize the context 310 of a person lying on a bed and sleeping based on the relationship between the attribute information of the person object and the attribute information of the bed object.

[0192] In addition, when the electronic device 100 identifies the child object and the sofa object, the electronic device 100 can obtain information that the attribute information of the child object is dynamic and the attribute information of the sofa object is static. The electronic device 100 can identify the context 315 that the child is sitting on the sofa based on the relationship between the attribute information of the child object and the attribute information of the sofa object.

[0193] In addition, when the electronic device 100 recognizes a person object and a chair object, the electronic device 100 can obtain information that the attribute information of the person object is dynamic and the attribute information of the chair object is static. The electronic device 100 can recognize the context 320 of a person sitting on a chair based on the relationship between the attribute information of the person object and the attribute information of the chair object.

[0194] In addition, in the case where the electronic device 100 identifies the table object and the ceramic object, the electronic device 100 can obtain the attribute information of the table object and the attribute information of the ceramic object, both of which are static information. The electronic device 100 can identify the context 325 of the ceramic being placed on the table based on the relationship between the attribute information of the ceramic object and the attribute information of the table object. Here, the electronic device 100 can identify the context by identifying information about the height at which the specific object (ceramic) is located. In the case where the specific object is located at a predetermined threshold (e.g., 50 cm) or more from the floor, the processor 130 can identify that the context of the specific object (e.g., ceramic) is a dangerous context.

[0195] In addition, when the electronic device 100 recognizes a person object and a shower object, the electronic device 100 can obtain information that the attribute information of the person object is dynamic and the attribute information of the shower object is static. The electronic device 100 can recognize the context 330 of a person showering based on the relationship between the attribute information of the person object and the attribute information of the shower object.

[0196] Figure 3 The multiple contexts 310, 315, 320, 325, and 330 shown in the figure may correspond to the current context. In addition to the current context, the electronic device 100 may also identify a predicted context. The predicted context may indicate various situations that may occur in the future based on the current context. Figures 5 to 10 A control operation of the electronic device 100 corresponding to the current context and the predicted context is described.

[0197] Figure 4 is used to describe Figure 3 A view of a context recognized by an electronic device at a specific location in the.

[0198] refer to Figure 4 ,refer to Figure 3The various contexts 310, 315, 320, 325, and 330 described may be identified at specific locations 410, 415, 420, 425, and 430, respectively, on the moving route 305 of the electronic device 100. For example, the electronic device 100 may identify at least one context at a first location 410 on the moving route 305, taking into account the relationship between the attribute information of the human object and the attribute information of the bed object and the environmental information. In addition, the electronic device 100 may control the travel state of the electronic device 100 at the first location 410 based on the identified context. For example, the electronic device 100 may control the electronic device 100 to move to the first location 410 by moving along the moving route 305.

[0199] Figure 5 It is a view for describing an operation of an electronic device to change a traveling state on a specific moving route.

[0200] refer to Figure 5 , the electronic device 100 can identify the context of a person sleeping in a bed based on the relationship between the person object and the bed object and the environmental information. The information about the relationship between the person object and the bed object may be information about the person using the bed. In addition, the environmental information may be information about the person's sleep time. In the case where the electronic device 100 identifies the context of a person sleeping in a bed, the electronic device 100 may perform a control operation that does not interfere with the person's sleep. The electronic device 100 may perform a change from a basic travel state to a low-noise travel state so as not to interfere with the person's sleep. The low-noise travel state may indicate an operation of reducing the RPM of a motor of the electronic device 100 and an operation of reducing the moving speed of the electronic device 100. According to another embodiment, the electronic device 100 may perform an operation of reducing the screen brightness of a display included in the electronic device 100 or an operation of reducing the volume of a sound output from a speaker of the electronic device so as not to interfere with the person's sleep.

[0201] Here, the electronic device 100 may perform a control operation that does not disturb the sleep of a person when traveling along the moving route 305 only in a specific section 510 on the moving route 305. The specific section 510 may be an area from a specific position of the context in which a person is recognized to sleep in a bed to a position a predetermined distance away from the person object, and the electronic device 100 may obtain information about the specific section 510. According to another embodiment, the electronic device 100 may set the specific section 510 by using the traveling state information stored in the memory 110. The electronic device 100 may store information of a change in the traveling state of the electronic device 100 in the memory 110, and set the specific section 510 by using the stored information.

[0202] Figure 6 is a view for describing an embodiment in which an electronic device changes a moving route.

[0203] refer to Figure 6 , the electronic device 100 can identify the predicted context based on the person object and the bed object. Here, the predicted context may indicate a situation in which a person leaves the bed. The electronic device 100 can identify a space 610 where a person who leaves the bed may stand based on the predicted context. In addition, in a case where the space 610 where a person who leaves the bed may stand overlaps with the moving route 305, the electronic device 100 can obtain a new moving route 620 by excluding the moving route 305 that overlaps with the space 610. The electronic device 100 can control the traveling state of the electronic device 100 according to the new traveling route 620 to prevent collisions with people in advance. The electronic device 100 can control the traveling state of the electronic device 100 according to the predicted context to prepare for sudden unexpected situations (situations in which a person suddenly leaves the bed).

[0204] Figure 7 is a view for describing an operation of an electronic device monitoring an object at a specific location on a moving route.

[0205] refer to Figure 7 , the electronic device 100 can identify a predicted context in which a child may fall from a sofa based on a child object and a sofa object. Here, the electronic device 100 can identify that the relationship between the child object and the sofa object can correspond to a dangerous relationship. For example, the relationship between the sofa object and the child object may correspond to a dangerous relationship, while the relationship between the sofa object and the adult object may not correspond to a dangerous relationship.

[0206] In addition, the electronic device 100 can identify a space or position 710 where a child may fall based on the predicted context. Further, when the relationship between the sofa object and the child object is identified as a dangerous relationship, the electronic device 100 can change the travel state of the electronic device 100 to perform monitoring to observe the state of the child. Here, the electronic device 100 can stop at a specific position 415 and continue to observe (monitor) the child without moving along the moving route 305. Here, in the case where the electronic device 100 determines that the child is very likely to fall from the sofa, the electronic device 100 can control the speaker to output an alarm sound (warning sound).

[0207] In addition, the electronic device 100 can determine whether there is an object that may harm the child in the identified space or position 710. For example, in the case where there is a water glass in the space or position 710 from which the child may fall, the electronic device 100 can output an alarm sound (warning sound) through a speaker. Although the case where the electronic device 100 outputs an alarm sound (warning sound) has been described in the above example, the present disclosure is not necessarily limited to this, and various operations that can warn of the current situation can be performed by the electronic device 100. As an example, in the case where the processor 130 identifies that the current situation corresponds to a predetermined context, the processor can call, send a message, transmit an image, etc. to a specific user. For example, in the case where it is identified that two people are currently in a predetermined space and the predetermined space is on fire, the processor 130 can immediately send a fire message or a rescue message ("request to rescue two adults") to the emergency rescue center.

[0208] Figure 8 It is a view for describing an embodiment of changing the property of an object and changing an existing moving route.

[0209] refer to Figure 8 , the electronic device 100 can identify the predicted context of the chair being moved by a person based on the person object and the chair object. The electronic device 100 can identify the person object and the chair object at a specific position 420 on the moving route 305, and can change the existing moving route 305 of the electronic device 100 based on the predicted context. Specifically, the electronic device 100 can obtain information that the attribute information of the chair object is static. Here, the electronic device 100 can change the attribute information of the chair object from static to dynamic based on the relationship between the chair object and the person object. In addition, the electronic device 100 can determine that the chair object identified at the same time as the person object can be moved based on the dynamic attribute. In addition, the electronic device 100 can identify the spatial range 810 or position to which the chair object can move. In the case where the electronic device 100 determines that the identified spatial range 810 overlaps with the existing moving route 305, the electronic device 100 can set a new route 820 by excluding the overlapping portion of the existing moving route.

[0210] According to another example, suppose that there are two adults and one child sitting on a dining chair at 8 am, and one chair is empty. The electronic device 100 can identify that two adults and one child are currently sitting on the dining chair by using at least one of a camera, a microphone, an ultrasonic sensor, or a frequency sensor. In addition, the electronic device 100 can identify that 8 am is a meal time based on environmental information. The electronic device 100 can identify that three people are currently dining and identify the predicted context that the chair will be moved after the meal based on the relationship between the human object and the chair object and the environmental information. The electronic device 100 can determine the degree of danger of the predicted context by considering the number of collisions between the electronic device 100 and the chair, the frequency of occurrence, and the degree of damage when the collision occurs. The electronic device 100 can obtain the number of collisions, the frequency of occurrence, and the degree of damage when the collision occurs based on past behavior history information. The number of collisions can indicate the number of collisions between the electronic device 100 and the chair, and can indicate the possibility of the collision. The frequency of occurrence can be information indicating how many times the collision occurs during a specific period. The degree of damage can indicate the degree of damage to the electronic device 100 or the chair when the collision occurs. The electronic device 100 can separately obtain the degree of danger of each of the multiple chairs. In addition, the electronic device 100 can perform a control operation corresponding to the danger level of each chair. For example, the electronic device 100 can recognize that the danger level of the context corresponding to the chair where the adult sits is 10, the danger level of the context corresponding to the chair where the child sits is 8, and the danger level of the context corresponding to the empty chair is 3. In addition, the electronic device 100 can set a new route 810 to be 100 cm away from the chair where the adult sits, to be 50 cm away from the chair where the child sits, and to be 20 cm away from the empty chair. The electronic device 100 can control the electronic device 100 to move along the new travel route 810. The above route setting operation can be automatically performed by the electronic device 100.

[0211] Fig. 9 is a view for describing an embodiment in which an existing moving route is changed depending on a property of an object.

[0212] refer to Fig. 9 , the electronic device 100 can identify a ceramic object 901 and a table object 902 with one leg at a specific position 425. Here, it is assumed that attribute information indicating that the ceramic object 901 is fragile is stored in the memory 110. The electronic device 100 can identify the prediction context based on the relationship between the ceramic object 901 and the table object 902 with one leg, that is, the ceramic may be broken even with a small impact. Here, the relationship between the ceramic object 901 and the table object 902 with one leg may correspond to a dangerous relationship. However, the relationship between the ceramic object 901 and the table object with two or more legs may not correspond to a dangerous relationship.

[0213] In the case where the electronic device 100 identifies the ceramic object 901 and the table object 902 with one leg, the electronic device 100 can identify the space 910 or position from which the ceramic object 901 may fall. In addition, in the case where the electronic device 100 identifies that the space 910 from which the ceramic object 901 may fall overlaps with the existing moving route 305, the electronic device 100 can set a new route 920 by excluding the overlapping portion of the existing moving route 305 and the space 910 from which the ceramic object 901 may fall. The electronic device 100 can control the electronic device 100 to move along the new route 920. In addition, the electronic device 100 can output a message or an alarm sound for warning that the ceramics are damaged.

[0214] Fig.10 It is a view for describing an operation of an electronic device to change a traveling state on a specific moving route.

[0215] refer to Fig.10 , the electronic device 100 can identify the current context that the person is showering based on the person object 1001 and the shower object 1002 at the specific position 430. Here, the electronic device 100 can identify the context that the person is currently in the bathroom (or shower room) based on the shower object 1002. In addition, information indicating that the bathroom (or shower room) corresponds to a personal space (private space) may be included in the environmental information (spatial information). Considering the relationship between the person object 1001 and the shower object 1002 and the environmental information, the electronic device 100 can obtain a privacy protection context.

[0216] In the case where the electronic device 100 recognizes the privacy protection context, the electronic device 100 may change the travel state of the electronic device 100 in a specific section 1010 on the existing moving route 305. Here, the electronic device 100 may turn off the camera or temporarily stop the image capture operation in the specific section 1010 in order to protect privacy while continuing to control the electronic device 100 to move along the moving route 305. Here, the specific section 1010 may indicate a section in which the electronic device 100 can recognize the privacy protection context. According to another embodiment, information about the specific section 1010 may be stored in the memory 110. The electronic device 100 performs a control operation according to the privacy protection context, thereby possibly obtaining the effect of protecting individual privacy.

[0217] refer to Fig.10, the relationship between the shower object 1002 and the adult object may correspond to a privacy protection relationship. However, the relationship between the shower object 1002 and the child (or specific person object) may not correspond to a privacy protection relationship. For example, there may be a context in which a child or a patient who needs care needs to be observed showering. Therefore, even in the case where the shower object 1002 is recognized, the electronic device 100 may obtain different contexts depending on the different relationships of the objects.

[0218] Fig.11 is a view for describing an operation of an electronic device storing information about a member.

[0219] refer to Fig.11 According to another embodiment, the electronic device 100 may store information about members in the memory 110 based on a predetermined space. Here, the predetermined space may indicate a limited space such as a house or a company where the electronic device 100 is operated. The member may indicate a dynamic object that is repeatedly recognized in the predetermined space, and the dynamic object may be a person or a pet.

[0220] As an example, the electronic device 100 may set a house as a predetermined space and may set an adult male 1101, an adult female 1102, a teenage female 1103, a male child 1104, and a pet 1105 as members of the predetermined space. In addition, the electronic device 100 may store unique feature information of each member in the memory 110, such as each member's name, age, gender, height, face, fingerprint, appearance, voice, etc. The electronic device 100 may recognize whether a member exists in the predetermined space and where the member is located in the predetermined space by using the unique feature information of each member.

[0221] In addition, the electronic device 100 can continuously monitor the location information of each member and control the travel state of the electronic device 100 based on the location information of each member. For example, when the electronic device 100 recognizes that all members stored in the memory 110 do not exist in the predetermined space, the electronic device 100 can perform a specific control operation in the predetermined space. As an example, the electronic device 100 can perform a cleaning operation or an operation of a control system to interrupt indoor power supply based on the information that all members do not exist in the predetermined space. As another example, it is assumed that a teenage female 1103 and an adult female 1102 are present in a predetermined space. The electronic device 100 can identify the location of the adult female 1102 to report information about the teenage female 1103 to the adult female 1102. The electronic device 100 can analyze the behavior history to estimate the location of the adult female 1102 and can move to the location where the adult female 1102 is most often located at the current time.

[0222] Fig.12is a view for describing an embodiment in which an electronic device recognizes the presence or absence of a member and performs a control operation corresponding to the recognition result.

[0223] refer to Fig.12 , the electronic device 100 can identify whether each member stored in the memory 110 exists in a predetermined space. Fig.11 In the embodiment shown, it is assumed that the electronic device 100 sets the house as a predetermined space and sets an adult male 1101, an adult female 1102, a teenage female 1103, a male child 1104, and a pet 1105 as members of the predetermined space. Here, the electronic device 100 can identify that the adult male 1101 and the adult female 1102 correspond to the protector or the adult. In addition, the electronic device 100 can identify the child 1104 as the target to be protected. In the case where the electronic device 100 identifies that the target to be protected exists in the predetermined space and the protector does not exist in the predetermined space, the electronic device 100 can change the traveling state of the electronic device 100 to monitor the target to be protected. For example, in the case where the electronic device 100 identifies that the objects 1101 and 1102 corresponding to the adults do not exist in the house, and only the child 1104 and the teenage female 1103 exist in the house, the electronic device 100 can control itself to be located near the child 1104 to monitor the child 1104 intensively.

[0224] Fig.13 is a view for describing another embodiment in which an electronic device recognizes the presence or absence of a member and performs a control operation corresponding to the recognition result.

[0225] refer to Fig.13 , the electronic device 100 can identify the member entering the predetermined space and control the travel state of the electronic device 100. Fig.12 In the illustrated embodiment, the electronic device 100 monitors the target to be protected, a child 1104. Here, in the case where the electronic device 100 newly recognizes that a protector exists in a predetermined space, the electronic device 100 may not monitor the target to be protected. For example, suppose that a protector 1102 enters the house while the electronic device 100 is monitoring the child 1104 because the protector (or adult) does not exist. Once the electronic device 100 recognizes that the protector 1102 enters the house, the electronic device 100 may stop the operation of monitoring the child 1104.

[0226] The electronic device 100 can control the travel state of the electronic device 100 by using the relationship between the child to be protected 1104 and the protector. According to another example, in the case where the electronic device 100 recognizes that the target to be protected and the adult object that is not the protector exist in the predetermined space, the electronic device 100 can continue to monitor the target to be protected. Although the adult object is an adult, in the relationship with the target to be protected, the adult object does not act as a protector.

[0227] Fig.14 is a view for describing an operation of the electronic device to set a movement route based on information on movement of a member.

[0228] refer to Fig.14 , the electronic device 100 can control the travel state of the electronic device 100 based on the historical information of the member. As an example, the electronic device 100 can store the movement route of the member. The electronic device 100 can identify the main movement route by analyzing the movement route history of the member. In addition, the electronic device 100 can set the movement route of the electronic device 100 by considering the main movement route of the member. In the case where the electronic device 100 recognizes that the member exists in a predetermined space, the electronic device 100 can set the movement route of the electronic device 100 to avoid the main movement route of the member stored in the memory 110 so as not to hinder the main movement route of the member.

[0229] For example, it is assumed that the main movement route 1410 is stored in the memory 110 as one of the historical information of the pet 1105. The electronic device 100 can control the traveling state of the electronic device 100 to avoid the main movement route 1410 of the pet 1105. Specifically, the electronic device 100 can set the basic traveling route of the electronic device 100 not to overlap with the main movement route 1410 of the pet 1105. The electronic device 100 can control the electronic device 100 to move within a predetermined space while avoiding the main movement route 1410.

[0230] According to another example, only in the case where the electronic device 100 recognizes that the pet 1105 exists in the house and members other than the pet 1105 do not exist in the house, the electronic device 100 can control the travel state of the electronic device 100 in consideration of the main movement route of the pet 1105. In the case where a specific member other than the pet 1105 exists in the house, the electronic device 100 can recognize that the pet 1105 moves with the specific member instead of moving along the main movement route. Therefore, in addition to the main movement route 1410 of the pet object 1105, the electronic device 100 can also control the travel state of the electronic device 100 in consideration of the relationship between the pet 1105 and another object.

[0231] According to another example, the electronic device 100 may store a specific time interval during which the pet 1105 moves along the main movement route 1410 as environmental information. For example, the electronic device 100 may recognize that the pet 1105 moves along the main movement route only in the time interval from 8 a.m. to 8 p.m. Specifically, when the electronic device 100 recognizes that the pet 1105 is present in the house from 8 a.m. to 8 p.m., the electronic device 100 may set a new travel route of the electronic device 100 to avoid the main travel route 1410 of the pet.

[0232] Fig.15 It is a view for describing an operation of an electronic device setting a moving route to avoid collision with a member.

[0233] refer to Fig.15 , the electronic device 100 can control the travel state of the electronic device 100 based on the historical information of the member. Fig.14 As described, the electronic device 100 can set the movement route of the electronic device 100 to avoid the main movement route that is one of the historical information of the member. As an example, the electronic device 100 can control the travel state of the electronic device 100 to travel along the edge of the predetermined space. Specifically, the electronic device 100 can set a route for traveling along the edge of the predetermined space.

[0234] For example, it is assumed that the main movement route 1410 of the pet 1105 is stored in the memory 110. In a case where the electronic device 100 recognizes that the pet 1105 exists in the house, the electronic device 100 may set a movement route 1505 that travels along the edge of the house.

[0235] Fig.16 is a diagram for describing a plurality of operations performed by an electronic device to identify a traveling state corresponding to a context.

[0236] refer to Fig.16 , the electronic device 100 may receive various data required for object recognition (S1605). Here, the data may be an image obtained by a camera 120 included in the electronic device 100 or may be an image received from an external device. Then, the electronic device 100 may recognize the object based on the received data (S1610). Here, the electronic device 100 may recognize the object by using an artificial intelligence learning model. Specifically, the electronic device 100 may provide the received image data as input data to the artificial intelligence learning model, and the object corresponding to the image data may be indicated as result data obtained by using the artificial intelligence learning model. According to an embodiment, the electronic device 100 may recognize the object by using the image data. According to another example, the electronic device 100 may receive audio data and recognize the object by using the received audio data.

[0237] Then, the electronic device 100 can identify whether the identified object is a new object (S1615). The electronic device 100 can identify whether the identified object is a new object by comparing it with the data stored in the memory 110 to determine whether the identified object is a pre-stored object. For example, in the case where the identified object is not stored in the memory 110, the electronic device 100 can determine that the identified object is a new object. Then, in the case where the identified object is a new object, the electronic device 100 can obtain the properties and characteristics of the object (S1620). Here, the properties and characteristics of the object can be directly input by the user, or information about the properties and characteristics of the object can be received from an external server. According to another example, the electronic device 100 can analyze the received data and store the properties and characteristics of the object in the memory 110. In addition, in S1620, the update of the changed attribute information of the object or the changed environmental information can be performed.

[0238] In the case where the electronic device 100 determines in S1615 that the recognized object is not a new object, the electronic device 100 may identify the context based on the information of the object or the environmental information (S1625). The electronic device 100 may use an artificial intelligence learning model to identify the context. Specifically, the electronic device 100 may provide at least one of the information about the object (attribute information) or the environmental information as input data to the artificial intelligence learning model, and receive result data from the artificial intelligence learning model. Here, the result data may be context information. This will be referred to later. Fig.17 The specific operations in S1625 are described.

[0239] Then, the electronic device 100 may check (identify) whether the identified context is a new context (S1630). Here, in the case where the identified context is a new context not stored in the memory 110, the electronic device 100 may perform a context analysis and update operation on the new context (S1635). Fig.18 Describe the specific operations in S1635.

[0240] When the electronic device 100 determines in S1630 that the identified context is not a new context, the electronic device 100 may identify the travel state of the electronic device 100 corresponding to the context (S1640). As an example, the electronic device 100 may identify the travel state of the electronic device 100 suitable for the context and control the electronic device 100 to be in the corresponding travel state. Fig.19 Describe the specific operations in S1640.

[0241] Fig.17is a diagram for describing an operation of an electronic device to recognize a context based on information of an object or environmental information.

[0242] You can Fig.17 Specific instructions shown Fig.16 The electronic device 100 may analyze the information of the object and / or the environmental information to identify the context based on the information of the object or the environmental information (S1705). Then, in the case of identifying multiple objects, the electronic device 100 may analyze the relationship between the information of the multiple objects (S1710). Then, the electronic device 100 may analyze the relationship between the information of the multiple objects and the environmental information (S1715). Then, the electronic device 100 may identify the context based on the information obtained in S1705 to S1715 (S1720).

[0243] Referenced above Fig.17 A case where S1625 is constituted by S1705 to S1720 is described. However, one or more steps from S1705 to S1720 may be omitted. For example, the electronic device 100 may determine that the context can be identified based on the information in S1710, and thus the electronic device 100 may determine to omit S1715.

[0244] Fig.18 is a diagram for describing an operation of an electronic device analyzing and updating a context.

[0245] refer to Fig.18 , in the analysis and update of the context (S1635), the current context or the possible context (predicted context) can be identified (S1805). Then, in S1635, the degree of danger of each identified context can be determined (S1810). As an example, the electronic device 100 can obtain the degree of danger (danger level), safety level (safety level), protection necessity level and observation necessity level of the new context from the user. Here, the electronic device 100 can consider the possibility of collision, contact with a specific object, exposure frequency, etc. to determine the degree of danger. According to another example, the electronic device 100 can obtain information about the new context from an external server. According to another example, the electronic device 100 can directly analyze and thereby obtain information about the new context. Then, in S1635, the information about the final context can be stored in the memory 110 to update the existing context information (S1815).

[0246] Fig.19 is a flowchart for describing an operation of an electronic device identifying a traveling state corresponding to a context.

[0247] refer to Fig.19, when identifying the traveling state of the electronic device 100 corresponding to the context, the electronic device 100 may check (identify) whether the context is a context corresponding to the current situation (S1905). In the case where the identified context is a context corresponding to the current situation, the electronic device 100 may identify at least one of the danger level (danger level), safety level (safety level), protection necessity level, or observation necessity level of the current context (S1910). In the case where the identified context is a possible context, the electronic device 100 may identify the danger level (danger level), safety level (safety level), protection necessity level, or observation necessity level of the predicted context (S1915). The degree of danger, etc. may vary depending on whether the identified context corresponds to the context of the current situation or the context of the possible situation. For example, assume a situation where a child falls from a sofa. In the case of the current context, the danger level of the situation where the child falls from the sofa can be set to 100. In the case of the predicted context (possible context), the danger level of the situation where the child falls from the sofa can be set to 50. Even in the case of the same context, different degrees of danger may be obtained depending on whether the situation is a current situation or a possible situation.

[0248] In S1640, after obtaining information about the degree of danger, the electronic device 100 may determine whether the identified degree of danger is equal to or greater than a threshold value (S1920). Here, the threshold value may indicate a predetermined value and may be changed according to the user's setting. In the case where it is determined in S1920 that the obtained degree of danger is not equal to or greater than the threshold value, the electronic device 100 may perform control to maintain the existing travel state of the electronic device 100 (S1930).

[0249] In the case where it is determined in S1920 that the obtained danger level is equal to or greater than the threshold value, the electronic device 100 may change the travel state of the electronic device 100 (S1925). Then, the electronic device 100 may determine whether the changed travel state can be performed in the existing travel mode (S1935). In the case where it is determined in S1935 that the changed travel state can be performed in the existing travel mode, the electronic device 100 may perform a control operation corresponding to the changed travel state in the existing travel mode (S1945). In the case where it is determined in S1935 that the changed travel state cannot be performed in the existing travel mode, the electronic device 100 may perform a change from the existing travel mode to the travel mode corresponding to the changed travel state. Then, the electronic device 100 may perform a control operation corresponding to the changed travel state based on the changed travel mode (S1940).

[0250] For example, it is assumed that the electronic device 100 can operate in the first travel state or the second travel state in the first mode, and can operate in the third travel state or the fourth travel state in the second mode. In the case where the first travel state is the basic travel state and a change is performed from the first travel state to the second travel state, the electronic device 100 can maintain the first mode. However, in the case where the first travel state is the basic travel state and a change is performed from the first travel state to the third travel state, the electronic device 100 can perform a change from the first mode to the second mode.

[0251] Fig. 20 is a flowchart for describing a control method of an electronic device according to an embodiment.

[0252] refer to Fig. 20 , in the control method of the electronic device storing attribute information of an object and environmental information related to a predetermined space according to an embodiment, a plurality of objects may be recognized based on an image obtained by the camera 120 (S2005).

[0253] In the control method of the electronic device 100 , a context of at least one of a plurality of objects may be recognized based on a relationship between attribute information of a plurality of correspondingly recognized objects and environmental information ( S2010 ).

[0254] In the control method of the electronic device 100 , the travel state of the electronic device may be controlled based on the recognized context ( S2015 ).

[0255] Here, in controlling the traveling state of the electronic device (S2015), a danger level of at least one object corresponding to the identified context may be identified based on the identified context, and the traveling state of the electronic device may be controlled based on the identified danger level.

[0256] Here, in identifying the context (S2010), a possible context of at least one of the multiple objects can be predicted based on the relationship and environmental information, and in controlling the travel state of the electronic device (S2015), the travel state of the electronic device can be controlled based on the predicted context.

[0257] In addition, one of the multiple objects may be a static object with static properties, and another of the multiple objects may be a dynamic object with dynamic properties. Here, in the recognition context (S2010), at least one of the moving direction or moving distance of at least one of the multiple objects may be predicted based on the relationship between the static object and the dynamic object and the environmental information, and in controlling the travel state of the electronic device (S2015), the travel state of the electronic device may be controlled based on at least one of the predicted moving direction or predicted moving distance.

[0258] Furthermore, in recognizing the context ( S2010 ), in the case of recognizing that the dynamic object is close to or in contact with the static object, a context in which the static object may be moved by the dynamic object may be recognized.

[0259] Furthermore, in controlling the traveling state of the electronic device, a mode of the electronic device may be identified based on the identified context, and the mode of the electronic device may include at least one of a protection mode, a low noise mode, a monitoring mode, or a privacy mode.

[0260] In addition, in identifying the context (S2010), when private space information is obtained from one of a plurality of objects and another of the plurality of objects is a dynamic object, the context of the dynamic object can be identified based on the relationship between the plurality of objects and the private space information, and in controlling the movement state of the electronic device (S2015), control can be performed based on the identified context to place the electronic device in at least one of a private mode or a low noise mode.

[0261] In addition, in controlling the travel state of the electronic device (S2015), in a context in which one of the multiple objects is identified as dangerous to another of the multiple objects based on the relationship between the multiple objects, control can be performed to put the electronic device into at least one of a monitoring mode or a protection mode.

[0262] like Fig. 20 The control method of the electronic device shown can be used in a Figure 1 or Figure 2 The method may be executed on an electronic device having a configuration of, or may be executed on an electronic device having a different configuration.

[0263] The methods according to the various embodiments described above may be implemented in the form of an application installable in an existing electronic device.

[0264] Furthermore, the methods according to the above-described various embodiments may be implemented by performing software upgrade or hardware upgrade with respect to the electronic device.

[0265] Furthermore, the various embodiments disclosed above may be executed through an embedded server provided in the electronic device or an external server of the electronic device.

[0266] In addition, the control method of the electronic device according to the above-mentioned various embodiments can be implemented by a program and thus provided to the electronic device. In particular, the program including the control method of the electronic device can be stored and provided in a non-transitory computer-readable medium.

[0267] In a non-temporary computer-readable medium storing computer commands, the computer commands, when executed by a processor 130 of the electronic device storing attribute information of the objects and environmental information related to a predetermined space, cause the electronic device to perform operations, the operations including: identifying multiple objects based on images obtained by a camera 120; identifying a context of at least one of the multiple objects based on the relationship between the attribute information of the multiple corresponding identified objects and the environmental information; and controlling the travel state of the electronic device based on the identified context.

[0268] The various embodiments described above can be implemented in a computer or a computer-readable recording medium using software, hardware, or a combination of software and hardware. As an example, according to hardware implementation, the embodiments described in this disclosure can be implemented using at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a processor, a controller, a microcontroller, a microprocessor, and an electrical unit for performing other functions. In some cases, the embodiments described in this disclosure can be implemented by the processor 130 itself. According to software implementation, embodiments such as the processes and functions described in this disclosure can be implemented by separate software modules. Each software module can perform one or more functions and operations described in this disclosure.

[0269] Computer instructions for executing the processing operations of the electronic device according to the above-mentioned various embodiments may be stored in a non-transitory computer-readable medium. The computer instructions stored in the non-transitory computer-readable medium allow a specific device to execute the processing operations of the electronic device according to the above-mentioned various embodiments when executed by a processor of the specific device.

[0270] Non-transitory computer-readable media refers to a medium that stores data semi-permanently and can be read by a machine. Specific examples of non-transitory computer-readable media may include a compact disc (CD), a digital versatile disc (DVD), a hard disk, a Blu-ray disc, a universal serial bus (USB), a memory card, a ROM, etc.

[0271] Although the embodiments have been illustrated and described above, the present disclosure is not limited to the above specific embodiments, but can be variously modified by a person skilled in the art to which the present disclosure belongs without departing from the subject matter disclosed in the attached claims. These modifications should also be understood to fall within the scope and spirit of the present disclosure.

Claims

1. An electronic device, comprising: camera; A memory for storing attribute information and environment information of multiple objects; as well as A processor is configured to identify multiple objects based on images obtained by a camera, identify the relationship between the attribute information of the multiple objects, wherein the relationship indicates at least one of the following relationships between a first object and a second object among the multiple objects: a usage relationship, a protection relationship, or whether the second object can be moved by the first object, identify a first context of the first object based on the relationship between the attribute information of the multiple objects and environmental information, and control the travel state of the electronic device based on the first context.

2. The electronic device according to claim 1, wherein: The processor is further configured to identify a first danger level of the first object based on the first context, and control a travel state of the electronic device based on the first danger level.

3. The electronic device according to claim 1, wherein: The processor is also configured to identify a first predicted context of the first object based on the relationship and the environmental information, and control a travel state of the electronic device based on the first predicted context.

4. The electronic device according to claim 3, wherein: One of the plurality of objects is a static object having static properties, wherein another of the plurality of objects is a dynamic object having dynamic properties, and The processor is further configured to predict at least one of the moving direction or moving distance of at least one of the multiple objects based on the relationship between the static objects and the dynamic objects and the environmental information, and to control the travel state of the electronic device based on at least one of the predicted moving direction or the predicted moving distance.

5. The electronic device according to claim 4, wherein: The processor is further configured to identify a context in which the static object can be moved by the dynamic object based on the dynamic object being identified as being within a threshold distance of the static object.

6. The electronic device according to claim 1, wherein: The electronic device is configured to operate in a protection mode, a low noise mode, a monitoring mode, and a privacy mode, The processor is further configured to identify a current mode of the electronic device from a protection mode, a low noise mode, a monitoring mode, and a privacy mode based on the first context, and The processor is further configured to identify the current mode according to the travel status of the electronic device.

7. The electronic device according to claim 6, wherein: The processor is also configured to control the current mode of the electronic device to be at least one of a private mode or a low noise mode based on the fact that private space information indicating an unopened space is obtained from a first object, that a second object from among the multiple objects is a dynamic object, and the relationship between the multiple objects and the private space information.

8. The electronic device according to claim 6, wherein: The processor is also configured to control a current mode of the electronic device to at least one of a monitoring mode or a protection mode based on a context in which one of the multiple objects is dangerous to another of the multiple objects based on the relationship between the multiple objects.

9. The electronic device according to claim 1, wherein: Each of the plurality of objects corresponds to one of the plurality of users. The attribute information includes profile information of each of the plurality of users, and The processor is further configured to identify contexts of the multiple users based on the relationship between the multiple users and environmental information, and control the travel state of the electronic device based on the context.

10. The electronic device according to claim 1, wherein: At least one of the plurality of objects is a user, The attributes of each object of the plurality of objects as a user type include corresponding profile information of the user, and The processor is further configured to identify the context of each user based on the relationship between each user and other objects among the multiple objects, environmental information and the behavior history of the corresponding user, and control the travel state of the electronic device based on the identified context.

11. The electronic device according to claim 1, wherein: The environmental information includes at least one of spatial information, time information or position information related to the current space.

12. The electronic device according to claim 1, wherein: The processor is further configured to identify a first context of the first object based on a relationship between the attribute information of each of the plurality of objects and the environmental information.

13. A control method for an electronic device storing attribute information and environmental information of a plurality of objects, the control method comprising: Recognize multiple objects based on images; identifying a relationship between the attribute information of the plurality of objects, wherein the relationship indicates at least one of the following relationships between a first object and a second object among the plurality of objects: a usage relationship, a protection relationship, or whether the second object can be moved by the first object, Based on the relationship between the attribute information of the plurality of objects and the environment information, identifying a first context of the first object; as well as A travel state of the electronic device is controlled based on the first context.

14. The control method according to claim 13, further comprising identifying a first danger level of the first object based on the first context, in, The travel status of the control electronics is based on a first hazard level.

15. The control method according to claim 13, wherein: Identifying a first context includes identifying a first predicted context of the first object based on the relationship and the environmental information, and Wherein, controlling the travel state of the electronic device is based on the first predicted context.

Images