Electronic device, operating method thereof, and storage medium

Abstract

Disclosed is an electronic device traveling a traveling space. The electronic device comprises: at least one sensor; a driving unit; a projection unit; at least one processor including processing circuitry; and a memory storing instructions, wherein the instructions, when executed individually or collectively by the at least one processor, cause the electronic device to: receive data corresponding to a notification request; control the driving unit to move to a first location related to the notification request, when a condition based on at least one of context information and tracking information obtained through the at least one sensor corresponds to a first condition corresponding to the notification request, according to the received data corresponding to the notification request; and provide a notification corresponding to the notification request through the projection unit.

Description

Electronic device and its method of operation and storage medium

[0001] The present disclosure relates to an electronic device, a method of operation thereof, and a storage medium, and more specifically, to an electronic device, a method of operation thereof, and a storage medium that provides a service related to a user's notification request.

[0002] With the advancement of electronic technology, various types of electronic devices are being developed and distributed, and recently, mobile electronic devices including projectors (e.g., robots) are being developed. These mobile electronic devices provide various information to users by using projectors while moving through specific spaces (e.g., homes, restaurants, airports, etc.). A projector refers to an electronic device that projects light onto a projection surface (or screen) to form an image on the projection surface.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] The aspects of the present disclosure are to address at least the problems and / or disadvantages mentioned above and to provide at least the advantages described below. Accordingly, the aspects of the present disclosure provide an electronic device and a method for controlling the same that provide a service related to a user's notification request.

[0005] Additional aspects will be presented in part in the following description, and in part may become apparent from the description or be learned by practicing the presented embodiments.

[0006] An electronic device for driving in a driving space according to one or more embodiments of the present disclosure comprises at least one sensor, a driving unit, a projection unit, at least one processor including a processing circuitry, and a memory for storing instructions, wherein when the instructions are executed individually or collectively by the at least one processor, the electronic device receives data corresponding to a notification request, and according to the received data corresponding to the notification request, if a condition based on at least one of context information and tracking information obtained through the at least one sensor corresponds to a first condition corresponding to the notification request, the driving unit is controlled to move to a first location associated with the notification request, and a notification corresponding to the notification request is provided through the projection unit.

[0007] A method of operation of an electronic device driving in a driving space according to one or more embodiments of the present disclosure comprises: receiving data corresponding to a notification request; controlling a driving unit to move to a first position associated with the notification request if, according to the data corresponding to the notification request, a condition based on at least one of context information and tracking information obtained through at least one sensor corresponds to a first condition corresponding to the notification request; and providing a notification corresponding to the notification request through a projection unit.

[0008] A storage medium storing computer-readable instructions according to one or more embodiments of the present disclosure causes the electronic device to receive data corresponding to a notification request when the instructions are executed by at least one processor of the electronic device traveling in a driving space, and, according to the received data corresponding to the notification request, if a condition based on at least one of context information and tracking information obtained through at least one sensor corresponds to a first condition corresponding to the notification request, the electronic device to control a driving unit to move to a first position associated with the notification request and to provide a notification corresponding to the notification request through a projection unit.

[0009] The above and other aspect features and advantages of specific embodiments of the present disclosure will become more apparent from the following description, which is referenced together with the accompanying drawings.

[0010] FIG. 1 is a diagram schematically illustrating an electronic device according to one embodiment.

[0011] FIG. 2 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0012] FIG. 3 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.

[0013] FIG. 4 is a flowchart illustrating a method for providing a notification according to one embodiment.

[0014] FIG. 5 is a flowchart illustrating a method for identifying a first condition based on information about an object according to one embodiment.

[0015] FIG. 6 is a flowchart illustrating the operation of acquiring tracking information according to one embodiment.

[0016] FIG. 7 is a drawing for explaining a method of identifying an object according to one embodiment.

[0017] FIG. 8 is a drawing for explaining a method of providing a notification through a projection unit according to one embodiment.

[0018] FIG. 9 is a diagram illustrating a method for identifying whether a task according to one embodiment has been completed.

[0019] FIG. 10 is a block diagram showing the detailed configuration of an electronic device according to one embodiment.

[0020] It should be noted that the same reference numbers are used throughout the drawings to denote identical or similar elements, features, and structures.

[0021] The following description, made with reference to the attached drawings, is provided to facilitate a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. While various specific details are included to aid understanding, they should be considered merely illustrative. Accordingly, those skilled in the art will recognize that various changes and modifications to the various embodiments described herein may be made without departing from the scope and spirit of the present disclosure. Additionally, for the sake of clarity and brevity, descriptions of well-known functions and configurations may be omitted.

[0022] The terms and words used in the following description and claims are not limited to their bibliographic meanings and are used merely to enable the inventor to understand the present disclosure clearly and consistently. Accordingly, it is evident to those skilled in the art that the following description of various embodiments of the present disclosure is provided merely for illustrative purposes and is not intended to limit the present disclosure as defined by the appended claims and their equivalents.

[0023] It should be understood that, unless the context clearly indicates otherwise, the singular forms "a," "an," and "the" include plural referents. Thus, for example, a reference to "a component surface" includes a reference to one or more such surfaces.

[0024] Terms used in the specification will be briefly explained, and then the present disclosure will be explained.

[0025] Commonly used terms currently in use have been selected for use in the embodiments of this disclosure in consideration of their functions within this disclosure, and these terms may be changed according to the intent of those skilled in the art, case law, the emergence of new technology, etc. Additionally, in specific cases, terms may be selected at the applicant's discretion. In such cases, the meaning of such terms is mentioned in the relevant explanatory section of this disclosure. Accordingly, terms used in this disclosure should be defined based on their meaning and the content throughout this disclosure, rather than merely their names.

[0026] In the specification, expressions such as "have," "may have," "include," or "may include" indicate the presence of the relevant feature (e.g., a component such as a numerical value, function, behavior, or part) and do not exclude the presence of additional features.

[0027] Expressions such as "at least one of A or / and B" can represent "A or B" or "both A and B".

[0028] Expressions such as "first" and "second" used in this disclosure may refer to various components regardless of order or importance. These expressions are used solely to distinguish one component from another and do not limit said component.

[0029] Where a component (e.g., Component 1) is referred to as being "(functionally or telecommunicationally) coupled / coupled" or "connected" to another component (e.g., Component 2), it should be understood that the component may be directly coupled to the other component or coupled through another component (e.g., Component 3).

[0030] A singular expression may include a plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0031] In the present disclosure, a "module" or "a component / or" may perform at least one function or operation and may be implemented in hardware, software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "components / or" may be integrated into at least one module and implemented by a processor (not shown), except for a "module" or "component / or" that must be implemented in specific hardware.

[0032] Furthermore, terms such as "signal" in the specification may include not only electrical signals but also signals in the form of sound waves, and electrical signals may be analog signals or digital signals. For example, expressions such as "audio signal (or noise signal)" indicate a sound wave (or radio wave) signal when the signal is outside the electronic device, and an electrical signal when it is inside the electronic device depending on its location. Additionally, signal processing inside the electronic device described below may be digital signal processing, analog signal processing, or a signal processing method that combines analog and digital methods.

[0033] Additionally, terms such as "filter" in the specification refer to a device for removing specific components (e.g., a specific frequency range or a specific pattern), and the filter may be a digital filter or an analog filter.

[0034] It should be understood that the blocks of each flowchart and combinations of flowcharts can be executed by one or more computer programs containing computer-executable instructions. One or more computer programs may be stored entirely in a single memory device, or one or more computer programs may be divided into different parts stored in several different memory devices.

[0035] Any of the functions or operations described herein may be processed by a single processor or a combination of processors. A single processor or a combination of processors is a circuit that performs processing and includes circuits such as an application processor (AP, e.g., a central processing unit (CPU)), a communication processor (CP, e.g., a modem), a graphics processing unit (GPU), a neural network processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a Wi-Fi chip, a Bluetooth™ chip, a Global Positioning System (GPS) chip, a Near Field Communication (NFC) chip, a connectivity chip, a sensor controller, a touch controller, a fingerprint sensor controller, a display driver integrated circuit (IC), an audio codec (CODEC) chip, a Universal Serial Bus (USB) controller, a camera controller, an image processing IC, a microprocessor unit (MPU), a system-on-chip (SoC), an IC, etc.

[0036] FIG. 1 is a diagram schematically illustrating an electronic device according to one embodiment.

[0037] According to FIG. 1, an electronic device (100) according to one embodiment can travel in a driving space (10). According to one embodiment, the electronic device (100) can receive a notification request.

[0038] According to one example, the electronic device (100-1) may receive a notification request from a first user (1-1). According to one example, the notification request may be a request to provide information to a specific user, such as, "Tell my son to take out the soup in the kitchen and the side dishes in the refrigerator and eat them." When the notification request is received, the electronic device (100-1) may identify whether conditions corresponding to the received notification request are met. According to one example, conditions corresponding to the notification request may include at least one of whether the target user is detected within the driving space (10), whether the target time is reached, and whether the target user moves to the target location. When the son is detected and identified as moving to the kitchen, the electronic device (100-1) may move to the location corresponding to the notification request (e.g., the kitchen) and provide a notification through a projection unit.

[0039] According to one example, the electronic device (100-2) may receive a notification request from a second user (1-2). According to one example, the notification request may be, "Tell my daughter that she has to go to English academy at 6:00 PM." When the notification request is received, the electronic device (100-2) may identify whether conditions corresponding to the received notification request are met. For example, the electronic device (100-2) may identify whether the current time has reached the target time of 6:00 PM. The electronic device (100-2) may detect the daughter within the driving space (10). When the daughter is detected and the target time reaches 6:00 PM, the electronic device (100-2) may move to the area where the daughter is located and provide a notification through a projection unit.

[0040] FIG. 2 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0041] According to FIG. 2, the electronic device (100) may include at least one sensor (110), a driving unit (120), a projection unit (130), at least one processor (140) and a memory (150).

[0042] The electronic device (100) can travel through the travel space. In one example, the electronic device may be implemented as a mobile projector that provides images while moving through the travel space, but is not limited thereto, and in one example, the electronic device may be implemented as a robot that travels through the travel space.

[0043] At least one sensor (110) may include a plurality of sensors of various types. The sensor (110) may measure physical quantities or detect the operating state of an electronic device (100) and convert the measured or detected information into an electrical signal. The sensor (110) may include a camera (or a camera sensor), and the camera may include a lens that focuses visible light or other optical signals received by being reflected by an object onto an image sensor, and an image sensor capable of detecting visible light or other optical signals. Here, the image sensor may include a 2D pixel array divided into a plurality of pixels. Alternatively, at least one sensor (110) may include a temperature sensor or an infrared sensor.

[0044] The drive unit (120) may be configured to drive a wheel included in the electronic device (100). According to one example, the drive unit (120) may include a rotating shaft connected to the wheel and a motor for driving the wheel by transmitting power to the rotating shaft. Accordingly, the processor (140) can drive the wheel through the drive unit (120) to perform various driving operations such as moving, stopping, speed control, turning, and changing angular velocity of the electronic device (100).

[0045] The projection unit (130) is configured to project an image outward. According to various embodiments of the present disclosure, the projection unit (130) can be implemented using various projection methods (e.g., cathode-ray tube (CRT) method, liquid crystal display (LCD) method, digital light processing (DLP) method, laser method, etc.). For example, the CRT method is basically the same principle as a CRT monitor. The CRT method magnifies the image using a lens in front of the cathode-ray tube (CRT) to display the image on a screen. Depending on the number of cathode-ray tubes, it is divided into a single-tube type and a three-tube type, and in the case of the three-tube type, the red, green, and blue cathode-ray tubes can be implemented separately.

[0046] As another example, the LCD method displays an image by transmitting light from a light source through a liquid crystal. LCD methods are divided into single-panel and three-panel types. In the case of the three-panel type, light from a light source is separated into Red, Green, and Blue by a dichroic mirror (a mirror that reflects only specific colors of light and allows the rest to pass through), then passes through the liquid crystal, and finally converges back into one place.

[0047] As another example, the DLP method is a technique that displays images using a DMD (Digital Micromirror Device) chip. The projection unit of the DLP method may include a light source, a color wheel, a DMD chip, and a projection lens. Light emitted from the light source can acquire color as it passes through a rotating color wheel. The light passing through the color wheel is input into the DMD chip. The DMD chip contains numerous micro-mirrors and reflects the light input into it. The projection lens performs the function of magnifying the light reflected from the DMD chip to the size of an image.

[0048] As another example, the laser method includes DPSS (Diode Pumped Solid State) lasers and galvanometers. Lasers that output various colors utilize a laser formed by installing three DPSS lasers for each RGB color and superimposing their optical axes using special mirrors. The galvanometer includes mirrors and high-output motors to move the mirrors at high speeds. For example, a galvanometer can rotate the mirror at a maximum speed of 40 KHz / sec. The galvanometer is mounted according to the scanning direction; since projectors generally perform planar scanning, the galvanometer can also be positioned along the x and y axes.

[0049] Meanwhile, the projection unit (130) may include various types of light sources. For example, the projection unit (130) may include at least one light source among a lamp, an LED, and a laser. Additionally, the projection unit (130) may output an image with a 4:3 aspect ratio, a 5:4 aspect ratio, or a 16:9 wide aspect ratio depending on the use of the electronic device (100) or the user's settings, and may output an image with various resolutions such as WVGA (854*480), SVGA (800*600), XGA (1024*768), WXGA (1280*720), WXGA (1280*800), SXGA (1280*1024), UXGA (1600*1200), and Full HD (1920*1080) depending on the aspect ratio.

[0050] At least one processor (140) (hereinafter, processor) is electrically connected to at least one sensor (110), driving unit (120), projection unit (130), and memory (150) to control the overall operation of the electronic device (100). The processor (140) may be composed of one or more processors. Specifically, the processor (140) may perform the operation of the electronic device (100) according to various embodiments of the present disclosure by executing at least one instruction stored in the memory (150).

[0051] According to one embodiment, the processor (140) may be implemented as a digital signal processor (DSP) that processes digital video signals, a microprocessor, a Graphics Processing Unit (GPU), an Artificial Intelligence (AI) processor, a Neural Processing Unit (NPU), or a Time Controller (TCON). However, it is not limited thereto, and may include or be defined by one or more of a central processing unit (CPU), a Micro Controller Unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), or an ARM processor. Additionally, the processor (140) may be implemented as a System on Chip (SoC) or Large Scale Integration (LSI) with a built-in processing algorithm, or may be implemented in the form of an Application Specific Integrated Circuit (ASIC) or Field Programmable Gate Array (FPGA).

[0052] The memory (150) can store data necessary for various embodiments. Depending on the purpose of data storage, the memory (150) may be implemented in the form of a memory embedded in the electronic device (100) or in the form of a memory that can be attached to and detached from the electronic device (100). For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in a memory that can be attached to and detached from the electronic device (100).

[0053] Meanwhile, the memory embedded in the electronic device (100) may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one-time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash, etc.), hard drive, or solid state drive (SSD). Additionally, the memory that is detachable from the electronic device (100) may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory connectable to a USB port (e.g., USB memory), etc. there is.

[0054] According to one embodiment, the processor (140) may receive data corresponding to a notification request. According to one example, the notification request may be a command to provide a notification to a user present in the driving space. For example, the notification request may be a request to convey information to a specific user in the home, such as, "Tell my son to take a sandwich out of the refrigerator and eat it for lunch."

[0055] In one example, the data corresponding to the notification request may be voice data. In one example, the processor (140) may identify the notification request based on user voice received through a microphone (e.g., the microphone (195) of FIG. 10). For example, when the processor (140) receives user voice through the microphone, it may convert the received user voice into text through a preset algorithm (e.g., a STT (speak-to-text) algorithm) and identify the notification request based on the converted text.

[0056] According to one example, a notification request may include at least one keyword. According to one example, at least one keyword may be information related to at least one of a target user, a target time, and a target location corresponding to the notification request. According to one example, the target user may be the recipient of the notification. According to one example, the target time may be a time related to the context (or content) of the notification. According to one example, the target location may be a location related to the context of the notification.

[0057] For example, one can assume a case where the notification request is 'tell my son to take a sandwich out of the refrigerator and eat it for lunch.' In one example, the processor (140) can identify 'son,' 'lunch,' and 'refrigerator' as keywords included in the notification request. The processor (140) can identify the target user as 'son' based on the keyword 'son.' The processor (140) can identify the target time as 'lunch time' based on the keyword 'lunch.' The processor (140) can identify the target location as 'kitchen' based on the keyword 'refrigerator.' However, it is not limited thereto, and in one example, the target location may be an area within the driving space related to the 'refrigerator,' including 'an area located within a preset distance from the kitchen,' 'refrigerator,' or 'an area located within a preset distance from the refrigerator.'

[0058] According to one example, information regarding the target user, target time, and target location corresponding to each keyword may be stored in memory (150). For example, identification information regarding the user corresponding to the keyword 'son' may be stored in memory (150). Or, for example, time information corresponding to the keyword 'lunch' may be stored in memory (150). Or, for example, location information corresponding to the keyword 'refrigerator' may be stored in memory (150).

[0059] According to one example, the processor (140) may obtain context information. According to one example, the context information is defined as information that can characterize the state of an object, where 'object' may be a user. According to one example, the user's context information may include at least one of the current time, environment information, and user history information. According to one example, the processor (140) may obtain context information from an external device (e.g., a server). Alternatively, according to one example, the processor (140) may obtain context information based on sensing data obtained through at least one sensor (110). Alternatively, according to one example, the processor (140) may obtain context information based on sensing data obtained from at least one IoT (Internet of Things) device present in the driving space.

[0060] According to one example, the processor (140) may obtain tracking information of a user through at least one sensor (110). According to one example, the tracking information may include at least one of information on the current user's location within the driving space (e.g., a first sub-space within the driving space) and information on the current user's behavior, as information obtained based on an image of the user. According to one example, if at least one sensor (110) is implemented as a camera sensor, the processor (140) may track (or monitor) the user in real time using sensing data obtained through at least one sensor (110). According to one example, the processor (140) may obtain information on the user's location within the driving space and information on the user's behavior based on sensing data obtained by tracking the user in real time. This will be described later.

[0061] Meanwhile, according to one example, the processor (140) may obtain tracking information based on sensing data obtained from at least one IoT (Internet of Things) device present in the driving space. For example, at least one IoT device equipped with a camera sensor in the driving space (e.g., a TV or a refrigerator, etc.) may obtain sensing data related to the user, and the processor (140) may receive sensing data from at least one IoT device through a communication circuit and obtain tracking information based on the received data. According to one example, the processor (140) may continuously obtain context information and tracking information by continuously tracking the user.

[0062] According to one example, the processor (140) can identify whether a condition based on at least one of the user's context information and the user's tracking information obtained through at least one sensor (110) corresponds to a first condition corresponding to the notification request.

[0063] According to one example, a condition based on at least one of the user's context information and the user's tracking information obtained through at least one sensor (110) may be a condition related to the user's movement or location. The first condition may be a condition related to keywords included in a notification request. According to one example, the first condition may be a condition related to whether the target user moves to the target location at the target time.

[0064] For example, one can assume a case where the notification request is 'tell my son to take a sandwich out of the refrigerator and eat it at lunch.' The processor (140) can identify whether the first condition is met based on whether the target user, the son, moves to the target location, the kitchen, at the target time, lunch time, based on keywords included in the notification request.

[0065] According to one example, among the first conditions, 'whether the target time has been reached' may be whether the current time is the target time. Alternatively, 'whether the target time has been reached' may be whether the current time is within a preset range from the target time. According to one example, the processor (140) may identify whether the current time has reached the target time based on the user's context information.

[0066] For example, a case can be assumed where the target time is lunch time (e.g., 12:00 PM). The processor (140) can identify whether the current time has reached lunch time based on context information. Alternatively, the processor (140) may identify that the target time has been reached if the current time has reached a time within a preset time (e.g., 30 minutes) from lunch time.

[0067] According to one example, the processor (140) can identify a target user within the driving space. According to one example, the processor (140) can identify a target user within the driving space that corresponds to a notification request among at least one user based on sensing data obtained through at least one sensor (110) and identification information corresponding to each of at least one user within the driving space. This will be described later.

[0068] According to one example, among the first conditions, 'whether to move to the target location' may include different actions of the target user moving to the target location, including departure to the target location and arrival to the target location. According to one example, the processor (140) may identify whether the target user moves to the target location based on the user's tracking information. For example, the case where the target location is a kitchen may be assumed. The processor (140) may identify whether the son moves to the kitchen based on the tracking information corresponding to the son.

[0069] Meanwhile, according to one example, if a keyword corresponding to a target location is not included in a notification provision location, the processor (140) may determine whether the first condition is satisfied by excluding conditions related to the target location.

[0070] According to one embodiment, the processor (140) may control the drive unit (120) to move to a first location associated with a notification request when a first condition is met. According to one example, the first location associated with the notification request may be a location for providing a notification. According to one example, the first location may be identified based on keywords included in the notification request. For example, one can assume a case where the notification request is "tell my son to take a sandwich out of the refrigerator and eat it for lunch." The processor (140) may identify the first location corresponding to the notification request as "kitchen." The processor (140) may control the drive unit (120) to move to the identified kitchen.

[0071] According to one embodiment, the processor (140) may provide a notification related to a notification request through the projection unit (130). According to one example, the notification related to the notification request may be of a different type of notification, including text-type information or voice-type information. For example, one can assume a case where the notification request is "tell my son to take a sandwich out of the refrigerator and eat it for lunch." The processor (140) may move to the kitchen and provide a text-type notification (for example, a notification corresponding to "take a sandwich out of the refrigerator and eat it") to a pre-set area within the kitchen (for example, the refrigerator or the floor). Alternatively, the processor (140) may output voice corresponding to "take a sandwich out of the refrigerator and eat it" through a speaker. Alternatively, the processor (140) may irradiate light onto the refrigerator through a light-emitting unit (for example, an LED (light emitting diode)).

[0072] According to the example described above, when an electronic device (100) confirms a notification request for a specific user, it can provide a notification to the user by taking into account the information included in the notification request. The electronic device (100) can move to a location related to the notification request and provide the notification directly to the user. The electronic device (100) can provide various types of notifications to the user. Accordingly, content related to the notification can be efficiently provided to the user.

[0073] FIG. 3 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.

[0074] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0075] According to FIG. 3, according to one embodiment, the operation method may include an operation (S310) of receiving data corresponding to a notification request. According to one example, an electronic device may receive data corresponding to a notification request. According to one example, an electronic device (100) may obtain a notification request. According to one example, the electronic device (100) may include a microphone, and the electronic device (100) may obtain a notification request through the microphone.

[0076] According to one embodiment, the operation method may include an operation (S320) of controlling a driving unit to move to a first position related to a notification request when, according to data corresponding to a received notification request, a condition based on at least one of context information and tracking information obtained through at least one sensor corresponds to a first condition corresponding to a notification request.

[0077] According to one example, the electronic device (100) may include at least one sensor (110). According to one example, when a notification request is received, the electronic device (100) may identify whether a condition based on at least one of context information and tracking information obtained through at least one sensor corresponds to a first condition related to the notification request, based on at least one of the user's context information and tracking information obtained through at least one sensor (110).

[0078] In one example, the first condition may be a condition related to keywords included in the notification request. In one example, the first condition may be a condition related to whether the target user moves to the target location at the target time.

[0079] According to one example, the electronic device (100) may control the driving unit (120) to move to a first location associated with a notification request when a first condition is met. According to one example, the first location associated with a notification request may be a location for providing a notification. According to one example, the first location may be identified based on keywords included in the notification request.

[0080] According to one embodiment, the operation method may include an operation (S330) of providing a notification corresponding to a notification request through a projection unit.

[0081] According to one example, the electronic device (100) may include a projection unit (130). According to one example, the electronic device (100) may provide a notification related to a notification request through the projection unit (130). According to one example, the notification related to the notification request may be a different type of notification including text-type information or voice-type information.

[0082] Returning to FIG. 2, according to one embodiment, the processor (140) can identify whether a first condition is satisfied based on at least one of context information, user location information based on sensing data obtained through at least one sensor (110), and user behavior information.

[0083] According to one example, tracking information may include user location information and user behavior information. According to one example, user location information within a driving space may be information about a subspace where the user is located among at least one subspace included in the driving space, but is not limited thereto, and according to one example, the location information may include information about the user's location within the subspace where the user is located.

[0084] According to one example, the processor (140) can identify the location information of a user based on sensing data obtained through at least one sensor (e.g., a camera sensor). For example, the processor (140) can identify the location of a user within a driving space while performing a patrol operation over a driving space using a camera sensor (an RGB camera or a LiDAR sensor).

[0085] Alternatively, according to one example, the processor (140) may identify user behavior information based on sensing data. According to one example, the behavior information may include information about the user's direction of movement or the user's movement line. According to one example, the processor (140) may continuously track a user present within a driving space, and the processor (140) may identify the behavior information of the first user based thereon. According to one example, the processor (140) may continuously update location information and behavior information by tracking the user.

[0086] According to one example, the processor (140) can identify whether the current time has reached the target time based on the user's context information. For example, the user's context information may include information about the current time. When the target time is identified based on keywords included in the notification request, the processor (140) can identify whether the current time has reached the target time by comparing the target time with the current time.

[0087] According to one example, the processor (140) can identify a target user within the driving space. According to one example, the processor (140) can identify a target user within the driving space that corresponds to a notification request among at least one user based on sensing data obtained through at least one sensor (110) and identification information corresponding to each of at least one user within the driving space. This will be described later.

[0088] According to one example, the processor (140) can identify whether the target user is moving to a target location based on the tracking information of the user. According to one example, the processor (140) can identify the location information and behavior information of the target user based on sensing data obtained through at least one sensor (110). The processor (140) can identify whether the target user is moving to a target location based on the identified location information and behavior information of the target user. For example, a case where the target location is a kitchen can be assumed. The processor (140) can identify whether the son is moving to the kitchen by identifying the son's current location and the son's movement path based on the location information and behavior information corresponding to the son.

[0089] FIG. 4 is a flowchart illustrating a method for providing a notification according to one embodiment.

[0090] According to FIG. 4, according to one embodiment, the operation method may include an operation (S410) of identifying whether a target user moves to a target location based on location information and behavior information of a target user corresponding to a notification request.

[0091] According to one example, the electronic device (100) can identify a target user based on the acquired notification request when a notification request is acquired. According to one example, the electronic device (100) can acquire location information and behavior information of the target user based on sensing data acquired through at least one sensor (110). According to one example, the electronic device (100) can identify whether the target user is moving to a target location within a driving space based on the acquired location information and behavior information of the target user.

[0092] For example, one can assume a case where the notification request is 'tell my son to take a sandwich out of the refrigerator and eat it for lunch.' The electronic device (100) can identify 'son,' 'lunch,' and 'refrigerator' as keywords included in the notification request. The electronic device (100) can identify the son as the target user within the driving space. The electronic device (100) can identify the son's location information and behavior information within the driving space through at least one sensor (110). The electronic device (100) can identify whether the son is moving from his current location to the target location, the 'kitchen.'

[0093] According to one embodiment, the operation method may include an operation (S420) of controlling the driving unit (120) to move to a first position when it is identified that it is moving to a target position.

[0094] According to one example, the electronic device (100) can identify a first location based on keywords included in a notification request. According to one example, if the electronic device (100) identifies that a target user is moving to a target location, the driving unit (120) can be controlled to move to a first location corresponding to the notification request.

[0095] For example, if a notification request includes 'refrigerator', the electronic device (100) can identify 'kitchen' as the target location corresponding to the notification request based on information stored in memory (150). If the electronic device (100) identifies that the son is moving to the kitchen, the driving unit (120) can be controlled so that the electronic device (100) moves to the first location, the kitchen.

[0096] Meanwhile, depending on the example, the target location and the first location may differ. For instance, even if the target location is identified as a 'kitchen,' the first location may be identified as a 'refrigerator' located within the kitchen or an 'area located at a pre-set distance from the refrigerator.'

[0097] According to one embodiment, the operation method may include an operation (S430) of providing a notification related to a notification request through a projection unit (130).

[0098] According to one example, the electronic device (100) may provide a notification related to a notification request at a first location. For example, the electronic device (100) may move to the kitchen and then provide a text-type notification (e.g., a notification corresponding to 'take a sandwich out of the refrigerator and eat it') in a pre-set area within the kitchen (e.g., the refrigerator or the floor around the refrigerator). Alternatively, the electronic device (100) may output a voice corresponding to 'take a sandwich out of the refrigerator and eat it' through a speaker. Alternatively, the electronic device (100) may irradiate light onto the refrigerator through a light-emitting part (e.g., an LED (light emitting diode)).

[0099] Meanwhile, according to one embodiment, the electronic device (100) may identify whether the target user is moving to the target location when it is identified that the target time corresponding to the notification request has been reached based on the user's context information. According to one example, when the target user is identified, the electronic device (100) may identify whether the current time has reached the target time based on the target user's context information. When the current time has reached the target time, the electronic device (100) may identify whether the target user is moving to the target location.

[0100] For example, the electronic device (100) can identify whether the son moves to the kitchen when the current time reaches the target time, which is ‘lunch time’. Or, for example, the electronic device (100) can identify whether the son moves to the kitchen when the current time reaches a time that is a preset time (e.g., 30 minutes) before ‘lunch time’.

[0101] FIG. 5 is a flowchart illustrating a method for identifying a first condition based on information about an object according to one embodiment.

[0102] According to FIG. 5, according to one embodiment, the operation method may include an operation (S510) of obtaining information about at least one object included in the driving space based on sensing data obtained through at least one sensor (110) while performing a patrol operation on the driving space.

[0103] According to one example, the electronic device (100) can perform a patrol operation (or a function corresponding to a patrol) over a driving space. According to one example, while performing a patrol operation over a driving space, the electronic device (100) can obtain information about at least one object included in the driving space based on sensing data obtained through at least one sensor (110).

[0104] According to one example, information about an object may include location information and identification information for each object existing within a driving space. According to one example, the location information of an object may include information about a sub-space where the object is located within the driving space and a location where the object is placed within the sub-space. According to one example, the identification information of an object may be information about the type of the object. For example, the type of the object may include not only types of home appliances such as refrigerators, remote controllers, and TVs, but also types other than home appliances such as clothes, flowerpots, and laundry, but is not limited thereto. According to one example, an electronic device (100) may input an image corresponding to an object into a preset object type classification algorithm to obtain identification information of the object.

[0105] According to one example, the electronic device (100) can acquire location information and identification information for each of at least one object existing in the driving space based on sensing data. According to one example, the electronic device (100) can store the acquired location information and identification information in map information for the driving space.

[0106] According to one example, map information for a driving space may be stored in the memory (150). According to one example, the map information for a driving space may include identification information for each of at least one sub-space within the driving space. According to one example, the map information for a driving space may include location information and identification information for each of at least one object existing within the driving space. According to one example, the electronic device (100) may update the map information for a driving space using the location information of each of at least one object obtained.

[0107] For example, map information may include location information within the driving space for each sub-space, including 'living room', 'shoe cabinet', 'kitchen', and 'master bedroom' included in the driving space, along with identification information corresponding to each sub-space. According to one example, map information may include information regarding the type of object included in each sub-space and the location of each object within the sub-space.

[0108] According to one example, the electronic device (100) may update map information corresponding to the driving space based on sensing data obtained through at least one sensor (110). For example, the electronic device (100) may update map information corresponding to the driving space by periodically performing a patrol operation.

[0109] According to one embodiment, the operation method may include an operation (S520) of identifying a first condition based on information about at least one acquired object. According to one example, when information about at least one object is acquired, the electronic device (100) may identify a first condition based thereon.

[0110] For example, one can assume a case where the notification request is, "Tell my daughter to wear a yellow padded jacket because it is cold in the morning." The electronic device (100) can identify "daughter," "morning," and "yellow padded jacket" as keywords corresponding to the notification request. Based on the keywords, the electronic device (100) can identify the target user as the daughter, identify the target time as the morning time (e.g., 8:00 AM), and identify the target location as the "area where the yellow padded jacket is located." The electronic device (100) can identify the "area where the yellow padded jacket is located" based on information about at least one object. According to one example, location information and identification information regarding the yellow padded jacket may be included in map information regarding the driving space, and the electronic device (100) can identify the area where the yellow padded jacket is located within the map based on this. The electronic device (100) can identify whether the first condition is satisfied by identifying whether the daughter moves to the area where the yellow padded jacket is located. This will be explained in detail through Fig. 8.

[0111] Meanwhile, according to one example, when the electronic device (100) is in an out-of-box experience (OOBE) state, it can obtain information about objects while performing a patrol operation over the driving space. For example, when the electronic device (100) is in an OOBE state, it can obtain location information and identification information for each of at least one object existing within the driving space while performing a patrol operation over the driving space.

[0112] According to one example, when a new object is identified within the driving space, the electronic device (100) may update map information about the driving space based on information about the identified object. According to one example, when a new object is identified or the OOBE state is present, the electronic device (100) may provide content related to the OOBE through the projection unit (130). According to one example, the content related to the OOBE may be content intended to assist in the operation of the electronic device (100). However, it is not limited thereto.

[0113] Meanwhile, according to one embodiment, the type of object may include a dynamic object or a static object. According to one example, when a dynamic object is identified as a target object corresponding to a notification request, the electronic device (100) may update the location of the target object within the driving space and identify whether a first condition is satisfied based on the updated location. According to one example, the target object may be an object related to the notification request among the objects existing within the driving space. According to one example, when the first condition is satisfied based on the updated location, the electronic device (100) may identify a first location based on the updated location and move to the first location to provide a notification. Alternatively, according to one example, when a static object is identified as a target object corresponding to a notification request, the electronic device (100) may identify whether the first condition is satisfied based on the location of the identified target object based on information stored in the memory (150).

[0114] Returning to FIG. 2, according to one embodiment, when a notification related to a notification request is provided, the processor (140) can identify feedback information related to the notification request based on at least one of tracking information, sensing data obtained through at least one external device, and sensing data obtained through at least one sensor (110).

[0115] For example, at least one external device may be at least one IoT (Internet of Things) device existing within the driving space, but is not limited thereto, and at least one external device may be a server.

[0116] For example, feedback information related to a notification request may include information on whether the target user has completed the task associated with the notification request. For instance, if the notification request is "Tell my daughter to wear a yellow padded jacket because it is cold in the morning," the task could be "daughter wearing a yellow padded jacket."

[0117] According to one example, the processor (140) can identify feedback information related to the notification request after the notification is provided. For example, the processor (140) can identify whether the yellow padding remains in its original location based on sensing data obtained through at least one sensor (110) after the notification corresponding to 'Tell my daughter to wear a yellow padding because it is cold in the morning' is provided to the target user, the daughter. If the processor (140) identifies that the yellow padding remains in its original location, it can identify that the task related to the notification request has not been performed. Alternatively, for example, the processor (140) may identify that the task related to the notification request has not been performed if, based on the daughter's tracking information, it identifies that the daughter is not wearing the yellow padding.

[0118] According to one example, the processor (140) may provide a notification to a user based on the feedback information when the feedback information is identified. According to one example, if it is identified that a task has not been performed, the processor (140) may provide information related to the feedback to the input user. According to one example, the input user may be a user who has entered a notification request. The processor (140) may identify the location of the input user within the driving space and provide information related to the feedback (e.g., information that a task has not been performed) to the input user at the identified location through the projection unit (130).

[0119] Alternatively, depending on one example, the processor (140) may provide guidance information to the target user if it is identified that the task has not been performed. For example, the processor (140) may move to the target user's location based on the target user's tracking information and provide a notification again at the moved location.

[0120] Alternatively, according to one example, the processor (140) may provide information related to feedback to the input user when it is identified that the execution of a task has been completed. For example, the processor (140) may identify the location of the input user within the driving space and provide information related to feedback (e.g., information that the execution of a task has been completed) to the input user at the identified location through the projection unit (130).

[0121] According to one embodiment, the processor (140) may provide guidance information related to a notification request through the projection unit (130) if the first condition is not satisfied. According to one example, the guidance information may be information that guides the performance of a task corresponding to the notification request. For example, in the case of a notification request corresponding to "Tell my daughter to wear a yellow padded jacket because it is cold in the morning," the processor (140) may project light to the location of the yellow padded jacket within the driving space. Alternatively, the processor (140) may move to the location of the daughter within the driving space and provide a notification through the projection unit (130) at the location of the daughter.

[0122] According to one example, the processor (140) may provide information to the input user indicating that no notification was provided if the first condition is not met. For example, if the first condition is not met, the processor (140) may identify the input user (e.g., mother) in the driving space and provide the input user with information including 'daughter is not wearing a yellow padded jacket' via the projection unit (130).

[0123] FIG. 6 is a flowchart illustrating the operation of acquiring tracking information according to one embodiment.

[0124] According to FIG. 6, according to one embodiment, the operation method may include an operation (S610) of identifying a target user corresponding to a notification request among at least one user based on sensing data obtained through at least one sensor (110) and identification information corresponding to each of at least one user in the driving space.

[0125] According to one example, the electronic device (100) can acquire image information corresponding to each of at least one user based on sensing data acquired through at least one sensor (110). According to one example, at least one sensor (110) may be implemented as an RGB (red, green, blue) camera sensor. For example, the image information may include at least one of information regarding head detection of the user and information regarding body pose detection of the user.

[0126] According to one example, reference image information corresponding to each of at least one user may be stored in memory (150), but is not limited thereto, and according to one example, the electronic device (100) may obtain reference image information from an external device (e.g., a server). The electronic device (100) may identify at least one user by comparing image information obtained through at least one sensor (110) with reference image information stored in memory.

[0127] According to one example, the electronic device (100) can acquire identification information corresponding to each of at least one user. According to one example, the identification information may include at least one of face information and voice information corresponding to the user.

[0128] According to one example, reference identification information corresponding to each of at least one user may be stored in memory (150), but is not limited thereto, and according to one example, the electronic device (100) may obtain reference identification information from an external device (e.g., a server). The electronic device (100) may obtain identification information based on sensing data obtained through at least one sensor (110). The electronic device (100) may identify at least one user by comparing the obtained identification information with the reference identification information stored in memory.

[0129] Alternatively, according to one example, the electronic device (100) may identify at least one user based on image information and identification information, respectively.

[0130] According to one example, when an electronic device (100) receives a notification request, it can identify a target user corresponding to the received notification request. When a target user is identified, the electronic device (100) can identify a target user among at least one user existing in a driving space by comparing at least one of the image information and identification information received by the electronic device (100) with reference image information and reference identification information corresponding to the identified target user. According to one example, the electronic device (100) may also identify a target user among at least one user existing in a driving space using a learned neural network model.

[0131] According to one embodiment, the operation method may include an operation (S620) of obtaining tracking information of an identified target user. According to one example, when a target user is identified, the electronic device (100) may obtain tracking information of the identified target user.

[0132] FIG. 7 is a drawing for explaining a method of identifying an object according to one embodiment.

[0133] Referring to FIG. 7, according to one embodiment, the electronic device (100) can obtain information about at least one object included in the driving space (710) based on sensing data obtained through at least one sensor (110) while performing a patrol operation for the driving space (710).

[0134] According to one example, the electronic device (100) can obtain identification information for an object present in the driving space (710) based on sensing data obtained through at least one sensor (110). For example, it may be assumed that at least one sensor is implemented as a camera sensor. The electronic device (100) can identify an object present in the driving space (710) based on image information obtained through the camera sensor. The electronic device (100) can identify an object by comparing image information corresponding to an object with reference image information corresponding to a plurality of objects.

[0135] According to one example, reference image information corresponding to a plurality of objects may be reference image information (or sample image information) corresponding to each type of a plurality of objects. According to one example, reference image information corresponding to each type of a plurality of objects may be stored in memory (150), but is not limited thereto, and according to one example, reference image information may be stored in an external device (e.g., a server). According to one example, the electronic device (100) may, of course, obtain identification information based on image information corresponding to an object by using a preset image classification algorithm.

[0136] According to one example, the electronic device (100) may acquire location information for an object present in a driving space (710) based on sensing data acquired through at least one sensor (110). According to one example, the electronic device (100) may identify the location of an object within the driving space (710) based on map information and sensing data for the driving space (710).

[0137] According to one example, the electronic device (100) can update the map information so that the information about the acquired object is included in the map information if the information about the acquired object is not included in the existing map information.

[0138] According to one example, when a new object is identified, the electronic device (100) can provide content related to the OOBE through the projection unit (130).

[0139] According to one example, the electronic device (100) may identify the type of the remote control (730) based on image information corresponding to the remote control (730). For example, the electronic device (100) may identify whether the remote control (730) is a TV (720) remote control (730) or an air conditioner remote control (730) by comparing the image information corresponding to the remote control (730) with reference image information.

[0140] Alternatively, according to one example, the electronic device (100) may identify the type of the remote control (730) based on information about objects around the newly identified remote control (730). For example, if the electronic device (100) identifies that a TV (720) exists around the newly identified remote control (730), it may identify the newly identified remote control (730) as a TV (720) remote control (730).

[0141] FIG. 8 is a drawing for explaining a method of providing a notification through a projection unit according to one embodiment.

[0142] Referring to FIG. 8, according to one embodiment, the electronic device (100) can provide a notification through the projection unit (130).

[0143] For example, one can assume a case where the notification request is 'Tell my daughter to wear a yellow padded jacket because it is cold in the morning.' The electronic device (100) can identify 'daughter,' 'morning,' and 'yellow padded jacket' as keywords corresponding to the notification request. Based on the keywords, the electronic device (100) can identify the target user as the daughter, identify the target time as the morning time (e.g., 8:00 AM), and identify the target location as the 'area where the yellow padded jacket is located.'

[0144] According to one example, the electronic device (100) can identify the 'area where the yellow padding is located' based on information about at least one object. According to one example, the location information about the object may include location information about the object within at least one sub-space included in the driving space (810), and information about the location where the object is placed within the sub-space where the object exists. For example, the location information about the yellow padding may include information about a first sub-space (e.g., a first room (830)) as the sub-space where the yellow padding is located, and a location where the yellow padding is placed within the first sub-space (e.g., a closet (820)). According to one example, the information about the yellow padding may be included in map information about the driving space (810).

[0145] According to one example, the electronic device (100) can identify whether the target user, the daughter, moves to the first room (830) when the current time reaches the target time of 8:00 AM. According to one example, the electronic device (100) can identify that the first condition is satisfied if it is identified that the daughter moves to the first room (830) based on the acquired sensing data. According to one example, the electronic device (100) can control the driving unit (120) to move to the first location (e.g., closet (820)) associated with the notification request when the first condition is satisfied. According to one example, the electronic device (100) can provide information including "It is cold in the morning, so wear a yellow padded jacket" as a notification associated with the notification request through the projection unit (130). For example, the electronic device (100) can display the above-mentioned information on the closet (820) or on the floor through the projection unit (130).

[0146] According to one example, the electronic device (100) can identify feedback information related to the notification request after the notification is provided. For example, the electronic device (100) can determine whether the daughter is wearing a yellow padded jacket. If the electronic device (100) determines that the daughter is not wearing a yellow padded jacket or that there is no change in the position of the yellow padded jacket, it can identify that the task has not been completed. The electronic device (100) can provide information related to the feedback to the input user (e.g., the mother). Alternatively, the electronic device (100) may provide the notification again to the target user.

[0147] For example, if the first condition is not met or the task is not completed, the electronic device (100) may identify the location of the target user, move to the identified location, and provide a notification to the target user.

[0148] Alternatively, according to one example, the electronic device (100) may identify that the first condition is not satisfied if, based on the acquired sensing data, it is identified that the daughter has not moved to the first room (830) for more than a preset time even though the target time has been reached. According to one example, if the first condition is not satisfied, the electronic device (100) may provide guidance information related to a notification request to the daughter. For example, the electronic device (100) may move to a first area within a preset distance from the daughter and provide a notification to the daughter through the projection unit (130).

[0149] Alternatively, according to one example, the electronic device (100) may provide a notification based on the target user's operational state or operational pattern. For example, a case where the first condition is not met may be assumed. The electronic device (100) may identify that the target user is currently sleeping in the master bedroom based on the target user's tracking information. The electronic device (100) may move to the master bedroom and provide a notification based on the identified operational state of the target user. Alternatively, for example, the electronic device (100) may identify the operational pattern of the target user going out at a specific time (e.g., 7 PM) based on the target user's tracking information. If the target time is 7 PM, the electronic device (100) may identify the target location as the 'front door' based on the target user's operational pattern. If the electronic device (100) identifies that the target user is moving to the front door, it may move to the front door and provide a notification.

[0150] FIG. 9 is a diagram illustrating a method for identifying whether a task according to one embodiment has been completed.

[0151] Referring to FIG. 9, according to one embodiment, the electronic device (100) can identify whether a task related to a notification request has been completed.

[0152] For example, one can assume a case where the notification request is 'Tell my son to take the flowerpot on the veranda to the master bedroom at lunchtime.' The electronic device (100) can identify 'son,' 'lunchtime,' 'flowerpot,' 'veranda,' and 'master bedroom' as keywords corresponding to the notification request. Based on the keywords, the electronic device (100) can identify the target user as the son (90), identify the target time as lunchtime (e.g., 12:00 PM), and identify the target location as the veranda and 'master bedroom.' For example, the electronic device (100) can identify the target object as the 'flowerpot.' For example, the target object may be an object related to the notification request among the objects existing within the driving space (910).

[0153] According to one example, the electronic device (100) can identify whether the task related to the notification request has been completed after the notification has been provided to the son (90). For example, the electronic device (100) can identify whether a flowerpot (930) exists at the target location, the veranda, based on sensing data obtained through at least one sensor (110) after a preset time has elapsed since the notification was provided. If the flowerpot (930) does not exist at the target location, the veranda, the electronic device (100) can move to the target location, the master bedroom, and identify whether the flowerpot (930) exists in the master bedroom. If the flowerpot (930) exists in the space within the master bedroom, the electronic device (100) can identify that the task related to the notification request has been completed.

[0154] Alternatively, for example, a case can be assumed where the notification request is 'tell my son to take out side dishes from the refrigerator (920) and eat them at lunch.' The electronic device (100) can identify 'son,' 'lunch,' 'refrigerator,' and 'side dishes' as keywords corresponding to the notification request. Based on the keywords, the electronic device (100) can identify the target user as the son (90), the target time as lunchtime, the target location as the 'refrigerator (920),' and the target object as the 'side dishes.'

[0155] According to one example, the electronic device (100) can identify whether a task has been completed based on at least one of the sensing data obtained through an external device and the sensing data obtained through at least one sensor (110) after a notification is provided to the son (90). For example, the electronic device (100) can obtain sensing data related to the closing or opening of the door from the refrigerator (920). The electronic device (100) can identify that the task has been completed if sensing data corresponding to the opening of the door is obtained from the refrigerator (920) within a preset time after the notification is provided to the son (90).

[0156] Alternatively, for example, a case can be assumed where the notification request is 'tell my son to fold and store the clothes when the laundry is finished.' The electronic device (100) can identify 'son,' 'laundry finished,' 'clothes,' and 'store' as keywords corresponding to the notification request. Based on the keywords, the electronic device (100) can identify the target user as the son (90) and the target location as the 'washing machine' and 'closet.' The electronic device (100) can identify the target time as the 'time when the laundry is finished.'

[0157] According to one example, the electronic device (100) may obtain information regarding the end time of washing from the washing machine. Alternatively, according to one example, the electronic device (100) may obtain information regarding the opening or closing of the washing machine door after the end of washing based on sensing data obtained from the washing machine. According to one example, the electronic device (100) may identify whether the first condition is satisfied based on information regarding the end time of washing. According to one example, the electronic device (100) may identify whether the task is completed based on sensing data obtained from the washing machine.

[0158] Returning to FIG. 2, according to one embodiment, the processor (140) can identify a first device among at least one IoT (internet of things) device present in the driving space.

[0159] According to one example, the electronic device (100) may obtain information about at least one IoT device present in the driving space from an external device (e.g., a server). According to one example, the information about at least one IoT device may be identification information of each device and location information of each device within the driving space. Alternatively, according to one example, the electronic device (100) may be in a state of communication connection with each of at least one IoT device and may obtain information from each of at least one IoT device. According to one example, the first device may be a device including a display.

[0160] According to one example, the electronic device (100) can identify a first device that is closest to the target user based on information about at least one IoT device.

[0161] According to one embodiment, the processor (140) may transmit information about the notification to the first device through a communication circuit (e.g., the communication circuit (180) of FIG. 10). According to one example, the electronic device (100) may transmit information about the notification to the first device through a communication circuit when the first device is identified. According to one example, the notification may be provided through the first device. According to one example, the first device may provide information related to the notification through a display or output a voice related to the notification through a speaker.

[0162] Meanwhile, according to one embodiment, there may be various embodiments in which the electronic device (100) performs an operation corresponding to a user voice signal received through a microphone.

[0163] For example, the electronic device (100) can control the display based on a user voice signal received through a microphone. For example, when a user voice signal for displaying content A is received, the electronic device (100) can control the display to display content A.

[0164] As another example, the electronic device (100) can control an external display device connected to the electronic device (100) based on a user voice signal received through a microphone. Specifically, the electronic device (100) can generate a control signal to control the external display device so that an action corresponding to the user voice signal is performed on the external display device, and can transmit the generated control signal to the external display device. Here, the electronic device (100) can store a remote control application for controlling the external display device. And, the electronic device (100) can transmit the generated control signal to the external display device using at least one communication method among Bluetooth, Wi-Fi, or infrared. For example, when a user voice signal for displaying content A is received, the electronic device (100) can transmit a control signal to the external display device to control content A to be displayed on the external display device. Here, the electronic device (100) may refer to various terminal devices capable of installing a remote control application, such as a smartphone or an AI speaker.

[0165] As another example, the electronic device (100) may use a remote control device to control an external display device connected to the electronic device (100) based on a user voice signal received through a microphone. Specifically, the electronic device (100) may transmit a control signal to the remote control device to control the external display device so that an operation corresponding to the user voice signal is performed on the external display device. Then, the remote control device may transmit the control signal received from the electronic device (100) to the external display device. For example, when a user voice signal for displaying content A is received, the electronic device (100) transmits a control signal to the remote control device to control the display of content A on the external display device, and the remote control device transmits the received control signal to the external display device.

[0166] Meanwhile, the electronic device (100) can receive user voice signals in various ways.

[0167] According to one embodiment, the electronic device (100) can receive a user voice signal through a microphone included in the electronic device (100).

[0168] According to another embodiment, the electronic device (100) may receive a user voice signal from an external device including a microphone. Here, the external device may refer to a remote control device or a smartphone, etc. Here, the received user voice signal may be a digital voice signal, but may be an analog voice signal depending on the implementation example. The electronic device (100) may receive the user voice signal through a wireless communication method such as Bluetooth or Wi-Fi.

[0169] Meanwhile, the electronic device (100) can convert user voice signals in various ways.

[0170] According to one embodiment, the electronic device (100) can obtain text information corresponding to a user voice signal from an external server. Specifically, the electronic device (100) can transmit a user voice signal (audio signal or digital signal) to an external server. Here, the external server may refer to a speech recognition server. Here, the speech recognition server can convert the user voice signal into text information using STT (Speech To Text). Then, the external server can transmit the text information corresponding to the converted user voice signal to the electronic device (100).

[0171] According to another embodiment, the electronic device (100) may independently acquire text information corresponding to a user voice signal. Specifically, the electronic device (100) may directly apply a Speech To Text (STT) function to a digital voice signal to convert it into text information and transmit the converted text information to an external server.

[0172] Meanwhile, an external server can transmit information to the electronic device (100) in various ways.

[0173] According to one embodiment, an external server may transmit text information corresponding to a user voice signal to an electronic device (100). Specifically, the external server may be a server that performs a voice recognition function of converting a user voice signal into text information.

[0174] According to another embodiment, an external server may transmit at least one of text information corresponding to a user voice signal or search result information corresponding to text information to an electronic device (100). Specifically, the external server may be a server that performs a search result providing function that provides search result information corresponding to text information, in addition to a voice recognition function that converts a user voice signal into text information. As an example, the external server may be a server that performs both a voice recognition function and a search result providing function. As another example, the external server may perform only a voice recognition function, and the search result providing function may be performed on a separate server. The external server may transmit text information to a separate server to obtain search results and obtain search results corresponding to text information from the separate server.

[0175] Meanwhile, the electronic device (100) can communicate with external devices and external servers in various ways.

[0176] According to one embodiment, a communication module for communication with an external device and an external server can be implemented in the same way. For example, the electronic device (100) communicates with the external device using a Bluetooth module, and the external server can also communicate using a Bluetooth module.

[0177] According to another embodiment, a communication module for communication with an external device and an external server may be implemented separately. For example, the electronic device (100) may communicate with an external device using a Bluetooth module and with an external server using an Ethernet modem or a Wi-Fi module.

[0178] FIG. 10 is a block diagram showing the detailed configuration of an electronic device according to one embodiment.

[0179] According to FIG. 10, the electronic device (100') may include at least one sensor (110), a driving unit (120), a projection unit (130), at least one processor (140), a memory (150), a display (160), a user interface (170), a communication circuit (180), a speaker (190), a microphone (195), and an input / output interface (196). A detailed description of the configurations shown in FIG. 10 that overlap with the configuration shown in FIG. 2 will be omitted.

[0180] The display (160) may be implemented as a display including a self-emissive element or as a display including a non-emissive element and a backlight. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, or a QLED (Quantum dot light-emitting diodes). The display (160) may also include a driving circuit, a backlight unit, etc., which can be implemented in the form of an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (160) may be implemented as a touch screen combined with a touch sensor, a flexible display, a rollable display, a 3D display, a display in which a plurality of display modules are physically connected, etc. The processor (140) can control the display (160) to output an output image obtained according to the various embodiments described above. Here, the output image may be a high-resolution image of 4K or 8K or higher. According to one embodiment, the output image may be a game image.

[0181] According to one embodiment, the display (160) may include a plurality of haptic elements. The haptic elements may be implemented as motors to provide haptic feedback (e.g., vibration feedback) to a user, but are not limited thereto. According to one example, the display (160) may include a predetermined number of haptic elements. For example, the display (160) may include a predetermined number of haptic elements corresponding to a predetermined number of sub-regions of the display, but is not limited thereto, and it is obvious that the display may include a number of haptic elements different from the number of sub-regions corresponding to the display.

[0182] The user interface (170) is a configuration for the electronic device (100') to perform interaction with the user. For example, the user interface (170) may include at least one of a touch sensor, a motion sensor, a button, a jog dial, a switch, a microphone, or a speaker, but is not limited thereto.

[0183] The communication circuit (180) can input and output various types of data. For example, the communication circuit (180) can transmit and receive various types of data to and from an external device (e.g., source device), an external storage medium (e.g., USB memory), an external server (e.g., web hard drive) through communication methods such as AP-based Wi-Fi (Wi-Fi, Wireless LAN network), Bluetooth, Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), Optical, Coaxial, etc.

[0184] According to one example, the communication circuit (180) may include a BLE (Bluetooth Low Energy) module. BLE refers to Bluetooth technology capable of transmitting and receiving low-power, low-capacity data in a 2.4 GHz frequency band with a range of about 10 m. However, it is not limited thereto, and the communication circuit (180) may include a Wi-Fi communication module. That is, the communication circuit (180) may include at least one of a BLE (Bluetooth Low Energy) module or a Wi-Fi communication module.

[0185] According to one embodiment, the speaker (190) may be composed of a tweeter for reproducing high-frequency sound, a midrange for reproducing mid-frequency sound, a woofer for reproducing low-frequency sound, a subwoofer for reproducing ultra-low-frequency sound, an enclosure for controlling resonance, and a crossover network for dividing the frequency of an electrical signal input to the speaker into bands.

[0186] According to one embodiment, the speaker (190) can output an acoustic signal to the outside of the electronic device (100'). The speaker (190) can output multimedia playback, recording playback, various notification sounds, voice messages, etc. The electronic device (100') may include an audio output device such as the speaker (190), but may include an output device such as an audio output terminal. In particular, the speaker (190) can provide acquired information, information processed or produced based on the acquired information, response results to user voice or operation results, etc., in the form of voice.

[0187] A microphone (195) may refer to a module that acquires sound and converts it into an electrical signal, and may be a condenser microphone, a ribbon microphone, a moving coil microphone, a piezoelectric element microphone, a carbon microphone, or a MEMS (Micro Electro Mechanical System) microphone. Additionally, it may be implemented in omnidirectional, bidirectional, unidirectional, subcardioid, supercardioid, or hypercardioid modes. According to one embodiment, an electronic device (100') may include a microphone (195) and an inner microphone, and the microphone (195) may be a microphone located relatively outside the body. According to one example, the electronic device (100') may acquire an audio signal including external noise through the microphone (195). According to one embodiment, the microphone (195) may be positioned in a direction opposite to the direction in which the speaker (190) emits sound.

[0188] The input / output interface (196) may be any one of the following interfaces: HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), or DVI (Digital Visual Interface). The input / output interface (196) may input and output at least one of audio and video signals. Depending on the implementation example, the input / output interface (196) may include separate ports for inputting and outputting only audio signals and for inputting and outputting only video signals, or it may be implemented as a single port for inputting and outputting both audio and video signals. Meanwhile, the electronic device (100') may transmit at least one of the audio and video signals to an external device (e.g., an external display device or an external speaker) through the input / output interface (196). Specifically, an output port included in the input / output interface (196) can be connected to an external device, and the electronic device (100') (100') can transmit at least one of audio and video signals to the external device through the output port.

[0189] According to the example described above, when an electronic device (100') confirms a notification request for a specific user, it can provide a notification to the user by taking into account the information included in the notification request. The electronic device (100') can move to a location related to the notification request and provide the notification directly to the user. The electronic device (100') can provide various types of notifications to the user. Accordingly, content related to the notification can be efficiently provided to the user.

[0190] Meanwhile, according to the exemplary embodiments of the present disclosure, the various embodiments described above may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include a display device (e.g., a display device (A)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code provided or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means only that the storage medium does not contain a signal and is tangible, and does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0191] Additionally, according to one embodiment, the method according to the various embodiments described above may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed online in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or provided on a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0192] Additionally, each component (e.g., module or program) according to the various embodiments described above may be composed of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in the various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the functions performed by each of the respective components prior to integration in the same or similar manner. The operations performed by the module, program, or other components according to the various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations added.

[0193] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device that travels in a driving space, At least one sensor; Driving unit; Projection unit; At least one processor including processing circuitry; and Includes memory for storing instructions; and When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Receive data corresponding to the notification request, and According to the data corresponding to the received notification request, if a condition based on at least one of context information and tracking information obtained through at least one sensor corresponds to a first condition corresponding to the notification request, the driving unit is controlled to move to a first position associated with the notification request. An electronic device that provides a notification corresponding to the notification request through the projection unit.

2. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Identify whether the condition corresponds to a first condition corresponding to the notification request based on at least one keyword included in the notification request, and The above at least one keyword is, An electronic device comprising information related to at least one of a target user, a target time, and a target location corresponding to the above notification request.

3. In Paragraph 1, The above tracking information is, Includes the user's location information and the user's behavior information, and When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that identifies whether the condition corresponds to a first condition corresponding to the notification request based on at least one of the context information, the user's location information based on sensing data obtained through at least one sensor, and the user's behavior information.

4. In Paragraph 3, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, When movement of the target user to the target location is identified based on the location information and behavior information of the target user corresponding to the above notification request, the driving unit is controlled to move to the first location, and An electronic device that provides a notification corresponding to the notification request through the projection unit.

5. In Paragraph 4, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that identifies the movement of the target user to the target location when, based on the above context information, the current time is identified as corresponding to the target time corresponding to the above notification request.

6. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, While performing a function corresponding to a patrol of the above driving space, information about at least one object included in the driving space is obtained based on sensing data obtained through at least one sensor, and An electronic device that identifies whether the condition corresponds to a first condition corresponding to the notification request based on information about at least one object obtained above.

7. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that, when a notification corresponding to the above notification request is provided, identifies feedback information corresponding to the notification request based on at least one of the tracking information, sensing data obtained through at least one external device, and sensing data obtained through at least one sensor.

8. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that, if the above condition does not correspond to the first condition corresponding to the notification request, provides guide information corresponding to the notification request through the projection unit.

9. In Paragraph 1, The above-mentioned at least one sensor is, Includes a camera sensor, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Based on sensing data acquired through the at least one sensor and identification information corresponding to each of the at least one user within the driving space, a target user among the at least one user who corresponds to the notification request is identified, and An electronic device for obtaining tracking information of the identified target user.

10. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that identifies whether a task corresponding to a notification request is completed based on at least one of sensing data obtained through at least one external device and sensing data obtained through at least one sensor.

11. A method of operation of an electronic device traveling in a driving space, A step of receiving data corresponding to a notification request; A step of controlling a driving unit to move to a first position associated with the notification request if, according to data corresponding to the received notification request, a condition based on at least one of context information and tracking information obtained through at least one sensor corresponds to a first condition corresponding to the notification request; and A method of operation comprising the step of providing a notification corresponding to the notification request through a projection unit.

12. In Paragraph 11, The method includes the step of identifying whether the condition corresponds to a first condition corresponding to the notification request based on at least one keyword included in the notification request; The above at least one keyword is, A method of operation comprising information related to at least one of a target user, a target time, and a target location corresponding to the above notification request.

13. In Paragraph 11, The above tracking information is, Includes the user's location information and the user's behavior information, and The step of identifying whether the above first condition is satisfied is, A method of operation for identifying whether the condition corresponds to a first condition corresponding to the notification request based on at least one of the context information, the location information of the user based on sensing data obtained through at least one sensor, and the behavior information of the user.

14. In Paragraph 13, A step of controlling the driving unit to move to the first location when the movement of the target user to the target location is identified based on the location information and behavior information of the target user corresponding to the above notification request; and A method of operation comprising the step of providing a notification corresponding to the notification request through the projection unit.

15. In a storage medium storing computer-readable instructions, the instructions, when executed by at least one processor of an electronic device traveling in a travel space, cause the electronic device, Receive data corresponding to the notification request, and According to the data corresponding to the received notification request, if a condition based on at least one of context information and tracking information obtained through at least one sensor corresponds to a first condition corresponding to the notification request, the driving unit is controlled to move to a first position associated with the notification request. A storage medium that causes to provide a notification corresponding to the above notification request through a projection unit.

Images