Electronic device and operation method thereof

Abstract

An electronic device according to one embodiment of the present disclosure comprises: a memory for storing one or more instructions; and at least one processor for executing the one or more instructions, wherein the at least one processor can, when audio received through a microphone corresponds to a preset condition, identify the presence of a user within a sensing range of Wi-Fi CSI, and, when the presence of the user within the sensing range of the Wi-Fi CSI is identified, identify whether the user is present within a first sensing range based on ultrasonic waves.

Description

Electronic devices and their operating methods

[0001] The present disclosure relates to an electronic device capable of detecting the approach of a user and a method of operating the same.

[0002] Advances in artificial intelligence and electronic device technology are driving further growth in the smart home device market. Among these smart home devices, the market for screen-equipped smart displays is particularly notable. Smart displays can visually provide users with useful information, such as their current location, weather, and date, provide immediate responses to user queries, and offer a variety of content through media applications. As these smart displays are developed and produced in various screen sizes and increasingly larger, development is also underway to integrate smart display technology into existing televisions, offering smart display functionality.

[0003] Meanwhile, smart displays with small screens don't consume much power, allowing them to remain on for extended periods of time, allowing users to instantly access their features. However, TVs with relatively large screens consume significantly more power. Furthermore, considering the warranty period for display panels, keeping the TV screen always on can be inefficient.

[0004] In one embodiment of the present disclosure, an electronic device may include a memory storing one or more instructions and at least one processor executing one or more instructions. In one embodiment of the present disclosure, when audio received through a microphone corresponds to a preset condition, the electronic device may identify the presence of a user within a detection range of wireless fidelity (WiFi) channel state information (CSI). In one embodiment of the present disclosure, when the presence of a user within the detection range of the WiFi CSI is identified, the electronic device may identify whether a user is present within a first detection range based on ultrasound, by having the at least one processor execute one or more instructions.

[0005] In one embodiment of the present disclosure, the method of operating an electronic device may include a step of identifying the presence of a user within a detection range based on WiFi CSI when audio received through a microphone corresponds to a preset condition. In one embodiment of the present disclosure, the method of operating an electronic device may include a step of identifying the presence of a user within a first detection range based on ultrasound based on the presence of a user within the detection range based on WiFi CSI, based on the presence of a user within the detection range based on ambient sound being human sound.

[0006] In one embodiment of the present disclosure, a program for performing an operating method of an electronic device on a computer can be recorded on a computer-readable recording medium.

[0007] FIG. 1 illustrates an example in which an electronic device detects a user's approach and provides useful information in one embodiment of the present disclosure.

[0008] FIG. 2 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment of the present disclosure.

[0009] FIG. 3 is a drawing showing an example of a detection range of an electronic device according to one embodiment of the present disclosure.

[0010] FIG. 4 is a diagram showing an example of the operation of an electronic device when an ambient sound is identified by a person in one embodiment of the present disclosure.

[0011] FIG. 5 is a diagram illustrating an example of an operation of an electronic device when an ambient sound caused by a person is identified and the presence of a user within a detection range based on WiFi CSI is identified in one embodiment of the present disclosure.

[0012] FIG. 6 is a diagram illustrating an example of an operation of an electronic device when an ambient sound caused by a person is identified, the presence of a user within a detection range based on WiFi CSI is identified, and the presence of a user within a detection range based on ultrasound is identified in one embodiment of the present disclosure.

[0013] FIG. 7A is a diagram illustrating an example of a first state of an electronic device according to one embodiment of the present disclosure.

[0014] FIG. 7b is a diagram illustrating an example of a second state of an electronic device according to one embodiment of the present disclosure.

[0015] FIG. 8 is a drawing showing an example of a method of operating an electronic device according to one embodiment of the present disclosure.

[0016] FIG. 9 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment of the present disclosure.

[0017] FIG. 10 is a diagram illustrating an example of an operation of an electronic device when a user is located outside a WiFi CSI-based detection range in one embodiment of the present disclosure.

[0018] FIG. 11 is a diagram showing an example of a configuration of an electronic device according to one embodiment of the present disclosure.

[0019] FIG. 12 is a diagram showing an example of a configuration of an electronic device and an input / output device connected to the electronic device according to one embodiment of the present disclosure.

[0020] This disclosure may be subject to various modifications and various embodiments. Specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the disclosure.

[0021] When describing embodiments, detailed descriptions of related known technologies are omitted if they are deemed to unnecessarily obscure the main point. Furthermore, numbers (e.g., "first," "second," etc.) used in the description of embodiments are merely identifiers used to distinguish one component from another. Furthermore, unless the context clearly indicates otherwise, the singular forms "a," "an," and "the" are understood to include plural references.

[0022] It should also be understood that the blocks and combinations of flowcharts in each flowchart can be executed by one or more computer programs containing computer-executable instructions. The one or more computer programs may be stored entirely in a single memory, or may be stored in separate portions across multiple different memories.

[0023] All functions or operations described in this document may be performed by a single processor or a combination of processors. A single processor or a combination of processors is a circuitry that performs processing, and may include circuitry such as an Application Processor (AP), a Communication Processor (CP), a Graphical Processing Unit (GPU), a Neural Processing Unit (NPU), a Microprocessor Unit (MPU), a System on Chip (SoC), or an Integrated Chip (IC).

[0024] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail to facilitate implementation by those skilled in the art. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. Before proceeding with a detailed description of the invention, the terms used herein are defined or understood as follows.

[0025] When a component is referred to herein as being "connected" or "connected" to another component, it should be understood that the component may be directly connected to or connected to the other component, but may also be connected or connected via another component in between, unless otherwise specifically stated. Furthermore, "connection" may include a wireless connection or a wired connection.

[0026] In addition, in this specification, components expressed as 'unit', 'module', etc. may be two or more components combined into one component, or one component may be divided into two or more components with more detailed functions. In addition, each component described below may additionally perform some or all of the functions performed by other components in addition to its own main function, and of course, some of the main functions performed by each component may be performed exclusively by other components.

[0027] In the present disclosure, the expression 'at least one of a, b, or c' can refer to 'a', 'b', 'c', 'a and b', 'a and c', 'b and c', 'all of a, b, and c', or variations thereof. In the present disclosure, the expression 'a or b' can refer to 'a', 'b', 'a and b', or variations thereof. In the present disclosure, the expression 'a (or, b, c)' can refer to 'a', 'b', 'c', 'a and b', 'a and c', 'b and c', 'all of a, b, and c', or variations thereof.

[0028] In the present disclosure, "identification" may include identifying target information by directly determining, directly generating, or acquiring or receiving target information from an external source. For example, "an electronic device identifying A" may include identifying A by the electronic device directly determining or generating A. For example, "an electronic device identifying A" may include identifying A by receiving, being provided with, or acquiring A (or information about A) from another external electronic device, server, or system.

[0029] FIG. 1 illustrates an example in which an electronic device detects a user's approach and provides useful information in one embodiment of the present disclosure.

[0030] In one embodiment of the present disclosure, when a user (110) approaches the electronic device (100), the electronic device (100) may be configured to provide a specific function or a specific service to the user (110). When the user (110) is not using the electronic device (100) or is not in close proximity to the electronic device (100), the electronic device (100) may maintain a low-power state with low power consumption. Thereafter, when the electronic device (100) detects or recognizes the approach of the user (110), the electronic device (100) may exit the low-power state and provide a service to the user (110). For example, the electronic device (100) may detect the approach of the user (110) and execute a specific application. For example, the electronic device (100) may detect the approach of the user (110) and output a wired or wireless broadcast through at least one of a display or a speaker.

[0031] In one embodiment of the present disclosure, the electronic device (100) may detect the approach of a user (110) and provide specific information to the user (110). For example, the electronic device (100) may output visual elements (e.g., images, text, graphics, user interface (UI), etc.) representing specific information through a display. For example, the electronic device (100) may display visual elements on the display. For example, the electronic device (100) may output auditory elements (e.g., voice, notification, etc.) representing specific information through a speaker. For example, the electronic device (100) may output specific information in a form recognizable by the user (110) (e.g., haptic, document, etc.) through another output device (e.g., vibration generating device, printer, projector, etc.).

[0032] In one embodiment of the present disclosure, the type of specific information output upon user (110) access may be predetermined, set, or defined. For example, the type of specific information may be set by user input. In one embodiment of the present disclosure, the type of specific information output upon user (110) access may be adaptively determined. For example, the type of specific information may change depending on the environment at the time of output.

[0033] In one embodiment of the present disclosure, specific information output upon user (110) access may include at least one of real-time information and pre-stored information. For example, the specific information may include information input, generated, or collected in real time. For example, the specific information may include information pre-stored in memory. For example, the specific information may include information input, generated, collected, or updated periodically.

[0034] For example, referring to FIG. 1, the electronic device (100) can output date information, time information, weather information, smart home control information, and memo information through a display. For example, the electronic device (100) is a display device (e.g., a TV device) that includes or has a display built in, and can display date information, time information, weather information, smart home control information, and memo information on the display. For example, the electronic device (100) can display the output information on the display in the form of visual elements such as letters, numbers, identifiers, images, icons, and graphics.

[0035] In the present disclosure, various methods may be provided for an electronic device (100) to detect or recognize the approach of a user (110). For example, the electronic device (100) may detect or recognize the approach of a user (110) using a sensor, module, or device built into, included in, or connected to the electronic device (100).

[0036] In one embodiment of the present disclosure, the electronic device (100) can detect or recognize the approach of a user (110) by using a microphone (hereinafter, “microphone”) built into, included in, or connected to the electronic device (100). For example, the electronic device (100) can identify that the user (110) is located nearby by using ambient sounds (e.g., voices, footsteps, rustling sounds, etc.) input through the microphone. However, when only ambient sounds input through the microphone are used, the electronic device (100) cannot identify how close the user (110) is, and may incorrectly identify that the user (110) is located nearby even when sounds are generated by something other than the user.

[0037] In one embodiment of the present disclosure, the electronic device (100) can detect or recognize the approach of a user (110) by using a speaker and a microphone built into, included in, or connected to the electronic device (100). For example, the electronic device (100) can identify that a user (110) is located nearby by using a phase shift between an ultrasonic signal (e.g., 18 to 20 KHz) output (or transmitted) through the speaker and an ultrasonic signal input (or received) through the microphone. Ultrasonic waves are sound waves in the inaudible range, but when the electronic device (100) constantly transmits and receives ultrasonic signals, some younger users or pets may perceive the ultrasonic waves auditorily, and thus, may experience auditory stress.

[0038] In one embodiment of the present disclosure, an electronic device (100) may detect or recognize the approach of a user (110) using a motion sensor. In this case, electronic devices (100) that previously did not need to have a built-in or included motion sensor, or be connected to a motion sensor via wired or wirelessly, must additionally have a built-in or included motion sensor, or be connected to a motion sensor via wired or wirelessly, solely for the purpose of detecting a user (110). Therefore, additional costs may be incurred.

[0039] In one embodiment of the present disclosure, a method may be provided for an electronic device (100) to detect or recognize the approach of a user (110) using WiFi (wireless fidelity) channel state information (CSI). WiFi CSI is channel state information and may include parameters measured while a signal is transmitted in a WiFi-based wireless channel. For example, WiFi CSI may include signal strength, signal phase, signal arrival time, beamforming feedback, channel interference degree, multipath, etc. One or more embodiments in which the electronic device (100) detects or recognizes the approach of a user (110) using WiFi CSI may be described later in the description of FIGS. 2 to 12 .

[0040] FIG. 2 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment of the present disclosure.

[0041] In explaining Fig. 2, any explanation that overlaps with the explanation given above in Fig. 1 may be omitted.

[0042] FIG. 2 may illustrate an example of a method by which an electronic device detects or recognizes a user's approach. Referring to FIG. 2 , a method (200) according to one embodiment of the present disclosure may include steps 210 and 220. In one embodiment of the present disclosure, steps 210 and 220 of the method (200) may be executed by at least one processor included in the electronic device. The method (200) for operating the electronic device is not limited to that illustrated in FIG. 2 , and in one or more embodiments, additional steps not illustrated in FIG. 2 may be included, or some steps may be omitted.

[0043] In step 210, the electronic device can identify the presence of a user within the detection range of the WiFi CSI if the audio received through the microphone corresponds to a preset condition. For example, the detection range of the WiFi CSI may include a range in which the presence or absence of an object (e.g., a person) or movement can be detected using the WiFi CSI. In the present disclosure, the terms “detection range based on WiFi CSI,” “detection range of WiFi CSI,” or “detection range of WiFi CSI” may be used interchangeably.

[0044] In one embodiment of the present disclosure, an electronic device may receive audio via a microphone. For example, the electronic device may identify whether the received audio corresponds to a preset condition. For example, the preset condition may include conditions regarding whether the audio corresponds to ambient sounds of the electronic device, the volume, frequency, state of the audio, and the class of ambient sounds to which the audio corresponds. For example, the electronic device may identify whether the received audio corresponds to ambient sounds of the electronic device.

[0045] In one embodiment of the present disclosure, an electronic device can identify whether an ambient sound input through a microphone is a human sound. For example, if the received audio corresponds to an ambient sound, the electronic device can identify whether the ambient sound is a human sound based on at least one class associated with the ambient sound. For example, the electronic device can record the ambient sound (or audio received through a microphone) at regular intervals to identify whether the ambient sound is a human sound. To this end, power can be supplied to the microphone. The ambient sound can be converted into an electrical signal (e.g., electrical data) by the microphone and input to the electronic device. For example, the ambient sound can include sounds in the audible frequency band or sounds in the inaudible frequency band.

[0046] In one embodiment of the present disclosure, a microphone may be mounted, installed, built-in, or included in an electronic device as a component, input device, or module of the electronic device. In one embodiment of the present disclosure, the microphone may be included as a component in a device separate from the electronic device or may be a separate device itself, and the separate device may be connected to the electronic device via wired or wirelessly. In this case, the electronic device may receive data corresponding to ambient sound from the microphone (or a device including a microphone). For example, the microphone may be included in a remote control device (e.g., a Bluetooth remote control, a WiFi remote control) that transmits and receives data with the electronic device via wired or wirelessly. In this case, the remote control device may convert the input ambient sound from an analog signal to a digital signal and transmit it to the electronic device via Bluetooth, infrared, or WiFi communication. For example, the microphone may be included in a client device (e.g., a smartphone, an AI speaker) that transmits and receives data with the electronic device via wired or wirelessly. An application that controls the electronic device may be installed or executed on the client device. In this case, the client device can convert the incoming ambient sound from an analog signal to a digital signal and transmit it to an electronic device via wireless or wired communication.

[0047] In one embodiment of the present disclosure, an electronic device may obtain information about a class of ambient sound (or a class corresponding to the ambient sound, a class related to the ambient sound) (e.g., a sound class, an audio class) to determine whether the ambient sound is a sound made by a person. The ambient sound may be classified into one or more classes using a learning model, a classification algorithm, or the like. For example, the electronic device may classify the ambient sound into at least one class among a plurality of classes, and may determine whether the ambient sound is a sound made by a person using the information about the class of the ambient sound. For example, the electronic device may receive information about the class of the ambient sound from an external device, a server, or a system, and may determine whether the ambient sound is a sound made by a person. In this case, the external device, the server, or the system may classify the ambient sound into at least one class among a plurality of classes, and provide information about the class of the ambient sound to the electronic device.

[0048] For example, if an ambient sound is classified into a class corresponding to a horn sound, the electronic device can determine that the ambient sound is not a sound caused by a human being. For example, if an ambient sound is classified into a class corresponding to footsteps, the electronic device can determine that the ambient sound is a sound caused by a human being. To this end, information regarding whether each of the multiple classes corresponds to a sound caused by a human being or a sound caused by something other than a human being can be stored in the memory of the electronic device.

[0049] In one embodiment of the present disclosure, an electronic device can identify whether a microphone-input ambient sound is a human sound by receiving information from an external device, server, or system regarding whether the microphone-input ambient sound is a human sound. In this case, the external device, server, or system can determine whether the microphone-input ambient sound is a human sound and provide the determination result to the electronic device.

[0050] In one embodiment of the present disclosure, an electronic device can identify the presence of a user within a detection range based on WiFi CSI. In one embodiment of the present disclosure, the electronic device can identify the presence of a user within the detection range based on WiFi CSI based on identifying that an ambient sound is a sound caused by a person (or that the ambient sound is not caused by something other than a person). For example, after identifying that an ambient sound is a sound caused by a person, the electronic device can identify the presence of a user within the detection range based on WiFi CSI.

[0051] In one embodiment of the present disclosure, an electronic device may acquire WiFi CSI and use the WiFi CSI to determine whether a user is present within a detection range based on the WiFi CSI. For example, the electronic device may transmit and receive WiFi signals (e.g., data) with an access point (AP), a soft AP, or a router device through a WiFi channel, and may acquire WiFi CSI for such WiFi channel. The WiFi CSI may be acquired periodically, aperiodically, or continuously. The WiFi CSI may dynamically change due to external factors such as changes in the physical environment, changes in the distance between a WiFi receiver and a transmitter, interference from external radio waves, or weather. Changes in the physical environment may include the presence, location, type, or movement of objects within a space where WiFi signals are transmitted and received.

[0052] For example, an electronic device can determine whether a user is present within a detection range based on WiFi CSI based on whether there is a change in WiFi CSI, the type of change, or the degree of change. For example, an electronic device can determine whether a user is present within a detection range based on WiFi CSI using a machine learning model or algorithm based on the acquired WiFi CSI. For example, an electronic device can obtain information on whether a user is present within a detection range based on WiFi CSI by inputting WiFi CSI into a machine learning model or algorithm. The machine learning model can include a model trained to detect the presence of an object or the movement of an object within a specific detection range (e.g., a target space) by using WiFi CSI as an input.

[0053] In one embodiment of the present disclosure, an electronic device can identify the presence of a user within a WiFi CSI-based detection range by receiving information about the presence of a user within a WiFi CSI-based detection range from an external device, server, or system. In this case, the external device, server, or system can determine whether a user is within the WiFi CSI-based detection range and provide the determination result to the electronic device.

[0054] In one embodiment of the present disclosure, the electronic device may change, or control to change, from a first state in which power consumption is lower than a preset value to a second state in which power consumption is higher than a preset value based on the presence of a user within the detection range of the WiFi CSI. For example, the first state may include a state in which power consumption is lower than a normal state based on at least one of the speaker or the display not operating. For example, the second state may include a state in which power consumption is higher than a normal state based on at least one of the speaker or the display operating. For example, the normal state may include a predefined reference state or a state in which a specific function can be provided. For example, the normal state may include a state in which power consumption is as high as a preset value.

[0055] In step 220, if the presence of a user within the detection range of the WiFi CSI is identified, the electronic device can identify whether a user is present within the first detection range based on ultrasound. For example, if the received audio corresponds to a preset condition and a user is present within the detection range of the WiFi CSI, the electronic device can output an ultrasonic signal of a first intensity through a speaker. For example, the electronic device can identify whether the user is present within the first detection range based on the outputted ultrasonic signal of the first intensity and the ultrasonic signal input through the microphone.

[0056] In one embodiment of the present disclosure, an electronic device can identify the presence of a user within a first ultrasonic detection range based on the presence of a user within the detection range based on WiFi CSI, based on the presence of a user within the ambient sound. For example, if the electronic device identifies the presence of a user within the first ultrasonic detection range based on the presence of a user within the detection range based on WiFi CSI, the electronic device can identify the presence of a user within the first ultrasonic detection range.

[0057] To this end, in one embodiment of the present disclosure, the electronic device may output an ultrasonic signal of a first intensity through a speaker. For example, at least one processor of the electronic device may control a speaker inherent or included in the electronic device to generate the first ultrasonic signal. For example, the electronic device may transmit a signal, information, data, or message requesting or instructing the output of an ultrasonic signal to a speaker device connected to the electronic device via wire or wirelessly. In this case, the speaker device may generate an ultrasonic signal of the first intensity according to the received signal, information, data, or message.

[0058] An ultrasonic signal output through a speaker can be reflected by any object and input to a microphone. In one embodiment of the present disclosure, an electronic device can identify the presence of a user within a first ultrasound-based detection range by using the output ultrasonic signal of the first intensity and the ultrasonic signal input to the microphone. For example, the electronic device can determine whether a user is present within the first ultrasound-based detection range by using the output ultrasonic signal of the first intensity and the ultrasonic signal input to the microphone. For example, the electronic device can determine whether a user is present within the first ultrasound-based detection range by using the phase difference between the ultrasonic signal output through the speaker and the ultrasonic signal input to the microphone.

[0059] In one embodiment of the present disclosure, an electronic device can identify the presence of a user within a first ultrasonic-based detection range by receiving information about the presence of a user within a first ultrasonic-based detection range from an external device, server, or system. In this case, the external device, server, or system can determine whether a user is within the first ultrasonic-based detection range and provide the determination result to the electronic device.

[0060] In one embodiment of the present disclosure, an electronic device may output or control to output a screen corresponding to the presence of a user based on the presence of a user within a first ultrasonic detection range. For example, the electronic device may output a specific screen on a display based on the presence of a user within the first ultrasonic detection range. For example, if the electronic device identifies that a user is present within the first ultrasonic detection range, the electronic device may output a specific screen on the display. For example, at least one processor of the electronic device may control a display inherent or included in the electronic device to output (e.g., display) a specific screen. For example, the electronic device may transmit a signal, information, data, or message requesting or instructing the output of a specific screen to a display device connected to the electronic device by wire or wirelessly. In this case, the display device may output (e.g., display) a specific screen based on the received signal, information, data, or message.

[0061] In one embodiment of the present disclosure, the electronic device can change or control to change from a second state in which power consumption is higher than a preset value to a first state in which power consumption is lower than a preset value based on the absence of a user within a first detection range based on ultrasonic waves.

[0062] Although FIG. 2 illustrates that the electronic device performs step 220 after performing step 210, this is not limited thereto. For example, the electronic device may perform step 210 after performing step 220, or may perform steps 210 and 220 in parallel.

[0063] FIG. 3 is a drawing showing an example of a detection range of an electronic device according to one embodiment of the present disclosure.

[0064] In explaining Fig. 3, any explanation that overlaps with the explanation given above in Fig. 1 or Fig. 2 may be omitted.

[0065] In one embodiment of the present disclosure, an electronic device (e.g., a TV device) (100) can identify the presence of a user within a sound-based detection range (310). For example, the electronic device (100) can detect or determine the presence of a user within the sound-based detection range (310) by using information about ambient sounds input through a microphone (e.g., input sound pressure level, type, frequency, length, characteristics, class, etc. of the ambient sounds). For example, if the ambient sound is a sound made by a person and the input sound pressure level of the ambient sound exceeds a threshold value, the electronic device (100) can determine that a user is present within the sound-based detection range (310).

[0066] In one embodiment of the present disclosure, the sound-based detection range (310) may vary depending on the hardware settings, software settings, or surrounding environment (e.g., surrounding structures, etc.) of the detecting device. For example, the sound-based detection range (310) may be determined by the type, location, and directionality of the microphone or electronic device (100). For example, the sound-based detection range (310) may be determined by the sensitivity level on the hardware or the sensitivity level on the software. For example, the sound-based detection range (310) may vary depending on the surrounding noise environment. For example, in an environment with loud surrounding noise, the sound-based detection range (310) may be reduced.

[0067] Although Fig. 3 illustrates a sound-based detection range (310) as a single range, the sound-based detection range (310) may include multiple detection ranges (e.g., detection ranges according to sound pressure levels of ambient sounds). For example, a wide detection range may be formed for a high sound pressure level, i.e., a loud sound, and a narrow detection range may be formed for a low sound pressure level, i.e., a soft sound. The sound-based detection range (310) illustrated in Fig. 3 may represent a detection range for an average sound pressure level, a medium sound pressure level, a highest frequency sound pressure level, or a highest sound pressure level of sounds that may be generated by a human.

[0068] In one embodiment of the present disclosure, the electronic device (100) can identify the presence of a user within a detection range (320) based on WiFi CSI. WiFi CSI is channel state information of a WiFi wireless signal and may include information on the strength of a WiFi signal, phase information of a WiFi signal, noise level information, etc., and WiFi CSI may change depending on changes in the wireless environment, the presence of other wireless devices, and the presence of obstacles. For example, WiFi CSI may change depending on changes in the structure of a space where WiFi signals are transmitted and received, reflected, refracted, interfered with, or attenuated, the addition or removal of obstacles, or the mobility of a mobile device.

[0069] Accordingly, the electronic device (100) can detect or determine whether a user exists within a WiFi CSI-based detection range (320) by using WiFi CSI. For example, the WiFi CSI can be used after noise is removed. For example, the electronic device (100) can determine whether a user exists within a WiFi CSI-based detection range (320) by using a model trained to detect (or determine) the presence of a person within a WiFi CSI-based detection range (320) with the input of the WiFi CSI. For example, the electronic device (100) can determine whether a user exists within a WiFi CSI-based detection range (320) depending on whether or how much the WiFi CSI changes. For example, if the current WiFi CSI changes by a threshold amount or more from the reference WiFi CSI, the electronic device (100) can determine that a user exists within the WiFi CSI-based detection range (320). The reference WiFi CSI can include WiFi CSI for a state in which a user does not exist.

[0070] In one embodiment of the present disclosure, the electronic device (100) can identify the presence of a user within an ultrasonic-based detection range (330). For example, the electronic device (100) can output (or transmit) an ultrasonic signal through a speaker (or an ultrasonic signal generator), and the output ultrasonic signal can be reflected, refracted, interfered with, scattered, or attenuated and input through a microphone. The electronic device (100) can detect or determine the presence of a user within the ultrasonic-based detection range (330) by using the ultrasonic signal output through the speaker and the ultrasonic signal input through the microphone.

[0071] For example, the electronic device (100) can detect or determine whether a user is present within an ultrasonic-based detection range (330) by using information (e.g., frequency, intensity, speed, phase) about an output ultrasonic signal and an input ultrasonic signal. For example, the electronic device (100) can detect or determine whether a user is present within an ultrasonic-based detection range (330) by using a phase difference, a difference in transmission and reception time, or a difference in intensity between the output ultrasonic signal and the input ultrasonic signal. For example, the electronic device (100) can detect or determine whether a user is present within an ultrasonic-based detection range (330) by using a model learned to detect whether a user is present within an ultrasonic-based detection range (330) by using information about an output ultrasonic signal and information about an input ultrasonic signal as inputs.

[0072] The ultrasonic-based detection range (330) can be determined based on the intensity of the output ultrasonic signal. For example, the stronger the output ultrasonic signal, the wider the ultrasonic-based detection range (330) can be formed. Conversely, the weaker the output ultrasonic signal, the narrower the ultrasonic-based detection range (330) can be formed. Accordingly, the electronic device (100) can output a strong ultrasonic signal through the speaker to detect a user located at a distant location.

[0073] Although each detection range is depicted in an ideal, theoretical, or fixed form in Figure 3, this is not a limitation. For example, the shape or width of each detection range may vary depending on various factors such as spatial structure, obstacles, WiFi signal conditions, ultrasonic signal conditions, and the surrounding environment. For example, each detection range may be variable.

[0074] In Fig. 3, the WiFi CSI-based detection range (320) is illustrated as covering the widest range, and the ultrasonic-based detection range (330) is illustrated as covering the narrowest range. However, as described above, each detection range may vary, and thus is not limited thereto. For example, the sound-based detection range (310) or the ultrasonic-based detection range (330) may cover the widest range. For example, the WiFi CSI-based detection range (320) or the sound-based detection range (310) may cover the narrowest range.

[0075] Although FIG. 3 depicts each detection range in two dimensions on a plan view, this is not a limitation. For example, at least one detection range may include a three-dimensional (e.g., spherical) range.

[0076] FIG. 4 is a diagram showing an example of the operation of an electronic device when an ambient sound is identified by a person in one embodiment of the present disclosure.

[0077] In explaining Fig. 4, any explanation that overlaps with the explanation given above in any one of Figs. 1 to 3 may be omitted.

[0078] FIG. 4 may show an example of the operation of an electronic device (100) in a case where the ambient sound (410) input through a microphone in one embodiment of the present disclosure is a sound by a person, but there is no user within the detection range based on WiFi CSI.

[0079] In one embodiment of the present disclosure, the electronic device (100) can receive ambient sound (410) through a microphone associated with the electronic device (100). The microphone associated with the electronic device (100) may include a sound sensor, a sound input module, or a sound input device capable of converting ambient sound into an electrical signal. For example, the microphone associated with the electronic device (100) may include a microphone built into or included in the electronic device (100). For example, the microphone associated with the electronic device (100) may include a microphone device connected to the electronic device (100) by wire or wirelessly. For example, the microphone associated with the electronic device (100) may include a microphone built into or included in a device connected to the electronic device (100) by wire or wirelessly.

[0080] The fact that ambient sound by a person is input through the microphone, i.e., that the ambient sound (410) is a sound by a person, can be considered as the presence of a user within the sound-based detection range. Therefore, in one embodiment of the present disclosure, the electronic device (100) can identify whether the ambient sound (410) is a sound by a person in order to identify the presence of a user within the sound-based detection range. To this end, for example, the electronic device (100) can extract features from the ambient sound (410) and classify the ambient sound (410) into at least one of a plurality of classes using the extracted features. To classify the ambient sound (410), the electronic device (100) can use a learning model, a statistical model, a classification algorithm, or a signal processing algorithm for classifying input sounds.

[0081] The plurality of classes may include classes corresponding to sounds produced by non-human entities or classes corresponding to sounds produced by humans. For example, one or more of the plurality of classes may be predetermined to correspond to sounds produced by humans. For example, one or more of the plurality of classes may be predetermined to correspond to sounds produced by non-human entities.

[0082] In one embodiment of the present disclosure, if the ambient sound (410) is classified into a class corresponding to a sound made by a person, the electronic device (100) can identify (or determine) the ambient sound (410) as a sound made by a person. In one embodiment of the present disclosure, if the ambient sound (410) is not classified into a class corresponding to a sound made by a non-human entity, the electronic device (100) can identify (or determine) the ambient sound (410) as a sound made by a person. For example, if the ambient sound (410) is classified into a speech sound class, a footstep sound class, a laughter sound class, etc., the electronic device (100) can identify (or determine) the ambient sound (410) as a sound made by a person. For example, if the ambient sound (410) is classified into a dog barking class, the electronic device (100) can identify (or determine) the ambient sound (410) as not being a sound made by a person. For example, if the ambient sound (410) is not classified into a predefined class, the electronic device (100) can identify (or determine) the ambient sound (410) as a sound made by a human.

[0083] In one embodiment of the present disclosure, based on identifying that the ambient sound (410) is a human sound, the electronic device (100) can identify whether a user is present within a detection range based on WiFi CSI. To this end, the electronic device (100) can acquire WiFi CSI periodically or aperiodically. For example, the electronic device (100) can collect, calculate, receive, or generate WiFi CSI for WiFi signals transmitted and received with an AP device. The electronic device (100) can use the acquired WiFi CSI to identify (or determine) whether a user is present within a detection range based on WiFi CSI.

[0084] In one embodiment of the present disclosure, if it is determined that there is no user within the detection range based on WiFi CSI, the electronic device (100) may maintain its existing state. For example, the existing state of the electronic device (100) may include a state with low power consumption (or a state with low power consumption) (e.g., a power saving mode, a low power mode), and for example, may include a state in which power is not supplied to a speaker or display.

[0085] For example, a detection range based on WiFi CSI may correspond to a specific indoor or outdoor space (e.g., a home, an office, a bedroom, etc.) in which the electronic device (100) is located. Accordingly, the absence of a user in the detection range based on WiFi CSI may be regarded as the absence of a user in the corresponding specific indoor or outdoor space. For example, if a user does not exist in a specific indoor or outdoor space, the electronic device (100) may continue to detect the presence of a user within the detection range based on WiFi CSI. For example, if a user does not exist in a specific indoor or outdoor space, the electronic device (100) may not perform additional operations to detect the user until another ambient sound is input to the microphone.

[0086] For example, as illustrated in FIG. 4, when a sound from outside the house is transmitted into an empty house, the TV device can identify whether the ambient sound (410) from outside the house is a sound caused by a person. If the ambient sound (410) is not a sound caused by a person, the TV device may not perform additional actions to detect a user or change its state until another ambient sound is input through the microphone. If the ambient sound (410) is a sound caused by a person, the TV device can identify that no user is present inside the house based on WiFi CSI.

[0087] FIG. 5 is a diagram illustrating an example of an operation of an electronic device when an ambient sound caused by a person is identified and the presence of a user within a detection range based on WiFi CSI is identified in one embodiment of the present disclosure.

[0088] In explaining Fig. 5, any explanation that overlaps with the explanation given above in any one of Figs. 1 to 4 may be omitted.

[0089] FIG. 5 illustrates an example of an operation of an electronic device (100) in a case where an ambient sound (410) input through a microphone is a sound made by a person and a user (110) is present within a detection range based on WiFi CSI, according to one embodiment of the present disclosure. Referring to FIG. 5 , the electronic device (100) identifies the sound of boiling water input through the microphone as a sound made by a person, and thus can identify whether a user (110) is present inside a house using WiFi CSI.

[0090] In one embodiment of the present disclosure, when the electronic device (100) identifies that the ambient sound (410) input through the microphone is a sound made by a person, the electronic device (100) can identify (or determine) whether a user (110) is present within a detection range based on WiFi CSI. For example, the electronic device (100) can identify (or determine) whether a current user (110) is present within a detection range based on WiFi CSI using the current WiFi CSI.

[0091] In one embodiment of the present disclosure, based on identifying the presence of a user (110) within a detection range based on WiFi CSI, the electronic device (100) may transition or change from an existing state to another state. For example, the electronic device (100) may transition or change from an existing state with low power consumption (e.g., power saving mode) to another state with high power consumption. For example, the electronic device (100) may transition from an existing state in which the speaker is powered off to a state in which the speaker is powered on. For example, the electronic device (100) may transition from an existing state in which the connection interface module with the speaker is powered off to a state in which the connection interface module with the speaker is powered on.

[0092] In one embodiment of the present disclosure, based on identifying that a user (110) is present within a WiFi CSI-based detection range, the electronic device (100) can identify whether a user (110) is present within a first ultrasonic-based detection range (510). To this end, the electronic device (100) can output an ultrasonic signal through a speaker associated with the electronic device (100). For example, the electronic device (100) can transmit an inaudible ultrasonic signal through the speaker for a certain period of time.

[0093] For example, a speaker associated with an electronic device (100) may include an output module or output device capable of generating, transmitting, or outputting an ultrasonic signal. For example, a speaker associated with an electronic device (100) may include a speaker built into or included in the electronic device (100). For example, a speaker associated with an electronic device (100) may include a speaker device connected to the electronic device (100) by wire or wirelessly. For example, a speaker associated with an electronic device (100) may include a speaker built into or included in a device connected to the electronic device (100) by wire or wirelessly.

[0094] An ultrasonic signal output through a speaker may be reflected by surrounding objects, and the reflected ultrasonic signal may be input through a microphone associated with the electronic device (100). In one embodiment of the present disclosure, the electronic device (100) may determine whether a user (110) exists within an ultrasonic-based first detection range (510) using the output ultrasonic signal and the input ultrasonic signal. For example, the ultrasonic-based first detection range (510) may include a proximity area (e.g., an area within 3 m) in which the user (110) can actually use the service of the electronic device (100). The absence of the user (110) within the ultrasonic-based first detection range (510) may be regarded as meaning that the user (110) is not located in proximity to the electronic device (100).

[0095] For example, when the user (110) is located outside the first ultrasound-based detection range (510), the output ultrasound signal may not be reflected by the user (110), and the ultrasound signal input to the microphone may not include the ultrasound signal reflected by the user (110). In this case, the electronic device (100) may determine that the user (110) is not present in the first ultrasound-based detection range (510) using the input ultrasound signal that does not include the ultrasound signal reflected by the user (110). For example, even when the user (110) is located outside the first ultrasound-based detection range (510), the output ultrasound signal may be reflected by the user (110) and input to the microphone. In this case, the electronic device (100) may determine that the user (110) is present outside the first ultrasound-based detection range (510) using the input ultrasound signal.

[0096] In one embodiment of the present disclosure, if there is no user (110) within the first ultrasonic-based detection range (510), the electronic device (100) may maintain its current state. For example, the electronic device (100) may continue to detect the presence of the user (110) within the first ultrasonic-based detection range (510). In one embodiment of the present disclosure, if there is no user (110) within the first ultrasonic-based detection range (510), the electronic device (100) may transition to a state with low power consumption. For example, the electronic device (100) may transition to a state in which the speaker is powered off. For example, the electronic device (100) may not perform additional operations to detect the user (110) until another ambient sound is input to the microphone.

[0097] For example, as illustrated in FIG. 5, if a user (110) is boiling water in a kitchen far from the TV device, the TV device can identify that the ambient sound (410) of the water boiling sound is a human sound, identify that the user (110) is present inside the house based on WiFi CSI, and identify that the user (110) is not located close to the TV device based on ultrasonic signals.

[0098] FIG. 6 is a diagram illustrating an example of an operation of an electronic device when an ambient sound caused by a person is identified, the presence of a user within a detection range based on WiFi CSI is identified, and the presence of a user within a detection range based on ultrasound is identified in one embodiment of the present disclosure.

[0099] In explaining Fig. 6, any explanation that overlaps with the explanation given above in any one of Figs. 1 to 5 may be omitted.

[0100] FIG. 6 illustrates an example of an operation of an electronic device (100) in a case where, in one embodiment of the present disclosure, an ambient sound (410) input through a microphone is a sound caused by a person, a user (110) is present within a detection range based on WiFi CSI, and a user (110) is present within a detection range based on ultrasonic waves. Referring to FIG. 6 , the electronic device (100) identifies a footstep sound input through a microphone as a sound caused by a person, and identifies that a user (110) is present inside a house, and thus can identify whether the user (110) is located close to the electronic device (100) using an ultrasonic signal.

[0101] In one embodiment of the present disclosure, when the ambient sound (410) is a sound caused by a person and the presence of a user (110) within a WiFi CSI-based detection range is identified, the electronic device (100) can identify whether the user (110) is within a first ultrasonic-based detection range (510). When the user (110) is located within the first ultrasonic-based detection range (510), an ultrasonic signal output through a speaker can be reflected by the user (110) and input to a microphone. The electronic device (100) can determine whether the user (110) is within the first ultrasonic-based detection range (510) by using the output ultrasonic signal and the ultrasonic signal reflected and input to the microphone.

[0102] For example, the presence of a user (110) within the first ultrasonic-based detection range (510) may be considered as the user (110) being located in proximity to the electronic device (100). Accordingly, the electronic device (100) may detect the approach of the user (110) by identifying that the user (110) is located within the first ultrasonic-based detection range (510).

[0103] In one embodiment of the present disclosure, when a user (110) is present within an ultrasonic-based first detection range (510), the electronic device (100) may perform a predetermined operation. For example, when the electronic device (100) detects the approach of the user (110), the electronic device (100) may output specific information through an output device associated with the electronic device (100). For example, the output device associated with the electronic device (100) may include an output device built into or included in the electronic device (100). For example, the output device associated with the electronic device (100) may include an output device connected to the electronic device (100) by wire or wirelessly. For example, the output device associated with the electronic device (100) may include an output device built into or included in a device connected to the electronic device (100) by wire or wirelessly.

[0104] For example, when the electronic device (100) detects the approach of a user (110), the electronic device (100) may output a specific screen through a display associated with the electronic device (100). For example, when the electronic device (100) detects the approach of a user (110), the electronic device (100) may output a visual element (e.g., a graphic, an icon, an identifier, a character, an image, an index, etc.) representing specific information through a display associated with the electronic device (100). For example, when the electronic device (100) detects the approach of a user (110), the electronic device (100) may output an auditory element (e.g., a voice, a notification, a sound, etc.) representing specific information through a speaker associated with the electronic device (100).

[0105] In order to output a screen or visual element on a display associated with an electronic device (100), power must be supplied to the display or the connection interface module with the display. For example, the electronic device (100) may transition from a state in which the display is powered off (e.g., in a power-saving mode) to a state in which the display is powered on. For example, the electronic device (100) may transition from a previous state in which the connection interface module with the display is powered off to a state in which the connection interface module with the display is powered on.

[0106] In one embodiment of the present disclosure, based on identifying the presence of a user (110) within a detection range based on WiFi CSI, the electronic device (100) may transition to a state in which the display is powered on. For example, the state in which the display is powered on may include a state in which the frame rate control (FRC) module is powered on, and the LCF backlight can be turned on. The FRC module may be an on-board surface-mounted device (SMD) chip, and may include a module for reducing artifacts caused by slow discharge of the liquid crystal and converting broadcast signals to a higher frame rate. For example, based on identifying the presence of a user (110) within a first detection range (510) based on ultrasound, the electronic device (100) may transition to a state in which the display is powered on. For example, the electronic device (100) may transition to a power-on state of the display before identifying the presence of a user (110) within the first ultrasonic-based detection range (510).

[0107] If power is applied to the display before identifying the presence of a user (110) within the first ultrasonic detection range (510), the electronic device (100) can output a screen or visual element on the display as soon as it identifies the presence of a user (110) within the first ultrasonic detection range (510). On the other hand, if power is applied to the display to output a screen or visual element after identifying the presence of a user (110) within the first ultrasonic detection range (510), a delay time may occur between the time of the user's (110) approach and the time of the display's output. Therefore, by applying power to the display in advance before identifying the presence of a user (110) within the first ultrasonic detection range (510), the delay time can be reduced.

[0108] For example, as illustrated in FIG. 6, when a user (110) walks close to a TV device, the TV device can identify that the ambient sound (410) of footsteps is a human sound, identify that the user (110) is present inside the house based on WiFi CSI, and identify that the user (110) is located close to the TV device based on ultrasonic signals. The TV device can output a screen including visual elements representing useful information on the display.

[0109] FIG. 7A is a diagram illustrating an example of a first state of an electronic device according to an embodiment of the present disclosure. FIG. 7B is a diagram illustrating an example of a second state of an electronic device according to an embodiment of the present disclosure.

[0110] In one embodiment of the present disclosure, the electronic device (100) may include at least one processor (710), a high-definition multimedia interface (HDMI) module (720), a tuner module (730), a speaker (740), and a display (750). For example, the HDMI module (720) may perform a digital interface function for receiving or transmitting digital video and audio signals. For example, the tuner module (730) may receive and process terrestrial or cable TV broadcast signals.

[0111] The state of the electronic device (100) may include a power consumption state of the electronic device (100), and the electronic device (100) may be set to one of a plurality of power consumption states at one point in time. In one embodiment of the present disclosure, the power consumption state of the electronic device (100) may indicate whether power is supplied to sensors, modules, or devices inherent in, included in, or wirelessly connected to the electronic device (100). For example, a state of low power consumption (or a state of low power consumption) may include a state in which power is supplied to a relatively small number of sensors, modules, or devices. For example, a state of high power consumption (or a state of high power consumption) may include a state in which power is supplied to a relatively large number of sensors, modules, or devices.

[0112] In one embodiment of the present disclosure, the electronic device (100) may transition from a first state with low power consumption to a second state with high power consumption based on identifying (or determining) that a user is present within a detection range based on WiFi CSI. For example, the first state may include a state in which at least one of the speaker (740) or the display (750) is powered off, and the second state may include a state in which at least one of the speaker (740) or the display (750) is powered on. For example, when it is determined that a user is present within a detection range based on WiFi CSI, the electronic device (100) may transition to a second state in which power is supplied to the speaker (740) to output an ultrasonic signal through the speaker (740).

[0113] In one embodiment of the present disclosure, the electronic device (100) may transition from a second state with high power consumption to a first state with low power consumption based on identifying (or determining) that a user is not present within a first ultrasonic-based detection range. For example, if it is determined that a user is not present within the first ultrasonic-based detection range, the electronic device (100) may cut off power to at least one of the speaker (740) and the display (750). For example, if the user is located far away from the electronic device (100) and the electronic device (100) does not need to execute a function requiring the speaker (740) or the display (750), the electronic device (100) may enter a state in which the speaker (740) or the display (750) is turned off, thereby preventing unnecessary power consumption.

[0114] Referring to FIG. 7A, an electronic device (100) in a first state with low power consumption may have power supplied to the processor (710), but not to the HDMI module (720), the tuner module (730), the speaker (740), and the display (750). In the first state, the electronic device (100) may operate with low power consumption by not supplying power to the HDMI module (720), the tuner module (730), the speaker (740), and the display (750). Since power is not supplied to the HDMI module (720), the tuner module (730), the speaker (740), and the display (750) in the first state, functions of the electronic device (100) that require operation of the HDMI module (720), the tuner module (730), the speaker (740), or the display (750) may be deactivated.

[0115] Referring to FIG. 7B, the electronic device (100) in the second state with high power consumption may have power supplied to the processor (710), the speaker (740), and the display (750), but not to the HDMI module (720) and the tuner module (730). The power consumption of the electronic device (100) in the second state may be higher than in the first state, but may be lower than in the state in which power is supplied to all of the processor (710), the HDMI module (720), the tuner module (730), the speaker (740), and the display (750). In the second state, since power is supplied to the speaker (740) and the display (750), functions of the electronic device (100) that require operation of the speaker (740) or the display (750) may be activated to be executed or provided.

[0116] The electronic device (100) illustrated in FIGS. 7A and 7B includes, but is not limited to, a processor (710), an HDMI module (720), a tuner module (730), a speaker (740), and a display (750). For example, the electronic device (100) may include a microphone (or a connection interface module with a microphone) and a WiFi module, and at least one of the first state or the second state may include a state in which power is supplied to at least one of the microphone or the WiFi module.

[0117] In addition, although FIGS. 7A and 7B illustrate the first state and the second state as states of the electronic device, respectively, the states of the electronic device are not limited thereto. For example, the plurality of states of the electronic device may include more types of states. For example, the plurality of states may include a state in which power is supplied to the speaker (740) and the processor (710), and power is cut off to the HDMI module (720), the tuner module (730), and the display (750). For example, the plurality of states may include a state in which power is supplied to the display (750) and the processor (710), and power is cut off to the HDMI module (720), the tuner module (730), and the speaker (740).

[0118] FIG. 8 is a drawing showing an example of a method of operating an electronic device according to one embodiment of the present disclosure.

[0119] FIG. 8 may illustrate an example of the operation and interaction of a processor and each module included in an electronic device according to one embodiment of the present disclosure. In one embodiment of the present disclosure, the electronic device may include a microphone module (810), at least one processor (812), a WiFi module (814), an ambient sound module (816), a power module (818), an ultrasonic transceiver module (820), a speaker module (822), and a display module (824). For example, at least one processor (812) may include an application processor (AP).

[0120] Referring to FIG. 8, a method (800) according to one embodiment of the present disclosure may include steps 830 to 854. The method (800) of operating an electronic device is not limited to that illustrated in FIG. 8, and in one or more embodiments, steps not illustrated in FIG. 8 may be further included, or some steps may be omitted.

[0121] Referring to FIG. 8, in step 830, the processor (812) can identify ambient sounds from the microphone module (810). For example, the processor (812) can identify ambient sounds input through the microphone module (810). For example, the ambient sounds can be recorded at regular intervals. For example, the microphone module (810) can convert ambient sounds into electrical signals, and the processor (812) can identify the ambient sounds converted into electrical signals. The ambient sounds converted into electrical signals can be stored in the memory of the electronic device.

[0122] In one embodiment of the present disclosure, the processor (812) can use the ambient sound module (816) to identify whether an ambient sound is a human sound. For example, the processor (812) can control the ambient sound module (816) to recognize human presence. For example, the ambient sound module (816) can determine whether an ambient sound is a human sound.

[0123] Referring to FIG. 8, at step 832, the processor (812) may provide or input ambient sounds (e.g., ambient sound data) to the ambient sound module (816). For example, the processor (812) may request, command, or instruct the ambient sound module (816) to determine whether the ambient sounds are human sounds.

[0124] Referring to FIG. 8, in step 834, the processor (812) can recognize presence from the ambient sound module (816). For example, the ambient sound module (816) can determine whether the ambient sound is a sound caused by a person and output or return the determination result. For example, if the ambient sound is a sound caused by a person, the ambient sound module (816) can output or return information (e.g., a result value) indicating that the ambient sound is a sound caused by a person to the processor (812). Accordingly, the processor (812) can identify that the ambient sound is a sound caused by a person, i.e., recognize presence, based on the return information from the ambient sound module (816).

[0125] In one embodiment of the present disclosure, the processor (812) can identify the presence of a user within a WiFi-based detection range as a presence is recognized. Referring to FIG. 8, the processor (812) can identify WiFi CSI from the WiFi module (814) in step 836, and identify presence in a target space in step 838. For example, the WiFi module (814) can obtain, calculate, collect, generate, or store WiFi CSI. For example, the WiFi module (814) can store WiFi CSI for a target space in which no person is present as a reference WiFi CSI. If the current WiFi CSI has changed by a threshold amount or more from the reference WiFi CSI, the processor (812) can determine that a user is present within the WiFi-based detection range. For example, since the WiFi-based detection range may correspond to the target space, the processor (812) can determine presence in the target space by determining the presence of a user within the WiFi-based detection range.

[0126] In one embodiment of the present disclosure, the processor (812) can identify information indicating the presence of a user within a WiFi-based detection range from the WiFi module (814). For example, the processor (812) can request, command, or instruct the WiFi module (814) to determine the presence of a user within the WiFi-based detection range. Accordingly, the WiFi module (814) can determine the presence of a user within the WiFi CSI-based detection range and output or return information indicating the presence of a user within the WiFi CSI-based detection range to the processor (812).

[0127] In one embodiment of the present disclosure, if the target space is identified as being in an occupied state, the processor (812) may change the state of the electronic device. For example, the processor (812) may request, command, or instruct the power module (818) to change the power consumption state (or mode). Accordingly, the power module (818) may apply or cut off power to at least one module.

[0128] Referring to FIG. 8, in step 840, the processor (812) may request the power module (818) to change from a low-power state (e.g., an ambient sound reception state) to an ultrasonic transmission / reception state. Accordingly, the power module (818) may supply power to the speaker module (822) in step 842, and supply power to the display module (824) in step 844. Although FIG. 8 illustrates that the power module (818) sequentially supplies power to the speaker module (822) and the display module (824), the present invention is not limited thereto. For example, the power module (818) may supply power to the speaker module (822) and the display module (824) simultaneously.

[0129] In one embodiment of the present disclosure, when a target space is identified as an occupant state, the processor (812) can identify the presence of a user within an ultrasonic-based detection range. Referring to FIG. 8, in step 846, the processor (812) can request (or command, instruct) the ultrasonic transmission / reception module (820) to transmit (or output) an ultrasonic signal. For example, the processor (812) can request (or command, instruct) the ultrasonic transmission / reception module (820) to transmit (or output) an ultrasonic signal of a specific intensity or frequency. Accordingly, in step 848, the ultrasonic transmission / reception module (820) can transmit (or output) an ultrasonic signal through the speaker module (822). For example, the speaker module (822) can generate an ultrasonic signal of a specific frequency and a specific intensity according to a command, request, control, or instruction of the ultrasonic transmission / reception module (820).

[0130] The ultrasonic signal generated by the speaker module (822) may be reflected by an external object and input through the microphone module (810). For example, the microphone module (810) may convert the ultrasonic signal reflected by the external object into an electrical signal or data. Referring to FIG. 8, in step 850, the ultrasonic signal input through the microphone module (810) may be provided to or input to the processor (812).

[0131] Referring to FIG. 8, in step 852, the processor (812) can identify the phase difference between the transmitted ultrasonic signal and the received ultrasonic signal from the ultrasonic transceiver module (820). The processor (812) can use the phase difference between the transmitted ultrasonic signal and the received ultrasonic signal to determine whether a user is present within the ultrasonic-based detection range. For example, if the phase difference between the transmitted ultrasonic signal and the received ultrasonic signal is constant for a certain period of time, the processor (812) can determine that a user is absent within the ultrasonic-based detection range. For example, if the phase difference between the transmitted ultrasonic signal and the received ultrasonic signal changes for a certain period of time, the processor (812) can determine that a user is present within the ultrasonic-based detection range.

[0132] In one embodiment of the present disclosure, when the processor (812) identifies the presence of a user within the ultrasonic-based detection range, the processor (812) may output specific information. Referring to FIG. 8, in step 854, the processor (812) may display a screen on the display module (824). For example, the processor (812) may transmit data, information, instructions, requests, or commands for screen output to the display module (824), and the display module (824) may display a screen according to the received data, information, instructions, requests, or commands.

[0133] In FIG. 8, the processor (812) recognizes presence using ambient sound and then uses WiFi CSI to identify occupancy. However, this is not limited to this. For example, the processor (812) may identify occupancy in a target space using WiFi CSI and then use ambient sound to recognize presence. In this case, if presence is recognized by ambient sound, the processor (812) may change its state or output ultrasonic waves.

[0134] FIG. 9 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment of the present disclosure.

[0135] In explaining Fig. 9, any explanation that overlaps with the explanation given above in any one of Figs. 1 to 8 may be omitted.

[0136] FIG. 9 illustrates an example of an operating method of an electronic device when the absence of a user within a detection range based on WiFi CSI is identified, but a loud ambient sound is identified. Referring to FIG. 9 , an operating method (900) of an electronic device according to an embodiment of the present disclosure may include steps 910 to 940. In an embodiment of the present disclosure, steps 910 to 940 of the method (900) may be executed by at least one processor included in the electronic device. The operating method (900) of the electronic device is not limited to that illustrated in FIG. 9 , and in one or more embodiments, steps not illustrated in FIG. 9 may be further included, or some steps may be omitted.

[0137] The detection range based on WiFi CSI may not completely cover the target space (e.g., an indoor space such as a home or office) where the user is to be detected. For example, the target space may include a space outside the detection range based on WiFi CSI or a shadow space for WiFi signals. In this case, the electronic device cannot detect a user existing in a space outside the detection range based on WiFi CSI or a shadow space among the target spaces based on WiFi CSI. Even if the electronic device identifies that a user is not present within the detection range based on WiFi CSI, the user may actually be present within the target space or may be entering the target space. Therefore, even if the electronic device identifies that a user is not present within the detection range based on WiFi CSI, the electronic device needs to reconfirm whether the user is present in the target space (i.e., whether the user is present) or whether the user is entering.

[0138] In step 910, the electronic device can identify the absence of a user within the WiFi CSI detection range. In one embodiment of the present disclosure, the electronic device can detect or determine the absence of a user within the WiFi-based detection range using WiFi CSI.

[0139] In step 920, the electronic device can identify whether the level of the received audio (or the ambient sound corresponding to the audio) is greater than or equal to a threshold. For example, the audio (or the ambient sound) in step 920 may include at least one of audio (or the ambient sound) received (or input) before identifying the absence of the user within the detection range based on WiFi CSI (i.e., step 910), audio (or the ambient sound) received (or input) during the identification, or audio (or the ambient sound) received (or input) after the identification. In one embodiment of the present disclosure, the electronic device can identify whether the level (e.g., input sound pressure level) of the ambient sound input through the microphone is greater than or equal to a predetermined threshold. For example, the electronic device can determine whether the level of the ambient sound input through the microphone is greater than or equal to a threshold (e.g., 50 dB). For example, the threshold may be determined based on the level of a sufficiently loud sound that can be considered to have originated within a target space.

[0140] In one embodiment of the present disclosure, an electronic device can identify whether the volume of an ambient sound corresponding to a specific class is greater than or equal to a threshold. For example, the specific class may include a class of sounds that may occur when a user enters a target space. For example, the specific class may include a class corresponding to the sound of a door closing, a class corresponding to the sound of shoes taking off, a class corresponding to the sound of a door lock, etc. For example, the electronic device can identify (or determine) whether an ambient sound input through a microphone corresponds to a specific class and identify whether the volume of the ambient sound corresponding to the specific class is greater than or equal to a threshold.

[0141] If the ambient sound level is greater than or equal to a threshold value, in step 930, the electronic device may output a second intensity ultrasonic signal through the speaker. In one embodiment of the present disclosure, the electronic device may output a second intensity ultrasonic signal through the speaker based on identifying the absence of a user within a detection range based on WiFi CSI and determining that the ambient sound level is greater than or equal to a threshold value. For example, the second intensity ultrasonic signal may be determined based on the structure and size of a target space or the position of the electronic device. For example, the second intensity ultrasonic signal may be stronger than the first intensity ultrasonic signal output when a user is present within a detection range based on WiFi CSI.

[0142] In one embodiment of the present disclosure, the electronic device can change from a first state in which power consumption is less than a preset value to a second state in which power consumption is more than a preset value when there is no user within the WiFi CSI detection range and the size of the received audio is greater than a threshold value.

[0143] An electronic device can identify the presence of a user within a second detection range based on ultrasound. Referring to FIG. 9, in step 940, the electronic device can identify the presence of a user within a second detection range based on ultrasound. In one embodiment of the present disclosure, the electronic device can identify the presence of a user within the second detection range based on ultrasound using an outputted ultrasonic signal of second intensity and an ultrasonic signal input through a microphone. For example, a second detection range based on an ultrasonic signal of second intensity may be wider than a first detection range based on an ultrasonic signal of first intensity. Accordingly, even if the absence of a user within a detection range based on WiFi CSI is identified, the electronic device can double-check whether a user exists or enters a target space based on sound and ultrasonic signals.

[0144] By identifying the presence of a user within a target space using an ultrasonic signal, the electronic device can identify whether the user is located in proximity. In one embodiment of the present disclosure, the electronic device can identify the presence of a user within a first ultrasonic-based detection range based on identifying the presence of a user within a second ultrasonic-based detection range. For example, if the presence of a user within a second ultrasonic-based detection range is identified, the electronic device can identify the presence of a user within the first ultrasonic-based detection range using an ultrasonic signal of a first intensity.

[0145] In one embodiment of the present disclosure, when the absence of a user within a detection range based on WiFi CSI is identified and the volume of an ambient sound input through a microphone increases, the electronic device can identify the presence of a user within a second detection range based on ultrasound. For example, when the volume of an ambient sound input through a microphone tends to increase, the electronic device can identify the presence of a user within the second detection range based on ultrasound. For example, when the input ambient sound is louder than the previously input ambient sound after the electronic device identifies the absence of a user within the detection range based on WiFi CSI, the electronic device can identify the presence of a user within the second detection range based on ultrasound.

[0146] FIG. 10 is a diagram illustrating an example of an operation of an electronic device when a user is located outside a WiFi CSI-based detection range in one embodiment of the present disclosure.

[0147] In explaining Fig. 10, any explanation that overlaps with the explanation given above in any one of Figs. 1 to 9 may be omitted.

[0148] FIG. 10 may illustrate an example of a method of operation of an electronic device when the absence of a user (110) within a detection range based on WiFi CSI is identified, but a loud ambient sound is identified, in one embodiment of the present disclosure.

[0149] As illustrated in FIG. 10, even if a user (110) exists in a target space (e.g., a house), if the user is located outside or in a shaded space of the WiFi CSI-based detection range (310), the electronic device may determine or identify that the user (110) does not exist within the WiFi CSI-based detection range (310). Therefore, even if it is identified that the user (110) does not exist within the WiFi CSI-based detection range (310), it is necessary to additionally detect whether the user (110) exists within the target space. In one embodiment of the present disclosure, even if it is identified that the user (110) does not exist within the WiFi CSI-based detection range (310), if a loud ambient sound by a person is generated, a sound of a person entering the target space is generated, or the ambient sound gradually becomes louder, it may be assumed that the user (110) exists or the user (110) is entering the target space.

[0150] To further confirm the presence of a user (110) in a target space, the electronic device may utilize ambient sound and ultrasound. In one embodiment of the present disclosure, even if there is no user (110) within a WiFi CSI-based detection range (310), if an ambient sound input through a microphone satisfies a predefined condition, the electronic device may additionally identify (or detect, determine) whether a user (110) is present within the target space using ultrasound. For example, based on identifying that an ambient sound above a microphone decibel level is input, the electronic device may identify whether a user (110) is present within a second ultrasound-based detection range. For example, based on identifying that an ambient sound corresponding to a specific class is input, the electronic device may identify whether a user (110) is present within a second ultrasound-based detection range. For example, based on identifying that an increasingly louder ambient sound is input, the electronic device may identify whether a user (110) is present within a second ultrasound-based detection range.

[0151] In one embodiment of the present disclosure, to identify whether a user (110) is present within a second detection range based on ultrasound, the electronic device may output an ultrasonic signal of a second intensity through a speaker. To output the ultrasonic signal of the second intensity through the speaker, power may be supplied to the speaker associated with the electronic device. For example, even if there is no user (110) within the detection range based on WiFi CSI (310), if an ambient sound input through the microphone satisfies a predefined condition, the electronic device may transition from a low power consumption state to a high power consumption state. The low power consumption state may include a state in which power is cut off to the speaker associated with the electronic device or a connection interface with the speaker. The high power consumption state may include a state in which power is supplied to the speaker associated with the electronic device or a connection interface with the speaker.

[0152] In one embodiment of the present disclosure, the electronic device may output specific information based on identifying the presence of a user (110) within a second ultrasonic detection range. For example, if the user (110) is within the second ultrasonic detection range, the electronic device may output a screen including visual elements corresponding to the specific information on the display. For example, if the user (110) is within the second ultrasonic detection range, the electronic device may output an auditory element corresponding to the specific information through a speaker.

[0153] In one embodiment of the present disclosure, the electronic device may detect the presence of a user (110) in a target space using a second intensity ultrasonic signal, and then additionally detect the presence of the user (110) in a proximity area using a first intensity ultrasonic signal. For example, the electronic device may identify whether the user (110) is also present within a first ultrasonic-based detection range (510) based on identifying that the user (110) is present within a second ultrasonic-based detection range. The electronic device may output specific information based on identifying that the user (110) is also present within the first ultrasonic-based detection range (510). For example, the second ultrasonic-based detection range may correspond to a target space, and the first ultrasonic-based detection range (510) may correspond to a proximity area with the electronic device.

[0154] FIG. 11 is a diagram showing an example of a configuration of an electronic device according to one embodiment of the present disclosure.

[0155] In explaining Fig. 11, any explanation that overlaps with the explanations given above in Figs. 1 to 10 may be omitted.

[0156] The electronic device (1100) illustrated in FIG. 11 is an electronic device that outputs an image (or video), and may include a TV, a smart monitor, a smart display, an outdoor monitor, a gaming monitor, an electronic picture frame, a laptop, a desktop computer, a wearable device, a tablet PC (personal computer), an e-book terminal, a digital broadcasting terminal, a PDA (personal digital assistant), a PMP (portable multimedia player), a navigation device, an MP3 player, and the like. In one embodiment of the present disclosure, the electronic device (1100) may include, but is not limited to, at least one processor (1110), a memory (1120), a communication module (1130), an input / output interface module (1140), an input device (1150), and an output device (1160).

[0157] The processor (1110) is electrically connected to components included in the electronic device (1100) and can execute operations or data processing related to control and / or communication of the components included in the electronic device (1100). In one embodiment of the present disclosure, the processor (1110) can load and process requests, commands, or data received from at least one of the other components into a memory and store the processing result data in the memory. The processor (1110) is a component that controls a series of processes so that the electronic device (1100) operates according to the above-described embodiments, and may be configured with one or more processors.

[0158] One or more processors included in the processor (1110) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. One or more processors included in the processor (1110) may include at least one of a general-purpose processor such as a central processing unit (CPU), a Micro Processor Unit (MPU), an application processor (AP), a Digital Signal Processor (DSP), a graphics-only processor such as a GPU (graphic processing unit) or a VPU (vision processing unit), an artificial intelligence-only processor such as an NPU (neural processing unit), or a communication-only processor such as a CP (communication processor). When one or more processors included in the processor (1110) are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0159] The processor (1110) may include various processing circuits and / or multiple processors. For example, the term "processor" as used in this disclosure, including the claims, may include various processing circuits including at least one processor. One or more of the at least one processor may be configured to perform one or more functions of the present disclosure, individually and / or collectively in a distributed manner. In this disclosure, when "processor," "at least one processor," or "one or more processors" is described as being configured to perform multiple functions, this may include situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor performs all of the functions. Furthermore, the at least one processor may include a combination of processors that perform various functions in a distributed manner. The at least one processor may execute program instructions to achieve or perform various functions.

[0160] The processor (1110) can write data to the memory (1120), read data stored in the memory (1120), and in particular, process data according to predefined operation rules or artificial intelligence models by executing a program or at least one instruction stored in the memory (1120). The processor (1110) can process input data or control other components to process it according to the data, operation rules, algorithms, methods, or models stored in the memory (1120). The processor (1110) can perform operations of predefined operation rules, algorithms, methods, or models stored in the memory (1120) using the input data.

[0161] The memory (1120) is electrically connected to the processor (1110) and may store one or more modules, algorithms, operating rules, models, programs, instructions, or data related to the operation of components included in the electronic device (1100). For example, the memory (1120) may include any non-transitory computer-readable recording medium. For example, the memory (1120) may store one or more modules, algorithms, operating rules, models, programs, instructions, or data for processing and controlling the processor (1110). The memory (1120) may be configured as a storage medium or a combination of storage media such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc., but is not limited thereto. The memory (1120) may not exist separately and may be configured to be included in the processor (1110). The memory (1120) may be configured as a volatile memory, a nonvolatile memory, or a combination of a volatile memory and a nonvolatile memory. A program or at least one instruction for performing operations according to the above-described embodiments may be stored in the memory (1120). The memory (1120) may also provide stored data to the processor (1110) at the request of the processor (1110).

[0162] In one embodiment of the present disclosure, the memory (1120) may store data and / or information identified, acquired, generated, or determined by the electronic device (1100). For example, the memory (1120) may store data and / or information identified, acquired, generated, or determined by the electronic device (1100) in a compressed form. In one embodiment of the present disclosure, the memory (1120) may store predefined or determined information.

[0163] In one embodiment of the present disclosure, an electronic device (1100) may include a module that performs (or is used to perform) at least one operation. Some modules of the electronic device (1100) that perform at least one operation may be composed of multiple sub-modules or may constitute a single module.

[0164] Some modules that perform at least one operation of the electronic device (1100) may be implemented as hardware modules, software modules, and / or a combination thereof. The software modules included in the electronic device (1100) may be included in the memory (1120). In one embodiment of the present disclosure, the modules included in the memory (1120) may be executed by the processor (1110) to perform operations. For example, the modules (i.e., software modules) included in the memory (1120) may include programs, models, or algorithms that are executed according to the control or instructions of the processor (1110) and are configured to perform operations that derive output data for input data.

[0165] The electronic device (1100) may include a communication module (1130) (e.g., a communication interface module, a transmission / reception module) for communicating with other devices, servers, or systems. In one embodiment of the present disclosure, the communication module (1130) of the electronic device (1100) may support the establishment of a wired or wireless communication channel (or connection, link) with another external device or server and the performance of communication through the established communication channel.

[0166] In one embodiment of the present disclosure, the communication module (1130) may receive signals, information, requests, and / or data (e.g., image data, sound data) from another external electronic device or server via wired communication or wireless communication, or transmit signals, information, requests, and / or data to another external electronic device or server. According to one embodiment of the present disclosure, the communication module (1030) may include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module), and may communicate with an external electronic device or server via at least one network, for example, a short-range communication network (e.g., Bluetooth, WiFi direct, WiFi P2P, WiFi Aware, or IrDA (infrared data association)) or a long-range communication network (e.g., a cellular network, the Internet, or a computer network (e.g., a LAN or a WAN)) using any one of the communication modules. For example, a communication module for communicating with a server and a communication module for communicating with another electronic device (e.g., a remote control device) may be different or the same.

[0167] In one embodiment of the present disclosure, the communication module (1130) may include a WiFi module (1132). For example, the WiFi module (1132) may support wireless network access of the electronic device (1100). For example, the WiFi module (1132) may support the electronic device (1100) to transmit and receive data with an external router device, an access point (AP) device, or a non-AP device via WiFi signals. For example, the electronic device (1100) may use the WiFi module (1132) to connect to the Internet and stream online content or browse the web via a wireless network. For example, the WiFi module (1132) may periodically or aperiodically collect, produce, or generate information (e.g., WiFi CSI) about a WiFi signal or WiFi channel used for WiFi communication of the electronic device (1100). For example, the WiFi module (1132) may acquire WiFi CSI without the control of the processor (1110).

[0168] In one embodiment of the present disclosure, the communication module (1130) may include a tuner module (1134). For example, the tuner module (1134) may receive and process TV broadcast signals. For example, the tuner module (1134) may receive TV broadcast signals via an antenna, cable, satellite, or other receiving device. For example, the tuner module (1134) may receive signals in various frequency bands and process signals of a frequency corresponding to a channel selected by a user. For example, the tuner module (1134) may process digital or analog signals so that a broadcast screen can be output on a display of an electronic device.

[0169] The input / output interface module (1140) is a module that supports connection and data transmission / reception between the electronic device (1100) and other electronic devices, and can manage and process input and output of data. For example, the input / output interface module (1140) can process transmission / reception of media content data, such as voice or video, between the electronic device (1100) and external devices. For example, the input / output interface module (1140) may include an input interface module (e.g., an input port), an output interface module (e.g., an output port), or an input / output integrated interface module (e.g., an input / output integrated port). For example, the electronic device can receive or input an audio signal from an external device (e.g., a microphone) using the input port. For example, the electronic device can output or transmit a video or audio signal to an external device (e.g., a display, a speaker, etc.) using the output port. For example, the input / output interface module (1140) may include an HDMI (High-Definition Multimedia Interface) module (1142) (e.g., HDMI port), DP (display port), RGB (Red-Green-Blue) port, DVI (Digital Visual Interface), Thunderbolt, etc.

[0170] An input device (1150), an output device (1160), or an input / output integrated device may be included in the electronic device (1100) as a module constituting the electronic device (1100). For example, the input device (1150) may include a microphone (1152), a keyboard, a mouse, a camera, a touch pad, etc., and the output device (1160) may include a display (1162), a speaker (1164), a headset, a projector, etc., and the input / output integrated device may include a touch screen, etc. For example, the speaker (1164) may output both signals in the audible frequency band and signals in the inaudible frequency band. For example, the electronic device (1100) may include a speaker that outputs signals in the audible frequency band and a speaker (e.g., an ultrasonic transmitter) that outputs signals in the inaudible frequency band, respectively.

[0171] The electronic device (1100) is not limited to that illustrated in FIG. 11, and may include more components than those illustrated in FIG. 11, or components illustrated in FIG. 11 may be omitted.

[0172] FIG. 12 is a diagram showing an example of a configuration of an electronic device and an input / output device connected to the electronic device according to one embodiment of the present disclosure.

[0173] In explaining Fig. 12, any explanation that overlaps with the explanation given above in any one of Figs. 1 to 11 may be omitted.

[0174] In one embodiment of the present disclosure, the electronic device (1100) may be connected to at least one of an external memory device, an external communication device, an external input / output interface device, an external input device, or an external output device, either wired or wirelessly. For example, the electronic device (1100) may utilize functions of the external device connected to the device by interacting with the external device connected to the device via wired or wireless means. For example, the electronic device (1100) may include a set-top box (STB), a Blu-ray disk player, a digital versatile disk (DVD) player, a game device, a digital camera, a camcorder, a streaming device, a home theater, and the like.

[0175] Although FIG. 11 illustrates an example in which an electronic device (1100) includes a microphone (1152), a display (1162), and a speaker (1164), the present invention is not limited thereto. For example, as illustrated in FIG. 12, the electronic device may be connected to an external microphone (1210), an external display (1220), or an external speaker (1230) via a wire or wireless connection. For example, the electronic device may receive data (e.g., sound data) from the external microphone (1210) or transmit data (e.g., image data, sound data) to the external display (1220) or the external speaker (1230).

[0176] In one embodiment of the present disclosure, the electronic device can identify ambient sounds input through an external microphone (1210). For example, the external microphone (1210) can convert waves of the ambient sounds into ambient sound data (e.g., electrical signals), and the electronic device can receive the sound data from the external microphone (1210).

[0177] In one embodiment of the present disclosure, the electronic device can output a screen through an external display (1220). For example, the electronic device can output a visual element through the external display (1220). For example, the electronic device can transmit information, data, requests, or instructions regarding a visual element or screen to be output to the external display (1220), and the external display (1220) can output the visual element or screen according to the received information, data, requests, or instructions.

[0178] In one embodiment of the present disclosure, the electronic device can output an ultrasonic signal or an auditory element through an external speaker (1230). For example, the electronic device can transmit information, data, requests, or instructions regarding an ultrasonic signal or an auditory element to be output to the external speaker (1230), and the external speaker (1230) can output the ultrasonic signal or the auditory element according to the received information, data, requests, or instructions.

[0179] In one embodiment of the present disclosure, the electronic device may include an output interface (e.g., an output port) for outputting data to an external microphone (1210), an external display (1220), or an external speaker (1230). For example, the output port may include a port for outputting image data, a port for outputting sound data, or a port for outputting image data and sound data together. In one embodiment of the present disclosure, the electronic device may include an input interface (e.g., an input port) for receiving data from an external microphone (1210), an external display (1220), or an external speaker (1230). In one embodiment of the present disclosure, the electronic device may communicate with the external microphone (1210), the external display (1220), or the external speaker (1230) by wire or wirelessly to output data to the external microphone (1210), the external display (1220), or the external speaker (1230).

[0180] In the present disclosure, overlapping descriptions in FIGS. 1 to 12 may be omitted, and one or more embodiments described above in FIGS. 1 to 12 may be applied / implemented in combination with each other. In the present disclosure, an operation described as being performed by an electronic device may be executed / performed by a module included or stored in the electronic device, executed / performed by at least one processor of the electronic device, or performed by the control of at least one processor of the electronic device using a module included or stored in the electronic device.

[0181] According to one embodiment of the present disclosure, even a TV without a motion sensor can detect a user using its existing microphone and speaker. Therefore, since no additional motion sensor is required, production costs can be reduced, power consumption can be reduced, and the device can be made lighter. For example, if sounds related to human actions are recognized through the microphone, the speaker can additionally use ultrasound to detect the user approaching the TV. Upon receiving this, the TV screen can automatically turn on and display useful information such as the weather, schedule, or memos.

[0182] Detecting a user solely based on sounds related to human behavior may be inaccurate due to noises from neighbors or upstairs, or from outside. According to one embodiment of the present disclosure, a user can be detected more accurately by utilizing not only ambient sounds but also WiFi CSI and ultrasound. Furthermore, if occupancy is determined solely based on sounds related to human behavior and speakers are powered on and ultrasound is transmitted, the speaker's lifespan may be shortened, and power relay noise may be generated during the power-on process. According to one embodiment of the present disclosure, by utilizing both sound and WiFi CSI to determine occupancy, the need to power speakers and transmit ultrasound can be minimized.

[0183] In one embodiment of the present disclosure, an electronic device may include a memory that stores one or more instructions and at least one processor that executes one or more instructions. In one embodiment of the present disclosure, when the audio received through a microphone corresponds to a preset condition, the electronic device may identify the presence of a user within a detection range of wireless fidelity (WiFi) channel state information (CSI). In one embodiment of the present disclosure, when the presence of a user within the detection range of the WiFi CSI is identified, the electronic device may identify whether the user is present within a first detection range based on ultrasound, by the at least one processor executing one or more instructions.

[0184] In one embodiment of the present disclosure, when at least one processor executes one or more commands, the electronic device can output an ultrasonic signal of a first intensity through a speaker if the received audio corresponds to a preset condition and a user is present within the detection range of the WiFi CSI. In one embodiment of the present disclosure, when at least one processor executes one or more commands, the electronic device can identify whether the user is present within the first detection range based on the ultrasonic-based signal of the first intensity output and the ultrasonic signal input through the microphone.

[0185] In one embodiment of the present disclosure, at least one processor executes one or more commands, thereby controlling the electronic device to output a screen corresponding to the presence of a user based on the presence of a user within the first ultrasonic-based detection range.

[0186] In one embodiment of the present disclosure, at least one processor executes one or more instructions, so that the electronic device can be controlled to change from a first state in which power consumption is lower than a preset value to a second state in which power consumption is higher than the preset value, based on the presence of the user within the detection range of the WiFi CSI.

[0187] In one embodiment of the present disclosure, at least one processor executes one or more instructions, thereby controlling the electronic device to change from a second state in which power consumption is higher than a preset value to a first state in which power consumption is lower than the preset value, based on the absence of the user within the first detection range based on the ultrasound.

[0188] In one embodiment of the present disclosure, the first state may include a state in which power consumption is lower than the normal state power consumption based on at least one of the speaker or the display not being operated. In one embodiment of the present disclosure, the second state may include a state in which power consumption is higher than the normal state power consumption based on at least one of the speaker or the display being operated.

[0189] In one embodiment of the present disclosure, at least one processor executes one or more instructions so that the electronic device can identify whether the ambient sound is a human sound based on at least one class associated with the ambient sound, if the audio received through the microphone corresponds to an ambient sound of the electronic device.

[0190] In one embodiment of the present disclosure, when at least one processor executes one or more commands, the electronic device can output an ultrasonic signal of a second intensity through a speaker if the user is not present within the WiFi CSI detection range and the volume of the received audio is greater than a threshold value. In one embodiment of the present disclosure, when at least one processor executes one or more commands, the electronic device can identify the presence of a user within a second detection range based on ultrasound using the outputted ultrasonic signal of the second intensity and the ultrasonic signal input to the microphone. In one embodiment of the present disclosure, the second detection range based on ultrasound can be wider than the first detection range based on ultrasound.

[0191] In one embodiment of the present disclosure, when at least one processor executes one or more instructions, the electronic device can change from a first state in which power consumption is less than a preset value to a second state in which power consumption is greater than the preset value when the user is not present within the WiFi CSI detection range and the size of the received audio is greater than a threshold value.

[0192] In one embodiment of the present disclosure, the ambient sound may correspond to a specific class.

[0193] In one embodiment of the present disclosure, at least one processor executes one or more instructions, so that the electronic device can identify whether a user is present within the first ultrasonic-based detection range based on whether a user is present within the second ultrasonic-based detection range.

[0194] In one embodiment of the present disclosure, at least one processor executes one or more instructions, so that the electronic device can transition from a second state with a high power consumption level to a first state with a low power consumption level based on the absence of a user within the second ultrasonic-based detection range.

[0195] In one embodiment of the present disclosure, a method of operating an electronic device may include a step of identifying the presence of a user within a detection range of wireless fidelity (WiFi) channel state information (CSI), if audio received through a microphone corresponds to a preset condition. In one embodiment of the present disclosure, the method of operating an electronic device may include a step of identifying whether a user is present within a first detection range based on ultrasound, if the presence of a user within the detection range of the WiFi CSI is identified.

[0196] In one embodiment of the present disclosure, the step of identifying whether a user exists within the first detection range based on ultrasound may include the step of outputting an ultrasonic signal of first intensity through a speaker if the received audio corresponds to a preset condition and a user exists within the detection range of the WiFi CSI. In one embodiment of the present disclosure, the step of identifying whether a user exists within the first detection range based on ultrasound may include the step of identifying whether a user exists within the first detection range based on the outputted ultrasonic signal of first intensity and an ultrasonic signal input through the microphone.

[0197] In one embodiment of the present disclosure, a method of operating an electronic device may include a step of controlling a screen corresponding to the presence of a user to be output based on the presence of a user within the first detection range based on ultrasound.

[0198] In one embodiment of the present disclosure, the step of identifying whether a user exists within the first detection range based on the ultrasonic wave may include a step of controlling a change from a first state in which power consumption is lower than a preset value to a second state in which power consumption is higher than the preset value based on the presence of the user within the detection range of the WiFi CSI.

[0199] In one embodiment of the present disclosure, a method of operating an electronic device may include a step of controlling the electronic device to change from a second state in which power consumption is greater than a preset value to a first state in which power consumption is less than the preset value, based on the absence of the user within the first detection range based on the ultrasonic wave.

[0200] In one embodiment of the present disclosure, the first state may include a state in which power consumption is lower than the normal state power consumption based on at least one of the speaker or the display not being operated. In one embodiment of the present disclosure, the second state may include a state in which power consumption is higher than the normal state power consumption based on at least one of the speaker or the display being operated.

[0201] In one embodiment of the present disclosure, a method of operating an electronic device may include, if audio received through the microphone corresponds to an ambient sound of the electronic device, a step of identifying whether the ambient sound is a sound made by a human based on at least one class associated with the ambient sound.

[0202] In one embodiment of the present disclosure, the operating method of the electronic device may include a step of outputting an ultrasonic signal of a second intensity through a speaker when the user is absent within the WiFi CSI detection range and the volume of the received audio is greater than or equal to a threshold value. In one embodiment of the present disclosure, the operating method of the electronic device may include a step of identifying the presence of the user within a second detection range based on ultrasound using the outputted ultrasonic signal of the second intensity and the ultrasonic signal input to the microphone. In one embodiment of the present disclosure, the second detection range based on ultrasound may be wider than the first detection range based on ultrasound.

[0203] In one embodiment of the present disclosure, a method of operating an electronic device may include a step of changing from a first state in which power consumption is less than a preset value to a second state in which power consumption is greater than the preset value, if the user is not present within the WiFi CSI detection range and the size of the received audio is greater than a threshold value.

[0204] In one embodiment of the present disclosure, the ambient sound may correspond to a specific class.

[0205] In one embodiment of the present disclosure, a method of operating an electronic device may include a step of identifying whether the user is present within the first ultrasonic-based detection range based on the presence of the user within the second ultrasonic-based detection range.

[0206] In one embodiment of the present disclosure, a method of operating an electronic device may include a step of transitioning from a second state having a high power consumption level to a first state having a low power consumption level based on the absence of a user within the second ultrasonic-based detection range.

[0207] In one embodiment of the present disclosure, a program for performing an operating method of an electronic device on a computer can be recorded on a computer-readable recording medium.

[0208] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0209] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

Claims

1. In an electronic device (1100), A memory (1120) storing one or more instructions; and comprising at least one processor (1110), By the at least one processor (1110) executing the one or more instructions, the electronic device, When the audio received through the microphone (1152) corresponds to a preset condition, the presence of a user within the detection range of WiFi (wireless fidelity) CSI (channel state information) is identified, An electronic device that identifies whether the user is present within the first detection range based on ultrasonic waves when the presence of the user is identified within the detection range of the WiFi CSI.

2. In paragraph 1, The electronic device, by the at least one processor (1110) executing the one or more instructions, If the received audio corresponds to a preset condition and there is a user within the detection range of the WiFi CSI, an ultrasonic signal of the first intensity is output through the speaker (1164). An electronic device that identifies whether the user is present within the first detection range based on the first intensity ultrasonic signal output above and the ultrasonic signal input through the microphone (1152).

3. In paragraph 1 or 2, The electronic device, by the at least one processor (1110) executing the one or more instructions, An electronic device that controls a screen corresponding to the presence of a user based on the presence of the user within the first detection range based on the ultrasonic waves.

4. In any one of paragraphs 1 to 3, The electronic device, by the at least one processor (1110) executing the one or more instructions, An electronic device that controls a change from a first state in which power consumption is lower than a preset value to a second state in which power consumption is higher than the preset value based on the presence of the user within the detection range of the WiFi CSI.

5. In any one of paragraphs 1 to 4, The electronic device, by the at least one processor (1110) executing the one or more instructions, An electronic device that controls a change from a second state in which power consumption is higher than a preset value to a first state in which power consumption is lower than the preset value based on the absence of the user within the first detection range based on the ultrasonic wave.

6. In paragraph 4 or 5, The first state is a state in which power consumption is less than the normal state power consumption based on at least one of the speaker (1164) or the display (1162) not operating, An electronic device, wherein the second state includes a state in which power consumption is greater than the normal state power consumption based on operation of at least one of the speaker (1164) or the display (1162).

7. In any one of paragraphs 1 to 6, The electronic device, by the at least one processor (1110) executing the one or more instructions, An electronic device, wherein when audio received through the microphone corresponds to ambient sound of the electronic device, the electronic device identifies whether the ambient sound is a sound made by a human based on at least one class associated with the ambient sound.

8. In any one of paragraphs 1 to 7, The electronic device, by the at least one processor (1110) executing the one or more instructions, If the user is not present within the WiFi CSI detection range and the size of the received audio is greater than a threshold value, a second intensity ultrasonic signal is output through the speaker (1164). Using the ultrasonic signal of the second intensity output above and the ultrasonic signal input to the microphone (1152), the presence of the user within the second ultrasonic-based detection range is identified, An electronic device wherein the second ultrasonic-based detection range is wider than the first ultrasonic-based detection range.

9. In any one of paragraphs 1 to 8, The electronic device, by the at least one processor (1110) executing the one or more instructions, An electronic device that changes from a first state in which power consumption is less than a preset value to a second state in which power consumption is more than the preset value when the user is not present within the WiFi CSI detection range and the size of the received audio is greater than a threshold value.

10. In any one of paragraphs 1 to 9, The electronic device, by the at least one processor (1110) executing the one or more instructions, An electronic device that identifies the presence of the user within the first ultrasonic-based detection range based on the presence of the user within the second ultrasonic-based detection range.

11. In the operating method of an electronic device (1100), A step of identifying the presence of a user within the detection range of WiFi (wireless fidelity) CSI (channel state information) when audio received through a microphone (1152) corresponds to a preset condition; and A method comprising a step of identifying whether the user is present within a first detection range based on ultrasonic waves when the presence of the user within the detection range of the WiFi CSI is identified.

12. In paragraph 11, The step of identifying whether the user is present within the first detection range based on the ultrasound is as follows: If the received audio corresponds to a preset condition and there is a user within the detection range of the WiFi CSI, a step of outputting an ultrasonic signal of the first intensity through a speaker (1164); and A method comprising a step of identifying whether the user is present within the first detection range based on the ultrasonic signal of the first intensity outputted above and the ultrasonic signal input through the microphone (1152).

13. In paragraph 11 or 12, A method comprising a step of controlling a screen corresponding to the presence of a user to be output based on the presence of the user within the first detection range based on the ultrasonic wave.

14. In any one of paragraphs 11 to 13, The step of identifying whether the user is present within the first detection range based on the ultrasound is as follows: A method comprising a step of controlling a change from a first state in which power consumption is lower than a preset value to a second state in which power consumption is higher than the preset value based on the presence of the user within the detection range of the WiFi CSI.

15. A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 11 to 14 on a computer.

Images